Multithreading
Relevant resources
Documentation
Web
Podcasts
Sample code - Apple
Sample code - class
Multithreading is hard. It is one of the most challenging topics for modern computer science as we increase computing power, not by upping clock speeds, but by adding parallel processing cores and massively parallel GPUs.
For the definition of a thread, we can look to the Threading Programming Guide:
Threads are a relatively lightweight way to implement multiple paths of execution inside of an application.
Threads let you perform more than one action at the same time, and they execute independently of one another. On multicore or multi-CPU systems, each thread can run on a different core, spreading out the computational load. On a single core, instructions from threads are interleaved according to the scheduling algorithm in the OS kernel.
While the iPhone is not multicore, there are many situations where you would like to use multiple threads to perform some process in the background, while leaving your user interface responsive. Also, at some point iPhone OS devices will go multicore, so designing your application for multiple threads can lead to them to run faster on those future devices.
Run loops and the main thread
To understand why you would want to use threads on the iPhone, it helps to understand how your program executes on the device. When your application first starts, it creates and starts in motion a run loop. This run loop cycles around for the duration of your application, taking in user interface and system events, handling timers (which we'll talk about in a bit), and performing user interface updates, among other functions.
All actions of the run loop are performed on the main thread of your application. Any methods executed due to the run loop (which is effectively everything in a normal application) are performed on the main thread. This includes user interface updates and all touch events.
What this means is that if you perform a computationally expensive operation, or one that waits for a network service to respond, your application's interface will stop updating and will become unresponsive until that operation is completed. Performing these operations on a background thread (asynchronously) will leave the interface completely responsive.
One thing to watch for in multithreading your application is that all user interface updates must be performed on the main thread. In order to guarantee that user interface updates be performed on the main thread, you need to either call them or methods containing them using -performSelectorOnMainThread:withObject:waitUntilDone:. For example,
[self performSelectorOnMainThread:@selector(enableButtons) withObject:nil waitUntilDone:YES];
will send the message [self enableButtons] on the main thread, and will block the currently executing thread until that method finishes on the main thread. All actions within the -enableButtons method will run on the main thread, including all user interface updates.
Note that this method lets you run methods with one or fewer arguments. For more complex methods, you will need to construct an NSInvocation or use a proxy. An example of this is the following category method on NSObject:
- (void)performSelectorOnMainThread:(SEL)aSelector withObject:(id)arg response:(NSObject **)response error:(NSError **)error waitUntilDone:(BOOL)wait;
{
NSInvocation *invocation = [[NSInvocation invocationWithMethodSignature:[self methodSignatureForSelector:aSelector]] retain];
[invocation setTarget:self];
[invocation setSelector:aSelector];
[invocation setArgument:&arg atIndex:2];
[invocation setArgument:&response atIndex:3];
[invocation setArgument:&error atIndex:4];
[invocation retainArguments];
[invocation performSelectorOnMainThread:@selector(invoke) withObject:NULL waitUntilDone:wait];
[invocation release];
}
This method lets you perform a selector on the main thread that returns a response as an NSObject double pointer in the second argument, as well as returning an NSError as a double pointer in the third argument. This is done by constructing an NSInvocation based on the selector passed in, manually setting the arguments for the invocation, then performing -invoke on the invocation on the main thread. This can be modified to support any number of arguments.
To avoid all of this, you can use proxies, such as Peter Hosey's main-thread performing category or Dave Dribin's DDFoundation project. For example, Peter Hosey's version of a multi-argument selector performed on a main thread would look like the following:
[[obj performOnMainThread_PRH] doThing:42 argument:4 otherArgument:nil];
One thing to watch out for with these updates performed on the main thread is that they, too, can be blocked by other methods running on the main thread. This can jam up your entire application if you have a background thread that is waiting for something performed on the main thread to be finished, while something else on your main thread is waiting for your thread run its course.
Means of avoiding explicit multithreading
Before we cover how to create and control background threads in your application, it helps to know ways that you can avoid the complexity and debugging issues that threads bring.
Normally, if you update the user interface, then immediately perform a time-consuming operation, the interface will not update until after the operation is finished. This is because user interface updates happen at the end of the current run loop cycle. To delay your operation until after this update occurs, you can use code like the following:
[self performSelector:@selector(delayedAction) withObject:nil afterDelay:0.01];
The tiny delay simply means that it will be performed at the start of the next pass through the run loop, letting your UI updates happen first. This is still not an ideal situation, because your application will be unresponsive during the calculation, but it may be good enough for certain situations.
For regular actions that need to be performed at certain times, or that can be spread out over a certain duration, you can use an NSTimer. For example,
secondsTimer = [NSTimer scheduledTimerWithTimeInterval: 1.0f target: self selector:@selector(respondToTimer) userInfo:nil repeats:YES];
will create a timer that triggers the method -respondToTimer on self once a second. This timer is retained by the current run loop and will terminate and be released when you call
[secondsTimer invalidate];
A final way to avoid explicit multithreading is to rely on another framework to do it for you. Examples include Core Animation, which performs animations on a background thread, or an asynchronous NSURLConnection, which handles receipt of data in the background. These frameworks abstract away their multithreading from you, letting you take advantage of performance and responsiveness gains without worrying about thread management and safety.
Manual NSThreads
While there is much you can do without needing multiple threads in your application, at some point you will run into a case that requires you to offload some operation to the background in order to improve the performance of your application or improve its responsiveness. The first way you can do this is to manually create threads.
Creating threads is not as hard as it sounds. You can run a method on a background thread using a single command. Either
[NSThread detachNewThreadSelector:@selector(countingThread) toTarget:self withObject:nil];
or
[self performSelectorInBackground:@selector(countingThread) withObject:nil];
will do the same thing: perform [self countingThread] on a background thread. This method will run to completion on a background thread, in parallel with anything you keep doing on the main thread.
Your overall application has a global autorelease pool in place (look to main.m to see how this created), but your newly spawned thread does not. If you use an autoreleased object within this thread, you will see warnings like this appearing all over your console:
_NSAutoreleaseNoPool(): Object 0xf20a80 of class NSCFNumber autoreleased with no pool in place - just leaking
To prevent these objects from being leaked, you'll need to create a new autorelease pool at the start of your thread:
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
and release it at the end:
[pool release];
Beyond this basic support for firing off methods as new threads, you can create NSThread instances for manual management of the running state of particular threads. You can subclass NSThread, override the -main method with your own code to be run in the thread, and execute it with -start. Honestly, I've never used this approach, instead going to NSOperation for more complex multithreaded tasks, which we'll talk about later.
If you have the need to delay the execution of some part of your thread for a defined time, you can use several functions and methods. These include
usleep(10000);
which causes your code to delay by 10 milliseconds (10000 microseconds) before proceeding, and
[NSThread sleepForTimeInterval:0.01];
which does the same (sleep for 10 milliseconds, or 0.01 seconds). These can be used within a while() loop to simulate a timer on a background thread.
One thing to be aware of when dealing with threads is that any method called from within the background-threaded method will also run on that same thread. This includes methods called indirectly, such as those responding to notifications or key-value observing. Therefore, you might need to be careful about how you access properties or trigger notifications to avoid unexpected behavior.
Thread safety and locking
Separate threads within your application share common memory, which is one of the reasons why you'd use threads over separate tasks in a desktop application (the iPhone doesn't let you run other executables, so there's not much choice there). This shared memory lets you perform a calculation in a background thread, update a value as that thread progresses, and use that same value for display in your main interface.
However, there are many problems that can occur when changing shared values in different threads. Behind memory-related crashes (under-retention and out-of-memory problems) and thrown exceptions (bad selectors, index-out-of-bounds errors, etc.), I'd say thread safety issues were the #3 cause of crashes in iPhone applications. Threading issues are also the hardest to debug because they are often nondeterministic (the same sequence of events will crash sometimes, but not always) and can lead to subtle, non-crashing errors in your application.
The first thing to be aware of are the structures and frameworks that are and are not thread-safe. For a list of which Cocoa classes are safe to access simultaneously from multiple threads, see Appendix A: Thread Safety Summary in the Threading Programming Guide. As was mentioned previously, all user interface updates must be performed on the main thread and are not thread safe.
What is meant by thread safety? Because threads can share memory for data structures, they can access those structures at the same time. This can lead to a "race condition", where the outcome of the operation depends on the specific timing in which the multiple threads take action on this shared structure. For example, if one thread adds a new object to an NSMutableArray, then another array tries to access that array mid-addition, it may find the array in an unusable state and crash the application (or worse, get a garbage value back and keep running).
There are several ways of avoiding problems with shared data structures. We already talked about one, in the context of dealing with user interface updates: forcing actions to be performed on the main thread. By guaranteeing that all methods where a data structure will be accessed are running on the main thread when they do so, multiple accesses to that data structure will not happen. This is not the greatest approach, performance-wise, because frequent accesses on the main thread can cause your application to stutter and you'll lose some of the advantages that multithreading gives you. It also opens you up to potential halting bugs in your application where just the right conditions can force multiple threads to wait forever for the other to finish.
The second, and most common, way of handling these issues is to place locks around access to the shared data structures. The simplest way to do this in Objective-C is to use the @synchronized directive:
- (void)myMethod:(id)anObj
{
@synchronized(anObj)
{
// Alter data structure here
}
}
This locks multiple accesses from various threads to the -myMethod: method so that only one at a time can run the instructions within the @synchronized block. However, what you gain in simplicity you lose in performance, according to Christopher Wright's benchmarks, among others. It is for this reason that I discourage you from using this in production code.
A much more performant (3X faster than @synchronized) way of doing this is to use NSLock. NSLock is a class that lets you perform locking around your shared data structures using code like the following:
[myLock lock];
// Alter data structure here
[myLock unLock];
The NSLock needs to be initialized (using [[NSLock alloc] init]) somewhere in your code, then released later. Every access to the same data structure needs to be wrapped in calls to one particular NSLock. This way, if a thread tries to -lock the NSLock while another already has, it will freeze at that point until the other thread has unlocked the NSLock.
An even lower-level way of doing this, which is ~2X faster than NSLock in Christopher Wright's benchmarks, uses pthread mutexes. Rather than use an NSLock class, you create a mutex using
pthread_mutex_t tallyMutex;
pthread_mutex_init( &tallyMutex, NULL );
and lock data access with
pthread_mutex_lock(&tallyMutex);
// Alter data structure here
pthread_mutex_unlock(&tallyMutex);
This approach is not more complex than dealing with NSLocks, and can be significantly faster.
All locking like this comes with some cost in terms of performance. Mike Ash describes ways around locks in his appearance on Late Night Cocoa Episode 032, and we'll see a way that we can change the architecture of our applications to remove the need for locks in the next section.
The iPhone, with its single core, and the Macs, with their (typically) multiple cores, can perform differently when testing your application. You can force your Mac to run as a single core machine by running the CPUPalette application found at /Library/Application Support/HWPrefs/CPUPalette.app and using it to disable all but one processor core on your system.
NSOperations and NSOperationQueues
Firing off single background threads for tasks you know will run once and do so to completion is fine, but what happens when you have 10, 20, or 100 discrete tasks that you want to run in parallel? Each thread has a significant amount of overhead associated with it, so running all 100 at once will lead to a severe slowdown. Also, what if certain tasks depend on others being completed? Balancing all of this yourself quickly becomes a code nightmare
Apple has recognized this, and created a solution in the form of the NSOperation and NSOperationQueue classes. NSOperations are self-contained tasks which are performed on a background thread. These tasks are added to an NSOperationQueue, which governs how they are executed. The NSOperationQueue manages the NSOperations so that only a certain number of operations are running at a given time. Threads for these operations are re-used so that that tasks can be performed with in parallel with a minimum of overhead.
NSOperations can also have dependencies, which are obeyed by the NSOperationQueue when determining the order in which to execute operations. All dependencies of an operation will be satisfied before that operation is executed.
Apple is heavily encouraging developers to not use manual threads, but to switch to NSOperations where possible. This can be seen in the new Concurrency Programming Guide, which is centered around the use of NSOperations. NSOperations and NSOperationQueues can cause you to think about your parallel operations in a more logical way, leading to performance gains
The easiest way to create an NSOperation is by using an NSInvocationOperation:
NSInvocationOperation* theOp = [[NSInvocationOperation alloc] initWithTarget:self selector:@selector(myTaskMethod:) object:data];
This creates an NSOperation that will execute [self myTaskMethod] when its turn in the NSOperationQueue comes up.
For more involved operations, where you might like to have self-contained data or additional processing, you can subclass NSOperation. You can create your own initializer to populate any data for the operation. To implement the code to be executed when the operation is run, you need to override the -main method and place this code there.
Because NSOperations run on a separate thread, you'll need to remember to set up an autorelease pool within the -main method of any custom NSOperation, or within the method called from an NSInvocationOperation.
Once you have an NSOperation, you will need to add it to an NSOperationQueue for execution. Typically, this queue will be constructed and maintained by your view controller, or even an application delegate or singleton. To add an operation to a queue, you use code like the following:
[calculationQueue addOperation:calculationOperation1];
As soon as operations are added to the queue, they will start executing according to the scheduling performed by the queue. Once an operation is added to the queue, you should not modify it in any way.
There are a number of things that you can do to tweak the execution order of operations. First among them is the ability to set the priority of operations. If you have some operations that you would like to be performed preferentially before others, you can set the queue priority using code like the following:
[calculationOperation6 setQueuePriority:NSOperationQueuePriorityHigh];
In this case, the operation has been given a high priority, which will cause it to run ahead of other standard priority operations. The priority levels are NSOperationQueuePriorityVeryLow, NSOperationQueuePriorityLow, NSOperationQueuePriorityNormal, NSOperationQueuePriorityHigh, and NSOperationQueuePriorityVeryHigh. Note that you can't change the priority of an operation once it has been added to a queue.
To set up a dependency between one or more operations, you can use code like the following:
[calculationOperation2 addDependency:calculationOperation1];
This will make it so that calculationOperation1 must finish before calculationOperation2 can run. You can add as many dependencies as you would like to an NSOperation, even to operations that will be executing on a different queue, leading to very clean flow control of multithreaded code. However, you need to make sure that you do not create circular dependencies, because they will stall indefinitely in the queue.
A NSOperationQueue by default tries to run a reasonable number of simultaneous operations for your given hardware. You can tweak this to determine the maximum number of concurrent operations that will run in the queue using code like the following:
[calculationQueue setMaxConcurrentOperationCount:3];
This lets up to three operations run at once in the queue.
Using the concurrent operation count, you can actually avoid needing locks around shared resources, a problem we discussed earlier. For a shared resource, like a file that needs to be written to, you can create an NSOperationQueue and set its maximum concurrent operation count to 1. For any actions that need to write to or read from this source, you add them as operations to this queue. With a single-wide queue, you will be guaranteed that only one thread at a time will hit this resource, eliminating the need for any locking mechanism. Because this all takes place in user space, you can realize significant performance gains over kernel-level locks.
Another great reason to start using NSOperations and NSOperationQueues now is that they fit perfectly with Grand Central Dispatch (GCD). Currently only available on Snow Leopard, GCD is a system-wide load-balancing technology that Apple has created to improve the performance of ths OS as a whole. GCD uses C structures called blocks and a series of dispatch queues to break parallel tasks down into small chunks and balance them across multiple cores. Basically, it extends the concepts of NSOperationQueues to the entire system, and any application using GCD benefits from having its load scaled depending on the available resources of the system.
GCD makes complex thread management much easier, and because blocks are similar in scope to NSOperations and dispatch queues to NSOperationQueues, if your application uses them in Snow Leopard you automatically get the benefits of GCD in your application for free. I believe that it is only a matter of time before GCD works its way onto the iPhone, so if you start using NSOperations and NSOperationQueues now, you'll reap the rewards of this when that happens.