Unit testing
Relevant resources
Documentation
Web
Podcast
Video
Sample code - class
Unit testing is a process where you write code that automatically tests the code within your application by running it against a set of input parameters, verifying that the end result is what you expect for each provided condition. This testing code is run as part of a suite that can be run every time you build your application. The goal of unit testing is to provide another layer of verification that your application runs as expected, in addition to compiler-time checking for syntax errors or undefined symbols, and code analysis tools like the Clang Static Analyzer.
Unit testing has not caught on among Cocoa programmers like it has for most other development platforms. Why this is true is still open for speculation, but it may be due to the fact that Cocoa developers tend to focus heavily on user interfaces, which are much more difficult to unit test, as opposed to data models or controller code. Wil Shipley argues the anti-unit-testing position in his article "Unit testing is teh suck, Urr.", which led to the pro-unit-testing response by Bill Bumgarner "Unit testing.", where he describes how the Core Data team was able to accelerate development using unit tests.
Unit tests are particularly important for those practicing test-driven development (TDD). Roughly, this development style advocates writing failing tests first, then write the code that will make these tests pass. The advocates of this approach claim that it provides cleaner, better-organized code, and by default it gives you tests for most of the inner functioning of your application. However, the tests do take time to write and they are made of code that you will need to debug and maintain in addition to your main application code.
Unit tests are not a perfect indicator that your application is bug-free. They can only test the exact cases you program them for and can't anticipate anything outside of that. However, they can be useful as regression tests to make sure that you do not reintroduce bugs. If someone submits a bug report, you can create a unit test that checks for that specific bug and make it pass to make sure that this bug won't recur in the future.
Using Doxygen to generate API documentation
Before we discuss writing handling unit tests in Xcode, it might be useful to point out ways of generating documentation for your code. Why would we discuss this first? Peter Hosey points out, in his video on Cocoa unit testing, that before you write any unit tests you should provide complete documentation as to how the method you're testing should behave. What inputs it should take, what results it should produce, what error states there are for the method, etc. need to be laid out before you know what to test.
Doxygen is probably the most popular of the documentation-generation tools among Cocoa programmers. When combined with doxyclean, it can create API documentation that mimics Apple's style.
Doxygen uses comments to generate the documentation for your application. It can recognize and handle most Objective-C language structures, using comments like the following:
/** @brief Adds two numbers and provides the sum.
*
* @param firstNumber The first number to add.
* @param secondNumber The second number to add.
* @return The sum of the two arguments.
**/
- (NSInteger)add:(NSInteger)firstNumber to:(NSInteger)secondNumber;
The Core Plot framework has a documentation policy which explains the proper usage of most of the documentation elements.
To build documentation for your project, first download Doxygen and install it in /Applications. Second, create a doxygen.config file (a sample one is included in the UnitTestExample application). Finally, you'll need to set up the script to run to compile and install your documentation (again, an example is present in the UnitTestExample application).
When you build documentation for your project, it will be created and installed in Xcode. You may need to restart Xcode to see it.
Unit testing in Xcode
Xcode includes built-in support for unit testing, which makes it easy for you to create and run these tests, as well as integrate them with your build process. Xcode's support for unit testing is build upon SenTestKit, a unit testing framework originally created by Sen:te and now distributed with Xcode. Following the naming convention of other xUnit test frameworks (where x stands for the language), this is also referred to as OCUnit.
To add unit tests to a project in Xcode, choose the menu option Project | New Target... and choose the Unit Test Bundle target type from the Cocoa Touch options. This will add a new target to your project that we will use to run our unit tests.
There are two general classes of unit tests: logic tests and application tests (also referred to as independent and dependent tests). Logic tests simply run against a part of your application's code, where application tests inject themselves into the full running application. On the iPhone, application tests only run on the device, so they can be much less convenient to use.
I'll only cover logic tests here. For a logic test, your next step will be to add unit test classes. Typically, you will want to add one class for each class you wish to test in your application. To add such a class, add a new file of the type Objective-C test case class from the iPhone OS grouping, making sure that this new class is added to the unit test target, not your application.
This will add a subclass of SenTestCase, which is where you will create each one of your tests for the corresponding class. Within this class, you'll create methods starting in -test. Each of these methods will be run as a separate test when your unit test bundle is executed. In general, it is recommended to have at least one test method for each method in the corresponding class. For example,
- (void)testAdd;
{
NSInteger sum = [operatorToTest add:1 to:2];
STAssertEquals(sum, 3, @"1 + 2 should equal 3, but instead returned %d", sum);
}
will test the -add:to: method of operatorToTest. To report a failure of this test, we use the STAssertEquals() macro, which fails and logs the third argument to the console if the first two arguments are not equal.
Several macros are defined for unit testing under Xcode. These include:
Note that you should not use NSAssert() or assert() in place of these macros in your unit tests.
Failures of tests appear as build errors when you switch to the unit test target and build it. If you double-click on the error within the Build Results window, you will be taken to the line where the appropriate assertion failed.
You'll notice that I did not declare the operatorToTest in this test. That's because I set it up ahead of time. If you have something that you would like to be set up ahead of the tests being run, then torn down afterward, the SenTestCase class has the methods -setUp and -tearDown that you can override to do this:
- (void)setUp
{
operatorToTest = [[MATCMathOperator alloc] init];
}
- (void)tearDown
{
[operatorToTest release];
}
Unit tests are great for testing data models, but you might be wondering how to best unit test Core Data. One available option is to set up an in-memory Core Data store in the -setUp method, then break it down in the -tearDown method. This will allow for fast testing without requiring a database to be maintained on disk.
You can make these unit tests run every time you compile your application by going to your application target, inspecting it, and clicking the plus button under the Direct Dependencies grouping in the General tab. Select the unit test target from the list to make it a direct dependency of your application target. This means that it must be built before your application can be. Any unit test errors will stop your build, forcing you to address them.
It's worth noting here that while this may add some overhead, it's generally regarded that unit test that take a very long time to run are probably architected incorrectly. Perhaps they require far too much setup to run or maybe your underlying code is not designed properly.
If you are dealing with more complex, interrelated objects (like network connections), you may wish to use mock objects for this. The OCMock framework provides a means to create these mock objects within the OCUnit testing environment. For how to use this on the iPhone, see Colin Barrett's "OCMock and the iPhone" article.
Debugging unit tests
If you build this unit test target, you may notice that you are getting errors about an executable not being present. In order to debug our unit tests, we'll need to create a custom executable for them. This can be done by selecting the Project | New Custom Executable... menu option and naming the executable. Give it a path of Developer/usr/bin/otest, relative to the current SDK.
Find the executable, inspect it, and go to the Arguments tab. Add
-SenTest Self
$(SRCROOT)/$(BUILT_PRODUCTS_DIR)/[executable_name].octest
and make sure they appear in exactly that order in the list. You should replace [executable_name] with whatever you called your custom executable.
As an alternative to using the built-in unit test support, Google has their own unit testing implementation in the Google Toolbox for Mac, which also works on the iPhone (see google-toolbox-for-mac: iPhone Unit Testing). This has some extensions to the base unit testing provided by SenTestKit, including some user interface unit testing.