After having a particularly good experience exploring and implementing unit tests in Visual Studio 2012 using the code coverage analysis found in the Premium SKU I decided it was time (past time!) to get a similar handle on testing my iOS applications.
Now I love Xcode but the tooling is, in my opinion, not as mature as Visual Studio in many respects: debugging experience, testing experience, analysis, and integration for 3rd-parties to name a few.
When it comes to unit testing and code coverage with Xcode, the story isn’t nearly as nice as that found in Visual Studio. For instance, your only option for running unit tests is to run all test cases at once. There is no test runner or dedicated UI and there is no way to run individual tests or easily toggle tests (other than individually toggling the membership of .m files in the test target). To put it simply, it’s a far cry from Visual Studio’s Test Explorer, much less something like CodeRush’s Test Runner. See the update at the bottom of the post for clarification on this.
When it comes to visualizing code coverage with Xcode – specifically iOS apps – there are several steps to take and a 3rd-party application is required.
The first step is to configure your iOS app target for code coverage. Select your app target in Xcode (not the unit test target) and, under Build Settings, enable the following settings for the Debug configuration:
- Generate Test Coverage Files
- Generate Profiling Code
- Instrument Program Flow
Now, as long as your project is setup with a unit test target, you can click Product and then Test to run your unit tests.
This is where things start to get not-as-pretty-as-Visual-Studio. What this did was generate a bunch of files in a deeply nested location on the file system, namely:
~/Library/Developer/Xcode/DerivedData/[Project Name]-[Random Characters]/Build/Intermediates/[Target Name].build/[Configuration Name]-iphonesimulator/[Target Name].build/Objects-normal/i386
Yes. Really. For example, this is the path for a sample I created:
If you check the contents of this folder after taking the above steps, you should now see a collection of DIA, GCDA, GCNO, and related files.
The next step is to find a friendly way to view these results. There is nothing for this built into Xcode or provided by Apple. This is where a nice open source application called CoverStory comes in handy. This application loads the files generated by Xcode and displays an output similar to other code coverage tools.
You can download CoverStory here. After running the app, click the File menu, then Open, and navigate to the same ridiculous path mentioned previously (press ⌘⇧. to show hidden files & folders in Finder). This should display the breakdown of code coverage for your application.
Selecting .m files will show the individual lines of code covered by the unit tests.
Finally, depending on the linker flags your project uses there may be additional steps required to generate GCDA files with iOS 6. I’ve covered steps for working around that issue in a separate blog post here.
My next step will be to look into GHUnit, an alternative to the unit testing bundled into the Xcode IDE (OCUnit). While not integrated into Xcode, GHUnit does offer several improvements such as the ability to run individual tests or failed tests.
UPDATE: Martin Pilkington was kind enough to call to my attention that there is another way to enable and disable tests and test cases using Xcode schemes. Click on Product, then Scheme, then Edit Scheme (or press ⌘<). On the left of the scheme sheet select the Test action. On the right, in the list under Tests, you can expand the individual test targets, test cases, and tests, and enable and disable them via the checkbox in the last column.
You can also multi-select multiple tests and test cases and right-click to disable or enable a group.
Thanks Martin! And while this is absolutely better than toggling the target membership of the .m test case files, I’d still like the ability to easily run a single test or test case. Right now, to target coverage of a single test or test case, I have to disable hundreds (on a larger project) of tests, then work on my one test, only to re-enable hundreds of tests afterwards.
What I’d personally like is the sort of simple features found in other unit test runners: easily select and run a single test, a single test case, failing tests, or all tests (without having to manually and individually select which tests to run).