CTest to run executables

In my previous blog post I mentioned some sample code I wrote to show how to use the Boost Unit Test Framework. This sample code, which I published at Bitbucket in repo cmake-boost-test, also serves another purpose, namely to show how to use CMake to build the code.

Since writing that post I have updated the CMake build description - called the filelist - to generate a Makefile target that runs the test executables. Before these changes, I had to run the executables explicitly, for example:

$> make
[ 14%] Built target my-boost-test
[ 42%] Built target my-dynamic-boost-test
[ 71%] Built target my-static-boost-test
[100%] Built target my-static-name-boost-test
$> ./my-boost-test
Running 2 test cases...
/home/pieter/bitbucket.org/cmake-boost-test/tests.h(10): error in "TestThatFails": check false failed

*** No errors detected

With the updated filelist, I can execute all tests with a single command:

Running tests...
Test project /home/pieter/bitbucket.org/cmake-boost-test/build/beck
    Start 1: my-boost-test
1/4 Test #1: my-boost-test ....................   Passed    0.01 sec
    Start 2: my-static-boost-test
2/4 Test #2: my-static-boost-test .............   Passed    0.23 sec
    Start 3: my-static-name-boost-test
3/4 Test #3: my-static-name-boost-test ........   Passed    0.07 sec
    Start 4: my-dynamic-boost-test
4/4 Test #4: my-dynamic-boost-test ............   Passed    0.23 sec

100% tests passed, 0 tests failed out of 4

Total Test time (real) =   0.57 sec

This is made possible by CTest, which is distributed as a part of CMake. CTest can do a lot more than what I use it for at the moment, "automate updating, configuring, building, testing, performing memory checking, performing coverage"[1]. The remainder of this post describes how I use CTest. You can find this description also in the README of the aforementioned repo.

To generate the Makefile target test, I added the following lines to the CMake listfile:

enable_testing()

# The first parameter specifies the name of the test. The second parameter the
# path to the executable to execute.
add_test(my-boost-test ${CMAKE_CURRENT_BINARY_DIR}/my-boost-test)
# When the second parameter specifies an executable target, you can omit the
# path as is done in the following examples
add_test(my-static-boost-test my-static-boost-test)
add_test(my-static-name-boost-test my-static-name-boost-test)
add_test(my-dynamic-boost-test my-dynamic-boost-test)

The command make test runs ctest. If you use ctest directly, you can specify which tests to run. For example,:

$> ctest -R my-boost-test

only executes my-boost-test and gives me the following output:

Test project /home/pieter/bitbucket.org/cmake-boost-test/build/beck
    Start 1: my-boost-test
1/1 Test #1: my-boost-test ....................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.03 sec

The option -R tells ctest to only run the tests whose name are matched by the regular expression that follows the option. So in the previous example, the command ctest -R my-boost would have executed the same test.

The above is not even the tip of the iceberg of functionality that CTest provides. To get a brief description of all the CTest options, execute:

$> ctest --help

For a more elaborate description in HTML, execute:

$> ctest --help-html