Next: , Previous: Command-line Arguments, Up: Usage



6.3.2 Test Suite

The first program one should run is the automated test suite, which will verify that the build is valid and that the platform is able to execute Monod. To build the test suite, run make testsuite from the main directory. The product is the executable testing/alltests. In the testing/ directory, run ./alltests --help to view the command line options. In addition to the options described in Command-line Arguments, the following options are available:

--verbose
Display the description of each test run.
--runwait n
Set the number of seconds to run each long test.
--filter s
Run only tests whose description matches the regexp s.
To run all the tests, simply run ./alltests in the testing/ directory. The output will look like this:
     mathieu% ./alltests
     .................
     Delay for the swarm test 1
     Second delay for the swarm test 1
     Third delay for the swarm test 1
     Fourth delay for the swarm test 1
     .
     First delay for the swarm test 2
     Second delay for the swarm test 2
     ....
     First delay for the incubator test 1
     Second delay for the incubator test 1
     Executed 4665 bindings in one second
     .
     First delay for the incubator test 2
     Second delay for the incubator test 2
     Executed 1005 bindings in one second
     .
     First delay for the incubator test 3
     Executed 3663 bindings in one second
     .
     First delay for the incubator test 4
     Executed 114 bindings in one second
     .
     First delay for the incubator test 5
     Executed (299, 299) bindings in 0.1 second
     .
     Ran: 27 tests in: 4.95 Seconds
     OK
     mathieu%

The important piece of information is the OK on the last line, which indicates that all the tests were run successfully. This should be the case for any freshly obtained distribution. Any error in the test run will be very apparent, as in the following output:

     mathieu% ./alltests
     .................F
     First delay for the swarm test 2
     Second delay for the swarm test 2
     ....
     
          [...]
     
     .
     First delay for the incubator test 5
     Executed (234, 233) bindings in 0.1 second
     .
     ======================================================================
     FAIL: Testing Util:17:Simple residents with no exposure
     
     OUnit: Just created swarm should not be running.
     not equal
     ----------------------------------------------------------------------
     Ran: 27 tests in: 4.55 Seconds
     FAILED: Cases: 27 Tried: 27 Errors: 0 Failures: 1
     mathieu%

The F on the first line is an early indication of the test failure, while the details are given at the end of the entire run. The details are used by a developer to locate the precise test which failed and begin debugging.

Running this executable will create the file monod.log, containing the logging output according to the log level.

Be aware that the test suite may succeed or fail depending on the value of the logging level set through the command-line! Indeed, certain tests check for time-dependent results. Since the log level affects the performance, some results can be skewed. In general, running with Emerg will provide the best results.

You can also increase the length of time each long test is supposed to run with the --runwait option. If you set the log level to log many entries, you can increase the runwait to make the test run longer to make sure it ends up passing. It is also often useful to simply increase this parameter to find possible deadlocks that happen only probabilistically. Note that sometimes deadlocks manifest themselves in ways that the test suite does not recognize, by simply stalling the test. Having a CPU activity monitor running alongside the test is often useful: if the activity drops while the test is still running, there's probably a deadlock somewhere.

Also, the success of the tests may depend on whether the executable was byte-compiled or compiled to native code, and whether the OCaml runtime is using native threads or not. Again, in order to smooth out differences, increase the --runwait parameter. It seems like the default parameter of one second may not be quite enough in all cases.