Browse Source

Help:test prop WILL_FAIL clarify and example

This makes the WILL_FAIL behavior explicit. Users could be confused
because some other build systems treat equivalent test properties
as allowing a failure, but still pass on a passing test, and may
catch SIGABRT unlike CMake.

We show an example of how to wrap a test so that sigabrt, segfault,
etc. are also handled by WILL_FAIL.
scivision 2 years ago
parent
commit
7b4983536a
1 changed files with 31 additions and 4 deletions
  1. 31 4
      Help/prop_test/WILL_FAIL.rst

+ 31 - 4
Help/prop_test/WILL_FAIL.rst

@@ -1,8 +1,35 @@
 WILL_FAIL
 WILL_FAIL
 ---------
 ---------
 
 
-If set to true, this will invert the pass/fail flag of the test.
+If ``true``, inverts the pass / fail test criteria. Tests for which
+``WILL_FAIL`` is ``true`` fail with return code 0 and pass with non-zero
+return code. Tests that exceed the timeout specified by :prop_test:`TIMEOUT`
+still fail regardless of ``WILL_FAIL``.
 
 
-This property can be used for tests that are expected to fail and return a
-non-zero return code. Note that system-level test failures such as segmentation
-faults or heap errors will still fail the test even if ``WILL_FALL`` is true.
+Caveat: system-level test failures including segmentation faults,
+signal abort, or heap errors may fail the test even if ``WILL_FAIL`` is true.
+
+Example of a test that would ordinarily pass, but fails because ``WILL_FAIL``
+is ``true``:
+
+.. code-block:: cmake
+
+    add_test(NAME failed COMMAND ${CMAKE_COMMAND} -E true)
+    set_property(TEST failed PROPERTY WILL_FAIL true)
+
+To run a test that may have a system-level failure, but still pass if
+``WILL_FAIL`` is set, use a CMake command to wrap the executable run like:
+
+.. code-block:: cmake
+
+    add_executable(main main.c)
+
+    add_test(NAME sigabrt COMMAND ${CMAKE_COMMAND} -E env $<TARGET_FILE:main>)
+
+    set_property(TEST sigabrt PROPERTY WILL_FAIL TRUE)
+
+.. code-block:: c
+
+    #include <signal.h>
+
+    int main(void){ raise(SIGABRT); return 0; }