Running self-tests under valgrind
=================================

For projects written in C or similar languages, running the self-tests under Valgrind can reveal hard to find memory issues.  Gnulib supports two ways to make use of Valgrind: one that enables use of Valgrind at configure time, when configure found it to be present; and one at the discretion of the developer.

Using valgrind without developer intervention
---------------------------------------------

The valgrind-tests module searches for Valgrind at configure time and declares the LOG_VALGRIND automake variable for use with automake's LOG_COMPILER.

After importing the valgrind-tests module to your project, you use it by adding the following to the Makefile.am that runs the self-tests:

     LOG_COMPILER = $(LOG_VALGRIND)

This will run all self-checks under valgrind.

Replace LOG_COMPILER with TESTS_ENVIRONMENT if you are using the old serial test harness.  The parallel test harness has been the default in automake since version 1.11.3, but if you are using an older automake, or put serial-tests in AM_INIT_AUTOMAKE/AUTOMAKE_OPTIONS you would still be using the serial test harness.

If you desire a project-wide decision that valgrind is not enabled by default, but still allow users to enable it with --enable-valgrind-tests you may put the following in configure.ac before gl_INIT.

     gl_VALGRIND_TESTS_DEFAULT_NO

Valgrind options
----------------

The VALGRIND variable holds the name of the valgrind binary and some options passed to valgrind.  You may provide additional options that are passed to valgrind using the VALGRINDFLAGS variable, for example:

     ./configure VALGRINDFLAGS="--suppressions=~/local.supp"

Alternatively during build phase:

     make check VALGRINDFLAGS="--suppressions=~/local.supp"

This is useful if you have a valgrind suppression files that are needed to avoid triggering errors for known errors, typically in system libraries.

The VALGRIND variable include options that are useful when valgrind is run non-interactively through the test harness.  The default parameters are -q to silence the output, --error-exitcode=1 to cause valgrind errors to be treated as fatal errors, and --leak-check=full to check for memory leaks.

These options can be controlled through the DEFAULT_VALGRINDFLAGS variable.  For example, when configuring the package:

     ./configure DEFAULT_VALGRINDFLAGS="--quiet"

Alternatively, during the build phase:

     make check DEFAULT_VALGRINDFLAGS="--quiet"

That would have the effect of removing --error-exitcode=1 and --leak-check=full from the default options, thus causing any valgrind errors to be silently ignored, instead of causing fatal test failures.

As a developer you may use the variables in configure.ac before calling gl_INIT, like this if your program has deeply-nested call chains:

     gl_EARLY
     ...
     VALGRINDFLAGS="$VALGRINDFLAGS --num-callers=42"
     ...
     gl_INIT

Note that any user-supplied VALGRINDFLAGS value is preserved, which is usually what you want.

Finally, as a developer you may want to provide additional per-directory options to valgrind and the AM_VALGRINDFLAGS variable can be used for this.  For example:

     AM_VALGRINDFLAGS = --suppressions=$(srcdir)/local-valgrind.supp
     LOG_COMPILER = $(LOG_VALGRIND)

Using valgrind at the developer's discretion
--------------------------------------------

In this approach, you define a Makefile.am variable VALGRIND (or, more abstractly, CHECKER), that is usually set to empty. When you have configured and built the package and you decide that you want to run the tests with valgrind, you do so by modifying the definition of VALGRIND in the Makefile.

How to use Valgrind with shell scripts
--------------------------------------

It is not desirable to apply valgrind to shell scripts or other non-binaries, because

  * It is wasteful, and you usually don't want to look for memory leaks in bash.

  * On a bi-arch system, you may get an error message such as "valgrind: wrong executable class (eg. 32-bit instead of 64-bit)".

There are two ways to avoid this:

  * Using the Automake parallel-tests feature, you can use the following instead:

          TEST_EXTENSIONS = .pl .sh
          LOG_COMPILER = $(LOG_VALGRIND)

     Then valgrind will only be used for the non-.sh and non-.pl tests.

     For old automake (before 1.11.3), you will need AUTOMAKE_OPTIONS = parallel-tests to enable the parallel test harness.

  * You can make use of the build-aux/run-test script from Gnulib. Add these lines to your Makefile.am:

          LOG_COMPILER += $(SHELL) $(top_srcdir)/build-aux/run-test '$(LOG_VALGRIND)'

     Replace LOG_COMPILER with TESTS_ENVIRONMENT if you use the old serial test harness.

However, with this measure in place, binaries invoked through scripts will not be invoked under valgrind.  This can be solved by defining environment variables in the TESTS_ENVIRONMENT variable that are then used by the shell scripts.  For example, add the following:

     TESTS_ENVIRONMENT = VALGRIND='$(LOG_VALGRIND)'

And then modify the shell scripts to invoke the binary prefixed with $VALGRIND.