Using Herbgrind

Herbgrind analyzes binaries to find inaccurate floating point expressions. The binaries can come from anywhere—C source, Fortran source, even unknown origins. This tutorial runs Herbgrind on the benchmark programs that Herbgrind ships with.

Compiling the benchmark program

Herbgrind ships test binaries in its bench/ directory. You can build them with:

make -C bench all

Let's analyze the diff-roots-simple.out binary that you just compiled. Run Herbgrind on that binary with:

valgrind/herbgrind-install/bin/valgrind --tool=herbgrind bench/diff-roots-simple.out

This should produce output that looks like this:

==16725== Herbgrind, a valgrind tool for Herbie
==16725== 
==16725== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==16725== Command: bench/diff-roots-simple.out
==16725== 
1.578592e-07
==16725== 
Writing report out to bench/diff-roots-simple.out-errors.gh

The printed value, 1.578592e-07, is printed by the diff-roots-simple.out binary. Herbgrind writes its results to the named file, bench/diff-roots-simple.out-errors.gh. This file contains one record for each operation; the only operation found in diff-roots-simple.c is:

(FPCore ()
  :type binary64
  (- (sqrt (+ 1.000000 10000000000000.000000)) (sqrt 10000000000000.000000)))
subtraction in main at diff-roots-simple.c:12 (address 400A00)
43.129555 bits average error
43.129555 bits max error
Aggregated over 1 instances

The first line gives the expression inaccurately evaluated, and the second line gives its location. That line in diff-roots-simple.c is actually:

y = sqrt(x + 1) - sqrt(x);

Since this line of code is run only once, Herbgrind doesn't know that x is intended to be a variable, and instead inlines its value.

The next three lines of the output give the error incurred by the inaccurate computation: 43.1 bits of error over 1 instance of computing that expression.

Turning Herbgrind on and off

While running on diff-roots-simple.out, Herbgrind found inaccurate computations not only in diff-roots-simple.out but also in several GNU library calls. Herbgrind has a feature to avoid tracking floating point operations in libraries and other code not within your control by adding instrumentation to your source code.

Simply surround the numerically-interesting parts of your computation in the HERBGRIND_BEGIN() and HERBGRIND_END() macros:

// initialization code ...
HERBGRIND_BEGIN();
// numerical code ...
HERBGRIND_END();
// cleanup code ...

The diff-roots-simple.c example does this on lines 11 and 13. You can then run Herbgrind with the --start-off flag, which tells Herbgrind not to begin analyzing floating point operations until it sees a HERBGRIND_BEGIN() region:

valgrind/herbgrind-install/bin/valgrind --tool=herbgrind \
      --start-off bench/diff-roots-simple.out

The report file now contains only the inaccurate expression described before, and no library computations.

The HERBGRIND_BEGIN()/HERBGRIND_END() regions can be sprinkled anywhere in your source code; it's common to use them to start Herbgrind only after initializing your program and before cleaning up and outputting results. Herbgrind can be turned on and off multiple times.