Herbie 2.0 Release Notes

The Herbie developers are excited to announce Herbie 2.0! This release focuses clarity, transparency, and trade-offs.

What is Herbie? Herbie automatically improves the accuracy of floating point expressions. This avoids the bugs, errors, and surprises that so often occur with floating point arithmetic. Visit the main page to learn more about Herbie.

Join us! Join the Herbie developers on the FPBench Slack, at the monthly FPBench community meetings, and at FPTalks 2023.

Speed-accuracy trade-offs

The Accuracy vs Speed section of a Herbie report, which has an accuracy vs speedup Pareto plot on the left and a tabular view of the accuracy and speed of each of Herbie's alternatives on the right. — Herbie's new Accuracy vs Speed view, showing how multiple alternatives found by Herbie compare on these different axes.

Two years ago, Herbie 1.5 added the --pareto mode, wherein suggested multiple alternatives that trade off speed and accuracy. We've made this mode faster, clearer, and more reliable, and as a result, we've made this mode the default. Speed-accuracy trade-offs are central to numerics, and we're excited for Herbie to help users with these trade-offs.

New Metrics and Redesign

The metrics shown at the top of each Herbie report: percentage accurate, time, alternatives, and speedup. — The metrics now being showcased on a Herbie results page: percentage accuracy replaces the old "bits of error" metric, while "speedup" now highlights the speed of the fastest alternative found by Herbie that is also more accurate than the initial program.

We've cleaned up the report page significantly. Besides a visual redesign—we've refreshed and standardized the new fonts, colors, and spacing—we've switched to a new way of measuring accuracy and speed, which we think will be more intuitive for users and avoid some misconceptions we see users run into.

Derivations

Herbie can now explain step by step how it transformed your original input into the final program. This can help you understand how Herbie works, and potentially help us identify bugs or misbehaviors. More importantly, when Herbie comes up with something truly surprising, you can now have more confidence that the answer is correct.

Foundations for a Herbie Workbench

We've started building a prototype numerics workbench, codenamed "Odyssey". This release thus contains the beginnings of a REST API for interacting with Herbie. Our long-term goal is an interactive, task-oriented interface for working with floating-point accuracy, leveraging Herbie but also potentially other tools. A prototype is currently available in the VS Code store as the herbie-vscode plugin.

Other improvements

Inputs whose real-valued result rounds to floating-point infinity are now considered valid. This changes Herbie's accuracy estimates, sometimes significantly, but we found that typically this yields better final programs.
Most sections of the Herbie reports page now have "help" links directly to the relevant section of the documentation.
Many, many algorithms within Herbie, especially pruning, regimes, and binary search, have seen significant speed ups.
Graphs are now drawn using JavaScript, on the client side, making them look sharper, especially on high-DPI displays.
Herbie's localization pass was written to be more accurate on tricky cases, which improves Herbie's results.
Herbie's support for the tgamma and lgamma functions was rewritten to be more accurate (though it remains quite slow).

Try it out!

We want Herbie to be more useful to scientists, engineers, and programmers around the world. We've got a lot of features we're excited to work on in the coming months. Please report bugs or contribute.

If you find Herbie useful, let us know!