Combining Precision Tuning and Rewriting was published at ARITH 2021.
Precision tuning and rewriting can improve both the accuracy and speed of floating point expressions, yet these techniques are typically applied separately. This paper explores how finer-grained interleaving of precision tuning and rewriting can help automatically generate a richer set of Pareto-optimal accuracy versus speed trade-offs.
We introduce Pherbie (Pareto Herbie), a tool providing both precision tuning and rewriting, and evaluate interleaving these two strategies at different granularities. Our results demonstrate that finer-grained interleavings improve both the Pareto curve of candidate implementations and overall optimization time. On a popular set of tests from the FPBench suite, Pherbie finds both implementations that are significantly more accurate for a given cost and significantly faster for a given accuracy bound compared to baselines using precision tuning and rewriting alone or in sequence.
The paper describes the modifications we made to Herbie to support multiple precisions and even multiple number systems (like fixed-point or posits), how we modified Herbie's internals to optimize for both cost and accuracy, and shows that interleaving rewriting and precision tuning is the key for achieving state-of-the-art speed-accuracy trade-offs for mathematical expressions.
The paper is available in PDF format. TeX sources are available upon request.
The Herbie talk was delivered by Brett Saiki and Oliver Flatt at ARITH 2021 on . The talk was prerecorded, so you can see the original talk below. You can also view the slides in Google Docs or PDF format.