WebAssembly has been an LLVM target for a long time.
Every release of LLVM brings regressions, but also new optimization passes that can affect WebAssembly.
But WebAssembly cannot be directly executed by CPUs. In order to be executed, it has to go through yet another compiler, or just-in-time compiler. So, even without new optimization passes, a slight change in the LLVM output can impact the WebAssembly compiler, and have significant impact on performance.
Zig 0.11 includes LLVM 16, while Zig 0.12 merged LLVM 17 the day it was released. At the time of testing, the C library for WASI (wasi-libc) was exactly the same in both compiler versions.
So, how does LLVM 17 compare to LLVM 16 performance-wise?
The picture below represents the performance gains of LLVM 17 (zig cc 0.11) over LLVM 16 (zig cc 0.12), as a percentage.
This is the standard libsodium benchmark, using libsodium 1.0.19, run with 500 iterations for each test, on a Zen4 CPU, {under,over}clocking turned off.
Exact same libsodium version and build script, with the simd128 and bulk_memory extensions turned on.
WebAssembly modules were compiled to native code and executed using Intel’s WebAssembly runtime (iwasm). In the previous libsodium benchmarks, iwasm was previously shown to be the fastest runtime, along with WebAssembly to C transpilers (w2c2, wasm2c).
A few regressions, some improvements, but also some massive gains, +14% and +17%, for two similar tests. With the exception of these tests, there’s nothing really significant.
At that point, it may be worth looking an extra revision back. What performance gains did we get from upgrading from LLVM 15 to LLVM 16?
Let’s find out, by using zig cc 0.10, that used LLVM 15.
Whoops! LLVM 15 is much faster than LLVM 16 on the AEGIS tests. It’s likely that there was a nasty regression for the particular patterns triggered by the AEGIS tests in LLVM 16.
So, maybe we should skip LLVM 16 and compare LLVM 15 to LLVM 17:
Verdict:
- LLVM 15 was the fastest
- LLVM 16 had a significant regression
- LLVM 17 fixes the LLVM 16 regression, but doesn’t bring anything over LLVM 15. Performance is very similar.
Note that we just benchmarked code written in C. Maybe code written in Zig, Swift, Rust, etc. performs differently.
Discuss it on Reddit’s WebAssembly forum.