Skip to content

perf: Link dllbinaryen.so from libbinaryen.a instead of compiling Binaryen twice - #179

Merged
ospencer merged 2 commits into
grain-lang:mainfrom
mtelvers:link-so-from-static-archive
Sep 7, 2026
Merged

ospencer merged 2 commits into
grain-lang:mainfrom
mtelvers:link-so-from-static-archive

Conversation

@mtelvers

@mtelvers mtelvers commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

On every platform, the dune file compiles the whole Binaryen tree twice: once with -DBUILD_SHARED_LIBS=OFF to produce
libbinaryen.a, and then again with -DBUILD_SHARED_LIBS=ON to produce
dllbinaryen.so. Both runs configure into the same binaryen build
directory, and because the toggle changes the compile definitions, CMake
rebuilds all ~270 translation units the second time.

The second compile is redundant on these platforms:

  • Binaryen's CMakeLists.txt always adds -fPIC on non-Windows,
    non-Emscripten builds, so the objects in the static archive are already
    position independent.
  • The BUILD_SHARED_LIBS definition is only consulted under _MSC_VER in
    src/binaryen-c.h, so the objects are otherwise identical.
  • llvm_dwarf is an OBJECT library, so its objects are already inside
    libbinaryen.a.

This change makes the dllbinaryen.so / dllbinaryen.dll rules depend on
libbinaryen.a and link it with c++ -shared -Wl,--whole-archive (Linux and
other Unix, keeping the -Bsymbolic and --no-undefined flags Binaryen's own
CMake uses there), c++ -dynamiclib -Wl,-all_load (macOS), or
x86_64-w64-mingw32-g++ -shared -Wl,--whole-archive (mingw64, the compiler
the previous rule passed to CMake). c++ is the compiler CMake picks by default, so the shared object is
linked against the same C++ runtime the objects were compiled with (libstdc++
with GCC on Linux, libc++ with Clang on FreeBSD).

Verification (x86_64 Linux, GCC 13, and v129/v132 trees)

  • nm -D --defined-only on the relinked dllbinaryen.so lists exactly the
    same exported symbols as the libbinaryen.so produced by CMake's own
    shared build (25,600 symbols on v129, 15,511 on v132, zero differences).
  • dune build -p libbinaryen @install and dune runtest pass.
  • A bytecode executable linked against the test library loads
    dllbinaryen.so through ocamlrun and runs.
  • A C program linked against the relinked dllbinaryen.so creates,
    validates, optimises and prints a module.
  • The link command was also exercised on FreeBSD 15.1 (Clang 19, lld).
  • On macOS 15.5 (arm64, Apple clang 17) the relinked dylib exports exactly
    the same 10,480 symbols as CMake's libbinaryen.dylib, depends on the same
    libc++/libSystem, and passes the dlopen and C smoke tests.
  • On Windows (mingw64 OCaml 5.4, Cygwin-hosted MinGW GCC 14.4) the relinked
    dllbinaryen.dll exports exactly the same 14,388 symbols as CMake's DLL and
    imports the same runtime DLLs. The relink takes ~25 s where the second
    compile took ~20 min.

Effect

The full Binaryen compile is roughly 50 CPU-minutes on a fast x86_64 machine
and several CPU-hours on RISC-V, so this halves the cost of installing the
package. On the opam-repository CI, the RISC-V workers currently hit the
4-hour job timeout just after finishing the first compile; with this change
they finish with time to spare.

No platform still compiles Binaryen twice; the esy Windows build was not exercised, only the opam/Cygwin one.

Under Cygwin/MSYS2 CMake cannot find a usable C/C++ compiler on its own.
Previously the shared-library rule configured the same build directory
first with -DCMAKE_C_COMPILER/-DCMAKE_CXX_COMPILER/-DCMAKE_SYSTEM_NAME,
and the static rule inherited them from CMakeCache.txt. Now that the
shared library is linked from the static archive nothing seeds the
cache, so give the static rule the same settings on mingw64.
@mtelvers

mtelvers commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

The first CI run failed on all three Windows jobs (and on ubuntu-latest, 5.3.0 with an unrelated GitHub 504 fetching ppx_derivers). The Windows failures were caused by this change, and 2eaaa22 fixes them:

  • esy: No CMAKE_C_COMPILER could be found when configuring the static archive.
  • opam: CMake picked /cygdrive/c/mingw64/bin/cc.exe, which cannot compile a test program given Cygwin paths.

Previously the shared-library rule ran cmake first into the same binaryen build directory with -DCMAKE_C_COMPILER=x86_64-w64-mingw32-gcc -DCMAKE_CXX_COMPILER=x86_64-w64-mingw32-g++ -DCMAKE_SYSTEM_NAME=Windows, and the static rule's configure inherited those from CMakeCache.txt. With the second cmake run gone, nothing seeded the cache. The fix gives the mingw64 static rule the same three settings explicitly; other platforms are unchanged.

Verified in a fresh Cygwin + MinGW GCC 14.4 container with OCaml 5.4 (mingw64): dune build -p libbinaryen @install configures and compiles the static archive on its own, and the relinked dllbinaryen.dll exports 14,388 symbols and imports the same runtime DLLs as the CMake-built one.

@ospencer

ospencer commented Sep 7, 2026

Copy link
Copy Markdown
Member

Thanks so much!

@ospencer
ospencer merged commit 3126958 into grain-lang:main Sep 7, 2026
9 checks passed
@mtelvers
mtelvers deleted the link-so-from-static-archive branch September 7, 2026 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants