Same Builder, Different Binary

· 9 min read
Contents

Same builder image. Same optimization level 2. Same x86-64-v3. Same Serial GC. If eighteen native binaries come out of one GraalVM container with the same optimization level, the same target machine and the same collector, they were built the same way. That was the assumption behind every native row in this benchmark, and for the upgrade to GraalVM 25i4 I checked it the way it always gets checked: by reading the builder banner.

Each native-image run prints a configuration block before it compiles anything. This one is from a Quarkus build:

#17 22.65  Builder configuration:
#17 22.65  - Java version: 25.0.4.1.1+1, vendor version: GraalVM CE 25.4.4.1.1+1.1
#17 22.65  - Graal compiler: optimization level: 2, target machine: x86-64-v3
#17 22.65  - C compiler: gcc (redhat, x86_64, 14.3.1)
[...]
#17 22.65  Image configuration:
#17 22.65  - Garbage collector: Serial GC (max heap size: 80% of RAM)

All 18 build logs (Micronaut, Quarkus, Spring Boot, six branches each) print the same GraalVM version, the same optimization level: 2, the same x86-64-v3 and the same Serial GC. Four out of four checks matched, so they were equal.

They weren't. The banner never mentions how the binary gets linked.

An ELF file starts with a 16-byte e_ident array, followed by a 16-bit e_type (elf(5)). e_type says what kind of object the file is. A classic fixed-address executable is ET_EXEC. A position-independent executable (PIE) is ET_DYN, the same type as a shared library, because it can be loaded at any base address. On little-endian x86-64, one byte at offset 16 answers the question, and all three runtime images ship od. These runs are from before the fix below; the Quarkus image at that point was sha256:8bc5f73f8d99, built from the Wave 2 changes:

$ docker run --rm --entrypoint od quarkus-imperative-synth-app -An -tx1 -j16 -N1 /app/application
 02
$ docker run --rm --entrypoint od micronaut-imperative-synth-app -An -tx1 -j16 -N1 /app/micronaut-native-as-hell
 03
$ docker run --rm --entrypoint od springboot-imperative-synth-app -An -tx1 -j16 -N1 /app/spring-native-as-hell
 03

readelf spells out the same answer. Here it reads the Quarkus and Micronaut binaries copied out of those images:

$ readelf -h application | grep Type
  Type:                              EXEC (Executable file)
$ readelf -h micronaut-native-as-hell | grep Type
  Type:                              DYN (Position-Independent Executable file)

So Micronaut and Spring Boot came out PIE, and Quarkus didn't. All three binaries came out of the same builder image, and every one embeds the string GraalVM CE 25.4.4.1.1+1.1.

One more image was still sitting in the local docker cache: a Spring Boot native from early September, whose binary embeds GraalVM CE 25.0.2+10.1:

$ docker run --rm --entrypoint od benchrun-app -An -tx1 -j16 -N1 /app/spring-native-as-hell
 02

Spring Boot was non-PIE on the old builder and is PIE on the new one. Nobody added a linker option to a Spring Boot branch in between.

GraalVM 25.1 changed the default for every Linux build#

Between those two builds, GraalVM 25.1 shipped this line in its release notes:

Linux builds now produce position-independent executables (PIE) by default. This can be disabled with -H:NativeLinkerOption=-no-pie.

The benchmark moved from the CE 25.0.2 builder straight to 25i4 (GraalVM 25.4.4.1.1). That jump crosses 25.1, so the new default arrived with the image tag. Micronaut's native-image.properties carries only -H:+SharedArenaSupport --enable-native-access=ALL-UNNAMED, and the Spring Boot poms pass no build arguments at all. Neither project sets a link mode, so both got whatever the builder does by default, and the default had changed.

The same release turned on relative code pointers for PIE images. The JDK 25 release notes introduced them as an experimental option "to significantly reduce relocation entries in position-independent executables". Keep that in mind for the cost section below.

Quarkus passes -no-pie on its own#

Quarkus came out non-PIE because it chooses the link mode itself. All six Quarkus native build logs contain the same option in the generated native-image command line:

[...] -H:-AddAllCharsets --enable-url-protocols=http,https -H:NativeLinkerOption=-no-pie --enable-monitoring=heapdump,threaddump [...]

That option isn't in any file in the repository. Quarkus 3.40.1 generates it in NativeImageBuildStep:

String pie = "";

boolean isContainerBuild = nativeImageRunner.isContainerBuild();
if (!isContainerBuild && OS.LINUX.isCurrent()) {
    if (nativeConfig.pie().isPresent() && nativeConfig.pie().get()) {
        pie = detectPIE();
    } else {
        pie = detectNoPIE();
    }
}

detectNoPIE() checks that cc accepts -no-pie, and the result goes out as -H:NativeLinkerOption=-no-pie. The switch is quarkus.native.pie, and its javadoc gives the reason for the default: PIE is disabled where the system allows it "for performance reasons", with a link to a Red Hat article on PIE performance. Quarkus has a documented opinion here, and that's reasonable. The surprise is only that it's the one framework of the three with an opinion.

The first condition has a catch too. A Quarkus container build (quarkus.native.container-build=true) skips the whole branch and passes no linker option, so the link mode is whatever that builder image does by default. The benchmark's Dockerfile runs Maven inside the GraalVM builder image, which Quarkus treats as a local Linux build, so -no-pie applies. The same application built in container-build mode on a 25.1-or-later GraalVM builder would come out PIE. Quarkus's documented default container builder is mandrel, and I haven't checked what that one does.

What PIE costs here is unmeasured#

Mechanically, PIE trades a fixed load address for work at startup. The binary gets its base address at load time, and the dynamic linker patches every relative relocation by adding that base and writing the result back. Ulrich Drepper's How To Write Shared Libraries describes that operation. A native image is compiled code plus a pre-initialised image heap, and GraalVM added relative code pointers to cut its relocation count as a PIE.

How much that shows up in throughput, startup time or RSS for these 18 binaries, I don't know. Nothing has measured it yet. The rerun will, because each row will now record the link mode next to the numbers. Until then, any gap between Quarkus and the other two in the native rows may include the linker as well as the framework. Spring Boot's change from the 25.0.2 run also includes a link-mode change that Quarkus's doesn't, and so does Micronaut's, if it followed the old default.

A native comparison needs one link mode across the board. That leaves two consistent options.

  • All non-PIE. Add -H:NativeLinkerOption=-no-pie to Micronaut and Spring Boot. The native rows then line up with the 25.0.2 run, where PIE wasn't the default. The Quarkus build log from the 1 September run shows -H:NativeLinkerOption=-no-pie on the 25.0.2 builder, and the Spring Boot binary above reads 02, so the old-versus-new comparison stays a GraalVM-and-framework comparison. The catch on Spring Boot is that the flag has to live somewhere Spring's AOT step doesn't overwrite.
  • All PIE. Leave Micronaut and Spring Boot on the GraalVM default and get Quarkus to link -pie too. That follows where GraalVM went with 25.1 and gives up continuity with the 25.0.2 rows on link mode.

The obvious choice was non-PIE, because that keeps the history comparable. Before deciding, I had to ask what PIE actually was. Then I went with PIE wherever possible. Only after that did I read the Quarkus source for a way to do it, and found the switch: no second linker flag fighting the first, just one build-time property:

quarkus.native.pie=true

All six Quarkus branches rebuilt with it have -H:NativeLinkerOption=-pie in their build logs where -no-pie used to be (committed as 73886c80, 7ada2a42, 58772b55, 5b7a2f8d, 206aeb98 and 3113985c). The imperative three read:

$ docker run --rm --entrypoint od quarkus-imperative-jdbc-synth-app -An -tx1 -j16 -N1 /app/application
 03

$ docker run --rm --entrypoint od quarkus-imperative-synth-app -An -tx1 -j16 -N1 /app/application
 03
$ docker run --rm --entrypoint od quarkus-imperative-synth-more-io-app -An -tx1 -j16 -N1 /app/application
 03

And the reactive three:

$ docker run --rm --entrypoint od quarkus-reactive-r2dbc-synth-app -An -tx1 -j16 -N1 /app/application
 03
$ docker run --rm --entrypoint od quarkus-reactive-synth-app -An -tx1 -j16 -N1 /app/application
 03
$ docker run --rm --entrypoint od quarkus-reactive-synth-more-io-app -An -tx1 -j16 -N1 /app/application
 03

Eighteen binaries, one link mode.

The fix that outlasts this decision is in the harness. On all 18 branches, scenario.sh (the block is identical on every tip as of 16 October) now reads the same byte from the binary inside the running container and writes it into each scenario's config.txt, next to the image ID that actually ran:

case "$COMPOSE_FILE" in
  *java*) echo "pie=n/a" ;;
  *)
    _bin=$(docker inspect -f '{{index .Config.Entrypoint 0}}' "$_app_cid" 2>/dev/null || true)
    case "$(docker exec "$_app_cid" od -An -tx1 -j16 -N1 "$_bin" 2>/dev/null | tr -d ' \n' || true)" in
      03) echo "pie=true" ;;
      02) echo "pie=false" ;;
      *) echo "pie=unknown" ;;
    esac ;;
esac

A dry run of the first version of these lines, on a Micronaut branch before they were rewritten and copied to the other twelve:

--- config.txt dry run:
app_image=sha256:3cc0034c562cb90e96f30e5f068e64ecc8f2d959e2a555fa5e117d1b9f852e22
pie=true

The banner, the repository config and the image tag all said the 18 builds were the same, and each of them was accurate as far as it went. The one place the difference showed was a single option buried in Quarkus's generated command line. config.txt already carried mem_limit, java_tool_options, native_extra_args and the builder image digest for every scenario. Now it carries pie as well.

That matters for the rerun, which is the first time anything will measure PIE on these binaries. Whatever startup or RSS gap it finds between the frameworks, the row will say which link mode it ran with. Until then, nobody gets a number for what PIE costs here, me included.

Filed under: java, graalvm, native-image, quarkus, micronaut, spring-boot, benchmarks

Sources#