Blog

HotSpot Strikes Back: Is JEP 544 The End Of The GraalVM Startup Monopoly?

Exciting times for the HotSpot JVM: JEP 544: Ahead-of-Time Code Compilation was just proposed for Java 28. This promises to give the HotSpot AOT cache a startup performance that will rival GraalVM, without any of its hardships. But training the AOT cache is different from pre-compilation with GraalVM, so how do they really compare?

AOT Cache, The Backstory

Java has been trying to improve startup speed for more than 20 years, starting with class data sharing (CDS) in Java 5, adding application classes with AppCDS in Java 10 and streamlining the training process in Java 13. However, none of this actually made a really impressive impact in terms of numbers and adoption.

In 2022, Mark Reinhold announced "Project Leyden", aiming to massively improve Java startup. Fast-forward to Java 24, which included JEP 483, the HotSpot AOT cache. The AOT cache uses the ideas of CDS but adds a more general mechanism that also stores the linked state of classes and archived heap objects after a training run.

Java 25 made the AOT cache usable with JEP 514 and added method profiling result caching with JEP 515. Now, with the proposed JEP 544 to cache compiled methods, Project Leyden is on its home stretch to deliver the promised killer improvements for Java startup.

How Much Faster?

As with most performance improvements, the actual numbers can vary wildly depending on what an application is actually doing. A good real-world example is the install4j IDE. It needs to load 10K classes at startup before it can show its main window. On my MacBook, this used to take around 3 seconds. With the AOT cache in Java 25, it comes up after 1.5 seconds. Twice as fast is an incredible achievement and matters a lot for user experience, moving straight from "sluggish" to "snappy".

With Java 28, we are promised another such improvement, just by upgrading the JDK. A sub-second startup time for a complex IDE is incredibly fast, and we will need to permanently retire any old preconceptions about Java's slow startup. This is the performance graph shown in the JEP:

This is actually mind-blowing and underscores how bad things were before.

What About GraalVM?

From its roots as a polyglot VM and a C2 HotSpot compiler replacement, GraalVM has morphed into a native image generator for JVM bytecode. The ecosystem shows a lot of adoption, including Quarkus, Helidon and Spring Boot.

With GraalVM you get a single binary without the need for a JVM, and you get great performance, both for startup and for workloads.

If this is so awesome, why doesn't everyone use it? As with many good things, there is a catch, and it's the closed world requirement. Everything that the program can do needs to be known at build time. Dynamic behavior has to be declared upfront: reflection, dynamic proxies, resources, serialization, and the list goes on. Also, Swing apps are not supported by GraalVM native image at all (the install4j IDE is one).

Let's Compare Them

For the comparison, I've deliberately picked an app that favors GraalVM: BetterBranch is a nifty command line tool that reports the ahead/behind status of all local Git branches relative to the current branch, and it is deployed as a GraalVM binary.

Below are the numbers, measured with hyperfine on Linux. The "Real workload" column shows BetterBranch being run on its own repository, "Fast exit" runs the tool in a directory without a Git repository so it exits immediately with an error message.

JEP 544 is not in the JDK 28 early access builds yet, so the AOT code numbers were measured with the Project Leyden early access build.

For the workload column, the JVM was pinned to 2 cores, so the JIT compiler competes with the application for CPU. This is the setting that the JEP 544 performance graphs were measured in, and where cached AOT code has a real impact: it saves 33% of wall time compared to the cache without AOT code and 53% compared to the plain JVM.

So is it close? No, GraalVM is still far ahead. But the gap is shrinking substantially. If milliseconds count for the startup of your containerized application, pick GraalVM. If your application cannot cope with the restrictions of the closed world, use AOT, now available in Java 25 and becoming more amazing in Java 28.

How To Train?

One question for the AOT cache remains: How do you actually create the cache in practice? If you deploy your app internally, you can perform a training run in your build system or your deployment script. However, when you ship an app, what are the options? You must train on the system where the app actually runs.

For a desktop app, staging a training run on a customer's machine sounds quite impractical. You need to add a special VM parameter for the training run, and a different one to use a recorded AOT cache for subsequent runs. While the AOT cache speedup matters a lot for the UX of a desktop app, forcing the users to participate in such a dance is an unattractive proposition.

Enter the AOT cache feature of install4j: install4j launchers can optionally record an AOT cache when the application is started for the first time by passing -XX:AOTCacheOutput=<cache file> as a VM parameter. On subsequent runs, the cache is used with -XX:AOTCache=<cache file>.

By default, the cache is stored in a per-user cache directory. The cache file name includes a key derived from the application build and the Java runtime, so the cache is automatically regenerated after application updates and Java runtime changes. Caches of previous builds are cleaned up automatically, and the uninstaller removes all created caches.

If you want to end the training run programmatically, there is the HotSpotAOTCacheMXBean MBean for that since Java 25.0.4, on which you can invoke endRecording(), for example after the user interface has been displayed and initial background tasks have completed.

For command line applications, staging a training run in the installer with a fast exit path is the best option, because it avoids showing the currently unavoidable terminal output of the training to the user. We do this for the install4j command line compiler by executing it with the --version flag.

Verdict

To show the worst-case scenario for the JVM, I have deliberately chosen an example where the numbers flatter GraalVM. They do so because the app is very small. On larger apps, GraalVM's advantage of not having to bring up the JVM becomes less important. When the real work portion of an app grows, that fixed win shrinks proportionally and the AOT code cache will catch up. How much depends entirely on which app you measure. For our install4j IDE, the AOT cache without AOT code already yields a 50% startup time reduction compared to around 30% for the smaller command line app above.

What is evident is that startup times for the JVM with the AOT cache are getting more competitive with GraalVM, especially considering the restrictions that GraalVM requires. Java 25 is great now, and Java 28 will be even better in this respect.

If you ship software and want to take advantage of install4j's AOT cache recording feature to boost your startup times, try it out, no registration required.