GraalVM native images are slower right now, but there's no reason that has to be the case in the future. I think we all have realized over the last 2 decades that though JIT compilers (like the JVM) should be theoretically faster than a traditional native code compiler, this is rarely (if ever) the case. I wouldn't be surprised if over the next 10 years most JVM code (including Java, Scala, etc) is compiled to native images.
It's both clear that Oracle is pushing GraalVM hard and that it's what the consumer wants. If performance was identical, I can't think of a single reason why anyone would choose the regular JDK over GraalVM. It's not like people are slinging jars over the network between linux and windows... hell, now with WSL, that's not even a good reason either.
Actually the JIT compiler often outperforms native code when hot because the latter doesn’t typically get compiled with PGO and the JIT is effectively PGO at runtime.
In addition, the JIT can perform optimisations that a native compiler can’t do, because the JIT can take different decisions if a field is null repeatedly, and thus elide the inclusion of code that supports the non null case leading to potential further optimisations due to dead code elimination and additional inlining opportunities. This is something that PGO won’t give you – the native code compiler will have to be defensive and compile everything, not just what it measured on a few runs of test cases.
In the JIT’s case, when an assumption is violated, it falls back to the interpreter and recompiles with the new knowledge of the change and thus includes more code than before.
The speed advantages of the JIT tend not to be seen on comparison sites like Debian’s language shootout because they don’t exclude start up time and rarely run the code a sufficient number of times to trigger the JIT’s compilation (10,000 method calls by default) to become hot. It’s why the JVM world has tools like JMH to do micro benchmarking.
Finally in the Graal case, it’s actually possible to use the Graal compiler as the hotspot replacement so you can see the difference in speed and behaviour between the two. Graal has some nice wins but also some disadvantages; it’s good at fixing some of Scales’s generated code which is why Twitter are big fans, but on regular Java code the advantages are not as pronounced.
The main benefit to Graal is the reduction in startup time, not execution speed at runtime. For long running servers you are better off with the JVM’s JIT for performance; for short start up times (like AWS Lambda) you could be better off with jaotc or Graal.
And if using a recent Java version (meaning not 8), that JIT code is cached between executions and can even take PGO data into account.
There are a couple of talks from the .NET team regarding the same issue, and how they are planning that for full support of all .NET features, the way forward is a mix of AOT/JIT.
Android team also learned the hard way that AOT only wasn't as good idea as they thought, hence why ART now does all combinations, interpreted, JIT and AOT with PGO and code cache.
HotSpot uses a JIT compiler that dynamically recompiles code at runtime to speed up hot code paths. GraalVM native compilation cannot do this, and given the nature of Java's non-reified generics, it's very hard to get equivalent speeds at runtime. Profile-guided optimization is the only hope to recover some of that performance gap, and it's currently only available in GraalVM EE. I'm not sure how the benchmarks compare to HotSpot.
If the above RAM figures are correct (⅕ ram for native), and of JIT performance is 20% faster than native, then I’m pretty sure that it’s cheaper to add 20% more CPU than 5x more RAM. Especially now that we are talking Gravitons and chiplets.
Why would type erasure hurt performance? Certainly expressiveness is curtailed due to Java's generics design (or lack thereof), but I wasn't aware of any performance implications.
Whenever a generic class method returns an erased type and the caller tries to use it, javac has to insert a runtime check (cast) so the bytecode is typesafe. It also forces boxing of primitive types.
Because of erasure, there is only one copy of List#get(int), and it returns Object (because there's no other upper bound). The source might say
List<String> l = ...
String s = l.get(0);
which looks typesafe, but to pass the verifier and load, the bytecode has to accept an Object and cast it to String before using it, just like Java programmers had to write by hand before generics (this really sucked).
There's not even anything stopping you from putting non-Strings in that List instance via reflection or generic casts (if you ignore the warning) or using a really old javac. You could use a checked collection but that just adds a runtime check during writes that ensures the runtime check during reads will never fail (but the bytecode verifier still won't let you remove it!)
I should have written ArrayList#get. It's possible to write a class that implements List<String> in a typesafe way, but callers using the standard List interface receive an Object and have to downcast.
It's not very easy to compare java's performance with that of other compiled languages, because of the GC. The closest native analogue is go, which is much slower than java, although it's not quite a fair comparison since go uses a concurrent GC by default and java does not.
Java is quite fast for what it does, and I don't expect any runtime to make it significantly faster (aside the fact that java isn't slow, and its performance problems are memory usage and lack of value types, which native vs. JIT has no impact on).
Languages like Ocaml and Haskell are both around Java speed, though. And even though both languages have more primitive GCs than Java, in practice, they tend to use less memory and have lower latency.
I can only think of one situation when using Ocaml that I had to think about the GC (I was allocated huge floating point arrays), while I can't even count the number of times I had to deal with a runaway heap or unacceptably long pauses. The last JVM version I've really used was 7, though. Perhaps things have gotten better?
This point has to be noted. There are languages other than Go with optimizing compilers, and those compiling to C. Go has focus on fast compiles and doesn't do much optimization. And no generics means many things are done using reflection / reimplemented poorly and can be slow.
OCaml, Haskell, SML with MLTon, Nim, D, Crystal etc.. have better performance than Java unless one benchmarks a tight numerical loop. Not to mention memory consumption.
JITs are oversold. JIT may be great for optimizing Python or Lua. Moreover java is just poorly designed from a performance standpoint - no value types, virtual by default.
Value types are coming, EA are available and virtual by default has long stopped being a problem thanks to StrongTalk optimization regarding de-virtualization across shared libraries and call caches.
That "better performance" isn't reflecting in market share.
That the beauty of "OCaml, Haskell, SML with MLTon, Nim, D, C" and lack effectiveness of JIT compilers have done very little for them to take market share from Java in any significant way.
In case of C, on the contrary it lost to Java the majority of the roles it owned during 90's enterprise distributed computing applications.
Yes, things have gotten a lot better. G1GC is the default now and we have free options like ZGC and ShenandoahGC for low latency options. Of course there are the IBM options and Azul as well.
Scalar replacement got better, and with Graal partial evaluation gives reduced heap allocation rates. Which can have big impacts.
Future is even more bright with inline (value) types going towards even less allocations and better memory layouts.
You mean partial escape analysis and scalar replacement for reduced heap allocation rates. Partial evaluation is something else, also pretty useful but for other reasons.
> I think we all have realized over the last 2 decades that though JIT compilers (like the JVM) should be theoretically faster than a traditional native code compiler, this is rarely (if ever) the case.
This is a very broad claim. Do you have data to support it? It would mean comparing production-quality JIT and AOT compilers for the same language, which are hard to find.
I'm vaguely aware that LLVM's JIT ambitions never worked out because compile times were too slow, which (IIRC) they tried to mitigate by running fewer optimizations in JIT mode, which meant the quality of the generated code wasn't as good as AOT. I wouldn't call this production quality though, and even so it wouldn't allow you to make your claim as broadly as you are.
AOT compilation doesn't make good use of PGO data (you need a very good test data set) and there is zero chance of inlining across dynamic libraries.
What most developers want is not having to pay for their tools, hence why so much love for GraalVM community edition.
Some form of AOT compilation has been part of commercial JDKs since early 2000's.
In fact, the JIT cache that is now also available for free on OpenJDK traces back to J/Rockit and similarly the JIT code caches now available on OpenJ9 go all the way back to WebSphere Real Time JVM.
Also this is another point, by holding on to Java 8, those JIT caches are not available to those that only want the free JDK variants.
However it is a warm feeling that despite the critic, here is a project that they have kept aliven from Sun Labs (aka MaximeVM) and not only have brought it to production, they have a long term roadmap to replace C2 with it (Project Metropolis).
> What most developers want is not having to pay for their tools, hence why so much love for GraalVM community edition.
Well I agree that developers do not want to pay in general. But Graal VM CE/EE are firmly in enterprise domain, so companies have to take a call on Graal.
Indeed. And when documents say "Unless low memory usage is top requirement just use regular JDK for better performance." It does not give much confidence in native solution.
Given Java's outrageous memory usage I think this is pretty compelling benefit. It is not easy to provision resources for Java applications relative to just about everything else.
Does java come with the tools you need to manually manage your memory to avoid using the garbage collector? In unity with c#, the garbage collector is a major problem that is annoying to work around
That's not what they are asking for. Well there is epsilon GC but rather than writing java that doesn't allocate, I'd rather write C++ or Go.
This is the problem with Java ecosystem. Horribly reinventing simpler things in complex ways, from frameworks to build systems, and selling them as next big thing ™.
Then you have either the pleasure to win the lottery and be part of the 70% statistic, or do tricks with 1GB memory allocations to force the GC to work, choices.
Value type is overrated, lots of game engines choose C# because of value type, any game in C# has good performance or memory usage? Unity games are memory hog...
Somehow I think they would still be a memory hog if written in C, better not mix the capabilities of the tooling with the skills of those that use them.
