When does JVM use JIT compilation

We saw in our previous post, three different scenarios of JIT compilation applied to our method. 

> JIT compilation was not applied at all when when the loop count was low

> We saw one JIT compile event log entry when loop is increased to 10K and 

> three entries (with two compilation id's) when loop count is increased to 1M.

Why is it varying depending on the no. of times our method is getting invoked? 

Why isn't the JIT compilation happening the same way all the time.  

Ideally it may be good to have JIT compilation done the same way all the time, but JIT compilation itself will consume resources, taking in CPU time and memory to do its job. 

The overhead incurred by JIT compilation, do not justify the benefit gained in executing compiled code produced, if the method is not getting invoked frequently. It will be better off to just interpret and execute that code. 

But if the method is getting invoked more frequently, it will be beneficial to compile that and convert it to native code.  

Hotspot JVM

For this reason, JVM keeps track of the sections of the code that are getting invoked frequently. Such code sections are are called hotspots. 

JVM only gives code sections that become hotspots for JIT compilation. The overhead incurred in JIT compilation of these hotspot code sections is justified by the performance gained in executing it multiple times over.

The name Hotspot JVM comes from this approach that the JVM takes to invoke JIT compilation. 

Optimization of compiled code:

Its one thing to compile the byte code to native assembly code. But the compiled code can also be optimized. Various levels of optimization are possible. Also more the no. of times the code is executed by the JVM, more information it has about the code which can be used for further optimization. 

The level to which optimization is done on the code during compilation, in itself adds an overhead, which can be justified only when the frequency of invocation is even higher. 

Tiered JIT compilation

JVM adopts tiered compilation of hotspot sections of the code. 

Initially code starts getting executed through interpretation

When the code section is identified as hot enough by JVM, it is compiled through JIT compilation, but less optimization is applied at this stage so as to avoid the overhead of optimization itself outweigh the benefit gained. Also some optimizations are possible only after the code section is invoked many more times.

When that code section has become even more hotter and JVM gains enough information for further optimization, aggressive optimization is applied to create the most optimized compiled code. 

This explains why when our method calculate() from the previous post is

• Invoked 100 times - JIT compilation is not applied
• Invoked 10K times - Single JIT compilation is applied
• Invoked 1M times - multiple JIT compilations are applied

Next we will move on to understand C1 & C2 JIT compilers and Tiered compilation options