The Artima Developer Community
Sponsored Link

The Hotspot Virtual Machine
How Hotspot Can Improve Java Program Performance and Designs
by Bill Venners
First Published in, May 1998

<<  Page 3 of 4  >>


Hotspot and adaptive optimization

The Hotspot VM is a collection of techniques, the most significant of which is called "adaptive optimization." In fact, this technique gives Hotspot its name.

The original JVMs interpreted bytecodes one at a time. Second-generation JVMs added a JIT compiler, which compiles each method to native code upon first execution, then executes the native code. Thereafter, whenever the method is called, the native code is executed. The adaptive optimization technique used by Hotspot is a hybrid approach, one that combines bytecode interpretation and run-time compilation to native code.

The Hotspot VM begins by interpreting all code, but it monitors the execution of that code. As mentioned earlier, most programs spend 80 to 90 percent of their time executing 10 to 20 percent of the code. By monitoring the program execution, the Hotspot VM can figure out which methods represent the program's "hot spot" -- the 10 to 20 percent of the code that is executed 80 to 90 percent of the time.

When the Hotspot VM decides that a particular method is in the hot spot, it fires off a background thread that compiles those bytecodes to native and heavily optimizes the native code. Meanwhile, the program can still execute that method by interpreting its bytecodes. Because the program isn't held up and because the Hotspot VM is only compiling and optimizing the "hot spot" (perhaps 10 to 20 percent of the code), the Hotspot VM has more time than a traditional JIT to perform optimizations.

The adaptive optimization approach yields a program in which the code that is executed 80 to 90 percent of the time is native code as heavily optimized as statically compiled C++, with a memory footprint not much bigger than a fully interpreted Java program. In other words, fast. The Hotspot VM keeps the old bytecodes around in case a method moves out of the hot spot. (The hot spot may move somewhat as the program executes.) If a method moves out of the hot spot, the VM can discard the compiled code and revert back to interpreting that method's bytecodes.

As you may have noticed, Hotspot's approach to making Java programs run fast is similar to the approach programmers should take to improve a program's performance. Hotspot, unlike a regular JIT compiling VM, doesn't do "premature optimization." Hotspot begins by interpreting bytecodes. As the program runs, Hotspot "profiles" the program to find the program's "hot spot," that 10 to 20 percent of the code that gets executed 80 to 90 percent of the time. And like a good programmer, the Hotspot VM just focuses its optimization efforts on that time-critical code.

Adaptive inlining

But there is a bit more to the adaptive optimization story. The adaptive optimization approach taken by Hotspot is tuned for the run-time characteristics of Java programs -- in particular, of "well- designed" Java programs.

According to David Griswold, Hotspot manager at JavaSoft, "Java is a lot more object-oriented than C++. You can measure that; you can look at the rates of method invocations, dynamic dispatches, and such things. And the rates [for Java] are much higher than they are in C++." Now this high rate of method invocations and dynamic dispatches is especially true in a well-designed Java program, because one aspect of a well-designed Java program is highly factored, fine-grained design -- in other words, lots of compact, cohesive methods and compact, cohesive objects.

This run-time characteristic of Java programs, the high frequency of method invocations and dynamic dispatches, affects performance in two ways. First, there is an overhead associated with each dynamic dispatch. Second, and more significantly, method invocations reduce the effectiveness of compiler optimization.

Method invocations reduce the effectiveness of optimizers because optimizers don't perform well across method invocation boundaries. As a result, optimizers end up focusing on the code between method invocations. And the greater the method invocation frequency, the less code the optimizer has to work with between method invocations, and the less effective the optimization becomes.

The standard solution to this problem is inlining -- the copying of an invoked method's body directly into the body of the invoking method. Inlining eliminates method calls and gives the optimizer more code to work with. It makes possible more effective optimization at the cost of increasing the run- time memory footprint of the program.

The trouble is that inlining is harder with object-oriented languages, such as Java and C++, than with non-object-oriented languages, such as C, because object-oriented languages use dynamic dispatching. And the problem is worse in Java than in C++, because Java has a greater call frequency and a greater percentage of dynamic dispatches than C++.

A regular optimizing static compiler for a C program can inline straightforwardly because there is one function implementation for each function call. The trouble with doing inlining with object- oriented languages is that dynamic method dispatch means there may be multiple function (or method) implementation for any given function call. In other words, the JVM may have many different implementations of a method to choose from at run time, based on the class of the object on which the method is being invoked.

One solution to the problem of inlining a dynamically dispatched method call is to just inline all of the method implementations that may get selected at run-time. The trouble with this solution is that in cases where there are a lot of method implementations, you get an exploding size problem.

One advantage Hotspot's adaptive optimization approach has over static compilation is that, because it is happening at runtime, it can use information not available to a static compiler. For example, even though there may be 30 possible implementations that may get called for a particular method invocation, at run-time perhaps only two of them are ever called. The Hotspot approach enables only those two to be inlined, thereby reducing the exploding size problem.


As used here, static compilation means compilation on the developer's computer, resulting in a native executable. It contrasts with dynamic compilation, which is done on the user's computer at run-time. A traditional C++ compiler is an example of static compilation. JITs and Hotspot are examples of dynamic compilation.

Because more information is available at run-time, Hotspot's adaptive optimization approach can yield better optimization than is possible with a static compiler. This means that a Java program running on Hotspot could potentially be faster than the same Java program compiled to a native executable. Hotspot's adaptive optimization technique enables Java programs that have high method invocation frequency (lots of focused, cohesive objects and methods) to perform well, even though they are being delivered as bytecodes in class files.

Other improvements

In addition to the adaptive optimization technique, the Hotspot VM includes several other improvements that address performance bottlenecks in current VMs. For example, Sun claims that interface method invocations in Hotspot will not have a performance disadvantage compared to other method invocations. Improved thread synchronization will make invoking a synchronized method much faster, only a little more expensive than invoking a non-synchronized method. And lastly, Hotspot uses a generational garbage collection algorithm, which reduces the cost of collecting large numbers of short- lived objects.

<<  Page 3 of 4  >>

Sponsored Links

Copyright © 1996-2018 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use