The micro benchmark starts a Java thread which performs a given number of tasks, or "events". Each event consists of a fixed number of object allocations. Each of the allocated objects are approximately 64k in size, which puts some pressure on the memory management system. The application measures the execution time of each event with System.nanoTime() and prints out the execution times with System.out.println(). At the end of the run the application also prints out the total time it took to execute all the events. The execution time of each event represents the latencies in the application, while the total run time represents the overall throughput.

Today I will show you how you can do some basic tuning of BEA JRockit just by looking at the output of your application. Of course, I know the inner workings of the JVM quite well, but I'll pretend that the JVM is a black box with a few knobs and handles and no output except the Java application itself. I use the micro benchmark described above as my example application.

For this example I choose to run with a fixed heap size. I've adjusted the application to hold about 100-150 MB of live data. I've heard that the heap size shuold be at least twice the amount of live data, so I set the heap size to 350 MB.

In my first run only the heap size is set, which means that BEA JRockit uses the default garbage collection mode, which is optimized for high application throughput.

The total execution time time for this run is 209594 ms. When I plot the latencies in a spread sheet I see that most are below 50 ms, but a considerable number of latencies are scattered between 50 and almost 120 ms, as seen in figure 1.

Figure 1

Now, let's say that my application is a transaction system and that I don't want the transactions to take longer than 50 or 60 ms to execute. The variations in transaction times when running in throughput mode would thus not be acceptable. One reason for variations in the latencies could be that the garbage collector pauses the Java execution now and then. Let's try this theory!

For my second run I use the command line option -Xgcprio:pausetime to select the garbage collection mode optimized for short pauses. The results don't look too good, the pause times are actually higher (see Figure 2), as well as the total execution time (246781 ms). Why is that? I check the documentation for -Xgcprio:pausetime and find that there is also a pause target parameter that you can use to define what you mean by "short pauses". By default the pause target is 500 ms. This is not even nearly good enough if I want the latencies down below 60 ms.

Figure 2

The lowest pause target setting for the pausetime mode is 200 ms. That works for larger applications where transactions can take a second or so, but for my small application that isn't enough. I must look for other options. How about the garbage collection mode for short and deterministic pauses? With this I can set much lower pause targets, so I decide to try that. The "deterministic garbage collector" is a part of BEA WebLogic Real Time (http://www.bea.com/framework.jsp?CNT=index.htm&FP=/content/products/weblogic/realtime/).

I do a run with -Xgcprio:deterministic and the default pause target setting, which is 30 ms. Immediately the "scatter" is reduced. Most latencies are below 70 ms, as you can see in Figure 3. The total execution time is 213843 ms, which isn't bad compared to the run in throughput mode.

Figure 3

I have however not quite reached my goal yet. By examining the graph of the latencies I notice that most of the latencies are between 5 and 30 ms. Maybe each "event" itself takes up to 30 ms to execute? Within that time we might get a couple of garbage collection pauses that total more than 30 ms, so of course the latencies can climb up tp 70 ms.

I decide to do one more run using the deterministic mode. This time I set the pause target to 10 ms, using the command line option -XpauseTarget:10ms. Finally the results look good - you can see this for yourself in Figure 4. The graph from the spreadsheet shows only one stray latency longer than 60 ms, and most are below 50 ms. One long latency of about 9500 I can live with, at least for now.

Figure 4

What about the total execution time? Actually, it looks very good: 200234 ms. Even if you allow for a variation of a few percent in the total execution time measurements, this shows that the deterministic mode is on par with or better than throughput mode for this particular application. With a few benchmarking rounds and some analysis of the results between each run I've found a garbage collection mode and pause target setting that not only gives me acceptable latencies but also gives me good performance.

Of course this is a micro benchmark. The results will most likely be different if I run another application, but now I at least know how to think and what to look for..

An ordinary garbage collection traverses all references in all objects in the heap, while a garbage collection of the nursery (young collection) only traverses references that have changed since the last garbage collection. Any "live" objects found in the nursery are promoted (moved) to the "old space". This frees up the entire nursery for swift new object allocation. Using a nursery will thus improve both garbage collection efficiency and object allocation speed.

The BEA JRockit Diagnostics Guide contains information on how to tune the nursery size, so I will not go into those details here. Instead I will talk a little about the rationales behind the dynamic nursery sizing heuristics and the problems that both BEA JRockit and anyone who attempts to tune the nursery size manually may encounter.

The Basics

Figures 1-3 illustrate a basic scenario with a nursery and an old space. Objects are allocated in the nursery, and when the nursery is full the objects that are still alive are promoted to the old space.

Let's say that each allocated object lives for exactly ten seconds, no more and no less. Thus, when the nursery becomes full the only live objects within the nursery are the ones that were allocated less than ten seconds ago. In a simple young collection (without a keep area) these are the objects that would be promoted. Ideally, the size of the nursery wouldn't matter at all. The young collections promote the same amount of objects as long as they're at least ten seconds apart. In this world, a larger nursery size would mean that each young collection frees a larger amount of memory. Also, the frequency of old collections would depend directly on the size of the old space - a smaller old space would mean more frequent old collections. Since the nursery is a part of the heap, and the old space is what is left of the heap when the nursery has been reserved, a larger nursery would allow fewer young collections before an old collection must be performed.

Figure 1: The Heap

Figure 2: "Before"

Figure 3: "After"

Your goal here is to get as much memory as possible freed by young collections rather than old collections. I won't go into the details on why, but surprisingly it turns out that a nursery size of approximately half of the free memory on the heap is nearly optimal in this sense.

Promoting objects just because they haven't had a chance to die yet when the young collection starts feels a bit silly. Thus BEA JRockit uses something called a "keep area". The last bit of the nursery is designated as the "keep area", and objects within the keep area won't be promoted until the second young collection after they were created, providing that they still are alive at that point. In our optimal world where objects always live exactly ten seconds, we would tune the keep area to fit exactly these objects, and nothing would ever be promoted. In this world, the nursery size could be increased almost indefinitely, since the old space only has to have space for a few long lived objects.

Figures 4-6 illustrate a young collection using a keep area. As you can see, no objects are promoted in this example.

Figure 4: The Heap

Figure 5: "Before"

Figures 6: "After"

The Quirks

Now, BEA JRockit has a little quirk: large objects can be allocated directly in old space. Thus, if your application allocates a lot of large objects, the old space may become full before even one young collection has been performed. Don't worry, you can fix that too. Just set the limit for what objects to consider "large" higher, using the option -XXlargeObjectLimit. Everything smaller than the large object limit will always be allocated in the nursery.

Another aspect in nursery sizing is garbage collection pause times. If your application is sensitive to latencies you may want to decrease the nursery size to shorten the young collection pause times. The pause time isn't linearly dependent on the nursery size, but you have a fair chance to get shorter pause times by reducing the nursery size.

So, why can't BEA JRockit do this all by itself? Well, it makes a fair attempt on some of it, but unfortunately the real world has a quirk of its own: the lifetimes of the objects vary. Sometimes they vary a lot. You will notice this also when you tune the nursery size by hand. What in theory sounds optimal can turn out to be a bit off, or even quite suboptimal when the inexactness of the real world is thrown in the face of neat models and calculations.

By default, BEA JRockit optimizes garbage collection for maximum throughput. In this mode the nursery size is adjusted automatically in runtime. The nursery can even be disabled if most of the allocated objects are large. The static generational parallel garbage collector (-Xgc:genpar) uses similar heuristics to adjust the nursery size. For both these garbage collectors you can override the dynamic nursery size by setting a nursery size on the command line (-Xns:). If you're using the garbage collection mode optimizing for short pauses (-Xgcprio:pausetime) or the static generational concurrent garbage collector (-Xgc:gencon) you will most likely want to tune the nursery size manually.

Read more about nursery sizing and garbage collection in the BEA JRockit Diagnostics Guide.

So, how do we minimize the impact of garbage collection? We can't get rid of it for good (believe me, many people have tried), but we can put it where it doesn't matter so much. Do garbage collection pauses matter to your application? Are you willing let the garbage collection run concurrently with the application and trade some overall application throughput for minimal garbage collection pauses? Sip it in small gulps or quaff it all in one go? Or do you want something in between?

BEA JRockit offers a selection of garbage collection "modes". There are currently three "dynamic" modes and a "static" mode. The dynamic modes optimize the application for a specific profile - maximum overall throughput, short pauses or very short and deterministic pauses. These garbage collection modes will select a garbage collection "strategy" based on how the application behaves and may change garbage collection strategies if the application's behavior changes. The "static" garbage collection mode will on the other hand let you select a single garbage collection strategy to be used throughout the run.

All applications want throughput, the higher the better. Thus the dynamic mode for maximum throughput is the default garbage collection mode in BEA JRockit. Some applications do however have an additional requirement that the latencies should be low. For example, a transaction may time out if a garbage collection interrupts it for too long, or a user will go and get some coffee and get lost on the way back if the graphical user interface suddenly freezes for garbage collection. The garbage collection in BEA JRockit needs to be tuned a bit for these applications. It costs a bit in overall throughput, but will save the day when the application becomes unusable if garbage collection pauses are too long.

The dynamic mode optimized for short pauses will do its best to keep the garbage collection pauses below a user defined pause target. The dynamic mode optimized for deterministic and very short pauses (available as a part of BEA WebLogic Real Time only) does this even better and for even lower pause targets. But sometimes you want something more... specific. Maybe you've noted that a generational parallel garbage collection strategy gives you just the right kind of garbage collection behavior, or maybe you are a firm believer in single generational garbage collection with a parallel mark phase and a mostly concurrent sweep phase. In that case you may want to try a static garbage collection mode.

Four static garbage collection strategies are available with the command line option -Xgc, but hidden within BEA JRockit there are no less than twelve different garbage collection strategies! These can be accessed with the command line option -XXsetGc. Four of these are the same as the ones that you can set with -Xgc, and the other eight are the obscure combinations where the mark phase and the sweep phase of the garbage collection use different strategies.

Now, if "mark and sweep", "generational", "mostly concurrent" and "parallel" are complete mumbo jumbo to you and you don't even want to know, I suggest that you select one of the dynamic garbage collection modes, or at least one of the four static strategies available with -Xgc. If you on the other hand look forward to running twelve rounds of benchmarking (plus some extra to vary the nursery size, because in this case size does matter) and want to see if you can find that optimal garbage collection strategy for your application, then -XXsetGc is your friend.

And if you're completely bewildered by now and want to know everything about what I'm talking about, I would recommend the sections called Understanding Memory Management and Tuning the Memory Management System in the BEA JRockit Diagnostics guide, as well as the documentation on the -Xgc, -XgcPrio and -XXsetGc command line options.

]]> http://dev2dev.bea.com/blog/noora/archive/2007/10/the_secrets_of.html 2007-10-03T16:04:45Z 2007-10-03T16:35:05Z