This month, I'll look at benchmarking and tuning your applications, and how to make your Java runtime more manageable. And, I offer some advice on how your developers can keep their focus on development work.

Q. Can you recommend a process for benchmarking and tuning applications?

A. To establish baselines on a production application, you need to determine its performance characteristics prior to deployment. The end result of this process is twofold. First, it allows you to tune the application to defined business objectives and verify it against a set of load tests that closely match expected production conditions. Second, it helps you to identify the application components that are critical to performance so that they can be monitored and tracked in production. The next step is critical to any capacity planning efforts as the expected load on the application grows.

I suggest that you take the following steps:

1. Create a set of test scripts that mimic user interactions.
2. Identify a performance characteristic to monitor (e.g., response time, CPU time, I/O activity).
3. Determine the characteristic's target performance level.
4. Using the test scripts, generate load on the application to establish a baseline for the performance characteristic under investigation.
5. Identify components of the application driving the performance characteristic and apply methods, one by one, to improve them.
6. After each applied performance improvement, rerun the tests to determine the level of improvement, if any, from the baseline.
7. Repeat steps 5 and 6 until the target performance level is achieved.

The importance of steps 3 and 4 cannot be overstated. Without a performance baseline, investigators cannot determine the effectiveness of any improvements nor can they judge if the performance goals can reasonably be attained. When both the generated load and the test environment sufficiently mirror production circumstances, I have found this process to be highly successful in helping ensure high application performance over the long run.

Q. How will the Java runtime become more manageable over time?

A. Historically, Java has done a great job of servicing the needs of the developer by rapidly rolling out new APIs and services developers need to build functionality and get the application functioning rapidly. However, when it comes to manageability the JVM is one area where there is room for improvement. For example, the Java runtime does not provide APIs for notification about garbage collection or for measuring thread performance.

What organizations need are APIs for performance and availability that specifically target production environments and incur minimal overhead. Those APIs are currently being specified - there are some new JSRs, such as JSR 174, that focus on exactly these issues. While the standards bodies address performance and availability, some JVM vendors have begun efforts to make their JVMs more manageable because they understand that there's an immediate need for a solution like this at the JVM level itself. (JRockit, for example, offers extra features for manageability.)

JMX offers some performance and availability management functionality, and organizations should consider coding APIs that allow them to utilize it. The primary strength of JMX, however, is in making the application servers themselves more manageable, and that's the focus of much of the current JMX research and implementation. Over time, manageability will become more and more a part of the standard Java runtime. This will be enabled by the emergence of new JSRs that make the JVM more manageable with low overhead in a production environment and that focus on performance, availability, and resource utilization.

Q. I'm a developer and I keep getting pulled into meetings to solve PRODUCTION application performance and availability problems regardless of their cause. What can I do to get out of this and get back to developing software?

A. This is a problem that many developers have asked me about. Too often, software developers get pulled in to solve production problems without much information about what the cause of the problem is. As a result, they spend needless time looking through thread dumps and log files trying to find a needle in the haystack, rather than spending their time developing software.

There is a need for a specific role within the company called a Level II application support manager. This person serves as a complement to the day-to-day operations people who handle typical management duties such as making sure the hardware boxes are rebooted on time or that DNS servers are available. When there's a problem with an application, the primary role of the application support manager is to perform triage. He or she should know the right person to call to get the problem fixed and be able to provide that person with information about why they were called in and why their area of expertise is needed. For example, they should be prepared to offer some trending information, some historical performance information, or some real-time performance information to whomever they call so that person isn't thrown into complete darkness.

If you have an application support manager to monitor the application and who can perform triage quickly and accurately when there's a problem, then the developers will be taking fewer calls, and the ones they do take will legitimately require their expertise. This will result in serious productivity gains for the development organization as well as serious gains in application availability on the operations side.

As always, I invite you to send an e-mail to asklew@syscon.com if you have any performance-related questions about JVMs, Java applications, WebLogic Server, or connections to back-end systems.

Copyright © 2003 SYS-CON Media, Inc.