Dev2Dev > Articles

---

Using Asynchronous Servlets to Deal with Hung Threads

by Francesco Marchioni
12/04/2007

Abstract

BEA WebLogic Server 9.2 and later versions expose an Abstract Asynchronous Servlet class, which allows you to decouple receiving a servlet request from sending its response. The class also provides a Future Response Servlet, which lets the server handle servlet responses with a different thread than the one that handles the incoming request. Both of these extensions to the traditional servlet model allow you to avoid hung threads and integrate long-running jobs with the servlet paradigm. This article introduces these two features, together with a few examples.

Introduction

Have you ever wanted to achieve a quality of service (QoS) in a Web application by controlling its response time? A quality of service requirement, in its simpler form, is about ensuring a response time within a certain number of seconds. If this requirement is not satisfied, a meaningful error message should be provided.

The traditional servlet threading model is quite simple: The application server allocates a certain number of threads, made available to the Web application (see Figure 1). When a new request comes in and is ready for service, a single thread is allocated. From then on, the servlet thread won't return to the pool until it has terminated all its tasks.

Figure 1. Traditional servlet threading model

This makes life difficult if the servlet has to run a long-running task. To solve this problem, it is very common to decouple Web requests from a long-running backend process using JMS or a Message Driven Bean in cases where processing results are not required to be sent back immediately in the response.

This sounds good in a "fire and forget" scenario. However, if you have to return something to the client, then you must deal with the fact that your servlet thread is a good candidate for becoming a hung thread.

The Problem

In most cases, a hung thread is one that has blocked on contacting a back-end system for data needed to dispatch its request. One typical situation is a remote database being connected via JDBC.

The key is to make sure that the application knows what to do if the resource it needs is getting slower and slower or if it's just not available. Most of the time, this isn't a concern because the code for timing out sockets to the remote database server is all handled by the data source and JDBC driver. However, when you have to write the timeout policy by yourself, the situation is significantly more complicated.

Many programmers dread the thought of handling network timeouts. A common fear is that a simple, single-threaded network client without timeout support will balloon into a complex multithreaded nightmare, with separate threads needed to detect network timeouts, and some form of notification process at work between the blocked thread and the main application.

This article shows how easy it is to write a Java Web application that handles timeouts gracefully, using BEA WebLogic Server's Future Response Model. This feature can prevent hung threads by decoupling response from incoming request and timing out unresponsive requests. To avoid this hung-thread scenario, WebLogic Server provides two classes that handle HTTP requests asynchronously by decoupling the response from the thread that handles the incoming request. The following sections describe these two classes.

The Abstract Asynchronous Servlet Class

Implementations of AbstractAsyncServlet decouple receiving a servlet request from sending its response. You implement the Abstract Asynchronous Servlet by extending the weblogic.servlet.http.AbstractAsyncServlet class. Here are the methods you have to implement:

• public boolean doRequest(RequestResponseKey rrk): This method is the initial point of contact for the servlet. It receives as input the RequestResponseKey class, which is a wrapper for the traditional servlet request. As you'll see later, you can use the request to determine the responsiveness of the servlet.
• public void doResponse (RequestResponseKey rrk, Object context): This method handles the servlet response. As stated in the documentation, this method will not necessarily be handled by the original thread.
• public void doTimeout (RequestResponseKey rrk): If the servlet response is not sent in a certain amount of time, the doTimeout() method will be triggered by the server.

The servlet class also makes a static method available to you:

• static void notify(RequestResponseKey rrk, Object context): Call this method to notify the server that a response should be sent for the key rrk.

Just implementing the abstract methods is not sufficient to achieve the decoupling. Decoupling the request from the response can be obtained using the TimerListener class in the doRequest() method. A new instance of the TimerListener class can be obtained using the factory pattern (actually from the TimerManagerFactory). A TimerListener thread can invoke notify() on an AbstractAsyncServlet instance to eventually trigger delivery of the response. Here's an example of how to create such a timer:

 TimerManagerFactory.getTimerManagerFactory().
     getDefaultTimerManager().schedule(new TimerListener() { 
       public void timerExpired(weblogic.timers.Timer arg0) { 
         try { 
           // This will trigger delivery of the response. 
          AbstractAsyncServlet.notify(rrk, null); 
         } 
         catch (Exception e)
        { 
         e.printStackTrace(); 
        } 
     } 
}, 1000);

    

The key here is that if this is in a doRequest() method, then the response will be delayed until the timer expires and notify() is called. In turn, notify() will call the doResponse() method. If notify() is not called within a certain time period, doTimeout() will be called instead.

Figure 2 shows the resulting diagram of interaction between clients and servers.

Figure 2. Request/response decoupling using AbstractAsyncServlet

Let's look at typical uses for this pattern. One scenario could be, as we said formerly, QoS applications where we need to provide a response to the client within a certain amount of time. But we could even benefit from a different use case like a delayed request handling: Imagine an application that must access a remote/slow resource (like a remote DB link) or a CPU intensive algorithm. In this kind of scenario, delaying the handling of the request (by a fixed amount of time) can help to prevent too many simultaneous accesses. Until now the only option was to wait, consuming a valuable thread.

Let's look at an example of this approach in action.

Pages: 1, 2, 3

Next Page »