What is asynchronous processing support in Servlet 3.0 ?

In traditional Servlet model, 1 Request is processed by 1 Thread. The Servlet container handles each request by spawning a new thread / by picking up an available thread for each request from the Web container’s thread pool. In this model the Servlet container can only handle new requests as long as it has free threads in this pool. As long as your own code is busy processing the request, the thread is not free.In some situations mostly for significant load on your application it might create problems.

In such situations it might be worth it to break this model. When a request arrives at the Servlet via such a Servlet container managed thread, and your code then asks for asynchronous execution by making a Servlet as asynchronous. In asynchronous Servlets you can return the thread calling the servlet to the container thread pool and continue the remainder of the request on another thread. This causes more number of free threads in the container pool to server requests from client.

To enable asynchronous processing on a servlet, set the parameter asyncSupported to true on the @WebServlet annotation as follows:

@WebServlet(urlPatterns={"/asyncservlet"}, asyncSupported=true)
public class AsyncServlet extends HttpServlet { ... }

The javax.servlet.AsyncContext class provides the functionality that you need to perform asynchronous processing inside service methods. To obtain an instance of AsyncContext, call the startAsync() method on the request object of your service method; for example:

public void doGet(HttpServletRequest req, HttpServletResponse resp) {
   ...
   AsyncContext acontext = req.startAsync();
   ...
}

This call puts the request into asynchronous mode and ensures that the response is not committed after exiting the service method. You have to generate the response in the asynchronous context after the blocking operation completes or dispatch the request to another servlet.

The answer to WHY and WHEN you would use this is not that straightforward. If you just wants to preserve threads then in the above case you would be exchanging one thread from one thread pool for the other (in this case the Servlet pool vs your own thread pool) and the net benefit wouldn't be that much. You could just as well have given your Servlet thread pool an extra thread.

Asynchronous processing should be used for requests that take a “long” time to process, especially when the request needs to wait for something. Often this is IO or some external computational logic. Contrary to synchronous request processing, this will not commit any response and will not close the connection. Instead, you can hand the computation over to another thread pool ,which can pick it up (perhaps in parallel) and thereby let go of the container thread.

In more advanced scenarios, you could however do more fine-grained managing of requests; divide them into multiple tasks and have a thread pool to serve those tasks. E.g. imagine 100 download requests for a 10MB file handled by 10 threads that round-robin give sends 100KB a time to each request.

To be a viable case for async the client should also be interested in some result coming from the service, or otherwise you can just respond synchronously, but continue processing afterwards in another thread.

But where we should not use async servlets? Perhaps not by default, since it adds some complexity and a bit more code is needed. Error handling and debugging also become a little harder. When the request and response are small, the request needs less external IO and the processing is short in such cases there are least benefit of using an asynchronous servlet.


How it works

The basic flow for a servlet that calls an external API using an async HTTP client (for example AsyncHttpClient) looks like this:

   1. The container calls the servlet.
   2. The servlet tells the container that the request should be asynchronous, by calling ServletRequest.startAsync. startAsync returns an AsyncContext.
   3. The asynchronous call to the external API is made.
   4. The service method of the servlet returns at this point. Now there is no thread associated with this request.
   5. On receiving the response from the external API the servlet creates a HTTPResponse accordingly and sent.
   6. The method AsyncContext.complete is called to tell the container that this request has been processed by the servlet.

import java.io.IOException;
import java.io.PrintWriter;

import javax.servlet.AsyncContext;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;

@WebServlet(urlPatterns = "/AsyncServlet", asyncSupported = true)
public class AsyncServlet extends HttpServlet {

  private static final long serialVersionUID = 1L;

  protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    response.setContentType("text/html;charset=UTF-8");
    final AsyncContext asyncCtx = request.startAsync();
    asyncCtx.start(new Runnable() {
      public void run() {
        String param = asyncCtx.getRequest().getParameter("param");
        ResponseEntity responseEntity = getResponseFromExternalAPI(param);
        HttpServletResponse response = (HttpServletResponse) asyncCtx.getResponse();
        PrintWriter out = response.getWriter();
    	out.write("Response >> " + responseEntity.getBody());
    	asyncCtx.complete();
      }
    });
  }
	
  // Mocking external api call
  private ResponseEntity getResponseFromExternalAPI(String param) {
    ResponseEntity responseEntity = new ResponseEntity<>("get response for param", HttpStatus.OK);
    return responseEntity;
  }
}

We can add AsyncListener implementation to the AsyncContext object to implement callback methods – we can use this to provide error response to client in case of error or timeout while async thread processing. We can also do some clean up activity here. You can set the timeout of a specific request through AsyncContext.setTimeout, otherwise a container default value will be used. Then you can “listen” for timeouts by attaching an AsyncListener through the method AsyncContext.addListener. The AsyncListener interface looks like this:

public interface AsyncListener extends EventListener {
  void onComplete(AsyncEvent);
  void onError(AsyncEvent);
  void onStartAsync(AsyncEvent);
  void onTimeout(AsyncEvent);
}
    
public interface AsyncListener extends EventListener {
  void onComplete(AsyncEvent);
  void onError(AsyncEvent);
  void onStartAsync(AsyncEvent);
  void onTimeout(AsyncEvent);
}

And the AsyncEvent like this:

public class AsyncEvent {
  public AsyncContext getAsyncContext() { ... }
  public ServletRequest getSuppliedRequest { ... }
  public ServletResponse getSuppliedResponse { ... }
  public Throwable getThrowable { ... }
}
    
public class AsyncEvent {
  public AsyncContext getAsyncContext() { ... }
  public ServletRequest getSuppliedRequest { ... }
  public ServletResponse getSuppliedResponse { ... }
  public Throwable getThrowable { ... }
}

The code that sends the response will get an IllegalStateException if the request timed out, since the AsyncContext will be completed. This needs to be handled and the easiest might be to just catch the exception.

 

 

JSP-SERVLET J2EE