jeudi 23 juin 2011

Keep your Tomcat Server from heavy loads : valving again !


In a previous post, we talked about the valve concept in Catalina engine.
Today, we'll see a good use case of Catalina valve concept : a valve to master load on Tomcat shoulders.

Contention ?


Well contention is a complex concept and can take place in various situations.
Basically, contention is a state where some running threads doing various things need some resource [1] badly (owned by another thread) : they cannot run anymore until they gain access to the resource .
Contention is a normal event in a JEE server, you have to share resources because you cannot allocate an unlimited amount of resource to every resource consumer : JEE applications run in finite worlds (even in a cloud), by its size, by its cost or both.
Generally, too much contention leads to heavy loads.

Performance assessment is not enough !


Performance or stress tests are a good practice to find out the load your app will endure.
But what if some day things are not working according to plan:

- there are plenty of extras users ?
- one key component of architecture, saying network, slow down dramatically ?
- one node or many nodes of your cluster crash ?
Using a simple but efficient approach we can use two criteria to assess load on our Tomcat Server :

- load average of the system
- number of sessions on your webapp
Indeed, your performance tests should tell you how many sessions your application is able to handle without compromising SLAs. We can use this to decide if a user can start a session or not.
But, in real world, performance tests cannot cover all paths and plan all cases, so we need another criteria.
Here comes the load average , if the load is higher than a specified value then we must delay the user request until the load average gets lower even if the number of sessions is below the max.

Master it with a valve


Using a valve allows you to control user requests before they reach the application.
Here is the skeleton :
public class EasyLoadValve extends ValveBase {

    private int maxLoadAvg = 1000;
    private static final int ONE_SECOND = 1000;
    private int timeout = 15 * ONE_SECOND;
    private Integer rejectedSession = 0;

    public int getTimeout() {
        return timeout;
    }

    public synchronized void setTimeout(int timeout) {
        this.timeout = timeout;
    }

    public double getMaxLoadAvg() {
        return maxLoadAvg;
    }
 
    public synchronized void setMaxLoadAvg(int maxLoadAvg) {
        this.maxLoadAvg = maxLoadAvg;
    }
 
    protected boolean hasSession(Request request) {
        return (request.getSession(false) != null);
    }

    public int getRejectedSession() {
        return rejectedSession;
    }

    public synchronized void setRejectedSession(int rejectedSession) {
        this.rejectedSession = rejectedSession;
    }


    protected double getLoadAvg() {

        OperatingSystemMXBean osBean = ManagementFactory.getOperatingSystemMXBean();
        double currentLoadAvg = osBean.getSystemLoadAverage();
        return currentLoadAvg;
    }
 
    public void invoke(Request request, Response response) throws IOException, javax.servlet.ServletException {

        // New session ?
        if (!hasSession(request)) {

            double curLoadAvg = getLoadAvg();
 
            // Système surchargé 
            if (curLoadAvg > maxLoadAvg) {
                 
                 // Reject session !!!
                 // The response is an error
                 response.setError();
                 
            } else {
                // Max atteint ?
                Context context = request.getContext();
                StandardManager standardManager = (StandardManager) context.getManager();
                int maxActiveSessionsAllowed = standardManager.getMaxActiveSessions();
                // Session illimitée
                if (maxActiveSessionsAllowed == -1) {
                    
                    getNext().invoke(request, response);
                    
                } else {
                    int activeSessions = standardManager.getActiveSessions();
                    if (activeSessions >= maxActiveSessionsAllowed) {
                      
                      // Reject session !!!
                      // The response is an error
                      response.setError();
                    } else {
                        getNext().invoke(request, response);
                    }
                }
            }
        } else {
            getNext().invoke(request, response);
        }
    }
}
Tomcat provides many ways to configure any valve :

- static way : use server xml to configure your valve at startup
- JMX way : use JMX (jconsole) to adjust valve parameters at runtime
Here is a piece of XML needed :
<Valve className="jee.architect.cookbook.perf.tomcat.easyload.EasyLoadValve" maxLoadAvg="4" timeout="15000"/>
Check Tomcat documentation to find the exact configuration pattern.

Polishing it


You can use a more user friendly approach by sending a waiting page.
Here is a rejectSession method which use an html page :
    protected void rejectSession(Request request, Response response) throws IOException, javax.servlet.ServletException {

        

        synchronized (this) {
            this.rejectedSession++;
        }

        ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
        InputStream inputStream = classLoader.getResourceAsStream("jee/architect/cookbook/perf/tomcat/easyload/toomuchload.html");
        BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
        StringBuilder pageBuilder = new StringBuilder();
        String line = null;
        while ( (line = reader.readLine() ) != null) {
            pageBuilder.append(line);
        }
        String page = pageBuilder.toString();

        String contextPath = request.getRequestURI();

        page = page.replace("URL", contextPath).replace("TIMEOUT", "" + this.timeout).replace("T_SECONDS", "" +  ((int) (this.timeout / 1000)));
        // The response is an error
        response.setError();

        try {
            response.reset();
        } catch (IllegalStateException e) {
            ;
        }

        response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
        response.setSuspended(false);
        try {
            try {
                response.setContentType("text/html");
                response.setCharacterEncoding("utf-8");
            } catch (Throwable t) {
                // DO SOMETHING SMART
            }
            Writer writer = response.getReporter();
            if (writer != null) {
                // If writer is null, it's an indication that the response has
                // been hard committed already, which should never happen
                writer.write(page);
            }
        } catch (IOException e) {
            ;
        } catch (IllegalStateException e) {
            ;
        }
    }


When load is too heavy, a new user will get this :
toomuch
This valve has been tested with success on Tomcat 5.0.23, Tomcat 6.0.32, Tomcat 7.0.8 (configuration is a bit different, see doc) powered by a Unix LIke SUN VM (beware of OperatingSystemMXBean on windows).
You can get the full source code on my github repository : check here (the associated parent pom is here).

Aucun commentaire:

Enregistrer un commentaire