Analyzing Traffic

Before beginning to optimize the configuration of your HTTP server and servlet container, it is fundamental that you analyse the profile of the traffic you expect your server to handle. This can be estimated or measured from an actual live server. The type of information that is useful gather includes:

Attribute Variations Comment
Request rate average, peak The number of requests per second
Connection rate average, peak The number of new connections established with the server per second
Simultaneous Users average, peak The number of users that a simultaneously interacting with the server.
Requests per page average The number of requests that is required to render a page of the webapp. Includes images and style sheets, but may be affected by client caching.
Page view time average The period of time that a typical user will view a page before requesting another from the webapp.
Session duration average The period of time that an average user will remain in contact with the server. This can be used to estimate session and memory requirements

Measuring Traffic

The most accurate way to measure the attributes listed above is to measure them on a live server that is handling real traffic for the webapp that you are trying to optimize. Statistics and log analysis can then be used to derive the information above.

Jetty supports statistics collection at both the server and context level. The following configuration excerpt shows how to turn on statistics for the server and for a particular web application:


  ...
  
    /myapp
    ./webapps/myapp
    false
  
  ...
  false
  ...

While statistics can be enabled as above, it is probably just as convenient to turn them on using a JMX agent to the Jetty MBeans. If Jetty is run with JBoss or within a JMX server, then a JMX agent can be used to configure and view statistics collection

Jetty HttpServer Statistics

The following statistics attributes are available on the org.mortbay.http.HttpServer class or via the associated MBean which is normally named like "org.mortbay:Jetty=0":

Attribute Comment
statsOn True if statistics collection is turned on.
statsOnMs Time in milliseconds stats have been collected for
statsReset() Reset statistics
connections Number of connections accepted by the server since statsReset() called
connectionsOpen Number of connections currently open that were opened since statsReset() called
connectionsOpenMax Maximum number of connections opened simultaneously since statsReset() called
connectionsDurationAve Sliding average duration in milliseconds of open connections since statsReset() called
connectionsDurationMax Maximum duration in milliseconds of an open connection since statsReset() called
connectionsRequestsAve Sliding average number of requests per connection since statsReset() called
connectionsRequestsMax Maximum number of requests per connection since statsReset() called
errors Number of errors since statsReset() called. An error is a request that resulted in an exception being thrown by the handler
requests Number of requests since statsReset() called
requestsActive Number of requests currently active
requestsActiveMax Maximum number of active requests since statsReset() called
requestsDurationAve Average duration of request handling in milliseconds since statsReset() called
requestsDurationMax Get maximum duration in milliseconds of request handling since statsReset() called.

Jetty HttpContext Statistics

The following statistics attributes are available on the org.mortbay.http.HttpContext class or via the associated MBean which is normally named like "org.mortbay:Jetty=0,HttpContext=0,context=/myappp":

Attribute Comment
statsOn True if statistics collection is turned on
statsOnMs Time in Milliseconds that stats have been collected for
statsReset() Reset statistics
requests Number of requests since statsReset() called
requestsActive Number of requests currently active
requestsActiveMax Maximum number of active requests since statsReset() called
responses1xx Number of responses with 1xx status (Informal) since statsReset() called
responses2xx Number of responses with 2xx status (Success) since statsReset() called
responses3xx Number of responses with 3xx status (Redirection) since statsReset() called
responses4xx Number of responses with 4xx status (Client Error) since statsReset() called
responses5xx Number of responses with 5xx status (Server Error) since statsReset() called

Estimating Traffic

It may not be possible to measure actual live traffic of a deployment to be optimized. In this case estimates must be made to obtain a traffic profile on which to base your optimization. The following work sheets give some examples of how this may be done:

Attribute Formula Example Comment
SimultaneousUsers - 1000 Estimated from marketing or other sources.
UserSessionDuration - 180 seconds Time a single user spends interactive with the webapp. Estimated from marketing, usage trials or other sources.
AvePageViewTime - 30 seconds Time between page requests from a single user. Estimated from marketing or usage trials or other sources.
PagesPerUserSession UserSessionDuration/PageViewTime 6
RequestsPerPageNoCache - 12 Calculated from inspection of HTML
RequestsPerPageCache - 3 Calculated from inspection of HTML and usage trials.
RequestsPerUserSession RequestsPerPageNoCache+ (RequestsPerPageCache* (PagesPerUserSession-1)) 27
RequestsPerSecPerUser RequestsPerUserSession/ UserSessionDuration 0.15
RequestsPerSec SimultaneousUsers* RequestsPerSecPerUser 150
ConnectionsPerUser - 2.5 Measured from usage trials with estimated browser mix.
AverageConnections SimultaneousUsers* ConnectionsPerUser 2500
ConnectionsPerSecond ConnectionsPerUser* SimultaneousUsers/ UserSessionDuration 13.88 Assuming persistent connections that will span entire user session. If connections will not span session the multiply by PagesPerUserSession
PeakRequestsPerSecond 2*ConnectionsPerSecond + (RequestsPerPageNoCache- RequestsPerPageCache) * SimultaneousUsers/ UserSessionDuration 77.76 Based on SimultaneousUsers doubling in UserSessionDuration. The formula represents double the normal requests rate, plus the additional load of the new users loading the initial page with no cache.

This work sheet is only indicative of an estimate process that can be used, specially the method for determining the peak request rate. If possible , several estimation techniques should be used and the worse case numbers assumed.

Clustered Traffic

When running a cluster of application servers, it is often desirable to be able to handle the max expected load in the advent of a node failure. Thus once the single node traffic has been estimated or measured, the traffic loads for failure modes can be calculated:

Nodes in Cluster Failed Nodes Load
2 1 200%
3 1 150%
3 2 300%
4 1 133%
4 2 200%

Generating Traffic

Once the expected traffic profile has been analysed, a test client can be used to generate load on the server that reflects realistic load. It is important to make sure that the test client used is generating realistic load:

  • Are persistent connections supported? Persistent connections are much more efficient than non persistent connections and a realistic mix should be used to represent the expected browser population using the server.
  • Are connections held idle for realistic times? Idle connections reduce latency for individual users at the expense of server resources. A test client that does not idle connections will not test the servers ability to balance these competing resource requirements.
  • Does the test client account for client caching and if-modified-since headers? Most pages of a webapp are rendered from a cluster of requests for the initial page and it's included resources such as images and style sheets. Most client browser will cache many of the included resources and may often issue no requests for them or a request with an if-modified-since header that can be responded to with a simple 304-Not-Modified response. Test clients that do not model client caching will be measuring an unlikely worse case scenario.
  • Is the test client run on a different machine to the server? Local networking has different characteristics to remote networking and a local test client will consume resources that could have been used by the server