Is it true that servlet containers service each request by creating a new thread? If that is true, how does a container handle a sudden dramatic surge in incoming requests without significant performance degradation?

Govind Seshadri

It is true that until recently, most containers serviced each request by creating a new thread. However, as you can imagine, this can place a significant burden on the server's resources, especially when there is a significant surge in the number of incoming requests. For example, if the server receives 500 simultaneous requests, the container has to then allocate the requisite memory, descriptors, etc. to create 500 new threads in order to service the requests, placing a severe burden on system performance and possibly even crashing the system in the event of resource exhaustion.

But nowadays, most servlet containers use a thread pool, thus relieving themselves from creating new threads. Instead, to service a request, they simply borrow a thread from the pool and when they are done, return it to the pool.

Typically, an administrator sets the upper and lower bounds for the thread pool. The upper bound on the number of threads used concurrently prevents the resource exhaustion problem associated with unlimited thread allocation. The lower bound can instruct the pool not to keep too many idle threads, freeing them if needed.

If the container maxed out to the thread upper limit, and a new request arrived, the new request will have to wait for a previous request to finish and free the thread used to service it.

If you are using Tomcat 4, the following configuration fragment from server.xml shows a thread pool setting:

<Connector className="org.apache.tomcat.service.PoolTcpConnector">
                value="10" />
Here, the pool has 3 configuration parameters:

max_threads - defines the upper bound to the for the concurrency, the pool will not create more then this number of threads.
max_spare_threads - defines the maximum number of threads that the pool will keep idle. If the number of idle threads passes the value of max_spare_threads the pool will kill these threads.
min_spare_threads - the pool will try to make sure that at any time there is at least this number of idle threads waiting for new requests to arrive. min_spare_threads must be bigger then 0.