Referring to FIG. 3, each server of the server pool 139 has a queue 134A. The queue 134A is a location storing an ordered list of TCP requests waiting to be processed by the server 130A. In one embodiment, the servers 130A-D of the server pool 139 are configured, for example by setting the ListenBacklog parameter in Apache, so that each server has a low queue capacity. In a standard Linux kernel, the ListenBacklog can be set at the lowest to 1. A queue capacity of 1 means that the server only accepts a request if it is able to begin processing the request as the next task. Otherwise, the server sends a rejection notification so that the load balancer 120 can send the request to another server that can process the request as its next task. This results in the best response time performance.
A queue capacity of 1 is preferred in some implementations, but queue capacities higher than 1 (e.g., 2) can also be used depending in part on the number of servers in the server pool 139 and the expected workload on them. Alternatively, a queue capacity of 0 may be preferred for servers that support such configuration. A queue capacity of 0 means that the queue accepts new requests only when a process is known to be ready to accept new work.