Keeping the queue capacity low functions to limit how disproportionate the workload between servers 130A-D in the server pool 139 can be. In other words, if the queue 134A of one server 130A of the server pool 139 is large, it is easier for a large queue of TCP requests to accumulate at that server while another server 103B-D may have no work to do. A low queue capacity caps the number of requests that are ageing in one queue rather than being processed by another server 130B-D that may be less busy. Another reason for a low queue capacity is to limit the time requests spend in the queue before processing starts. As a result, configuring servers 130A-D with low queue capacities can result in an overall improvement in response time performance. For example, if the request rates is low (for example one request per second average service rate), a queue length of 1 allows requests to sit in the queue unprocessed for up to a second (half of a second on average). However, at higher request rates, the time spent in the queue is less, so the difference in performance between a queue length of zero and one can become insignificant.

In one variation, although a queue capacity of one is preferred in some implementations, an exception would be cases where the system as a whole needs to serve a given number of requests, but the number of requests an individual server can handle is limited by resource constraints. In this case, instead of increasing the number of processes, the feedback loop, which will be described in more detail below, keeps the queue depth balanced across all nodes by increasing the queue depths, so that individual requests are served as quickly as possible.