Nginx Load Balancing Algorithms for Stateless APIs

For stateless APIs, round robin is a sound default in Nginx, and least connections is usually the better choice when request durations vary a lot. Hash-based methods are for cases where requests with the same key should reach the same server, such as caches. Because the application keeps no session state, you can pick the algorithm that balances load best without worrying about stickiness.

Which Load Balancing Methods Does Nginx Offer?

The open-source version of Nginx supports several methods in the upstream block. Each makes a different trade-off between simplicity and how evenly it spreads work.

• Round robin (default): sends requests to servers in turn, optionally weighted. It is simple and works well when requests cost about the same.
• least_conn: sends the request to the server with the fewest active connections, adjusted by weights. It adapts to uneven request duration.
• ip_hash: picks a server based on the client address, giving a rough form of stickiness.
• hash: picks a server by a key you define, such as a URI or header value, with optional consistent hashing.
• random: picks a server randomly, with an optional two-choice variant that favors the less loaded of two random picks.

upstream api_backend {
    least_conn;
    server 10.0.0.11:8080 weight=2;
    server 10.0.0.12:8080;
    server 10.0.0.13:8080 backup;
}

Some features, such as active health checks and certain session persistence methods, belong to the commercial edition. Check the Nginx documentation to see what your build supports.

When Is Round Robin Enough?

Round robin spreads requests evenly by count. If each request takes roughly the same time and servers are identical, every server ends up with a similar load. It has no overhead and is easy to reason about, which is a good reason to start with it.

Its weakness appears when request durations vary widely. If one server happens to receive several slow requests in a row, it accumulates work while others sit idle, and round robin keeps feeding it new requests anyway. Tail latency rises for the unlucky clients.

Weights help when servers differ in capacity. A server with twice the CPU can be given twice the weight, and round robin will send it twice the requests.

Why Is least_conn Often Better for APIs?

API workloads often mix fast lookups and slow reports. With least_conn, a server stuck with long requests has many open connections, so the balancer sends new requests elsewhere. This self-corrects uneven load without manual tuning.

It is not perfect. Active connections are only a proxy for load. With HTTP keepalive and connection reuse from the balancer to the backends, connection counts may not reflect actual work in progress. Measure the effect rather than assuming improvement.

When you scale out, newly added servers start with zero connections and receive a flood of requests immediately, which can overwhelm a cold server. Some configurations support a gradual ramp-up for new servers, so check what your edition provides.

When Should You Use Hash-Based Methods?

Hashing makes sense when the same key should hit the same server, because the server holds something useful for that key. A local cache is the classic example. Routing requests for the same resource to the same server raises cache hit rates.

• Use consistent hashing with the hash directive so that adding or removing a server remaps only a fraction of keys.
• Avoid ip_hash behind other proxies or NAT, where many users share one address and load becomes uneven.
• Remember that hashing can create hot spots when some keys are far more popular than others.
• Do not use hashing as a substitute for proper shared session storage. Stateless design is more resilient.

Whatever method you choose, configure failure handling. The max_fails and fail_timeout parameters mark servers as unavailable after repeated errors, and backup servers take over when the primary ones fail. Pair this with sensible proxy timeouts and retry rules, so a failing backend does not hurt users.

Frequently Asked Questions

What is the default load balancing method in Nginx?

Round robin. Requests are distributed to the servers in the upstream group in turn.

Is least_conn better than round robin?

Often for APIs with variable request times. For uniform requests the difference is small, so measure your own traffic.

Do stateless APIs need sticky sessions?

No. If state lives in a shared store, any server can handle any request, which simplifies scaling and failover.

How does Nginx detect unhealthy servers?

Open-source Nginx uses passive checks based on failed requests. Active health checks are part of the commercial offering.

Summary

Start with round robin, move to least connections when request duration varies, and use hashing only when key affinity helps. Configure failure handling and weights, and consult the Nginx documentation for what your version supports.