Nginx Proxy Timeouts and Retry Behavior for Upstream APIs

Nginx has separate timeouts for connecting to an upstream, sending a request, and reading a response, and each one applies to a different stage. Its retry behavior, controlled by proxy_next_upstream, can resend requests to another server after certain failures. Set timeouts based on real latency, and be careful with retries on non-idempotent requests, because they can cause duplicate side effects.

Which Proxy Timeouts Does Nginx Provide?

Three directives cover the main stages of a proxied request. Understanding what each measures prevents most misconfigurations.

• proxy_connect_timeout: how long Nginx waits to establish a connection with the upstream server.
• proxy_send_timeout: the maximum time between two successive write operations when sending the request to the upstream.
• proxy_read_timeout: the maximum time between two successive reads of the response from the upstream.

The send and read timeouts are not total deadlines. They measure gaps between operations, so a response that trickles in steadily can run longer than the configured value without triggering the timeout. If you need a hard limit on total request time, enforce it in the application or at another layer.

location /api/ {
    proxy_pass http://backend;
    proxy_connect_timeout 2s;
    proxy_send_timeout 10s;
    proxy_read_timeout 30s;
}

How Should You Choose Timeout Values?

Derive values from observed behavior, not round numbers. Connection establishment inside a data center should be fast, so a short connect timeout, such as a couple of seconds, detects dead hosts quickly. A read timeout should be somewhat above your slowest legitimate response, so real slow requests succeed while hung ones are cut off.

Keep the layers consistent. If Nginx waits 60 seconds but your load balancer in front gives up after 30, clients see errors from the outer layer while Nginx keeps working on abandoned requests. Align timeouts from the outside in, so each inner layer is shorter than the one wrapping it.

When Nginx times out waiting for the upstream, it returns a 504 Gateway Timeout. A 502 usually means the upstream closed the connection or sent an invalid response. Knowing the difference helps you decide whether to investigate slowness or crashes.

Percentiles are more useful than averages here. If the 99th percentile response time of an endpoint is eight seconds, a ten second read timeout leaves very little margin during a slow period. Look at several days of data, including peak hours, before you settle on numbers, and revisit them after major releases.

How Does proxy_next_upstream Retry Requests?

When you define several servers in an upstream block, Nginx can pass a failed request to the next server. The proxy_next_upstream directive lists which conditions trigger this, such as errors, timeouts, or specific status codes.

upstream backend {
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
}

location /api/ {
    proxy_pass http://backend;
    proxy_next_upstream error timeout;
    proxy_next_upstream_tries 2;
    proxy_next_upstream_timeout 10s;
}

By default, Nginx does not retry non-idempotent methods such as POST after the request has been sent to a server, unless you explicitly allow it. Verify this behavior in the documentation for your version before relying on it.

What Are the Risks of Retrying Requests?

A retry after a timeout is dangerous because you do not know whether the first server processed the request. If it did, the second attempt repeats the side effect. For payments, order creation, or email sending, that can mean duplicates.

• Limit retries with proxy_next_upstream_tries and a total time cap.
• Avoid enabling retries on non-idempotent methods unless the application deduplicates with idempotency keys.
• Do not retry on application errors such as 500 unless you are certain the operation is safe to repeat.
• Watch for retry amplification: when the backend is overloaded, retries add load and make recovery harder.

Passive health checks matter as well. Parameters such as max_fails and fail_timeout on upstream servers let Nginx stop sending traffic to a server that keeps failing, which reduces the number of requests that need a retry in the first place.

Frequently Asked Questions

What causes a 504 error in Nginx?

Nginx waited longer than the configured timeout for the upstream to respond. Check upstream performance and compare the read timeout with real response times.

Does proxy_read_timeout limit the total response time?

No. It limits the gap between two reads. A slowly streaming response can exceed it in total duration without being cut off.

Does Nginx retry POST requests?

Not by default after the request was sent, but the behavior is configurable. Use idempotency keys if you allow it.

Should I raise the timeout when I see 504 errors?

Only after understanding why the upstream is slow. Raising it can hide a real problem and tie up connections for longer.

Summary

Use short connect timeouts, read timeouts based on real latency, and cautious, bounded retries. Align timeouts across layers and protect non-idempotent operations. Confirm directive defaults and behavior in the Nginx documentation for your version.