Go / Gin

How to Fix Go net/http Client Leaking File Descriptors Under Heavy Load

5 min read by DebuggedIt

Quick answer

Your Go service makes a lot of outbound HTTP requests, and under sustained load, it starts hitting file descriptor limits or degraded performance over time....

Your Go service makes a lot of outbound HTTP requests, and under sustained load, it starts hitting file descriptor limits or degraded performance over time. This is almost always caused by not fully draining and closing HTTP response bodies, which prevents Go's net/http transport from reusing the underlying TCP connection β€” accumulating open file descriptors instead of recycling them.

The Problem

The service runs fine initially, but under sustained traffic, outbound requests start failing with a resource exhaustion error:

Get "https://api.example.com/data": dial tcp: lookup api.example.com: socket: too many open files

Checking the process's open file descriptor count over time confirms it's climbing steadily rather than staying stable:

$ lsof -p $(pgrep myapp) | wc -l
8213
$ lsof -p $(pgrep myapp) | grep TCP | wc -l
7940

Why It Happens

Go's net/http client is designed to automatically reuse (pool) underlying TCP connections across requests to the same host, which is significantly more efficient than opening a new connection for every single request. That connection reuse depends entirely on the response body being fully read and explicitly closed β€” if you don't do both, Go can't safely return the connection to the pool, and it either leaks the file descriptor entirely or forces a new connection on every subsequent request. Common causes:

  • Missing resp.Body.Close() β€” the single most common cause, especially on error paths where a developer checks the status code or an error condition and returns early without remembering to close the body first.
  • Closing the body without fully reading it first β€” closing an unread body still releases the connection back to Go's runtime, but not always back to the connection pool for reuse, leading to more new connections (and their underlying file descriptors) being opened than necessary.
  • Creating a new http.Client for every request instead of reusing one shared client, which defeats connection pooling entirely regardless of whether bodies are closed correctly, since each new client starts with its own empty connection pool.
  • No timeout configured on the client, allowing a slow or hung upstream to hold a connection (and its file descriptor) open indefinitely rather than failing and freeing the resource.

The Fix

Always close the response body, using defer immediately after checking the error from the request itself, so it happens regardless of which code path the function takes afterward:

resp, err := client.Get("https://api.example.com/data")
if err != nil {
    return err
}
defer resp.Body.Close()

body, err := io.ReadAll(resp.Body)
if err != nil {
    return err
}

For connections to actually be reused efficiently (not just avoid leaking), fully drain the body before closing it, even in cases where you don't need the body's content β€” this lets Go's transport put the connection back in the pool cleanly instead of discarding it:

resp, err := client.Get("https://api.example.com/data")
if err != nil {
    return err
}
defer func() {
    io.Copy(io.Discard, resp.Body) // drain any remaining bytes
    resp.Body.Close()
}()

Create and reuse a single http.Client for your application's lifetime, rather than constructing a new one per request or per function call β€” this is essential for connection pooling to have any effect at all:

var httpClient = &http.Client{
    Timeout: 10 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        100,
        MaxIdleConnsPerHost: 20,
        IdleConnTimeout:     90 * time.Second,
    },
}

func fetchData(url string) ([]byte, error) {
    resp, err := httpClient.Get(url)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()
    return io.ReadAll(resp.Body)
}

MaxIdleConnsPerHost is worth tuning explicitly for services making many concurrent requests to the same host β€” the default of 2 is quite low for high-throughput services and can force unnecessary new connections even when bodies are being closed correctly.

Always set an explicit timeout on the client, since a client with no timeout at all can hang indefinitely on a slow or unresponsive upstream, holding its file descriptor open the entire time:

Timeout: 10 * time.Second,

Still Not Working?

If you've fixed the obvious cases but still see climbing descriptor counts, audit every place in your codebase making HTTP requests, since a single missed defer resp.Body.Close() anywhere in a frequently-called code path is enough to cause a slow, steady leak that's easy to miss in code review:

grep -rn "http.Get\|http.Post\|client.Do\|client.Get\|client.Post" --include="*.go" .

Cross-check each result to confirm a corresponding Close() call exists nearby. For ongoing monitoring rather than a one-time audit, track open file descriptor count as a metric in production so a new leak introduced by a future code change is caught quickly rather than discovered during an incident:

func fdCount() (int, error) {
    entries, err := os.ReadDir(fmt.Sprintf("/proc/%d/fd", os.Getpid()))
    if err != nil {
        return 0, err
    }
    return len(entries), nil
}

Exposing this as a Prometheus gauge and alerting on sustained upward trends gives you an early warning well before the process actually hits its file descriptor limit and starts failing requests outright.