Linux/Bash

Linux Signals and How Services Should Handle SIGTERM

4 min read by DebuggedIt

Quick answer

Quick answer: SIGTERM is the polite request to terminate, and a service should catch it, stop accepting new work, finish or hand off what is in flight, and...

Quick answer: SIGTERM is the polite request to terminate, and a service should catch it, stop accepting new work, finish or hand off what is in flight, and exit within the platform's timeout. SIGKILL cannot be caught and ends the process immediately. In containers, make sure your process actually receives SIGTERM, because PID 1 and shell wrappers often swallow it.

How Do the Common Signals Differ?

Signals are asynchronous notifications the kernel delivers to a process. Programs can install handlers for most of them, which lets a service react instead of dying. Knowing which signals can be handled is the foundation of reliable shutdown behavior.

  • SIGTERM: the standard request to terminate. It can be caught, which allows cleanup, and it is what service managers send first.
  • SIGINT: sent by Ctrl+C in a terminal. It is usually treated like SIGTERM by servers.
  • SIGHUP: historically for a closed terminal, and by convention often used to ask a daemon to reload its configuration.
  • SIGKILL: cannot be caught, blocked, or ignored. The kernel ends the process immediately with no cleanup.

The reliable pattern is escalation. A manager sends SIGTERM, waits for a grace period, and then sends SIGKILL if the process is still running. Your job is to finish cleanup before that deadline, because anything not completed when SIGKILL arrives is lost: open transactions are abandoned, buffers are not flushed, and connections are reset.

What Should a Service Do on SIGTERM?

A well-behaved service follows a short sequence. It stops accepting new requests or jobs, marks itself unready so load balancers stop sending traffic, lets in-flight work finish within a bounded time, flushes logs and buffers, closes connections to databases and queues, and exits with a suitable status code. Exiting promptly once work is done matters, because slow shutdowns delay deployments.

Make the handler small and safe. A signal handler should usually set a flag or cancel a context, and let the main program perform the cleanup, since doing complex work directly inside a handler can be unsafe in some languages. Also keep the shutdown timeout shorter than the manager's grace period, so you finish on your own terms.

The grace periods differ by platform. Systemd waits for a configurable stop timeout before escalating, container runtimes wait a default number of seconds on a stop command unless configured otherwise, and orchestrators have their own termination grace setting. Check the documentation for the values in your environment and tune them to match your longest legitimate in-flight request.

Why Do Containers Ignore SIGTERM?

The main process in a container runs as PID 1, which the kernel treats specially: signals with default actions are not applied to PID 1 unless the program installs a handler. A process that works fine on a normal host can therefore ignore SIGTERM in a container if it never registered a handler, and the runtime has to wait and then kill it.

The second common cause is shell wrappers. If the entrypoint is a shell script that launches your program as a child, the shell becomes PID 1 and may not forward the signal. Use the exec form of the entrypoint or end the script with exec so your program replaces the shell:

#!/bin/sh
set -e
# prepare config, run migrations, etc.
exec /app/server --config /etc/app/config.yaml

For programs that spawn children or cannot handle signals correctly, a minimal init process can run as PID 1, forward signals, and reap zombie processes. Many runtimes offer an option to include one. In shell scripts you manage yourself, use trap to forward signals to child processes and wait for them.

Frequently Asked Questions

Can I catch SIGKILL?

No. SIGKILL and SIGSTOP cannot be caught or ignored. Design for the possibility of abrupt termination with idempotent operations and durable state.

Why does my container take ten seconds to stop?

Probably it ignored SIGTERM and the runtime waited out the grace period before sending SIGKILL. Ensure the process receives and handles SIGTERM, for example by using exec in the entrypoint.

What exit code should I return after SIGTERM?

A clean shutdown usually exits with zero. Some platforms treat the conventional signal-based codes as normal for terminated processes. Check your supervisor's documentation.

Should I handle SIGHUP?

Only if you support configuration reload. Otherwise, the default action terminates the process, which may surprise operators who expect a reload.

Summary

Handle SIGTERM by stopping new work, draining in-flight requests, and exiting before the grace period ends. Remember that SIGKILL cannot be handled, and make sure containers deliver signals to your process using exec or an init process. Verify timeout defaults in your platform's documentation.