Linux Log Rotation Strategies for Long-Running Services

Quick answer: rotate logs by size or time, compress old files, and cap retention so disks never fill. With logrotate, prefer signaling the service to reopen its log file after rotation, and use copytruncate only when the application cannot reopen files. In containers, write to stdout and let the runtime or platform handle rotation and limits.

Why Does Log Rotation Matter?

Logs grow without limit. A busy service can write gigabytes per day, and a full disk is a classic cause of outages: databases stop writing, services cannot create temporary files, and even logging into the machine can fail. Rotation keeps files at manageable sizes, makes old data easy to compress or archive, and enforces a retention policy automatically.

Rotation also helps operations. Smaller files are faster to search and transfer, and a predictable naming scheme lets tools find yesterday's logs. Retention matters for more than disk space: you may need to keep logs for a minimum period for debugging or compliance, and delete them after a maximum period to limit exposure of sensitive data.

A subtle problem is that a process writing to a file keeps its file descriptor open. When another program renames that file, the process keeps writing to the renamed file, not to a new one. That is why rotation must be coordinated with the application.

How Do You Configure logrotate Correctly?

Logrotate is the standard tool on most Linux distributions. Configuration lives in files that describe which logs to rotate, how often, and what to do afterward. A typical entry for a service looks like this:

/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 myapp myapp
    postrotate
        systemctl kill -s HUP myapp.service
    endscript
}

Each option solves a specific problem. Rotate keeps a set number of old files. Compress and delaycompress shrink old logs while leaving the most recent rotated file uncompressed, which avoids breaking a process that still has it open briefly. Missingok and notifempty prevent errors and pointless empty rotations. The create line makes a fresh file with the right owner and permissions.

The post-rotation script tells the service to reopen its log file, commonly via SIGHUP, though conventions vary, so check what your application supports. If the application cannot reopen files, the copytruncate option copies the log and truncates the original in place. It works without cooperation, but a small window exists where lines written during the copy can be lost, so use it only when needed.

What About Journald and Containers?

Services managed by systemd that write to stdout and stderr have their output captured by the journal, which has its own size and retention limits, set in its configuration. Review those limits, because defaults may be too small for debugging or too large for a small disk. Persistent storage of the journal also needs to be enabled if you want logs to survive reboots.

In containers, the best practice is to log to stdout and stderr and let the runtime collect the output. Container runtimes can rotate their own log files, and the default configuration may keep them unbounded, so set maximum size and file count explicitly. In orchestrated environments, a log collection agent usually ships logs to a central system, and the node's local log files are only a buffer.

• Choose size-based rotation for bursty services and time-based rotation for predictable ones, or combine both with a maximum size.
• Monitor disk usage and alert before it fills, since rotation settings can be misconfigured or logs can suddenly spike.
• Ship important logs to central storage, so local rotation never destroys the only copy.
• Avoid logging sensitive data, because retention multiplies the exposure.

Frequently Asked Questions

What is the difference between copytruncate and create?

Create renames the old file and creates a new one, which requires the app to reopen its log. Copytruncate copies the file and empties the original, which works without cooperation but can lose lines.

Why does my service keep writing to the old log file?

It still holds the original file descriptor after rotation. Signal it to reopen logs in a postrotate script, or use copytruncate if it supports neither.

How long should I keep logs?

It depends on debugging needs and legal requirements. Many teams keep recent logs locally for days or weeks and archive centrally for longer, subject to policy.

How do I limit Docker container logs?

Configure the logging driver options for maximum size and number of files, either per container or globally in the daemon settings. Check the documentation for your runtime.

Summary

Rotate logs with logrotate by size or time, compress and expire old files, and have services reopen logs after rotation. Prefer stdout logging in containers with explicit runtime limits, and monitor disk usage. Review the logrotate and journald documentation for options on your distribution.