Linux / Bash

Linux systemd Service Fails to Start on Boot

A systemd service that starts fine manually but fails specifically during boot almost always means it's starting before something it actually depends on is ready — network connectivity, a mounted filesystem, or another service — even though systemd's default ordering doesn't guarantee that dependency is available yet.

The Problem

Running systemctl start your-service manually after the system has finished booting works perfectly. But checking the service status after a fresh boot shows it failed:

systemctl status your-service
● your-service.service - Your Service
   Active: failed (Result: exit-code)

The service definition hasn't changed between the manual start and the boot-time attempt, which makes the discrepancy confusing at first.

Why It Happens

By default, systemd starts services in parallel as much as possible during boot, for faster startup times — but this means a service can start before something it implicitly depends on (without that dependency being explicitly declared) is actually ready. Common causes:

  • The service needs network connectivity, but starts before networking is actually up, especially common with services that connect to a remote database or API on startup
  • The service depends on a filesystem being mounted (a separate disk, a network share) that isn't mounted yet at the point systemd tries to start the service
  • The service depends on another custom service (like a database) that takes time to become ready, and systemd starts both in parallel without an explicit ordering dependency between them
  • Environment variables or configuration files the service needs aren't yet available at boot time if they're generated by another startup process that hasn't run yet

The Fix

First, check exactly why it failed using the service's actual logs from the failed boot attempt, not from a later manual run:

journalctl -u your-service -b

The -b flag limits output to the current boot, showing you specifically what happened during that failed startup attempt, which often reveals a connection error, a missing file, or a similar clue pointing directly at the actual dependency issue.

If the service needs network connectivity, add the appropriate target as both a dependency and ordering constraint in the unit file:

[Unit]
After=network-online.target
Wants=network-online.target

Note the distinction between After and Wants—After only controls ordering (start after this target, if it's going to start at all), while Wants actually ensures the target is activated. Using both together ensures the network target is both started and that your service waits for it specifically, rather than just for basic network device initialization, which network.target alone doesn't guarantee (it fires much earlier, before an actual IP address may be assigned).

If the service depends on another custom service (like a local database) also managed by systemd, declare that dependency explicitly:

[Unit]
After=postgresql.service
Requires=postgresql.service

Requires creates a stronger dependency than Wants — if the required service fails to start, this service won't start either, which is usually the correct behavior when the dependency is truly essential rather than optional.

If the service depends on a specific mount point being available, declare that dependency using the mount unit's name (paths translate to unit names with dashes replacing slashes):

[Unit]
RequiresMountsFor=/mnt/data

This is often simpler than manually referencing the specific mount unit name, and handles the translation automatically based on the path you actually need available.

After editing the unit file, reload systemd's configuration and restart the service to test the change:

sudo systemctl daemon-reload
sudo systemctl restart your-service

To properly test whether the fix actually resolves the boot-time issue (not just a manual restart, which doesn't reproduce the original timing problem), reboot the system and check the service status afterward, rather than assuming a successful manual restart confirms the fix:

sudo reboot
# after reboot:
systemctl status your-service

Still Not Working?

If the service still fails intermittently at boot despite correct dependency declarations, the dependency service itself might be marked as "started" by systemd before it's actually ready to accept connections (a common gap, since systemd's default notion of "started" for a simple service type is just "the process launched," not "the application inside is fully initialized"). For services that need to signal true readiness rather than just process launch, configure the dependency service with Type=notify if it supports the sd_notify protocol, or add a brief retry/wait loop in your dependent service's own startup logic as a pragmatic workaround when you don't control the dependency's own unit file configuration.

You might also like