Python

Python Multiprocessing "Daemonic Processes Are Not Allowed to Have Children"

This error is Python's multiprocessing module deliberately preventing a daemon process from spawning its own child processes — daemon processes are specifically designed to be terminated automatically when the main program exits, and allowing them to have their own children would create orphaned processes that never get properly cleaned up.

The Problem

Code that spawns a process from within another process fails with:

AssertionError: daemonic processes are not allowed to have children

This specifically happens when the process attempting to spawn a child is itself marked as a daemon process (daemon=True), often set without fully considering the implications this has for further nested process spawning.

Why It Happens

A daemon process in Python's multiprocessing module is one that gets automatically terminated when the main program exits, without waiting for it to finish naturally — useful for background tasks that shouldn't block program shutdown. Python enforces a specific rule: a daemon process cannot itself create additional child processes, since if the main program exits and forcibly terminates the daemon, any grandchild processes it had spawned would be abruptly orphaned with no clean way to also terminate them. Common causes of hitting this restriction:

  • A worker process created with daemon=True that itself tries to use a multiprocessing.Pool or spawn additional worker processes internally, not realizing this nested spawning is specifically disallowed for daemon processes
  • Using a process pool (like multiprocessing.Pool, which creates daemon worker processes by default) where a task running inside one of those pool workers itself tries to create another process
  • A misunderstanding of what "daemon" actually restricts, assuming it's purely about background execution without realizing the specific "no children" rule that comes attached to it

The Fix

If the nested process spawning is genuinely necessary, don't mark the outer process as a daemon — accept that you'll need to explicitly manage its lifecycle (joining it, or explicitly terminating it) rather than relying on daemon auto-cleanup:

from multiprocessing import Process

def outer_worker():
    inner = Process(target=inner_worker)
    inner.start()
    inner.join()

# Don't set daemon=True if this process needs to spawn its own children
p = Process(target=outer_worker, daemon=False)
p.start()
p.join() # explicitly wait for it, since it's not a daemon that auto-cleans-up

If you're using multiprocessing.Pool and hitting this because a pool worker itself tries to spawn a process, restructure the work so nested parallelism happens differently — rather than a pool worker spawning its own child process, use threading within that worker (threads don't have this same daemon-child restriction) if the nested work is more I/O-bound than CPU-bound:

import threading

def pool_worker(item):
    # Instead of spawning a new Process here, use a thread
    thread = threading.Thread(target=some_task, args=(item,))
    thread.start()
    thread.join()

If genuine nested multi-level process parallelism is required (not just I/O-bound work suitable for threading), consider using concurrent.futures.ProcessPoolExecutor at the outermost level only, structuring your work so that only top-level, non-daemon processes ever spawn further children, rather than trying to nest process pools multiple levels deep:

from concurrent.futures import ProcessPoolExecutor

def top_level_task(item):
    # This level can spawn further processes if truly needed,
    # since it's not itself a daemon
    with ProcessPoolExecutor() as inner_executor:
        results = list(inner_executor.map(sub_task, item.sub_items))
    return results

with ProcessPoolExecutor() as executor:
    results = list(executor.map(top_level_task, items))

Note that even this pattern of nested process pools has real overhead and complexity — before implementing deeply nested multiprocessing, consider whether the workload can be flattened into a single level of parallelism instead, which is both simpler to reason about and avoids this entire category of restriction and complexity.

Still Not Working?

If restructuring to avoid nested process spawning isn't practical for your specific workload, and you're on a Unix-like system, consider using multiprocessing.set_start_method('spawn') combined with carefully explicit process lifecycle management (never relying on daemon flags for the specific processes that need to have children), which gives you full manual control at the cost of needing to handle cleanup and termination logic yourself rather than relying on Python's automatic daemon cleanup behavior:

import multiprocessing
multiprocessing.set_start_method('spawn')
# then structure all process creation without daemon=True at any level
# that needs to spawn further children, with explicit join()/terminate() calls

You might also like