05 Background work

Chapter 05

Background work

Queue workers, a scheduler, one-shot startup tasks, and as many programs of your own as your app needs. There is no switch for any of these in the panel — you declare them as files in your own folder, which means they travel with your code.

Queue workers & scheduler are PHP-image only

The queue worker and scheduler need PHP, so they run on the Laravel image. The static (web-base) image runs a process supervisor too, so boot.conf and your own programs work there — but queue-worker.conf and scheduler.conf do not apply.

Where programs live

Create a directory .nimbus/supervisor/ at the root of your app and put program definitions in it. Because these ship in your repo, they land at <folder>/application/.nimbus/supervisor/. There are two places inside it, and which you use depends on whether the platform has a name for what you are running.

Three filenames are known to the image and mean something specific:

FilePurpose
boot.confRuns once when the container starts, then exits. Use it for warm-ups or one-time setup.
queue-worker.confLong-running queue workers, kept alive and restarted if they die.
scheduler.confThe task scheduler.

Everything else goes in daemons.d/, a subdirectory where every .conf file is loaded and you choose the names. An importer, a websocket server, a second queue on its own connection — anything the platform has no name for. There is no limit on how many, and it works on every site type, not only the PHP one.

.nimbus/supervisor/
├── queue-worker.conf        a known name — loaded
└── daemons.d/
    ├── importer.conf        loaded
    └── websocket.conf       loaded

The rule in both places is simply presence: the file exists, the program runs. Delete it, and it stops. There is no state stored anywhere in the panel and nothing to toggle.

Outside daemons.d/, an unknown filename is ignored in silence

There is no wildcard matching in .nimbus/supervisor/ itself. A file named worker.conf or queue-worker.conf.bak sitting loose there is never loaded — and never warned about either. Inside daemons.d/ the opposite holds, except that the extension must be exactly .conf: importer.conf.bak is not read either.

If a program is not running, check where the file is and how it is spelled before you look at what is in it. Neither mistake produces an error.

Program names must be unique across every file

The name is the one in the section header — [program:worker] — not the filename. Two files declaring the same one collide silently, and one of them simply never runs. Name each program after what it does and the problem does not arise.

What a definition looks like

These are supervisor program definitions. The image supplies no defaults, so each file must be complete on its own:

[program:queue-worker]
command=/usr/bin/php /var/www/virtual/app.example.com/application/artisan queue:work --sleep=3 --tries=3
directory=/var/www/virtual/app.example.com/application
user=www-data
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

Use absolute paths in command. Because your folder is mounted at the identical path inside the container, the literal site path is correct in both places — no guesswork about what the path looks like from the inside.

Log to /dev/stdout and /dev/stderr as above. Everything the container prints is collected together, so your worker output lands beside nginx and PHP errors instead of in a file nobody reads.

The rules your file must satisfy

Every file is validated at container start, in daemons.d/ exactly as under a known name. The checks are few and strict:

  1. It must parse as INI. A syntax error means the file is rejected.
  2. Only program sections are allowed[program:…], [group:…] or [eventlistener:…]. You cannot redefine the supervisor daemon itself.
  3. Programs run as www-data. Leave user= out and it is filled in for you. Set it to anything else and the file is rejected outright rather than quietly rewritten.

A rejected file is skipped, never fatal. The reason is printed to the container log and the rest of the site carries on serving. This matters: supervisor aborts on a bad config, so an unvalidated file would take nginx and PHP down with it and the site would go completely dark. One broken worker definition costs you that worker, not the site.

Making a change take effect

Programs are read at container start. Adding, editing or deleting a file does nothing until then. Restart the site from the panel — the same restart that clears the opcache after a deploy, so a deploy that adds a worker needs only the one restart.

Recreating the container for a settings change, or switching image, also re-reads them. None of those operations ever modify or remove your .nimbus/ directory.

Checking whether a program loaded

On start, the container logs one line listing which programs it loaded, and a separate SKIPPED line with the reason for each file it rejected — a rejected daemon appears as SKIPPED daemon-<name>.conf, so you can see which of yours it was. That log line is the fastest way to tell "my file has a typo in the name" from "my file has a typo inside it". Reading it needs log access from the box.

You cannot manage these programs from inside the container — the supervisor control socket is restricted to root, and your shell is not. Restarting the site is the supported way to apply a change.