Moving a Node.js application to a server and running it with node app.js works during development but is not enough for production. The moment you close the terminal window or your SSH connection drops, in most cases the process ends along with you; the kernel sends a SIGHUP signal to processes tied to the session, and unless a precaution such as nohup, screen or tmux has been taken, the application stops. In this article we look at why that problem exists, how systemd solves it and how a .service unit file is configured.

Why "node app.js" falls short in production

A Node.js process started directly in a terminal has three fundamental weaknesses. First, it depends on the session: when the SSH connection closes, the process closes too. Second, it cannot monitor itself: if the application crashes because of an unexpected error (an uncaught exception, memory exhaustion and so on), the site or API stays offline until someone notices, reconnects to the server and runs the command again by hand. Third, when the server is restarted (planned maintenance, a kernel update, a power cut) the application does not come back up on its own, because there is no automatic start mechanism.

All three problems come down to the same root: the operating system does not recognise that Node.js process as a "service". On Linux, the component that takes on this job is systemd.

What systemd service management changes

systemd is the init system that runs as PID 1 on most modern Linux distributions, including Debian, Ubuntu, CentOS/RHEL and Fedora — that is, the first process to start on the system and the one responsible for managing all the others. When you introduce an application to systemd with a unit file, the application's lifecycle becomes the operating system's responsibility: automatic start at system boot, restart on an unexpected crash, collection of the standard output/error streams in a central logging system (journald), and uniform status checking with systemctl.

The three sections of a unit file

A .service unit file consists of three sections and is usually written under /etc/systemd/system/.

SectionWhat it does
[Unit]The service's description and dependencies. <code>After=</code> states which target or service it starts after (for example <code>network.target</code>).
[Service]How the service is run: <code>ExecStart</code> the main command, <code>WorkingDirectory</code> the working directory, <code>User</code> which user it runs as, <code>Restart</code> and <code>RestartSec</code> the restart policy, <code>Environment</code> environment variables.
[Install]Defines which target the service is attached to with <code>systemctl enable</code> (usually <code>multi-user.target</code>), that is, when it comes into play at boot.

Type=simple means the process started by ExecStart is the service's main process; it is the right choice for programs that run in the foreground and do not put themselves in the background (fork/daemonize), such as Node.js. For programs that fork themselves like classic Unix daemons and leave a child process behind, Type=forking is used and a PIDFile usually has to be specified — but for a Node.js application that is almost never needed.

The ExecStart, WorkingDirectory, User and Restart directives

ExecStart should be written with absolute paths, like /usr/bin/node /var/www/myapp/server.js. Writing a relative path or just node server.js can leave systemd not knowing which directory to run the command from. WorkingDirectory solves exactly this: it states the directory from which the application's require/import calls and the node_modules lookup start, so ExecStart and WorkingDirectory must be consistent.

User decides which system user the service runs as. Running the application as a user with limited privileges (for example www-data or a user specific to the application) rather than root limits the privileges an attacker can obtain when a security hole is exploited.

When Restart=on-failure and RestartSec=5 are used together, systemd restarts the service automatically when it crashes or exits with a non-zero exit code, but puts a 5-second wait between each attempt. Without that wait, a broken service that keeps crashing could be restarted dozens of times per second and consume the CPU (a crash loop); RestartSec slows that loop down and keeps the system stable. Restart=always always restarts regardless of the reason for exiting (including a clean shutdown), while Restart=no turns automatic restarting off entirely. Environment variables (such as NODE_ENV=production) are defined with Environment="KEY=VALUE" lines, one line per variable.

The setup and monitoring workflow with systemctl

Once the unit file is ready, the order to follow is fixed:

  • Save the file as /etc/systemd/system/myapp.service.
  • sudo systemctl daemon-reload — tells systemd to re-read new or changed unit files.
  • sudo systemctl enable --now myapp — both enables the service so it starts automatically at boot and starts it immediately.
  • sudo systemctl status myapp — shows whether the service is running, the last few log lines and the exit code.
  • journalctl -u myapp -f — follows the service's live logs; this shows the application's standard output and standard error streams through journald, systemd's log management system.

Common mistakes

  • The wrong WorkingDirectory: even if the file path in ExecStart is correct, if WorkingDirectory does not point at the application's own root directory the dependencies inside node_modules cannot be found and the service crashes with a "Cannot find module" error.
  • Running as root for no reason: leaving the User line out makes the service run as root; that goes against the principle of least privilege and magnifies the impact of a security hole.
  • Forgetting to run daemon-reload after editing the unit file: systemd keeps using the old definition in memory even though the file has changed on disk; the changes do not take effect even if you run systemctl restart.
  • Never setting the Restart line: without a Restart directive the default behaviour is no; once the application crashes the service stays permanently in the "stopped" state and never comes back without someone intervening.
  • Using an aggressive restart policy without RestartSec: if RestartSec is not specified alongside Restart=always and the application crashes as soon as it starts, systemd tries to restart it many times per second; that consumes server resources for nothing.

Each of these directives may look like a small detail on its own, but set correctly together they take your application out of being dependent on a terminal and a single SSH session and make it part of the operating system.

If, instead of writing the unit file from scratch by hand, you would rather fill in the fields above and get output you can copy directly, you can enter these values in the form and use the tool. The generated file brings together the three sections and directives described in this article with the same logic; all you need to do is specify your own ExecStart path, working directory and user.