6. TLS and certificates

How the server's certificate works, how to check it, replace it, re-issue it with agents online, and how to turn encryption off and on.

6.1 How it works#

• In HTTPS mode, nginx serves the certificate and key in infra/docker/ssl/fullchain.pem and privkey.pem. No other container reads the private key.
• A certificate must name every address it is reached by. An IP address is matched only by an IP entry in the certificate, a name only by a DNS entry. A wildcard covers exactly one leftmost label and never an IP address. Otherwise the agent installer fails with curl error 60 (no alternative certificate subject name matches target ipv4 address).
• Agents verify the server. With a certificate from a CA the hosts already trust, nothing else is needed. With a self-signed certificate, each agent was given that exact certificate file when it was installed, and trusts only that file.
• When the certificate changes, agents 2.1.9.s and later can be handed the new one over their existing connection before the server switches, so they stay online. Older agents, and agents that are offline during the switch, must be given the new certificate by hand (6.6). Nothing is re-enrolled and no data is lost either way.
• ./lawliet never re-issues a certificate that a CA issued. It only re-issues self-signed ones.

6.2 Check the certificate#

On the server#

sudo ./lawliet status        # section "TLS certificate": coverage of every address
sudo ./lawliet tls status    # state, mode, subject, expiry, agents on plain HTTP
sudo ./lawliet tls           # subject and expiry; warns within 30 days; answer N to change nothing

What you see:

In the console#

1. Open Settings > Platform and find the card Certificate.
2. Press Check again to read the certificate again.
3. What you see: one of the statuses below, and the details under it.

The card also shows the subject, issuer (with a self-signed tag), key, validity, the names in the certificate, each address in use marked covered or not covered, the SHA-256 fingerprint with a copy button, and the commands to run on the server. The notification bell announces each change of state once.

6.3 Export the certificate for agent hosts#

Needed only for a self-signed certificate.

sudo ./lawliet tls --export                 # writes ./lawliet-server.pem
sudo ./lawliet tls --export /tmp/srv.pem    # or another path

What you see: File, Subject, Expires. The file contains the certificate only, never the key. Copy it to each agent host as /path/to/lawliet-server.pem (Linux, macOS) or C:\path\to\lawliet-server.pem (Windows), or change that path in the commands ./lawliet token prints.

If it fails: No certificate to export. This deployment is not serving TLS. means plain-HTTP mode.

6.4 Install a certificate from your CA#

Use this to renew a CA-issued certificate, or to replace a self-signed one with a CA-issued one.

Before you start#

• Get from your CA a certificate whose subjectAltName lists every address operators and agents use: the DNS name, and an IP: entry for each IP address agents connect by.
• Have the full chain (server certificate, then intermediates, ideally ending with the root) and the private key, in PEM, on the server.
• Keep a copy: sudo cp -a infra/docker/ssl /root/ssl-before-$(date +%F).

Procedure#

1. Run:

   sudo ./lawliet tls

2. Answer Replace the certificate? (y/N) with y.

3. Enter Path to the certificate (fullchain PEM): and Path to the private key (PEM):.

4. It checks that the key belongs to the certificate: OK Certificate and key match.

5. With no agents enrolled, it installs the pair, tests it with nginx and reloads nginx: OK Installed, OK nginx reloaded with the new certificate.

6. With agents enrolled, and the chain ending in its root certificate, it first hands that root to the agents (6.5, With agents enrolled) and switches when they hold it. If the chain does not end in its root, or the root is a public CA's, it installs the pair at once and prints the refresh commands for every agent (6.6).

7. Check: sudo ./lawliet status, and open the console in a new browser window.

The names and IPv4 addresses the new certificate covers are added to TRUSTED_HOSTS.

Handing a public CA's root to agents#

If the chain ends in a root that is in this server's system trust store (a public CA), it is not handed to agents: they would then trust every certificate that CA issues, for any name. The command says so. Add --allow-public-root only if you accept that:

sudo ./lawliet tls --allow-public-root

What you see: the same steps as the procedure above; the root is then handed to the agents with the certificate. The option works the same way with tls repair (6.5).

6.5 Re-issue a self-signed certificate (tls repair)#

Use when status, doctor or the console says the self-signed certificate does not cover an address, has expired, or expires within 30 days.

sudo ./lawliet tls repair

It makes a new self-signed certificate (RSA 2048, 825 days) that keeps every name the old one had and adds every address this server is reached by: the deployment address, the server's own IPv4 addresses, and localhost. When nothing is missing and the certificate is not near expiry it says The certificate already covers ... and changes nothing.

With no agents enrolled#

It re-issues and installs at once. What you see:

  OK Installed
  OK nginx reloaded with the new certificate
  Covers                     10.0.0.5, lawliet.corp.example, localhost
  Valid until                <date>
  Previous pair kept as      infra/docker/ssl/{fullchain,privkey}.pem.backup.<time>
  OK No agent was enrolled, so none needs anything.

With agents enrolled#

1. Run sudo ./lawliet tls repair. It lists up to 20 enrolled agents and explains what will happen.

2. Answer Re-issue the certificate now? (y/N) with y (or use --yes).

3. Handing the new certificate to the agents first. The server announces the new certificate in every agent's next heartbeat. Agents 2.1.9.s and later add it to the certificates they trust while the server still serves the old one. Every 10 seconds it prints:

     12 of 14 agent(s) hold the new certificate (85%)

4. When every agent that can take it holds it, it switches: OK Every agent that can be handed it holds it, then Switching to the new certificate, and OK Agents drop the previous certificate once they have checked the server with the new one.

5. If some agents still do not hold it after the wait (10 minutes by default), it lists them with their version and why (waiting, offline, old), and asks Switch now anyway? Those agents go offline until each is given the new certificate (y/N).

   • No (the default, and always with --yes): nothing is switched. The new certificate stays announced and agents keep taking it. Run the same command later to go on waiting.
   • Yes: it switches; then do 6.6 for the listed hosts.

6. Check: sudo ./lawliet status and the Environment page: agents should stay online.

Options#

Examples:

sudo ./lawliet tls repair --yes                   # wait up to 10 minutes for every agent
sudo ./lawliet tls repair --yes --timeout 1800    # wait 30 minutes
sudo ./lawliet tls repair --yes --ack 90          # switch once 90 % hold it
sudo ./lawliet tls repair --yes --force           # switch after the wait, whoever holds it

Resuming. Running tls repair again while a certificate is still announced resumes with the same certificate: Resuming: the certificate staged by an earlier run is still being handed to the agents.

Agents that trust the system store. Some agents trust their operating system's certificate store rather than a file Lawliet manages. The command lists them; they keep working only if that store trusts the new certificate.

While TLS is off tls repair works the same (443 keeps serving the certificate to agents enrolled over HTTPS) and leaves TLS off.

6.6 Give the new certificate to agents that did not take it#

For agents older than 2.1.9.s, agents that were offline during a switch, and every agent after a switch without hand-over. The command that switched prints these steps with the exact download line for your server.

1. On the server, export the certificate:

   sudo ./lawliet tls --export        # writes ./lawliet-server.pem

2. Copy lawliet-server.pem to each listed host.

3. On each Linux host, run the refresh line printed by the server. Its core is:

   sudo bash install.sh --refresh-ca --ca-cert /path/to/lawliet-server.pem

4. On each Windows host, in PowerShell as Administrator, run the printed line. Its core is:

   powershell -NoProfile -ExecutionPolicy Bypass -File .\install.ps1 -RefreshCa -CaCert 'C:\path\to\lawliet-server.pem'

What you see: the installer checks the server with the new certificate first (a wrong file changes nothing), replaces the agent's trust file (keeping the old one as server-ca.pem.previous) and restarts the agent. It reconnects within a minute and keeps its identity.

6.7 Withdraw a staged certificate (tls cancel)#

Use when you started a certificate change with agents enrolled and decided not to finish it.

sudo ./lawliet tls cancel

What you see: Withdrawn: agents remove the announced certificate at their next heartbeat. nginx was not changed. or No certificate was staged; nothing to withdraw.

It refuses when the staged certificate is already the one nginx serves (the switch has happened): The staged certificate is the one this server already serves, so it cannot be withdrawn; it is recorded as switched.

6.8 Turn encryption off and on#

On an air-gapped site where a self-signed certificate cannot be given to hosts in time, tls off serves the console and API on plain HTTP. It is a deliberate step down: operator passwords, session tokens, the enrolment secret and agent traffic cross the network unencrypted. Every console page says so until it is turned back on.

Turn encryption off#

1. Run:

   sudo ./lawliet tls off

2. Read the warning and answer Turn encryption off? (y/N) with y (or pass --yes).

3. It saves .env, .lawliet.conf and the certificate to .tls-snapshots/tls-switch-<time>/, recreates nginx and the backend, waits for health and checks the result.

4. What you see: OK Serving as configured, the console URL on http://, the HTTPS address kept for agents enrolled over https, and the red warning.

Turn encryption on#

1. Run:

   sudo ./lawliet tls on

2. It saves the configuration, keeps the certificate (or asks for one if infra/docker/ssl/ is empty), recreates nginx and the backend and verifies HTTPS, HSTS, the redirect, and that plain HTTP refuses callers that are not plain-HTTP agents.

3. If the self-signed certificate misses an address, it runs the tls repair check (6.5).

4. It then asks, each default yes (treat what crossed in clear as captured):

   Question	What yes does
   Sign every user out now (all sessions are revoked)?	Every session issued so far is refused; everyone signs in again
   Revoke these API keys? (only if any were used while off)	Revokes them. Issue new ones under Settings > API keys
   Rotate the agent enrolment secret?	New install commands (./lawliet token). Enrolled agents are unaffected

   --yes answers yes to all three.

5. Afterwards: everyone who signed in while encryption was off should change their password, and agents enrolled while off are listed under Environment to be reinstalled over HTTPS.

Agents still on plain HTTP are a critical finding. An agent's token also signs the commands it runs as root, and that token crossed the network in clear. While TLS is on and any agent still uses plain HTTP, the console shows a critical finding and sudo ./lawliet tls status counts them. Reinstall each over HTTPS.

Browsers and HSTS. A host name that received HSTS while TLS was on stays https-only in that browser until its HSTS entry is cleared; 443 keeps answering, so the console still works over https there. Browsers ignore HSTS for IP addresses.

If a switch fails#

The command checks the switched deployment for up to 3 minutes. If it does not behave as it should, it puts the saved configuration back automatically: Reverted: TLS is <before> again, with the configuration saved in <snapshot>. Read sudo ./lawliet logs nginx and sudo ./lawliet logs backend.

6.9 Move a plain-HTTP install to HTTPS#

A deployment installed with option 2 (plain HTTP on 3000 and 8000) is moved behind nginx by the same command:

sudo ./lawliet tls on

1. It explains the move and asks Continue? (Y/n).
2. It asks for the certificate (your own or self-signed, as at install).
3. It takes a database backup, replaces the containers, and starts nginx with HTTPS on 443. nginx takes over ports 3000 and 8000, so agents installed against http://<address>:8000 keep reporting, and port 3000 sends browsers to HTTPS.
4. It runs the certificate check and the three questions of 6.8.
5. What you see: OK Serving over HTTPS and the new console URL https://<address>.

Then open 443/tcp on the firewall for operators and agents, reinstall the agents over HTTPS one by one (Environment lists them), and close 3000 and 8000 when none is left.

6.10 If it fails#