Troubleshooting

The problems met most often, each with its cause and fix. Start with the first checks: most problems show up there.

First checks

./lawliet status              # what is unhealthy, uncovered addresses, and the build id
./lawliet logs backend -n 200
sudo ./lawliet diag           # one redacted archive to send to support

From the host that has the problem, check the path to the server:

curl -vk https://<server>/health          # lab mode: http://<server>:8000/health

A JSON answer with "status": "healthy" means the server is reachable and serving. Anything else is the network, a firewall or name resolution, not the agent. The agent installers run this check themselves and say what is wrong before installing anything.

"Certificate does not cover <ip>" or curl error 60

Symptom. ./lawliet status, start or doctor print certificate does not cover <ip>; run: sudo ./lawliet tls repair, the console's certificate card says an address in use is not named, or an agent install stops with curl: (60) and "no alternative certificate subject name matches target ipv4 address".

Cause. The host reaches the server by an IP address that the server's certificate does not list as an IP entry. A self-signed certificate made by 1.5.9.s named only the address typed at install, often a host name or localhost, and an update does not replace an existing certificate by itself when agents are enrolled.

Fix. On the server:

sudo ./lawliet tls repair

It re-issues the self-signed certificate for every address of this server and adds them to TRUSTED_HOSTS. With agents enrolled it lists them and asks first: give each the new certificate afterwards (below). A certificate from your own CA is never re-issued: ask the CA for one that names every address agents use and install it with sudo ./lawliet tls. Then run the agent install again with --ca-cert. See TLS and certificates.

Agents offline after a certificate change

Symptom. After ./lawliet tls or ./lawliet tls repair, hosts that were online show as offline, and their agent logs show TLS verification errors.

Cause. Each agent trusts exactly the certificate it was installed with, and no TLS client accepts a different self-signed certificate in its place.

Fix. Export the new certificate with sudo ./lawliet tls --export, copy lawliet-server.pem to each host and refresh its trust; nothing is reinstalled or re-enrolled:

# Linux, as root
sudo bash install.sh --refresh-ca --ca-cert /path/to/lawliet-server.pem
# Windows, PowerShell as Administrator
powershell -ExecutionPolicy Bypass -File install.ps1 -RefreshCa -CaCert C:\path\to\lawliet-server.pem

The installer is the one the server serves; tls repair prints the full download line. A wrong file changes nothing. On macOS, copy the file over /etc/lawliet/server-ca.pem and run sudo launchctl kickstart -k system/com.lawliet.agent. See when the server's certificate changes.

403 "Invalid host header"

Cause. The name or address in the request is not listed in TRUSTED_HOSTS. On 1.5.9.s the installer listed only the address typed at its prompt, so a server installed with a host name refused requests by IP, even with a certificate that named the IP.

Fix. sudo ./lawliet tls repair adds this server's addresses, and ./lawliet tls adds the addresses of a certificate it installs. For any other name, add it to TRUSTED_HOSTS in .env and run sudo ./lawliet restart. See trusted host names and addresses.

409 when enrolling with a code

Symptom. A host installed from an offline package is refused with 409 and never appears.

Cause. An enrolment code never takes over an agent already on record. The host's earlier agent is still listed in the console, for example because it was removed from the host but not from Lawliet, or its token was revoked.

Fix. Delete the old agent in the console (Agents, the host, Remove agent). The waiting agent then enrols by itself while the code is still valid; if the code has expired, make a new package.

403 on remote deployment

Symptom. A deployment from Agents > Deploy agents remotely connects and passes the preflight, then every host fails at the installing step with HTTP 403.

Cause. A defect in 1.5.9.s: the agent download refused the deployment's one-time token on any server with an enrolment secret.

Fix. Upgrade to 1.5.11.s, then retry the failed hosts. Until then, install those hosts with the command from Install agent or ./lawliet token, which is not affected.

Lockout protection messages

Lawliet stops before it can lock a domain account. These messages are that protection at work, not a fault:

More in directory sign-in troubleshooting.

Clock and NTP

Symptom. An agent is online but scans, upgrades and other commands never run, or the Agents page shows a warning or critical clock chip.

Cause. Every command to an agent is signed and time-limited. A host more than a minute behind the server refuses the server's commands and policies while looking healthy; from 5 minutes, domain logons fail too.

Fix. Keep the server and every host on NTP (port 123/udp). On an air-gapped network, point them all at an internal time source. Check with timedatectl on Linux and w32tm /query /status on Windows. The Agents page shows each host's offset; see host clocks.

"The licence is not valid yet"

Symptom. A new licence is refused with NOT_YET_VALID; the activation screen shows the server's time in UTC next to the licence's start time.

Cause. The server's clock is behind the licence's start date.

Fix. Set the host's clock (containers use it): sudo timedatectl set-ntp true with a time server, or sudo timedatectl set-time 'YYYY-MM-DD HH:MM:SS' without one. Then install the licence again. See licence health.

Docker on an air-gapped server

Symptom. ./lawliet install stops because Docker or Compose is missing, or it cannot load the images.

Cause. The bundle carries Lawliet's images but not Docker itself, and the installer does not install Docker. Images are built per CPU architecture.

Fix. Install Docker Engine 24 or newer with the Compose plugin from packages brought in on media, or from an internal mirror: containerd.io, docker-ce, docker-ce-cli, docker-buildx-plugin and docker-compose-plugin. Check docker info and docker compose version, and that the account you install with can use Docker. Check the bundle matches the CPU: uname -m prints x86_64 or aarch64. See Docker on a server without internet.

An agent never appears

In rough order of likelihood:

1. Enrolment refused (401). The enrolment token is wrong or missing, or an offline package's code has expired or was revoked. ./lawliet token shows the current token and prints fresh install commands.
2. Server unreachable. The health check above fails from the host: open port 443 from the host to the server (lab mode: 8000).
3. Certificate not trusted. See certificate does not cover.
4. Clock skew. See Clock and NTP.
5. The host was registered before (409). Remove its old record in the console (Agents, the host, Remove agent); the waiting agent then enrols by itself.

An agent that shows the IP address 127.0.0.1 is working: on a host with no default gateway the agent cannot find its own address. Hosts with a default gateway, even with no internet behind it, show their real address.

The login page does nothing

In lab mode the console on port 3000 calls the API on port 8000; open both from the workstation. In HTTPS mode everything is on 443. With encryption switched off by ./lawliet tls off, the console is on http:// port 80.

Licence messages

Licensing never needs internet access: a key is verified on the server. See activation.

Collect diagnostics for support

sudo ./lawliet diag writes one redacted archive with logs, health, the certificate summary, TRUSTED_HOSTS, time and NTP, the licence status, agents by status and clock, and the last failed remote deployments. It refuses to write the archive if a self-check finds a secret in it. Send it to [email protected] with the build id from ./lawliet status; a version number alone is not specific enough. See Diagnostics.