Skip to content

Troubleshooting

The DNS answers REFUSED

The qname is outside every served zone. Check zone (or the zones keys) and the trailing dot; dig @host _whoami.<zone> TXT +short confirms which zone the server is serving.

The DNS answers NXDOMAIN

The name is inside the zone but is not the apex. A GSLB serves the apex only (app.example.com.), not www.app.example.com.. NXDOMAIN includes the SOA so a resolver can cache the negative answer.

The DNS answers SERVFAIL

No backend is healthy in the matched pool and in the default pool. Check the state of each backend on the Backends page and test the probe from the Probes editor. A pool entirely in maintenance behaves the same way: there is nothing to serve.

AAAA returns nothing

The pool is healthy but no backend in it has an ipv6 address. That is NOERROR with an empty answer by design. Add ipv6 to a backend, or expect an empty answer.

A backend shows "no verdict" (unknown)

The probe could not conclude: timeout, supervision 5xx, unreadable JSON, missing environment variable. This is not DOWN and the backend keeps serving. Check the probe URL, the supervision API and the *_env variables.

A probe never succeeds but curl works

  • The DNS container may not reach the target network (probe runs inside it).
  • verify_tls: true with a self-signed certificate: set it to false.
  • expect.status does not include the code the target returns.
  • expect.path and expect.allowed must be provided together.

The save is refused and the file is restored

The response names the cause: validation error (400) or a node that did not acknowledge. Common causes:

  • A CIDR does not parse or references an unknown pool.
  • A probe declares both healthcheck_url and probe.
  • A node is live but cannot write /config (check ownership and the tmpfs mode) or cannot reload (see its log).
  • A node is silent: it stops blocking after ten seconds, and catches up later.

The Instances page is empty

No node has pushed a snapshot. In the lot 2 file mode there is no fleet: the page explains it. In HA mode, check GSLB_API_URL, the token and the network between the node and the API (docker logs shows sync pull failed).

The VIP does not move

  • net.ipv4.ip_nonlocal_bind = 1 missing on a node: the DNS cannot bind the VIP and fails at startup (cannot assign requested address).
  • VRRP (protocol 112) blocked between the hosts.
  • Both nodes at the same priority: the highest wins, and the second only takes over when the first stops advertising.
  • GSLB_VRRP_CHECK_ZONE does not resolve: the health script fails and the node lowers its priority (this is the intended reaction, not a bug).
  • The check is not running at all. keepalived has no way to report "unknown": if it cannot fork its script — no SETUID/SETGID capability, or a script_user that does not exist — it keeps its last verdict, which reads succeeded. Stop the DNS container and watch docker logs <keepalived> for VRRP_Script(chk_dns) failed; if nothing appears, the check is decorative. See examples/keepalived/README.md.
  • The VIP is still owned by another device: check with arping.

The journal is missing queries

The DNS counts what it answered; the console counts rows. If the second is lower, the node logged event queue full or outbox full with a count: it answered faster than it could push its events to the API (one worker buffers 5 000 events and drains eight batches per tick). The answer path is unaffected — statistics are dropped, never a query.

At a sustained high rate, look at the volume before blaming the queue: one answer is one row of ~162 bytes, so 1 000 q/s is 14 GB a day. Shorten GSLB_RETENTION_DAYS, and expect /api/stats to take seconds once a window holds millions of rows (see Operations).

The compose reports No such image: gslb:local

Every compose names its images ${GSLB_VERSION:-local}. Without the variable, the expected tag is local: either build the image (docker compose build) or put GSLB_VERSION=<release> in the env file passed with --env-file. A host that pins a release must set it in both env files (API and DNS) — a missing one fails loudly like this instead of silently reusing an older image.

Port 53 is already in use

0.0.0.0:53 conflicts with systemd-resolved (127.0.0.53/54) on most hosts. Bind the LAN address (GSLB_ADDR=192.0.2.10) or the VIP, not the wildcard.

The container is unhealthy but DNS answers

The healthcheck queries GSLB_HEALTHCHECK_ADDR (default: the first bind address). On a VIP node it must be the node address; if it points at the VIP on the backup, the check goes to the master and can report the wrong process.

The console shows a stale snapshot

The API's snapshot_age is over ten seconds. In file mode the DNS bridge writes /state/snapshot.json every two seconds; in HA mode each node pushes its own. Check the DNS container and the network to the API. A stale snapshot is shown as a gap in availability, never as healthy.

Logs are unreadable in a collector

Set GSLB_LOG_FORMAT=json on both the DNS and the API; uvicorn is redirected to the same handler. Text stays the default and is byte-for-byte lot 1.