Troubleshooting¶
The DNS answers REFUSED¶
The qname is outside every served zone. Check zone (or the zones keys) and
the trailing dot; dig @host _whoami.<zone> TXT +short confirms which zone the
server is serving.
The DNS answers NXDOMAIN¶
The name is inside the zone but is not the apex. A GSLB serves the apex only
(app.example.com.), not www.app.example.com.. NXDOMAIN includes the SOA so a
resolver can cache the negative answer.
The DNS answers SERVFAIL¶
No backend is healthy in the matched pool and in the default pool. Check the state of each backend on the Backends page and test the probe from the Probes editor. A pool entirely in maintenance behaves the same way: there is nothing to serve.
AAAA returns nothing¶
The pool is healthy but no backend in it has an ipv6 address. That is NOERROR
with an empty answer by design. Add ipv6 to a backend, or expect an empty
answer.
A backend shows "no verdict" (unknown)¶
The probe could not conclude: timeout, supervision 5xx, unreadable JSON, missing
environment variable. This is not DOWN and the backend keeps serving. Check the
probe URL, the supervision API and the *_env variables.
A probe never succeeds but curl works¶
- The DNS container may not reach the target network (probe runs inside it).
verify_tls: truewith a self-signed certificate: set it tofalse.expect.statusdoes not include the code the target returns.expect.pathandexpect.allowedmust be provided together.
The save is refused and the file is restored¶
The response names the cause: validation error (400) or a node that did not
acknowledge. Common causes:
- A CIDR does not parse or references an unknown pool.
- A probe declares both
healthcheck_urlandprobe. - A node is live but cannot write
/config(check ownership and the tmpfs mode) or cannot reload (see its log). - A node is silent: it stops blocking after ten seconds, and catches up later.
The Instances page is empty¶
No node has pushed a snapshot. In the lot 2 file mode there is no fleet: the
page explains it. In HA mode, check GSLB_API_URL, the token and the network
between the node and the API (docker logs shows sync pull failed).
The VIP does not move¶
net.ipv4.ip_nonlocal_bind = 1missing on a node: the DNS cannot bind the VIP and fails at startup (cannot assign requested address).- VRRP (protocol 112) blocked between the hosts.
- Both nodes at the same priority: the highest wins, and the second only takes over when the first stops advertising.
GSLB_VRRP_CHECK_ZONEdoes not resolve: the health script fails and the node lowers its priority (this is the intended reaction, not a bug).- The check is not running at all. keepalived has no way to report
"unknown": if it cannot fork its script — no
SETUID/SETGIDcapability, or ascript_userthat does not exist — it keeps its last verdict, which readssucceeded. Stop the DNS container and watchdocker logs <keepalived>forVRRP_Script(chk_dns) failed; if nothing appears, the check is decorative. Seeexamples/keepalived/README.md. - The VIP is still owned by another device: check with
arping.
The journal is missing queries¶
The DNS counts what it answered; the console counts rows. If the second is
lower, the node logged event queue full or outbox full with a count: it
answered faster than it could push its events to the API (one worker buffers
5 000 events and drains eight batches per tick). The answer path is unaffected —
statistics are dropped, never a query.
At a sustained high rate, look at the volume before blaming the queue: one
answer is one row of ~162 bytes, so 1 000 q/s is 14 GB a day. Shorten
GSLB_RETENTION_DAYS, and expect /api/stats to take seconds once a window
holds millions of rows (see Operations).
The compose reports No such image: gslb:local¶
Every compose names its images ${GSLB_VERSION:-local}. Without the variable,
the expected tag is local: either build the image (docker compose build) or
put GSLB_VERSION=<release> in the env file passed with --env-file. A host
that pins a release must set it in both env files (API and DNS) — a missing
one fails loudly like this instead of silently reusing an older image.
Port 53 is already in use¶
0.0.0.0:53 conflicts with systemd-resolved (127.0.0.53/54) on most hosts.
Bind the LAN address (GSLB_ADDR=192.0.2.10) or the VIP, not the wildcard.
The container is unhealthy but DNS answers¶
The healthcheck queries GSLB_HEALTHCHECK_ADDR (default: the first bind
address). On a VIP node it must be the node address; if it points at the VIP on
the backup, the check goes to the master and can report the wrong process.
The console shows a stale snapshot¶
The API's snapshot_age is over ten seconds. In file mode the DNS bridge writes
/state/snapshot.json every two seconds; in HA mode each node pushes its own.
Check the DNS container and the network to the API. A stale snapshot is shown as
a gap in availability, never as healthy.
Logs are unreadable in a collector¶
Set GSLB_LOG_FORMAT=json on both the DNS and the API; uvicorn is redirected to
the same handler. Text stays the default and is byte-for-byte lot 1.