Prompt and scope
Design an authoritative DNS control plane for a multi-tenant SaaS. Users submit A, AAAA, CNAME, and TXT records. The system validates conflicts, creates zone versions, notifies authoritative nodes, and supports AXFR/IXFR, DNSSEC, rollout, rollback, and audit.
What the interviewer evaluates
- Separating the control plane, authoritative serving plane, and recursive resolvers.
- Using immutable zone versions and atomic publication.
- Connecting SOA serial, NOTIFY, AXFR, and IXFR.
- Tenant isolation, authorization, DNSSEC key protection, and bad-release prevention.
- Handling lagging nodes, partial publication, negative caching, and rollback.
Clarifications to ask first
Confirm zone count, write rate, propagation SLO, DNSSEC, node geography, authorization and approval model, and whether old nodes only support AXFR. If unspecified, assume a globally distributed authoritative cluster with minute-level visibility.
Thirty-second answer framework
Write normalized records in the control plane, create an immutable zone version, validate it, and atomically switch authoritative nodes. Increment the SOA serial; NOTIFY triggers IXFR, with AXFR as fallback. Every release has state, audit evidence, and a rollback pointer. DNSSEC, tenant authorization, and propagation monitoring are release gates.
Step-by-step deep dive
1. Separate planes
The control plane stores tenants, zones, records, and versions. The serving plane answers only from a published version. Recursive resolvers are outside the write path, and TTL creates visibility delay. A version includes canonical zone data, SOA serial, checksum, signing state, and targets.
2. Design writes and validation
Write a candidate transaction and validate names, TTLs, type constraints, CNAME conflicts, delegation boundaries, and tenant quotas. Batch changes carry an idempotency id and optimistic version. A failed validation never creates a publishable version.
3. Generate and publish atomically
Generate the zone structure and run syntax, delegation, cycle, and DNSSEC checks. Store the validated content-addressed object and checksum. Nodes switch a version pointer atomically, avoiding mixed old and new answers.
4. Propagation and lagging nodes
The primary increments the serial and sends NOTIFY. Secondaries request IXFR, falling back to AXFR when the delta chain is unavailable or fails validation. Nodes report serial, checksum, and signing state. A lagging node is drained or serves a validated old version; it must never silently serve an empty zone.
5. TTL, negative caching, and rollback
TTL controls recursive caching, and NXDOMAIN responses are negatively cached. Make the new target ready before changing TTLs; deleting an authoritative record does not clear external caches immediately. Roll back to a validated old version with a higher serial rather than reusing an old serial.
6. Security and tenant isolation
Use zone-scoped RBAC, approvals, and dual control. Authenticate internal update APIs and prevent replay. Keep DNSSEC keys in KMS or HSM and block a release when signing fails. Audit actor, request, version, serial, and result without storing tenant secrets.
7. Observability and drills
Monitor publish latency, serial skew, IXFR/AXFR ratio, signing failures, SERVFAIL, NXDOMAIN changes, query success, and regional distribution. Drill primary failure, partition, missing delta history, bad zones, key rotation, and rollback while proving the old validated version remains available.
High-quality sample answer
I would keep tenant-isolated immutable zone versions in the control plane and let authoritative nodes serve one validated version. Validate names, TTLs, CNAME conflicts, delegation, and quotas with idempotent batch requests and optimistic versions. Generate a canonical zone, verify syntax, DNSSEC, and checksum, then atomically switch the pointer.
The primary increments the serial and sends NOTIFY; secondaries prefer IXFR and fall back to AXFR. Nodes report serial, checksum, and signing state. TTL and negative caching determine external visibility, while rollback uses a newer serial. RBAC, approvals, KMS/HSM, audit, SERVFAIL and serial-skew monitoring, and partition and key-rotation drills are release gates.
Common mistakes
- Treating recursive-cache delay as a control-plane failure.
- Editing a shared zone file directly and serving a partial write.
- Assuming NOTIFY means propagation is complete.
- Serving an empty zone after IXFR fails.
- Reusing an old SOA serial during rollback.
- Giving tenants direct access to DNSSEC private keys.
- Omitting NXDOMAIN and SERVFAIL monitoring.
Follow-up questions and responses
Follow-up 1: Why must the SOA serial increase?
Nodes and caches compare serials to determine freshness. Rollback also needs a higher serial so it is not mistaken for an old version.
Follow-up 2: What if IXFR cannot find the delta?
Request AXFR. If transfer fails, keep the validated version and alert; never serve an empty zone.
Follow-up 3: What do you do with a node far behind?
Compare serial and checksum, drain it or keep it on a validated old version, then catch up with AXFR and sample queries.
Follow-up 4: Can you publish unsigned data after DNSSEC signing fails?
If the zone contract requires signatures, block publication and repair the key or chain. Do not silently downgrade.