General interview: How do you diagnose HTTP 431 Request Header Fields Too Large?

1. Question and context

After login, a page reaches an API through a CDN, load balancer, and service mesh. Some users receive 431 Request Header Fields Too Large; an incognito window and curl work. Explain the status semantics, hop-by-hop diagnosis, remediation, and release validation. Assume HTTP/1.1 or HTTP/2 may be used and logs cannot contain full cookies or authorization values.

2. What the interviewer evaluates

• Distinguishing an oversized aggregate header block from one oversized field and knowing that 431 is a client-error response before request processing.
• Measuring through the CDN, gateway, proxy, and application instead of changing only the application server.
• Recognizing cookie growth, duplicate Set-Cookie values, long tokens, and forwarding headers as likely causes.
• Reducing client state, rotating credentials, and adding size observability without weakening security.

3. Questions to clarify first

1. Which hop generated 431, and can response headers, server identity, or edge logs identify it?
2. What are the failed request's total header bytes, largest field, and protocol?
3. Does the browser send old cookies, cross-subdomain cookies, or growing session data?
4. Does each layer limit aggregate headers, a field, the request line, or buffers, and are there HTTP/2 decoding limits?

4. A 30-second answer

431 means the server refused the request because the aggregate request headers or a field exceeded an allowed limit; RFC 6585 does not define one universal byte limit. Measure redacted totals and largest fields from the client through edge, gateway, proxy, and application, then find the first hop returning 431. When only browsers fail, inspect cookies, duplicate Set-Cookie, and authorization before raising limits. Remove client state, shorten or replace tokens with server-side session references, expire old cookies, and align budgets across hops. Raise a limit only after capacity and denial-of-service analysis.

5. Step-by-step answer

Step 1: Confirm the status and emitting hop

Record the traversed nodes, protocol, response headers, and correlation ID. Replay a minimal request directly to origin, around the CDN, through the gateway, and with redacted browser headers; compare where 431 first appears. Some proxies use 400 or a vendor code such as 494 for a similar limit, so combine status with node logs and configuration.

Step 2: Measure headers in bytes

Compute encoded byte length for every field and record total request headers, request line, largest field, and field count. Character count is not byte count, and a compressed HTTP/2 header block is not the decoded field total. Keep only field names, lengths, hash prefixes, and request IDs in logs; never record cookie values, bearer tokens, or a full Referer.

Step 3: Find browser-only sources

Cookies are a primary suspect: same-name cookies on several paths or subdomains, user data stored in cookies, and a new value appended on every response make later requests grow. Inspect Set-Cookie Domain, Path, expiry, and deletion behavior; verify the old name actually expires instead of being overwritten only on one path. Keep authorization tokens short and put mutable data in a controlled server-side session or secure store.

Step 4: Account for proxy hops and HTTP/2

Each hop may have different aggregate, per-field, and buffer limits, while forwarding adds X-Forwarded-*, tracing, and authentication fields. Publish the budget as a configuration contract and use the smallest limit as the client design ceiling. HTTP/2 uses HPACK for transport compression, but endpoints still decode fields and enforce limits; compression ratio does not prove business headers are safe. Alert on gateway rejections when the application receives nothing.

Step 5: Choose remediation and defenses

Delete duplicate cookies, reduce session contents, move state server-side, and send only an unguessable short reference. Set token length, rotation, and revocation policies; reject unknown or unusually large custom fields. Raise limits only after checking parsing memory, concurrency, and denial-of-service cost, with sensible per-tenant or per-route budgets.

6. Model answer

I would identify the first hop returning 431 and measure the failed request after redaction: total header bytes, largest field, request line, field count, and HTTP version. Browser-only failure points first to duplicate or cross-subdomain cookies, growing session data, or authorization length; changing the origin limit alone is unsafe because a CDN, load balancer, or mesh may reject earlier. I would expire old cookies, minimize client state, use a server-side session reference, and align hop budgets. HTTP/2 transport compression does not remove decoded limits. Production telemetry would aggregate size percentiles by hop and field name while storing lengths and hash prefixes, never credentials.

7. Common mistakes

• Only clearing browser cookies: It may be temporary; trace the setter, Domain, Path, and expiry logic to remove the duplicate Set-Cookie root cause.
• Assuming 431 always means 8 KB: RFC 6585 sets no universal limit; inspect each layer and measure bytes.
• Raising only the origin limit: An edge or gateway can still reject first, and parsing cost grows; align budgets and assess capacity.
• Logging complete request headers: This exposes cookies and tokens; retain names, lengths, hash prefixes, and correlation IDs.
• Assuming HTTP/2 compression removes limits: Endpoints decode fields and enforce limits; test transport compression separately from decoded size.

8. Follow-up questions and responses

Follow-up 1: Why does curl work while the browser fails?

Browsers automatically send every cookie, authentication, and tracking header matching the domain; curl is usually smaller. Export a browser request, remove fields by binary search, and measure each hop to separate client content from proxy limits.

Follow-up 2: Can a JWT be stored in a cookie?

It can, but every request pays the token's byte cost and several cookies can exceed a limit. Keep a short reference or minimal claims client-side, store mutable data and revocation server-side, and enforce length and rotation budgets.

Follow-up 3: When is raising the limit acceptable?

Only when the requirement is real, every hop can absorb parsing memory and concurrency, and capacity, timeout, and denial-of-service tests pass. Keep size telemetry, rate limits, and a rollback configuration; a higher ceiling is not the sole fix.