General Interview: How Would You Govern Kubernetes Pod-Level Resizing?

Prompt and context

The platform team is enabling Kubernetes Pod-level CPU and memory resize. Product teams want automatic tuning, finance worries about cost, and SRE worries that memory changes can restart containers. Create cross-team admission, audit, incident-response, and rollback policy.

What the interviewer tests

• Whether technical capability becomes clear ownership, permission, and budget boundaries.
• Whether risk tiers determine which workloads may resize automatically.
• Whether status conditions, restart policy, and SLOs become approval evidence.
• Whether audits and drills prove the policy works under an incident.

Questions to clarify

1. Which namespaces, environments, and workloads may resize?
2. Who approves limits, owns cost, and can freeze changes urgently?
3. How do you detect container resizePolicy, stateful connections, and memory-restart risk?
4. Does the organization already have quotas, change windows, audit, and incident command?

30-second answer

I would tier by environment, business criticality, and restart risk: development can tune automatically, while critical production services require approval. Policy fixes CPU and memory bounds, step, cooldown, namespace quota, and SLO guardrails, and requires the /resize subresource with owner, reason, and observedGeneration. Admission rejects unexplained or out-of-bounds requests; events distinguish Pending, InProgress, Infeasible, and Deferred. Audits connect cost to restarts, and regular drills practice freeze, rollback, and ownership handoff.

Step-by-step deep dive

Define risk tiers

Tier by environment, SLO, data state, connection interruptibility, and container resizePolicy. Critical stateful, payment, and control-plane workloads default to approved memory changes; low-risk stateless workloads may tune within limits.

Establish admission rules

Require namespace allowlists, resource bounds, step, cooldown, node capacity, quota, change window, and labels. Every request states metric source, target, and expected cost.

Bind policy to Kubernetes status

Require the controller to read desired/actual resources, observedGeneration, and resize conditions. Only completed InProgress with matching actual values is success; Pending, Infeasible, and Deferred stay visible with reasons.

Handle restart and rollback

Memory changes can restart containers, so services declare connection draining, state recovery, and maximum restart count. Exceeding a gate freezes automation and restores the last stable budget; Pod phase Running is not proof of business continuity.

Make cost and audit first-class

Record before/after CPU and memory, duration, cost estimate, owner, approver, and result. Finance views budget variance by namespace, team, and workload; SRE correlates SLO, OOM, and restart events.

Drill incidents and evolve governance

Practice node capacity shortage, controller outage, bad budgets, and broad rollback. Update policy, runbooks, and contacts afterward; every exception has an expiry so a temporary allowlist cannot become permanent.

Model answer

I would treat Pod resize as governed change, not an open switch. Tier by environment, SLO, state, and resizePolicy; critical stateful services require approval. Admission fixes namespace, bounds, step, cooldown, quota, capacity, and change windows, and requires /resize with owner, reason, and observedGeneration. The controller reports Pending, Infeasible, and Deferred and confirms success only from actual state. Memory restart gates require draining and recovery; breaches freeze and roll back. Audits connect cost, SLO, OOM, and restartCount, and drills rehearse freeze, rollback, and ownership handoff.

Common mistakes

• Writing resource limits without an approver, freeze authority, or owner.
• Allowing automatic memory resize in every production namespace.
• Ignoring /resize conditions and container restart policy.
• Treating a Running Pod phase as proof of uninterrupted business service.
• Omitting cost audit, exception expiry, and rollback drills.
• Finding owners and runbooks only after an incident starts.

Follow-up questions

Which services should default to no automatic memory resize?

Critical services that cannot drain quickly, have costly state recovery, or risk consistency on restart should require approval and a completed drill.

How do you stop teams bypassing policy by editing Pods?

Use admission, RBAC, field management, and audit to reject unauthorized writes; emergency permission is time-limited, traceable, and expires automatically.

Who decides when cost conflicts with SLO?

Policy sets priority and budget thresholds in advance; a named product and SRE owner decide above the threshold rather than leaving the controller to choose silently.

How do you know the policy works?

Compare rejected out-of-bounds requests, SLO regressions, OOM, restarts, budget variance, rollback time, and drill completion, then revise with each team.