Prompt and scope
You own a global SaaS where a database, queue, or deployment failure in one cluster can affect every tenant. Design a cell-based architecture: each cell is an independently operable full copy of the system serving a fixed tenant set, and an entry layer routes requests to the target cell by a stable mapping.
Cover tenant partitioning, the routing directory, compute and data boundaries inside a cell, cross-cell reporting, releases, migration, capacity, and disaster recovery. AWS describes cells as independent replicas and stores user-to-cell mapping in highly available storage; its bulkhead guidance describes routing by a partition key behind one endpoint.
What the interviewer evaluates
- Whether you define failure scope, tenant consistency, and acceptable degradation first.
- Whether a cell is a real failure domain instead of duplicated stateless services.
- Whether you identify routing, control-plane, shared-dependency, and cross-cell query risks.
- Whether new cells, rebalancing, and tenant migration have reversible steps.
- Whether cell-level capacity, error, and dependency metrics prove the blast-radius claim.
System-design interview references treat cell-based architecture as an isolation pattern for very large systems. A strong answer explains when the reliability benefit justifies duplicate infrastructure and operations; a cell is not a synonym for a microservice.
Clarifying questions before answering
- Is the target failure scope one tenant, one cell, one Availability Zone, or a region?
- May tenants share data, global search, or cross-tenant aggregates?
- Which operations require linearizability, and which reports may be delayed or eventually consistent?
- Can migration be briefly read-only? Are there data-residency constraints?
- What are availability targets, tenant count, growth, cell capacity, and release frequency?
30-second answer framework
I would map a stable tenant key to a fixed cell. Each cell owns compute, queues, caches, and primary data storage; the global control plane owns only versions, capacity, and mappings. An entry router reads a highly available directory and removes only the failed cell. Cross-cell reports use asynchronous aggregation so queries do not recreate a shared database. Cell creation, migration, and release use small, observable, reversible steps, with cell-level SLOs proving the failure boundary.
Step-by-step deep answer
Step 1: Define cell boundaries and failure assumptions
Define a cell as a complete copy that can be deployed and recovered independently: API, workers, cache, database, object-storage prefix, and monitoring. Shared identity, billing, or configuration services need an explicit failure budget. If a shared control plane blocks every cell when unavailable, the data plane is not fully isolated.
Step 2: Choose the partition key and routing directory
Use tenant_id as a stable partition key. The directory stores tenant-to-cell mapping, version, migration state, and capacity. The router reads a local cache first and refreshes from a highly available directory; version numbers and leases prevent old routes from writing to two cells during migration. Clients see one hostname while the router owns retry boundaries.
Step 3: Build the in-cell data plane
Each cell has its own primary database and message queue; tenant data is not synchronously written across cells. Replicas, caches, and object storage carry the cell identity, and backups retain that metadata. Global configuration uses read-only snapshots or versioned rollout instead of sending hot traffic to one global database.
Step 4: Handle cross-cell queries
Each cell emits a change stream to build aggregate tables consumed by a global analytics layer. Reports include data time and missing-cell markers rather than claiming real-time completeness. Strongly consistent cross-tenant operations should be narrowed, made asynchronous, or explicitly accept a larger shared failure domain; avoid two-phase commit on the request path.
Step 5: Design failure and degradation paths
Health checks cover in-cell dependencies, route reachability, and business correctness. When a cell fails, the directory marks it draining and stops new requests; read caches or asynchronous work may degrade when the product permits. Do not move tenants to an arbitrary cell until replication, idempotency, and authorization boundaries are proven, or failover can create duplicate writes.
Step 6: Add cells and rebalance capacity
Create an empty cell from the infrastructure template, run synthetic traffic and shadow reads, then admit a small tenant set. Capacity signals include CPU, database connections, queue delay, storage growth, and cost per tenant. Rebalancing freezes a mapping version, copies and verifies data, performs a brief write handoff, and watches old and new cells; on failure, roll back the mapping instead of deleting the old data.
Step 7: Govern releases and versions
Release the control plane, cell template, and business version separately. Canary in one cell, then expand by cell. Keep a compatibility window so a new version cannot write fields an old version cannot read. Make version distribution and errors visible by cell; global averages cannot hide one bad version.
Step 8: Verify blast radius and operating cost
Inject database, queue, routing-directory, and deployment failures into one cell and confirm only the expected tenants are affected. Track cell availability, error budget, cross-cell request share, migration rollback time, shared-control-plane dependency, and spare capacity. If cells are too few to isolate faults or duplicate operations cost more than the reliability gain, choose a simpler partition or bulkhead design.
Model answer
I would map tenant_id to a fixed cell. Each cell independently runs API, queues, caches, and primary storage; the control plane owns only versions, capacity, and mappings. A single-hostname router reads a versioned, highly available directory, and a failed cell drains instead of accepting unverified writes elsewhere. Cross-cell reporting is asynchronous through change streams. Migration uses copy, verification, a brief handoff, and a reversible mapping. I would prove the blast-radius claim with cell SLOs, shared-dependency share, rollback time, and failure drills; if isolation costs exceed its benefit, I would use a simpler bulkhead.
Common mistakes
- Duplicate only stateless services → the shared database remains a single point → draw every cell's data and queue boundary.
- Randomly move tenants during failure → duplicate writes or authorization mismatch → prove replication, idempotency, and route versions first.
- Aggregate across cells on the request path → a new global failure domain → use asynchronous aggregation with data-time markers.
- Use only global health checks → a bad cell is hidden by averages → record cell-level SLOs and version distribution.
- Change the route directly during migration → old and new writes overlap → use mapping versions, copy verification, and rollback.
- Give every dependency its own cell → uncontrolled cost and operations → quantify blast radius and capacity benefit first.
Follow-ups and responses
What if the routing directory fails?
Keep versioned local caches and read-only snapshots, restrict mapping changes, and let existing tenants continue on their original cells. Do not rebalance broadly until the directory recovers.
What if an admin report must be real time across cells?
Clarify the allowed delay and missing-data semantics. Strong consistency may require narrowing the operation or accepting a larger shared failure domain; normal reports should use timestamped asynchronous aggregates.
How do you migrate a tenant when a cell is full?
Copy and verify data, publish a new mapping version, coordinate a brief write handoff, and ramp traffic. Keep the old cell until consistency is proven; roll back the mapping if it is not.
Is it safe to deploy one version to only some cells?
Yes, with compatible schemas, version observability, and an explicit rollout order. Global success rate cannot replace per-cell error budgets.
How do you prove the failure did not spread?
Drill database, queue, routing, and deployment failures inside one cell. Record affected tenants, cross-cell requests, recovery time, and shared dependency calls.
When should you not use cell-based architecture?
When tenant scale and failure cost are low, or strong cross-tenant consistency dominates, duplicate resources and migration complexity may not pay back.
How can cells share identity and billing?
Keep shared services in a low-rate control plane, cache read-only results, and define degradation. Billing writes need idempotency keys and compensation so a shared-service fault does not spread to every data plane.