Prompt and context
This question tests whether you can turn “how long should we retain it?” into an executable, explainable, and recoverable data-lifecycle system. Versions, snapshots, replicas, backups, and downstream exports do not share one deletion semantic; Apache Iceberg snapshot expiration also requires checking that files are not still referenced by retained snapshots. Cover policy sources, precedence, executors, dependency discovery, deletion evidence, and failure recovery. Retention periods depend on specific law and contracts, so the system should support legal review rather than replace legal judgment.
What the interviewer is evaluating
- Whether you distinguish business retention, legal hold, deletion request, backup window, and technical cleanup.
- Whether you design versioned, scoped, approved, and exception-aware policies instead of hard-coding one TTL.
- Whether you handle hidden references in snapshots, replicas, caches, exports, downstream tables, and orphan files.
- Whether you provide dry runs, observable execution, deletion evidence, retries, rollback boundaries, and audit trails.
Clarifying questions to ask first
Confirm data classes, subject or tenant scope, business purpose, region, contract, and legal source. Does a policy apply to a table, column, record, object, or event? Who approves and releases a legal hold when it conflicts with a deletion request? What copies and recovery paths exist for lakehouse snapshots, topics, object storage, backups, and search indexes? Must the system prove physical deletion, or prove removal from online query surfaces while backups expire naturally? On failure, may execution pause, retry, or roll back metadata?
A 30-second answer framework
I would model policies with source, version, scope, precedence, and validity, separating ordinary retention, legal holds, deletion requests, and backup exceptions. A compiler maps catalog entities to snapshot, topic, object, index, and backup executors and first produces a dry-run showing impact and conflicts. Execution uses a dependency graph and reference checks so files needed by snapshots or recovery are not deleted, commits in batches, and records every step. The resulting evidence covers scope, times, failures, and the remaining backup window. Any uncertainty pauses the plan for review instead of silently skipping it.
Step-by-step deep dive
1. Model policy sources and precedence
A policy should include subject scope, data labels, processing purpose, retention duration, start event, region, source, version, approver, and exceptions. Represent deletion requests, legal holds, contract periods, and natural backup expiry as distinct constraints with a documented conflict and escalation path. Persist the input snapshot used for every calculation so the system can explain why a record remains.
2. Build an entity and reference catalog
The catalog links tables, partitions, snapshots, objects, topics, indexes, exports, and backups to owners and lineage. For a lakehouse, identify files referenced by retained snapshots, branches, or tags; for asynchronous pipelines, record whether deletion events reached downstream consumers. When a reference cannot be established, keep the data and create a work item instead of guessing.
3. Compile an executable plan
The compiler creates an expiry time, action, precondition checks, approval requirement, and rollback boundary for each entity. Actions may block new writes, hide data from queries, expire snapshots, remove orphan files, emit downstream deletion events, or wait for a backup window. Include a plan version and idempotency key so retries do not duplicate side effects; the dry run reports conflicts, estimated object count, and risk.
4. Design batched execution and guardrails
Partition execution by tenant, region, table, or partition and limit concurrency and deletion per batch. Before physical deletion, recheck active snapshots, recovery jobs, legal holds, and recent writes; high-risk actions need dual approval or a short freeze window. Support pause, resume, and exponential backoff so storage or catalog services are not overloaded. Record plan version, input digest, result, and error class for every batch.
5. Generate verifiable deletion evidence
Evidence is more than a “job succeeded” log. It lists policy version, covered entities, start and end time, processed and failed counts, reference checks, downstream acknowledgements, backup expiry, and unverifiable boundaries. Record snapshot expiration, file deletion, and index cleanup separately. If a backup cannot be physically deleted immediately, state that the online surface is removed, when the backup expires, and who approved that boundary.
6. Handle failure, recovery, and continuous audit
Commit metadata changes and physical deletion in stages, retaining necessary snapshots or a soft-delete window for recovery. If a policy is wrong, stop later batches, mark affected entities, and execute a compensating plan. Physically deleted data cannot be pretended reversible; recover it from an allowed replica or upstream rebuild. Monitor expiry backlog, deletion blocks, downstream acknowledgement delay, backup residue, and policy conflicts, and periodically review policy sources and permissions.
Model high-quality answer
I would model data labels, purpose, subject scope, start event, retention, region, source, version, approver, and exceptions, then define conflicts between ordinary retention, legal hold, deletion request, and backup window. The catalog would link tables, partitions, lakehouse snapshots, topics, indexes, exports, and backups; unknown references stay protected. A compiler creates versioned, idempotent plans and runs a dry run before batching by tenant and partition. Before deletion it rechecks active snapshots, recovery jobs, legal holds, and recent writes, and records every result. Evidence lists scope, policy version, deletion and downstream acknowledgement, failures, and backup expiry, distinguishing online removal from physical deletion. An anomaly pauses later batches for compensation or approval. Ongoing audits track backlog, blocked deletions, residue, and permissions.
Common mistakes
- Treating every storage system as one key-value store with a TTL.
- Keeping only a final expiry date and losing purpose, source, version, approval, and conflict context.
- Ignoring snapshots, branches, backups, indexes, and downstream copies before deletion.
- Calling a successful job a deletion proof without scope or failure boundaries.
- Omitting dry runs, pause, batching, idempotency, and human approval guardrails.
- Claiming physical deletion is immediately reversible or ignoring natural backup expiry.
- Presenting system configuration as legal advice and losing the legal or contract source.
Follow-up questions and responses
What if a legal hold conflicts with a deletion request?
Model them as separate constraints, apply the precedence confirmed by legal policy, and record approver, reason, start, and release time. The system may freeze deletion or restrict online access, but it must not invent a legal exception.
How do you prove snapshot expiry will not delete referenced files?
Read active snapshots, branches, tags, and recovery-job references before execution and process only snapshots and files outside the retained set. Scan again after submission and record the check. Unknown references remain protected and become work items.
What if backups cannot be deleted immediately?
Represent online removal, backup unavailability, and physical-media expiry as separate states. Record backup retention, encryption-destruction or natural-expiry plans, and access restrictions. Evidence must state the boundary and owner for data not yet physically deleted.
What happens after an accidental deletion?
Stop the policy and later batches, freeze the plan version and audit evidence, and assess affected entities. Recover from an allowed snapshot, backup, or upstream rebuild, then apply the corrected policy and record remediation, approval, and notification.