Prompt and context
This question fits backend, platform, and distributed-systems interviews. The interviewer wants the boundary: SQS treats the same deduplication ID as a duplicate within the deduplication interval and keeps tracking that ID even after a message is received and deleted; that does not make an external database, payment, or email side effect inherently exactly once.
What the interviewer is testing
A strong answer separates producer deduplication, FIFO ordering, consumer visibility timeout, deletion acknowledgement, and business idempotency. AWS defines the deduplication ID as a token that prevents duplicate delivery; when content-based deduplication is enabled and no ID is supplied, SQS can derive one by hashing the message body. The answer should cover the interval, retries, and observability instead of saying “FIFO never duplicates.”
Clarifying questions to ask first
Semantic boundary
Ask whether the requirement is queue-level duplicate suppression, at-least-once processing, or one-time business effect. Keep “one message” separate from “one charge.”
Deduplication input
Confirm whether the producer creates a stable MessageDeduplicationId, whether content-based deduplication is enabled, how message groups are chosen, and whether retries fit inside the five-minute interval.
Failure points
Draw the receive, process, side-effect write, and DeleteMessage timeline. Ask what happens on consumer crash, visibility timeout, network retry, and downstream timeout.
30-second answer framework
“The FIFO deduplication ID suppresses the same message from being accepted again during the five-minute interval and keeps tracking that ID; it addresses queue-level duplicate sends and ordering, not exactly-once external side effects. I would use a stable business key for an idempotency record, put the side-effect state and an outbox in one transaction or another retry-safe boundary, and delete the message only after success. A crash may cause redelivery, but the repeated attempt reads the completed record.”
Step-by-step deep answer
Step 1: State the testable guarantee
Say that the same deduplication ID is treated as a duplicate during the five-minute deduplication interval; SQS continues tracking the ID after receive and delete. Do not describe the interval as permanent deduplication.
Step 2: Separate producer and consumer controls
The producer uses a stable ID for one business command; content hashing works only when the body fully represents that command. The consumer needs a durable unique constraint on an order, payment intent, or command ID because a retry after the interval may still be the same business action.
Step 3: Cover the crash window
If the consumer writes the database and then crashes, the message may reappear after its visibility timeout. Put the side-effect state and idempotency key in one transaction, or use an outbox for replayable publication; delete only after successful commit.
Step 4: Handle ordering and poison messages
Use the same MessageGroupId when order matters and set a visibility timeout that matches the work. Move repeatedly failing messages to a dead-letter queue with the original ID, receive count, and failure reason so retries do not block the group forever.
Step 5: Verify and observe
Test producer retries, a consumer crash after the write, DeleteMessage timeout, and replay outside the interval. Monitor duplicate business keys, ApproximateReceiveCount, DLQ depth, visibility timeouts, and end-to-end latency; alerts should distinguish queue duplicates from business duplicates.
High-quality sample answer
I treat SQS’s five-minute deduplication as a transport safeguard, not a one-time payment guarantee. The producer creates a stable command ID for each payment intent and reuses it on retry. The consumer first enforces a unique command ID in an inbox table, then writes the order state and an outbox event in the same transaction, and deletes the message after commit. If the process crashes after the write, redelivery finds the completed record and does not charge again. I would monitor duplicate commands, receive counts, and the DLQ, and inject failures during in-window, out-of-window, and DeleteMessage retries.
Common mistakes
- Mistake: Saying FIFO means permanent exactly-once. → Why it fails: It ignores the five-minute interval and consumer crashes. → Fix: State the ID’s time boundary and add business idempotency.
- Mistake: Generating a new deduplication ID for every retry. → Why it fails: One command becomes multiple messages. → Fix: Reuse a stable command ID.
- Mistake: Deleting before processing completes. → Why it fails: A downstream failure becomes message loss. → Fix: Delete after a successful commit and allow redelivery on timeout.
- Mistake: Relying only on a body hash. → Why it fails: Timestamps or irrelevant fields can bypass deduplication. → Fix: Define an explicit stable ID for the business command.
Follow-ups and responses
Follow-up 1: What if the same command arrives after five minutes?
Use the durable business idempotency record to decide whether it completed; queue deduplication reduces short-window repeats but cannot replace long-lived business state.
Follow-up 2: Why not delete before processing?
Deleting first turns a downstream failure into message loss. Unless loss is explicitly acceptable and another source is reliable, process with retries before deletion.
Follow-up 3: What does an outbox solve?
It commits business state and a pending event in one database transaction so publication can retry safely; downstream consumers still need event-ID idempotency.
Follow-up 4: How would you prove no duplicate charge?
Inject failures after the write, during deletion timeout, and after an out-of-window replay. Verify the database uniqueness constraint, the payment provider’s idempotency key, and audit logs rather than queue metrics alone.