System Design Interview: Design a Subscription Billing System

Prompt and scope

Design a service that manages recurring subscriptions for a SaaS, media, or membership product. Customers can choose monthly or annual plans, convert from a trial to paid access, change plans mid-cycle, and recover from failed renewals. The system must produce auditable invoices, notify the product about entitlements, and process asynchronous events from a payment provider.

Public interview material presents “design a subscription billing system” as a system-design question about subscription-created and trial-expired events, bottlenecks, customer metrics, and system monitoring. Stripe’s documentation puts subscriptions, invoices, PaymentIntents, trials, prorations, revenue recovery, and webhooks in the same lifecycle. This article does not claim that a particular company always asks this question.

Assume 1 million active subscriptions, one billing-related event per subscription per day on average, renewal peaks up to 10x normal traffic, amounts stored in the smallest currency unit, and a provider that can time out, duplicate events, or deliver events after the local state has changed. The system must avoid duplicate charges, keep invoices traceable, and converge entitlement state after retries.

What the interviewer evaluates

A strong answer separates plans, subscriptions, invoices, payment attempts, and entitlements. Writing paid=true onto a customer row cannot express proration, grace periods, refunds, or multiple attempts for one invoice.

The next signal is an explicit idempotency boundary. Invoice creation, payment calls, webhook handling, and entitlement delivery can all retry. A queue alone does not prevent a second charge or a second authorization.

The interviewer also tests time and accounting semantics: billing anchors, trial end, time zones, leap months, immediate upgrades, end-of-period cancellation, and immutable amounts. Historical invoices must not change when a plan price changes.

Finally, the candidate should explain peaks, failed payments, out-of-order events, manual refunds, reconciliation gaps, and repair. The useful outcome is knowing what should have been charged, what was charged, and which entitlements are valid after recovery.

Clarifying questions before answering

• What is the billing anchor? Calendar month or a rolling date? This changes period-end calculation and handling of February or day 31.
• When does an upgrade charge? Immediately, next period, or by customer choice? This changes proration and pending-payment states.
• How does a downgrade work? Immediate refund, next-period effect, or account credit? This changes invoice lines and refund handling.
• When does a failed payment remove access? A read-only grace period and immediate suspension have different entitlement rules.
• Are tax, discounts, or usage charges in scope? If so, pricing inputs must be versioned before invoice finalization.
• What is the entitlement source of truth? The product can consume provider events directly, or the billing system can publish entitlement.changed events. The choice changes replay and consistency.
• Are multiple currencies, refunds, or manual adjustments required? If yes, use integer minor units and never overwrite historical invoices.

A 30-second answer

“I would separate plans, subscriptions, invoices, payment attempts, and entitlements. A subscription state machine decides the current period and next action; a billing engine creates an immutable invoice snapshot at each boundary and uses invoice_id as the payment idempotency key. Confirm payment through signed webhooks plus provider queries, deduplicate by event ID, and allow events to arrive out of order. Represent upgrades and downgrades as explicit positive and negative invoice lines. Failed payments enter a bounded, jittered retry state machine, and entitlements follow confirmed payment state plus a documented grace policy. I would prove the design with reconciliation, state invariants, and failure injection.”

Step-by-step solution

Step 1: Define objects and invariants.

Plan(id, version, currency, interval, unit_amount, trial_days)
Subscription(id, customer_id, plan_version, status, period_start, period_end)
Invoice(id, subscription_id, period_start, period_end, currency, total, status)
InvoiceLine(id, invoice_id, source, description, quantity, unit_amount, amount)
PaymentAttempt(id, invoice_id, attempt_no, provider_key, status, provider_id)
Entitlement(id, customer_id, feature, valid_until, source_invoice_id)

Plans may change, but an issued invoice stores its price version and currency snapshot. The key invariants are: a subscription period has at most one effective invoice; one provider result advances an invoice once; a late failure cannot overwrite confirmed payment; and replaying entitlement changes converges to one version.

Step 2: Drive work with a state machine.

trialing --trial_end--> active
active --renewal--> invoice_open
invoice_open --payment_succeeded--> paid
invoice_open --payment_failed--> past_due
past_due --retry_succeeded--> paid
past_due --retries_exhausted--> canceled
active --cancel_at_period_end--> canceling
canceling --period_end--> canceled

Commands or events with a version perform transitions; arbitrary web requests should not mutate status directly. Define entitlement policy for active, pastdue, and canceled: for example, pastdue may retain read-only access for three days while canceled removes paid features. Record the reason, source event, actor, and time for every transition.

Step 3: Generate invoices and handle plan changes.

Partition a billing worker by periodend. First create an invoice under a unique constraint such as (subscriptionid, period_start), then calculate its lines. For an upgrade on day 15, record the unused old plan as a negative line and the remaining new plan as a positive line. Use integer minor units and an explicit seconds-or-days rule; never subtract rounded floating-point values.

Freeze amount, tax, discount, usage snapshot, and plan version when the invoice is finalized. The period job can run twice: the unique constraint returns the original invoice, and the worker checks whether a payment attempt already exists. End-of-period cancellation changes cancelatperiod_end; it does not delete invoice history. Immediate cancellation creates a refund or credit adjustment.

Step 4: Separate payment idempotency from network timeouts.

provider_key = "invoice:" + invoice_id + ":attempt:" + attempt_no
POST payment-provider/charges
  Idempotency-Key: provider_key

Write a local PaymentAttempt before calling the provider. After a timeout, do not infer failure; retry with the same key or query the provider. Stripe documents that repeated idempotent requests return the first result and compare parameters to prevent accidental key reuse. The key must therefore represent one business operation, not one network attempt.

On success, a webhook or provider query records providerid, amount, currency, and completion time. A conditional database update allows only open or pastdue to move to paid; a late payment_failed goes to the audit record and cannot roll back confirmed payment. A money or currency mismatch enters reconciliation instead of automatically granting or removing access.

Step 5: Handle webhook duplicates and reordering with an inbox.

WebhookInbox(event_id PRIMARY KEY, received_at, payload_hash, processed_at)
Outbox(id, aggregate_id, event_type, payload, published_at)

Verify the signature, store the raw event and hash in WebhookInbox, and acknowledge a duplicate event_id. The processor uses object version, provider status, and local invoice state to decide whether to advance; arrival time is not an ordering guarantee. Stripe recommends subscription webhooks for asynchronous activity and entitlement changes, so internal processing must be replayable too.

A local transaction can write billing state, an entitlement-change intent, and an Outbox record together. A publisher then sends the record to the entitlement service. Consumers deduplicate by (aggregate_id, version), so provider retries, queue retries, and consumer restarts do not create a second effective authorization.

Step 6: Estimate capacity and isolate peaks.

With 1 million active subscriptions and one billing event per subscription per day, the steady rate is about 11.6 events/second; a 10x renewal peak is about 116 events/second. The harder load is simultaneous invoice lines, payment calls, webhooks, and notifications. Bucket period work by time, cap each batch, and add jitter to avoid a single top-of-hour spike.

Index subscriptions by subscription ID and period end; make payment and notification work asynchronous. Give large tenants, concentrated renewal days, and high-usage accounts separate quotas so one tenant cannot consume all payment concurrency. These numbers are starting assumptions; provider limits and failure tests must calibrate the design.

Step 7: Reconcile, repair, and observe.

Reconcile daily against invoices, provider transactions, and settlement exports. For every invoice check that line items sum to the total, payment amount equals amount due, and entitlement expiry does not exceed confirmed paid time. Create immutable adjustment or refund records for differences; never overwrite an old amount.

Track period-scan lag, invoice latency, unknown payment outcomes, retry count, pastdue age, webhook duplicate rate and processing lag, entitlement propagation delay, reconciliation dollar differences, and per-tenant payment failures. Audit records should link subscriptionid, invoiceid, paymentattempt_id, event ID, and trace ID so a complaint can be traced to the provider response.

Step 8: Prove the boundaries with a failure matrix.

Inject repeated period jobs; a crash after invoice creation; a provider charge followed by a lost response; duplicate and out-of-order webhooks; a failed proration payment; a trial ending without a payment method; concurrent cancellation and renewal; a consumer crash after Outbox publication; and a manual refund racing an automatic retry.

Acceptance checks are: one effective invoice per (subscriptionid, periodstart); no second charge for one provider_key; confirmed payment is not overwritten by a late failure; replayed entitlement events produce the same version; every reconciliation difference has a traceable adjustment; and every automated action can be reconstructed from audit records.

High-quality sample answer

“I would separate plans, subscriptions, invoices, payment attempts, and entitlements, and snapshot the plan price when an invoice is created. A subscription state machine handles trials, renewals, grace periods, and cancellation. A period worker creates one invoice under a unique key; plan changes become positive and negative invoice lines using integer minor units and an explicit proration rule.

Before calling the provider, I write an attempt and derive a stable invoiceid + attemptno idempotency key. A timeout uses the same key or a provider query; it never invents a new key. Webhooks are verified, stored in an inbox, and deduplicated by event ID. Conditional version updates prevent an out-of-order failure from overwriting confirmed payment. An Outbox reliably sends entitlement intent downstream.

I isolate payment concurrency, period scans, webhooks, notifications, and tenant quotas, and add jitter to renewal peaks. Daily reconciliation compares local invoices, provider transactions, and settlement. I would test duplicate period jobs, a successful charge with a lost response, reordered webhooks, failed proration, concurrent cancellation and renewal, and entitlement convergence after replay.”

Common mistakes

• Mistake: store only paid on the subscription → Why it fails: invoices, attempts, grace periods, and refunds disappear → Fix: model invoices, payment attempts, and entitlements separately.
• Mistake: create a new payment ID for every timeout retry → Why it fails: one business charge can execute twice → Fix: keep one idempotency key per operation and query unknown outcomes.
• Mistake: apply events by arrival time → Why it fails: webhooks can be duplicated, reordered, or delayed → Fix: store an inbox and use object versions plus conditional transitions.
• Mistake: recalculate old invoices from today’s plan price → Why it fails: a price change rewrites history → Fix: freeze price, tax, discount, usage, and plan version on the invoice.
• Mistake: cancel immediately after one failure → Why it fails: a transient provider outage becomes an entitlement outage → Fix: use a past_due grace policy and bounded dunning state machine.
• Mistake: delete period records on cancellation → Why it fails: audit, refunds, and reconciliation lose evidence → Fix: append cancellation and adjustment records while retaining history.
• Mistake: treat a database transaction as a payment transaction → Why it fails: the provider is outside the local transaction → Fix: close the loop with idempotent calls, webhooks, queries, and reconciliation.
• Mistake: scan every subscription at one exact time → Why it fails: locks, payment calls, and notifications spike together → Fix: bucket, jitter, queue, and quota the work.

Follow-ups and responses

Follow-up 1: The upgrade payment fails. What happens to access?

Keep the old entitlement, create a pendingupdate and a new invoice, and switch plans only after confirmed payment. If the product allows upgrade-before-collection, mark the new entitlement as grace-limited with an explicit expiry. In both policies, invoice and entitlement versions remain traceable; changing only planid is insufficient.

Follow-up 2: No provider webhook arrives for three days. How do you detect it?

Run a reconciliation job from local invoice state and provider queries. Alert on old open, past_due, or expired invoices without a terminal event; if the query is still unknown, pause automatic cancellation and route it to manual review. A webhook is a low-latency notification, not the only source of truth.

Follow-up 3: Cancellation and renewal arrive concurrently. Which wins?

Assign monotonic versions to subscription commands and serialize them with a conditional update or ordered partition. cancelatperiod_end can coexist with the current renewal; immediate cancellation checks for an open invoice or payment attempt before issuing a refund or void. The audit events must explain the final billing and entitlement result.

Follow-up 4: How do you add usage billing without double-counting?

Deduplicate immutable usage event IDs and create a usage snapshot per billing period. A duplicate increases receipt metrics but not billable quantity. Freeze the snapshot at settlement; route late events to the next period or a manual adjustment. Store source and aggregation versions in invoice lines so recalculation can be compared.