Prompt and scope
Your B2B SaaS provides reports, data exports, and APIs. Customers ask how current the data is, while source delays, backfills, customer filters, regional queues, and maintenance can change the result. Decide whether to publish a data-freshness SLA and define its scope, metric, exclusions, communication, credits, and post-launch validation.
Public cloud SLAs generally define measurement windows, service boundaries, exclusions, and remedies. Google Cloud's BigQuery SLA distinguishes data-delivery timing from factors outside the service boundary. Snowflake's data-quality monitoring example treats freshness as a measurable expectation rather than a vague “fast” promise. This question tests whether you connect customer value, an enforceable commitment, and the measurement system required to operate it.
What the interviewer is testing
- Separating freshness, completeness, correctness, and availability.
- Establishing whether customers make high-value decisions from fresh data.
- Defining a computable metric such as a percentile and coverage rate.
- Handling source delay, backfill, customer configuration, maintenance, and regions.
- Weighing trust gains against credit cost and misunderstanding risk.
- Designing tiers, a pilot, monitoring, appeals, and rollback.
Public data-product-manager hiring guidance also names data contracts and freshness SLAs as interview probes for product judgment, so a strong answer must connect metric definitions to roadmap trade-offs.
Clarifying questions
- Which datasets and customer workflows are in scope? Start with one measurable domain.
- Does “updated” mean received, processed, queryable, or exportable? Choose the customer-visible boundary.
- Is this a public SLO or a contractual SLA? Start with an SLO; a contract needs legal and credit budgets.
- Who controls source delay? Separate platform control, customer configuration, and third-party sources.
- Do customers care about average or tail latency? Use percentiles and breach duration.
Thirty-second answer
I would first confirm whether customers make operational, compliance, or automated decisions before a freshness deadline. If value is real and measurable, pilot a public SLO for one data domain and tier: define source-confirmation to customer-queryable time, percentiles, coverage, and exclusions. Build lineage, delay categories, and backfill metrics internally, then run a shadow report and a small customer trial. Upgrade to a contractual SLA only after reliability and explainability pass; otherwise publish status and recovery estimates without promising upstream outcomes we cannot control.
Step-by-step solution
Step 1: Establish customer value and decision risk
Ask what customers do with the data: operations, finance close, risk, inventory, or automation. A stale dashboard can cause missed windows, duplicate work, or compliance risk, and tolerances differ by workflow. If customers only download historical reports occasionally, strict freshness may be less valuable than visibility.
Rewrite “as fast as possible” as “available before a decision deadline.” That determines whether the promise is queryability, exportability, or only ingestion progress.
Step 2: Define computable freshness
Choose the start and end among event time, ingestion time, processing completion, and customer query visibility. Store timestamps for each record or batch and state clock, timezone, late-event, and replay semantics.
An interpretable metric might say: “At least 99 percent of eligible batches become queryable within 30 minutes of source confirmation during the service window.” Define eligible batches, window, and exclusions; “real time” is not a metric.
Step 3: Separate SLA, SLO, and status promises
A public SLO is a transparency tool and does not automatically create credits. A contractual SLA needs measurement, breach, credit, and uncontrollable-factor rules. Pilot an SLO first, observe customer understanding and operability, then consider a contract.
If customers only need current delay, a status page, last-success timestamp, and recovery estimate may be more useful than a legal promise. Match the commitment tier to the customer plan, data domain, and processing mode.
Step 4: Design tiers and exclusions
High-value near-real-time data, bulk exports, and historical backfills should not share one target. Tier by dataset, plan, region, or processing mode only if each tier has independent measurement and support. Exclusions may include customer-paused jobs, missing sources, announced maintenance, legal holds, or customer filters.
An exclusion is not a hidden failure. Give each one an identifiable code, customer-visible explanation, and recovery action, or customers will see arbitrary non-performance.
Step 5: Evaluate economics and credits
Estimate queue, storage, retry, cross-region, and support costs required to hit the target, then compare them with renewal risk, expansion value, and decision impact. Show the marginal cost of reducing tail latency so rare peaks do not force permanent overprovisioning.
If credits are offered, use a simple rule based on affected domain or bill share instead of case-by-case negotiation. Credits do not replace root-cause repair and need legal review and evidence retention.
Step 6: Build measurement and customer visibility
Record freshness, completeness, error, and backfill signals by dataset, tenant, region, source, and processing stage. A quality monitor can set a freshness expectation, but the product UI must distinguish queryability from completeness and correctness.
Show last successful update, current delay range, affected scope, recovery estimate, and data gaps. A green status alone is unsafe; when data is delayed or backfilling, customers should know whether it is suitable for automation.
Step 7: Pilot, validate, and stop
Select a stable source and a workflow with clear customer value. Compute the target in shadow mode without promising it, then ask customers of different sizes to interpret the definition and examples. Stop if misunderstanding drives wrong decisions, support volume spikes, the metric cannot be reproduced, or credits exceed budget.
Validation includes fewer manual refreshes, completion before customer deadlines, and correct fallback behavior, not just platform attainment.
Step 8: Govern and review after launch
Version the metric definition, domains, exclusions, credits, and effective date. Rebaseline when sources, architecture, or customer behavior changes. Review tail latency, customer outcomes, and cost monthly.
If the target cannot be reliably delivered, fall back to progress and status visibility instead of preserving a marketing number. Product, engineering, support, sales, and legal should own changes together so sales promises do not exceed the documented scope.
Model answer
I would first verify that customers make important decisions before a freshness deadline, then choose one measurable data domain. Define end-to-end time from source confirmation to query visibility using percentile, coverage, and service-window language; monitor completeness, correctness, and backfills separately. Pilot a public SLO rather than immediately promising contractual credits, tier by dataset or plan, and make source delay, customer pauses, and maintenance identifiable exclusions.
Internally I would build tenant- and stage-level delay and gap metrics and show customers last update, affected scope, and recovery estimate. After shadow measurement and customer comprehension testing, expose it to a small cohort. If the metric is irreproducible, support cost is excessive, or customers make wrong decisions, roll back to transparent status without retaining a false promise. Only then evaluate a contract SLA and credits.
Common mistakes
- Calling “real time” a promise without defining start and end timestamps.
- Using average latency instead of percentiles and breach duration.
- Combining freshness, completeness, and correctness into one green signal.
- Confusing a public SLO with a contractual SLA and credits.
- Hiding source, customer, and maintenance exclusions in vague language.
- Measuring platform attainment without checking customer decisions.
- Publishing a number without versions, audit evidence, and rollback.
- Letting sales promise more than engineering, support, and legal can deliver.
Follow-up questions
What if a customer asks for every dataset within five minutes?
Split the request by domain and decision workflow, then show feasibility and cost for each processing mode. Offer a measurable tiered pilot rather than an unconditional promise; uncontrollable sources need status and recovery estimates.
Why not report average freshness?
An average hides peak and tail delay, while customers may act exactly in the tail. Use percentiles, attainment, and sustained-breach duration, segmented by tenant and dataset.
If data is timely but incomplete, does the SLA pass?
Keep freshness and completeness separate. If the commitment only covers queryability, show completeness gaps clearly. High-risk automation may require both conditions before action.
Should upstream-provider delay always be excluded?
Define the control boundary and evidence first. Exclusions need identifiable causes and recovery actions. If customers cannot distinguish them, include some delay in the product target or provide more transparent status instead of a blanket disclaimer.
When do you publish an SLO versus sign an SLA?
Sign an SLA only when definitions are stable, measurement is auditable, and support and credit costs are budgeted for a real customer need. Before that, use an SLO, status page, and event notifications to validate value and operations.
What if attainment is high but complaints remain high?
Check whether the boundaries match customer perception, whether exclusions are too broad, whether completeness or export delay is omitted, and whether customers misunderstand tiers. Add task completion and wrong-decision cost to platform metrics.
How do you stop sales promises from exceeding the scope?
Put the versioned definition, domains, exclusions, and credit rules in a citable product artifact. Require customer-specific terms to pass product, engineering, and legal approval and to be recorded in the system.