Prompt and context
A platform can generate build-provenance statements for binaries and container images, while consumers verify the statement against the artifact digest with a CLI. Security wants production deployments to verify every artifact; developers worry that legacy pipelines, external builders, and offline networks cannot adopt immediately. Propose the product decision and launch plan, including what attestations do and do not prove, segmentation, defaults, migration, metrics, and rollback. The core skill is product governance and trade-off reasoning, so this is a product question.
What the interviewer evaluates
First, whether you distinguish “a statement exists” from “the statement is trusted”: verification still needs signatures, artifact digests, workflow identity, and policy context.
Second, whether you segment users by risk and capability instead of applying one mandatory switch to hobby projects, enterprise production, and regulated environments.
Third, whether you design a progressive migration from observe to warn, selective blocking, and default blocking with auditable exceptions.
Fourth, whether metrics cover security, developer experience, coverage, and business outcomes rather than the number of generated statements.
Fifth, whether the threat model covers compromised builders, incorrect policy, repackaged images, and offline verification.
Questions to clarify first
- Are the target users open source projects, ordinary SaaS, or regulated production platforms?
- Do builds come from GitHub Actions, third-party CI, developer machines, or several sources?
- Can production read online statements, or must it support offline and isolated networks?
- Are temporary exceptions allowed, who approves them, and when do they expire?
- Is verification done by the platform, cluster admission, or the customer pipeline?
- Is the goal supply-chain auditability, blocking unauthorized artifacts, or faster incident response?
A 30-second answer
“I define an attestation as a verifiable build-origin signal, not an automatic guarantee that source code is safe. I segment critical production, ordinary production, development, and external builds by risk and migration ability. I start with observe and warn, then selectively block and finally default-block high-risk production, with scoped, expiring exceptions. Metrics include valid verification coverage, unauthorized-artifact blocks, false blocks, deployment latency, exception rate, and incident-response time. Compromised builders, repackaging, and offline verification get separate threat models, while gray rollout keeps a fast, audited rollback.”
Detailed solution
Step 1: Define the user problem and trust boundary
An artifact attestation binds an artifact digest to claimed build provenance, helping a consumer identify the workflow, commit, and builder that produced it. It does not prove the source has no vulnerabilities or repair a compromised builder. Product copy and policies must separate provenance, signature verification, scanning, and deployment authorization.
Step 2: Segment users
Separate critical production services, ordinary production, development previews, and external contributors. Critical production needs strong verification and short exceptions; development prioritizes fast feedback; external builds may need an uploaded bundle or a trusted rebuild. Segmentation determines defaults, support cost, and migration order.
Step 3: Design the verification loop
The producer writes a statement after the build. The consumer verifies it against the artifact digest and checks repository, commit, branch, builder, and issuer against policy. Results must explain actionable reasons such as missing statement, digest mismatch, disallowed source, or expiry rather than returning only a boolean.
Step 4: Plan progressive policy
First measure coverage and failure reasons in read-only mode. Next warn in non-production. Then block selected production services, and finally expand to default blocking. Each stage needs exit criteria for false blocks, verification latency, legacy-pipeline migration, and support volume. An exception records owner, reason, scope, and expiry.
Step 5: Support multiple build origins
GitHub Actions can generate statements directly, but third-party CI, external builders, and offline builds need imported evidence or a trusted issuer flow. The product must document evidence format, digest binding, and verification commands so uploading a JSON file is not mistaken for a trusted statement.
Step 6: Define metrics and guardrails
Core metrics are valid production verification coverage, unauthorized-artifact block rate, false-block rate, deployment latency p95, migration time, and exception expiry rate. Guardrails include build failure rate, rollback time, support tickets, offline-customer blocks, and statement availability during security incidents. Statement count is a usage metric, not a security outcome.
Step 7: Prepare gray rollout and rollback
Enable by organization, repository, or service while retaining policy versions and decision logs. If verification is unavailable, distinguish fail-closed from controlled fail-open: critical paths may block briefly with manual approval, while low-risk environments may log and continue. Rollback disables enforcement without deleting existing evidence or audit records.
A high-quality sample answer
“The product goal is to prevent artifacts whose origin violates policy from reaching production, not to claim that provenance proves code is safe. I segment customers by risk and build capability and provide evidence paths for GitHub Actions, third-party CI, and offline builders. The launch moves through observe, warn, selective block, and default block, expanding only when false blocks, latency, and migration rates meet thresholds. Verification explains missing evidence, digest mismatch, and disallowed origin; exceptions require approval, expiry, and audit. I measure valid coverage, unauthorized blocks, false blocks, and incident-response time, with controlled fallback for verifier outages.”
Common mistakes
- Treating statement count as security → many statements may be invalid or unverified → measure valid verification and blocking outcomes.
- Mandatory enforcement on day one → legacy and offline users are blocked → use segmentation, gray rollout, and migration windows.
- Checking only a signature → any trusted issuer may pass → check workflow, commit, repository, and builder policy.
- Calling provenance a vulnerability scan → users misunderstand the protection → separate origin, vulnerability, and authorization claims.
- Giving no failure reason → developers cannot fix the issue → return structured reasons and remediation links.
- Permanent exceptions → enforcement quietly decays → scope, approve, and expire them.
- Ignoring verifier outages → one outage stops releases broadly → define fail-closed, controlled fail-open, and rollback.
- Supporting one CI only → multi-source customers cannot migrate → provide import, bundles, and trusted-issuer paths.
Follow-up questions
Follow-up 1: Does an attestation prove code was not tampered with?
It binds an artifact digest to claimed build origin so consumers can verify the match. It does not prove the builder, dependencies, or source are absolutely safe; permissions, scanning, and reproducible builds remain necessary.
Follow-up 2: Why not fail closed for everyone immediately?
Customers differ in build capability, evidence sources, and network conditions. Universal blocking can create outages and encourage teams to disable the policy; observation and gray rollout calibrate rules with real failures.
Follow-up 3: How do you measure false blocks?
Record the reason, service, policy version, and later manual approval for each block. Combine confirmed legitimate recovery with total blocks and analyze by customer tier; do not classify every exception as a false positive.
Follow-up 4: How do offline customers verify?
Let them download an evidence bundle and required trust roots, then verify the digest and origin locally. Document trust-root updates, revocation, and expiry behavior for the isolated environment.
Follow-up 5: What if the builder is compromised?
The statement describes the claimed origin; it does not prove that origin was uncompromised. Restrict workflow permissions, isolate builders, rotate issuer identities, and monitor anomalous origins and policy changes.
Follow-up 6: How do you stop exceptions becoming permanent?
Treat each exception as an expiring anomaly with a countdown and owner. Track expiry and repeated reasons by team, then fix migration gaps instead of permanently weakening policy.