Question and scenario
A normal replicated state machine usually assumes nodes crash, lose packets, or restart without actively forging different content. This control plane manages certificates, permissions, and routes, so a compromised node might send inconsistent logs to different peers or impersonate another identity. Start from the threat model and explain what Raft, authentication, and Byzantine protocols each solve.
What the interviewer is testing
- Can the candidate distinguish crash faults, network partitions, malicious nodes, and stolen identity keys?
- Do they understand Raft's safety assumptions, quorum intersection, and the extra communication and replica cost of Byzantine protocols?
- Can they include signatures, membership, auditing, key rotation, and recovery in the system boundary?
- Will they avoid treating encryption or a majority vote as a Byzantine-safety proof?
Clarifying questions to ask first
Ask how many nodes an attacker can control, whether identities can be forged, whether peers have trusted keys, and whether consistency or availability is more important. Clarify the value of the control-plane state, human approval, membership-change frequency, and inter-region latency. A controlled cluster with hardware-protected keys may only need crash fault tolerance; an untrusted supply chain or operator changes the answer.
A 30-second answer framework
Write the fault model and protected invariants first. Raft assumes non-malicious failures and fits crash tolerance in a controlled cluster. Authentication proves that a message came from a key, not that the holder is honest. If a few nodes can send conflicting values, choose a Byzantine protocol that is safe under the stated fault bound, accepting more replicas, authenticated broadcast, timeouts, and key-operations cost. If isolation, key protection, and human approval reduce the risk enough, keep Raft and document what it does not cover.
Step-by-step deep dive
- State the fault model. Separate crash, omission, partition, and active-malicious faults. Say whether collusion, identity forgery, message delay, or disk edits are possible; without this model, protocol comparison is empty.
- Define safety invariants. Examples include no honest replica committing two conflicting configurations, no rollback of a revocation, and auditable key publication. Set availability, finality, and recovery objectives separately.
- Check ordinary-consensus assumptions. Raft uses a leader, log matching, and majority commit to resist crashes. It does not stop a node with a valid identity from sending different content to different peers. TLS protects transport, not malicious endpoints.
- Evaluate Byzantine cost. In the classic unauthenticated oral-messages model, tolerating f Byzantine nodes needs a higher replica bound. Authenticated protocols, threshold signatures, and trusted hardware change engineering trade-offs but do not remove the fault-bound and membership assumptions.
- Control the boundary. Even with a Byzantine protocol, restrict membership, rotate and revoke keys, isolate control and data planes, retain evidence, and prepare safe recovery. A protocol constrains participants, not an administrator who edits a database directly.
- Verify from the threat. Inject forked messages, forged signatures, replayed configurations, delayed quorum responses, and node recovery. Check invariants, audit evidence, and recovery paths; fault drills should demonstrate that the selected protocol meets the real limit.
High-quality sample answer
I would not choose Byzantine fault tolerance merely because the cluster spans regions. First define the attacker. If nodes only crash or suffer network faults, Raft's leader, log matching, and majority commit are sufficient. If a node with a valid key can send conflicting configurations to different replicas, authenticated transport and majority voting do not prove honesty.
For a high-value control plane, define f, membership rules, non-conflicting commit, and non-reversible revocation as invariants. Then choose an authenticated Byzantine protocol that remains safe for that f. Compare replica count, signature checks, latency, and key operations with the risk. If isolation, hardware keys, two-person approval, and read-only recovery make the malicious-node risk acceptable, keep Raft and record the uncovered threat. References include Lamport's Byzantine Generals work, the Raft paper, and its formal specifications.
Common mistakes
- Calling a partition or crash a Byzantine fault without saying whether a node lies actively.
- Assuming TLS, signatures, or a majority alone prevents a malicious node with valid credentials.
- Reporting one replica number while ignoring the authentication model, collusion bound, membership, and key protection.
- Discussing protocol messages but ignoring administrators, databases, backups, and recovery bypasses.
- Replacing invariants, attack drills, and risk thresholds with the phrase “more secure.”
Follow-up questions and responses
Can a three-node cluster tolerate one Byzantine node?
Not without naming the protocol and authentication model. The classic unauthenticated oral-messages model has a higher replica bound; authenticated protocols, trusted hardware, and partial synchrony change the conditions. State the model, then f, quorum, and the safety argument.
Why does a majority vote not automatically solve malicious behavior?
A malicious node can send different values to different observers or impersonate logical roles. A majority is meaningful only when message authentication, view changes, evidence propagation, and quorum intersection satisfy the protocol assumptions.
When should you stay with Raft?
When nodes and keys are within a controlled boundary, crashes and network faults dominate, and manual recovery is acceptable, Raft is simpler and easier to verify. Add membership approval, key rotation, auditing, and fault drills, and state that malicious nodes are outside its guarantees.