Linux Interview: How Do You Make an Atomic File Update Crash-Safe?

Problem and Use Cases

This problem contains three different guarantees. Atomic visibility means a reader opening the target path sees one complete version, not an in-place partial rewrite. Crash durability means an acknowledged version remains reachable after reboot. Writer isolation decides what happens when two updates race. A single rename() addresses only the first guarantee under its filesystem contract.

Assume a local Linux filesystem that correctly implements fsync() and atomic same-filesystem replacement, one serialized writer, and previously acknowledged versions written by the same protocol. The 64 KiB size is an interview assumption. Network filesystems, faulty hardware that lies about cache flushes, multi-file transactions, and hostile directory modification require separate contracts.

Typical uses include configuration snapshots, local agent state, checkpoint manifests, package metadata, and editor saves. If the state spans several files or needs queries and concurrent transactions, an embedded database or journal is usually safer than extending this protocol by hand.

What the Interviewer Is Evaluating

The first signal is whether the candidate distinguishes write() success, rename() atomicity, and durable commit. Linux documents that write() can be partial and a successful return does not prove disk persistence. fsync(file) flushes the file's modified data and associated metadata, but does not necessarily persist its directory entry. That last edge requires fsync(parent_directory).

The second signal is ordering. New bytes must be durable before the namespace points the target name at them. The directory entry must then be durable before success is reported. Reversing or omitting either barrier creates a crash window.

The third signal is honest failure semantics. fsync() can reveal delayed EIO, ENOSPC, or quota failures. If the final directory sync fails, the rename may already be visible while durability is unknown. The API must return an indeterminate failure, not claim success or pretend it can always roll back.

Clarifying Questions Before Answering

• Does “atomic” cover process crashes or host power loss? Process termination alone leaves the kernel page cache alive; the prompt requires host-crash durability and therefore sync barriers.
• Is the target on a local filesystem with documented semantics? NFS, FUSE, overlay filesystems, and unusual storage stacks may provide different error and persistence behavior. Test the actual deployment contract.
• Are there multiple writers? This base design serializes writers. Unique temporary names prevent collisions but do not prevent last-writer-wins updates.
• Must several files change together? A rename commits one pathname replacement. Use a generation directory, journal, or database for a multi-file invariant.
• Can an older open file descriptor keep reading the old version? Yes. Rename changes the directory mapping; an already open descriptor still refers to the old inode. Readers must reopen when they need the latest generation.
• What constitutes valid content? Validate serialization, schema, checksum, and version before publication. Filesystem atomicity cannot make a malformed but complete payload correct.

30-Second Answer Framework

“I separate atomic visibility from durability. I open the parent directory, create a unique temporary file in that same directory with exclusive creation, write in a loop until all bytes are accepted, set required metadata, and fsync the temporary file. Then I close it, atomically rename it over the target, and fsync the parent directory before returning success. The file sync makes the new inode contents durable; rename publishes one complete version; the directory sync makes that name-to-inode change durable. I serialize writers, treat every syscall error—including the final directory-sync failure—as a non-success with an explicit indeterminate state, clean orphan temporary files during recovery, and test power-loss points after every step rather than using process kill as durability proof.”

Step-by-Step Deep Dive

Define the commit point from the caller's perspective: only a successful parent-directory fsync permits an acknowledged success. The implementation outline is:

durableReplace(parentDir, targetName, bytes):
  dirfd = open(parentDir, read-only | directory | close-on-exec)
  tmpName = uniqueSiblingName(targetName)
  tmpfd = openat(dirfd, tmpName, create | exclusive | write-only | close-on-exec, mode)

  writeAll(tmpfd, bytes)          // retry EINTR; advance after partial writes
  setRequiredMetadata(tmpfd)      // mode/ownership if part of the contract
  fsync(tmpfd)                    // check delayed I/O and allocation errors
  close(tmpfd)                    // check the result

  renameat(dirfd, tmpName, dirfd, targetName)
  fsync(dirfd)                    // persist the namespace change
  close(dirfd)
  return committed

The temporary file must be a sibling of the target. Same-directory placement makes the rename stay on one filesystem and avoids an EXDEV copy fallback. Use exclusive creation and a non-predictable process-local suffix; do not follow an attacker-controlled existing symlink. Set permissions before the file sync so required metadata joins the durable file state.

writeAll matters even for a regular file. A positive return smaller than the requested count is a partial write, so advance the buffer and continue. Retry only the appropriate interrupted operation. A successful write() means the data reached the kernel's accepted state, not stable media. close() alone is not the commit barrier, and errors may be reported later by fsync().

Next, sync the temporary file. fsync covers modified file data and inode metadata and waits for the storage device to report completion. fdatasync can reduce unrelated metadata work, but it still must persist metadata needed to retrieve the data, such as file size. For a simple and auditable answer, use fsync unless measurement and the exact filesystem contract justify fdatasync.

Only after that sync succeeds should renameat replace the target. Linux guarantees that, when the destination already exists, another process opening the destination does not observe a moment when it is missing. A reader with the old file already open may finish reading the old inode; a later open resolves to the new one. Both are complete versions, which is the desired read contract.

Rename updates directory metadata. File fsync does not necessarily make that directory entry durable, so open the parent directory and sync it after rename. The ordering is the proof:

new contents durable
  -> target name atomically points to new inode
  -> target-name mapping durable
  -> caller may receive success

Before rename, a crash may leave the old target plus an orphan temporary file. After rename but before directory sync, the visible target may be new, while post-reboot reachability is not yet promised. After a successful directory sync, the acknowledged target mapping and already-synced contents form the committed generation. Recovery removes only recognizable stale temporary files after validating ownership and naming; it never deletes the target merely because the previous operation returned an error.

Error reporting needs a state richer than Boolean success. Before rename, failure is not_committed and the old committed target remains authoritative. After rename, a directory-sync failure is indeterminate: the new file may be visible and may or may not survive a crash. Return an error, retain diagnostics, and let recovery inspect a generation number or checksum. Blindly writing the old value back creates another uncommitted transition and can overwrite a newer writer.

For multiple writers, place an advisory lock around read-modify-write only if all writers honor it, or store an expected generation and reject stale updates. A unique temporary file merely keeps staging files separate. Without serialization, two valid renames are individually atomic but the later one silently wins. For multiple related files, publish one immutable generation directory through a single durable pointer, add a write-ahead journal, or use SQLite; repeated renames do not create a multi-file transaction.

Strict durability pays for at least a file flush and a directory flush per committed update. If every update need not survive power loss, offer a separately named relaxed mode that uses temp-plus-rename for atomic visibility and explicitly allows loss of the latest version. For high update rates, batch several logical changes into one journal commit or use a database with group commit. Never silently weaken the contract to improve a benchmark.

Test the protocol at every boundary: partial write; EINTR; ENOSPC; file-sync EIO; crash before and after file sync; crash before, during, and after rename; and directory-sync failure. After reboot, assert that the target parses, matches either the last acknowledged generation or an allowed unacknowledged new generation, and is never a byte mixture. Also assert that every acknowledged generation survives. A process SIGKILL test checks cleanup and file-descriptor behavior, but only a controlled VM or storage crash test exercises loss of volatile caches.

High-Quality Sample Answer

“I would start by defining three contracts: readers see a complete version, an acknowledged update survives a host crash, and writers are serialized. The current target is assumed to have been committed by the same protocol.

I open the parent directory and create an exclusive, unique temporary file beside the target. I loop over write because it can return a short count, apply the required permissions, call fsync on the temporary file, and check every result. Then I rename the sibling over the target. That rename is the atomic visibility boundary: existing readers may retain the old inode, while new opens resolve to the complete new inode.

The rename is not yet my durable commit. It changed a directory entry, and syncing the file does not necessarily persist that entry. I therefore fsync the open parent directory and return success only after it succeeds. A crash before rename leaves the old target and perhaps a removable temp file. A failure after rename but before directory sync is indeterminate, so I return an error and inspect generation metadata during recovery rather than promising rollback.

I would fault-inject partial writes and every syscall failure, then crash a VM at every boundary. The invariant is that the target is always a valid old or new generation, never a torn mixture, and every acknowledged generation remains after reboot. If I need concurrent writers or a multi-file transaction, I add generation checks and locking or switch to a journal/database instead of claiming rename solves those problems.”

Common Mistakes

• Overwrite the target in place → a crash exposes truncation or mixed contents → stage a complete sibling and rename it.
• Assume one write() writes everything → short writes silently truncate the staged file → loop until all bytes are written or an error ends the operation.
• Call rename before syncing the temporary file → the durable name may point at incomplete or lost data → sync the new inode before publication.
• Treat rename as durable commit → atomic visibility does not persist the directory entry → sync the parent directory after rename.
• Use a temp directory on another mount → rename fails with EXDEV or degrades into non-atomic copy → create the temp file in the target directory.
• Ignore fsync errors → delayed storage failure becomes false success → propagate an explicit non-success or indeterminate outcome.
• Test only with SIGKILL → the kernel cache survives process death → add controlled host/storage crash injection.
• Let concurrent writers race → individually atomic updates still lose a generation → serialize or compare expected generations.

Follow-up Questions and Responses

Follow-up 1: Why is fsync(temp) insufficient?

It makes the temporary inode's contents and associated metadata durable under the filesystem contract. The target pathname is a directory entry. After rename, that mapping has changed, and Linux explicitly says a file sync does not necessarily persist the containing directory entry. Sync the parent directory before acknowledging.

Follow-up 2: Is rename() atomic if another process already opened the target?

The namespace replacement is atomic for path lookup. An existing descriptor continues to reference the old inode, while a new open resolves to the new inode. If a reader needs one generation across several reads, it should keep one descriptor open rather than reopen midway.

Follow-up 3: What should the API return when directory fsync fails after rename?

Return an error with an indeterminate commit state. The rename may already be visible, and the application cannot prove whether it will survive reboot. Record the intended generation and error, stop claiming success, and let recovery validate the target. An automatic blind rollback can destroy a later valid update.

Follow-up 4: Can fdatasync replace fsync?

It can when the platform contract and measurements justify it. It omits metadata unrelated to later data retrieval, but must still persist required metadata such as file size. It does not remove the need to sync the parent directory after rename. Prefer fsync in a generic answer because its contract is easier to audit.

Follow-up 5: How would you update three files atomically?

Three independent renames can expose mixed generations. Write all files under an immutable generation directory, make that directory durable, then atomically switch and sync one manifest or pointer; alternatively use a journal or embedded database. Readers resolve one generation and keep it for the operation.

Follow-up 6: How do two writers avoid lost updates?

Use a lock honored by every writer around the complete read-modify-commit sequence, or include an expected generation and reject a stale commit. Unique temp names only prevent staging collisions. Atomic rename does not compare business versions and naturally permits last-writer-wins.