Integrity and trust¶

This page records where docbank's integrity guarantees live and — just as deliberately — where they don't. Several of the decisions below have been proposed, evaluated, and declined during review; they are written down here so the rationale is auditable and doesn't have to be re-litigated from scratch each time.

Trust boundary¶

docbank is a single-user tool. The vault lives under a 0700 directory owned by the user, its location comes from the user's own environment (DOCBANK_HOME), and every docbank process runs with the user's privileges. Consequently:

• In scope: crashes and power loss at any instant; filesystem bit rot; accidental damage to the vault or source trees (a stray symlink where a document or blob file belongs, a deleted file, a manual edit inside ~/.docbank); concurrent docbank processes; a stale or tampered object being silently vouched for. The vault's own directory structure is trusted — see the read-guarantee note below.
• Out of scope: an adversary with the user's own privileges. Anyone who can race a running docbank process while rewriting the user's files already executes as the user and does not need docbank as a vector. Defenses whose only payoff is against that actor are declined by policy.

Which layer owns which guarantee¶

Accepted trade-offs¶

These were proposed (most more than once) and deliberately not taken.

No re-hash on the dedup fast path¶

blob.Write's dedup check verifies structure (regular file, expected size), not content. Re-hashing the existing blob on every duplicate import would double the I/O of the most common bulk-import case to detect exactly one thing the structural check misses: same-length corrupt bytes, i.e. bit rot or tampering.

That detection intentionally lives in docbank verify, and for a reason beyond cost: a corrupt blob is equally wrong for every node already pointing at it. Catching it only when a duplicate import happens to pass by is not an integrity guarantee — it's a coincidence. The systematic answer is a scan that covers every blob regardless of import traffic, which is precisely verify's contract.

Fail closed on an invalid canonical object¶

An existing wrong-sized file, symlink, or special file at a blob's canonical path is evidence of damage or manual modification, not a valid deduplication target. blob.Write returns a content-mismatch error and leaves that object unchanged; it does not replace a path whose identity may have raced or been tampered with.

Recovery is explicit: stop the daemon, move the suspect object outside the vault for diagnosis, restart the daemon, and re-add the content from a trusted source (or restore it from a verified backup). The durable write happens before the existing node is recognized as an idempotent ingest, so re-adding repairs a missing loose copy without creating a duplicate document. Run docbank verify afterward to confirm the vault.

No fd-relative directory traversal on import¶

docbank add walks source trees by path (filepath.WalkDir). A directory swapped for a symlink between classification and descent can redirect the walk — but only by an actor mutating the user's own source tree mid-import, which is outside the trust boundary above. The payoff (importing user-readable files into the user's own vault) does not justify platform-specific openat/fstatat machinery. The accidental case — a symlink sitting in the tree — is handled: it is classified, reported, and skipped, and the final open refuses to follow links regardless. The one explicit exception is a source argument whose final component is a symlink to a directory, which is common for cloud-storage roots. Docbank resolves that link before walking and continues on the resolved path, so retargeting the user-facing link after resolution cannot redirect the walk. Descendant symlinks are still skipped, while virtual naming and provenance retain the spelling the user supplied.

Watched inboxes use a stronger traversal boundary because they run repeatedly without an operator supervising each scan, and accidentally entering the live vault would create a self-ingestion loop. A watcher pins its resolved source with os.Root, opens every descendant relative to held directory descriptors, rejects symlink or reparse-point components, and refuses to cross a filesystem mount. Linux requires a kernel-provided mount ID; it does not substitute a device number that would mistake a bind mount for the source mount. Before any ingest, the watcher also compares every descended directory with the pinned vault-root identity. This catches a vault physically beneath the source even when separate bind aliases hide that ancestry from path comparison; the same identity check remains active during later scans.

Pre-release schema freedom¶

Docbank has not yet established a public storage compatibility boundary. Until the first public release, incompatible schema and JSONL changes replace the development shape directly while the format identifier remains version 1. Developer vaults created by earlier commits are disposable; do not build migrations, compatibility decoders, cutovers, or downgrade fences for them.

The first public release freezes that v1 contract. Any later incompatible change must begin by defining an explicit compatibility policy and real released-vault fixtures; that work is deliberately deferred until it is needed.

One vault lock holder¶

The daemon holds the portable vault file lock exclusively for its whole lifetime, acquired non-blocking at startup — a second daemon is refused immediately, never queued. Ordinary commands don't touch the lock at all: they are HTTP clients of the daemon. Maintenance (gc --run, verify, trash empty) is serialized against ordinary mutations by the daemon's in-process maintenance gate. See Ownership & Concurrency.

Next: Backup & Recovery covers the snapshot architecture these guarantees extend to; Troubleshooting applies them when something looks wrong.