How Docbank works¶

Docbank presents a familiar tree of directories and files, but it is not a folder of ordinary files. It is a document system built from two cooperating parts:

• SQLite metadata describes the tree people and agents work with: stable document identity, paths, versions, tags, provenance, trash state, and search indexes.
• Immutable content storage holds the bytes, addressed by their SHA-256 digest and stored either as individual files or inside sealed pack files.

Separating those parts makes large reorganizations cheap, deduplicates identical content, retains version history without copying mutable files in place, and lets backup rebuild the same logical vault independently of its physical layout.

Docbank does not assume one global archive per machine. A person or application may operate many independent vaults, each with its own identity, metadata, content, policy, and backup history.

The model in one minute¶

Five ideas explain most of the system:

1. A vault is one independent document collection. Its database and content store are one logical unit.
2. A node is a directory or file with a stable numeric ID. Its path can change without changing which document it is.
3. A file's content version is an immutable historical record with its own random UUID. Editing or reverting creates a new version; it never rewrites an old one.
4. A blob is the byte content of one or more versions. Its SHA-256 digest is its identity, so byte-identical files share storage automatically.
5. The catalog decides which bytes are authoritative and where Docbank may read them. A loose file or pack entry found on disk has no authority by itself.

flowchart LR
    CLIENTS["People and agents<br/>CLI · HTTP · embedded Go"]
    TREE["Virtual tree<br/>nodes · paths · tags · trash"]
    VERSIONS["Immutable versions<br/>version ID · size · media type"]
    CATALOG["Content catalog<br/>SHA-256 · authorized location"]
    CONTENT["Immutable bytes<br/>loose blobs or sealed packs"]

    CLIENTS --> TREE
    TREE -->|"current and prior versions"| VERSIONS
    VERSIONS -->|"content digest"| CATALOG
    CATALOG -->|"loose or packed"| CONTENT

The arrows matter. A path resolves to a node; a file node selects a current version; a version names a content digest; and the catalog selects an authorized physical representation. Skipping a layer would make mutable names or stray files into accidental authority.

Identity, addressing, and authority¶

These values answer different questions and are intentionally not interchangeable:

This is why an automation should remember node and version IDs, not paths alone. It is also why finding a correctly named file under blobs/ does not make that file part of the vault: only a committed catalog reference does.

How a write becomes authoritative¶

Every content-write surface follows one ordering rule: prove and publish bytes before committing metadata that names them.

1. Docbank streams the source into private staging while counting and hashing it. A verified upload or replacement must declare the expected size and digest; a mismatch ends the operation without granting node authority.
2. The content store syncs the completed bytes, closes them, publishes them under their digest, and syncs the containing directory.
3. One SQLite transaction creates or updates the node, records the immutable content version, and grants the blob catalog authority.

The entry points differ in what the caller can know and what evidence comes back:

If the process fails before step 3, durable bytes may exist without a database reference. They are harmless orphans: normal reads cannot see them, and garbage collection can reclaim them later. The reverse state—a committed version whose bytes were never durably published—is prevented by the ordering.

Tree mutations such as move, rename, trash, restore, tag assignment, reversion, and version pruning happen in SQLite transactions. Path-based mutation endpoints resolve the path and perform the change in the same transaction, so a concurrent move cannot silently redirect the operation.

How a read is proved¶

A read walks the authority chain in the opposite direction:

1. Resolve a path or node ID to a file node.
2. Select its current version, or address a historical version directly.
3. Resolve that version's digest through the catalog.
4. Stream the authorized loose blob or pack entry while verifying its recorded framing, size, and SHA-256 digest.

Receiving some bytes is not yet proof of a complete read. The typed clients accept content only after reaching terminal EOF and validating the terminal Content-Digest trailer. Embedded streaming follows the same rule: an early close is incomplete verification, not success.

Editing, deletion, and reclamation are separate¶

Docbank keeps logical decisions distinct from physical storage maintenance:

This staging provides a regret window without pretending storage is free. Users and agents can retain every edit by default, deliberately prune unwanted history, empty trash on their own schedule, and reclaim packed space only when repacking is worthwhile. Editing & Versions and Trash, GC, Repack & Verify give the command-level contracts.

Loose and packed bytes are one content store¶

New content is published as a loose, digest-named file. Packing later combines eligible small blobs into sealed immutable pack files, reducing filesystem enumeration and backup overhead. The catalog changes representation without changing the blob digest, version identity, or document path.

Readers therefore do not care whether content is loose or packed. Backup also captures logical content, not the source pack layout, and restore may publish a different valid representation under fresh catalog authority. See Loose & Packed Content for limits, maintenance, and the boundary between Docbank policy and Kit mechanics.

One owner, two integration modes¶

Exactly one process owns an open vault at a time:

• In daemon mode, docbank daemon run owns SQLite and content storage. The CLI is a thin authenticated HTTP client, as are other local tools and agents. Discovery and auto-start do not create a second storage owner.
• In embedded mode, one Go application owns a separately rooted vault through the Go package. It uses the same metadata, content authority, verified reads, locking, and packing rules without running a sidecar daemon.

A vault is not shared between the two modes concurrently. Hierarchical locks also prevent a daemon or restore from operating inside an already owned vault tree. Ownership & Concurrency, Daemon & Process Model, and Embed in Go describe those boundaries.

Backup reconstructs meaning, not a live database copy¶

Snapshot repositories are append-only and incremental. Each snapshot contains a complete deterministic JSONL description of the logical vault plus every catalog-authorized content blob. Unchanged objects are reused by digest across snapshots.

Restore verifies repository content, imports JSONL into a fresh current-schema database, reconstructs search and physical catalog state, checks SQLite and manifest statistics, and only then publishes the result. Source loose-versus- packed placement is not logical metadata. This makes backup a recovery contract rather than a fragile copy of a running SQLite file. See Backup & Recovery.

Integrity boundary¶

Docbank is designed to detect truncation, corruption, stale writes, malformed metadata, and incomplete recovery within its application and storage boundaries. SHA-256 identifies bytes; revisions and ETags bind mutations to observed state; catalog authority excludes stray storage; full verification reads and hashes content; and restore validates the logical relations before publication.

These mechanisms do not make a host administrator, compromised process, or someone able to rewrite both data and expected evidence harmless. The Integrity & Trust page states what is proved, when it is proved, and which threats require independent evidence.

Where to go next¶

The Roadmap gives high-level product direction. These pages explain implemented behavior and durable design intent; planned behavior is marked explicitly. Kata, rather than the documentation, is the authority for implementation work and status.