Composition: reads & watches#

docd single-routes an operation to one owner. But a read or watch whose path is an ancestor of one or more mounts has no single owner — the answer lives partly in the base store and partly in each mounted subtree beneath it. docd composes these.

Composed reads (Match)#

When a Match path is a strict ancestor of one or more mounts, docd fans the read across the base owner and every mount below it, reads each subtree, and merges them into one document before replying. Each source read is bounded by a timeout so one stalled controller cannot hang the client's read; the composition runs on a background goroutine so the client's request loop keeps serving.

A read with no mount beneath it — the common single-owner case — is single-routed normally. Root and .meta reads are out of scope for composition.

A composed read answers a body. A return asking for more — a node's path, its id, its iterType — is unsupported here, as it is for a read at or under a mount, which a controller answers with a body; a read docd passes through to logd answers it in full.

A source with nothing at its path contributes nothing rather than failing the read. Most paths exist in one source and not in the others, so absence is the ordinary case; the composed read reports not_found only when every source is absent, which is the same answer a direct read of that path gives. Any other failure from any source still fails the composition, because a merged document missing a subtree nobody could read is not a smaller answer, it is a wrong one.

Composed watches#

A composed Watch multiplexes several backend sub-watches into the client's one watch:

1. docd opens a sub-watch on the base owner and on each mount below the watched path, each with noInit set — their deltas are buffered as they arrive.
2. docd sends one composed initial snapshot (the merged subtrees at the current commit) as the watch's State event.
3. docd flushes the buffered deltas and then forwards live deltas.

That composed snapshot is a read, so it answers absence the way a read does: if no source has anything at the watched path, the watch is refused with not_found rather than being established on a document nobody wrote. A client that meant to wait for the path sets waitIfAbsent on its watch request — docd then establishes the watch on a null snapshot, and carries the flag to the controller owning the subtree, since a controller serving from its own logd session has to pass it on in turn.

Establishing the sub-watches before taking the snapshot is what makes the handoff gapless: any change between the snapshot and going live is buffered, not lost.

The snapshot is taken before the confirmation only when it could refuse — that is, when the client did not say waitIfAbsent. A client that did keeps the later read, because reading early buys it nothing and costs it the wait: a source that does not answer reads would hold up establishing a watch that client was willing to open on nothing. The sub-watches are up and buffering under either order, so neither misses an event. Failures other than absence still end the watch after it is established, since by then the client has a stream to be told on.

Delta rooting#

The watch stream follows one contract, the same one logd's does: the initial State event is the value at the watched path, and every Patch after it is a delta of that value, rooted at the same place, so a consumer applies each to what it holds.

A sub-watch on a mount below the composed path delivers deltas rooted at the mount's path; docd re-roots each under the fields between the mount and the composed path before forwarding it, so what the client receives is rooted where its watch is. The sub-watch on the composed path itself sees the whole subtree, mounts included, and is trimmed to what the composed path owns -- the mounts carry their own subtrees on their own streams -- by the same partition that splits a write across mounts. A delta that cannot be split, an operator above a mount boundary, is forwarded whole rather than dropped. (logd may lower an operation to the result it produced, so a watch consumer applies deltas rather than pattern-matching their surface form.)

Retain: routed, not composed#

A retain names several containers, and each has an owner. docd routes each rule to the owner of its container — logd for a base path, the controller whose mount holds it otherwise — in one request per owner with one now for all, and answers the rules in the order sent, each saying who ran it (owner), which mounts beneath its container it did not reach (under), and what the owner said if it refused (error). A controller that does not implement retain answers unsupported, and the result reports that for the rule; the request is an error only when nothing ran anywhere. Nothing is composed beneath a container.

Coordination#

Mount membership must stay fixed for a composed watch's lifetime, or its snapshot and its sub-watch set would disagree. docd enforces this with a small reader/writer coordinator:

• Writers are mount/unmount; readers are active watches. They conflict on overlapping paths — one path is a prefix of the other.
• Writer priority: once a mount at a path is pending, new overlapping watches block until it finishes, so a stream of arriving watches cannot starve a mount.
• The mount then waits forceAfter (0 = forever) for the already-active overlapping watches to drain; any that remain are force-ended so membership can change.

Event preservation & reconnect#

Event preservation — every state change delivered as a discrete delta — is a logd guarantee, resting on logd's single commit sequence: a single-route watch inherits it fully, and FromCommit replays the exact delta history.

Mounts share the commit sequence — that is what the transaction mechanism is for, multiple remote participants under one tx id — so a commit means the same thing to every mount, and a composed watch does resolve one cursor and replay every mount from it.

What a composed watch cannot replay across is a change of membership, and not for want of a sequence: the composition itself changed, so deltas from before it describe a different document and there is no single document a replay of the gap would be describing. docd handles it by re-initializing:

• a mount/unmount that changes membership ends the overlapping watch with a terminal event whose EndReason says which change: "session_mounted" or "session_unmounted";
• the client re-watches the same path — now composed over the new mount set — and receives a fresh composed State snapshot (a re-sync to current), not a replay of the gap.

A snapshot-diffing consumer reconciles the re-init with no lost state. The terminal event carries endReason — a code from the same vocabulary as an error's — and endMessage, the detail the code cannot hold: the floor a compacted replay left behind, the commit range a read failed over, the sub-watch failure a composed client cannot see for itself. It also carries the last delivered commit as a resume point (WatchEndedError.Commit in libctl) — exact for a single-route watch, whose events arrive in commit order. For a composed one it is a hint: live deltas from different mounts are forwarded as they arrive rather than merged in commit order, so the mark can sit above a lower commit still in flight from another mount, and resuming at it would step over that one.

logd ends a watch the same way when a schema commit changes the keying of an array at, under or above its path (keying_changed). docd ends a composed watch when logd ends one of its sub-watches that way, and the ending carries not docd's mark but the schema commit, where the watch has to start again: the state there is the first under the new keying, and a watch resumed from earlier would cross the change and end again.

So the contract is: watches are event-preserving while streaming; a membership change is a re-sync, not a replay. A watch that never spans a mount boundary is fully event-preserving via logd.