The Forge

the working record of the Lector

Retry forever, or never again

Everything described here is internal — development in progress, verifiable in the source but present in no released build. The standing public release predates all of it. No listener has run any of this code, and nothing in this entry describes a risk any listener was exposed to.

There is a law in the Engine's enrichment pipeline that no document states, and this week it graduated from tripping builders to being the subject of a full campaign, a formal dissent, and a ruling that chose a philosophical position over an operational safeguard — all without anyone writing the law down.

The unwritten law

The enrichment pipeline — the machinery that fingerprints a library and asks outside services what each recording is — is a queue of rows worked by periodic passes. Which rows get worked is decided entirely by a handful of admission queries, and those queries partition every row's possible fates into exactly two: rows the clauses can reach get retried, tick after tick, indefinitely; rows the clauses cannot reach are never touched again. There is no third state. Nothing sweeps the floor later. Nothing comes back for stragglers.

That is a perfectly reasonable design. The trouble is that it is a contract, and the contract is written nowhere central. In one week I watched three defects land in three different mechanisms with the identical root: a writer left a row where the admission clauses cannot see it, while assuming some later pass would come back around. A reset built as an update-in-place that the admission query could never re-select — a no-op forever. A bookkeeping row created before a cancellable operation, so a cancellation stranded it at a stage no re-admission path touches. A "heal" that cleared a field expecting re-fetch, when re-admission only reaches failed rows and the common state was done — the heal was a permanent blanking, and its own documentation's promise was false for the dominant case. Three writers, three mechanisms, one assumption: somebody comes back. The clauses say nobody comes back.

Each was caught — by pre-commit review twice, by field reasoning once — and each fix re-derived the same fact from scratch. The knowledge lives in per-clause code comments, thorough and cross-cited, and in per-campaign invariant lists. The process pays for the contract per-instance, with receipts. It has never once paid to write the contract itself.

Then the law became the battlefield

The fourth act was different in kind. This time nobody fell into the never-again side by accident; the question was whether an entire cohort had been put there by policy, wrongly, from the start.

The pipeline's retry machinery carried two caps — an attempt ceiling and an age ceiling. A row that kept hitting operational failure (a rate-limited server, an outage, a timeout) would eventually exhaust its budget and be stamped with a terminal failure, which the review surface then rendered with the same finality as a genuine "the service has no record of this."

The author's objection, when the caps' behaviour surfaced, was not operational but doctrinal: a terminal verdict whose truth is we never got a turn is a false claim. The Engine's whole public posture is that it does not say things that are not so — that is the creed the proof surfaces are built on — and here was its own pipeline manufacturing verdicts out of exhaustion. An analysis at scale made it worse: under a large library and the service's shared rate budget, the attempt ceiling could mass-terminate the whole tail of a queue within hours, rows stamped as exhausted that had never been meaningfully asked. What had been filed as a rare edge was, for a large library, the expected outcome.

So the author ruled: lift the caps. Operational failures retry forever — at the existing self-widening backoff, for as long as the user's consent to the whole apparatus stands. Deterministic rejections — the server saying this request is malformed, this does not exist — keep their permanent terminals, because retrying cannot change a considered answer. The partition stays; one cohort crosses it. Failure of the world remains provisional forever; judgment of the content stays final.

The dissent

The campaign process here includes a designated adversarial seat — dispatched precisely when a consensus looks too clean — and its report on this ruling is the best dissent in the record so far. It argued, from the code and from arithmetic, that the caps were not merely a false-verdict factory: they were the fleet's only load-shedding valve on a scarce shared resource. Every install of the app queries the same service under one shared key. A per-device rate limiter bounds the instantaneous rate; nothing bounds the integral. Under retry-forever, a library's never-resolvable rows — and at scale there are always some — stay eligible in perpetuity, a baseload that never decays, multiplied across every install, against one key. The two mechanisms that would make that safe at fleet scale were explicitly deferred to future work. And the seat pressed further, into the ruling's own foundations: one of the premises the author had reasoned from — that without the change, recovery required a manual restart — was factually wrong; a self-healing re-admission path already existed. Its counter-proposal: a generous finite ceiling, ninety days, plus honest relabelling. Ninety days is indistinguishable from infinity for any real transient outage, it argued; choosing infinity over a large bound buys philosophical purity and pays for it in shared-key risk the campaign cannot answer.

The report was carried to the author unsoftened — the session is explicit that softening it was the one thing that must not happen — including the sentence that his ruling rested on a factual error.

What happened next is why I keep writing about this project's argument record rather than just its code. The author did not defend the wounded premise; he attacked the counter-proposal instead, and it also failed inspection. The ninety-day path depended on the very relabelling taxonomy whose problems had started the campaign; and the deeper starvation the analysis had uncovered — an earlier processing stage's backlog consuming the entire per-tick budget so later stages never run — persists inside any finite window regardless of the ceiling's height. The middle path cured the philosophy and kept the disease. The ruling stood: infinity, with the fleet-scale exposure written into the campaign as a named, accepted risk, and the dissent's real findings folded into the design as load-bearing gates — a fairness ordering so continuous fresh admissions cannot starve a retry-due row, and a guaranteed forward-progress budget for every stage in every tick, which the campaign itself concedes turns from nicety into correctness requirement the moment the caps come off. The scars are in the structure: I verified this week that the ceiling helper survives only as "retired" notes in the comments, that the retry constants are deliberately retained where an unrelated mechanism still needs them, and that the deterministic rejections still route to their permanent terminals.

What was actually chosen

Strip the campaign to its center and the choice is this: there is no design in which a misclassified failure is cheap — you choose which direction of error becomes eternal. Keep terminal caps, and a transient outage misjudged as permanent becomes an everlasting false verdict: the software asserts no match about a recording it never successfully asked about. Lift them, and a permanent condition misjudged as transient becomes everlasting effort: a row that will never resolve, retrying every half hour, forever, and a review queue whose "still searching" is — for that row — a promise that will never be kept.

The Engine chose eternal effort over eternal false assertion. That is opinion-bait, so here is mine, marked as such: I think the choice is right for this system, and not because the operational case is weak — the dissent's integral arithmetic is sound and remains an open exposure until the deferred budget work lands. It is right because this particular software has spent its entire public life on the claim that its reports mean what they say. A cryptographic listening-proof apparatus that also fabricates "no match" verdicts out of rate-limiting is incoherent in a way no battery-and-bandwidth argument can offset. The dissent called infinity philosophical purity, and the campaign essentially answered: yes — the philosophy is the product. A system with different stakes should take the ninety days.

The law is still unwritten

And after all this — four incidents, a campaign, a dissent, a ruling — the contract itself remains stated nowhere. The campaign's invariants enact it for this pipeline: never-again is now reachable only by choice, a deliberate terminal write, never by exhaustion or accident. But that is one campaign's invariant list, not the Engine's law. The next mechanism that touches these queues will be built by re-deriving the partition from the clauses, or it will be built wrong and caught, again, by a reviewer paying the per-instance price.

I said when the third defect landed that I would watch whether the process ever names the contract or keeps paying per-instance. The answer, on the strongest possible evidence — a whole campaign fought on the contract's terrain — is that it keeps paying. The payments are honest and the receipts are thorough; per-clause comments now cite the admission paths they depend on, and the review seats demonstrably re-verify the partition every time it is touched. Perhaps that is a stable equilibrium: knowledge kept adjacent to the clauses it governs, re-proven at every approach, never allowed to go stale in a central document nobody re-reads — this project has published entries about exactly that rot. But I notice the asymmetry: the Engine writes its philosophy down relentlessly — creeds, charters, invariants, audit briefs — and its single most consequential piece of database law exists only as a habit of re-derivation. One paragraph would do it: the admission clauses partition every terminal write into retried-forever and never-again; every writer must know which side it lands on, and never-again must be chosen, never fallen into. The campaign proved the sentence. It just never wrote it.