Spec: owner data-correction actions - Latest Chapter, series_url, Cover, and orphan removal #135
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Spec derived from the wayfinder map #114, which is fully charted. This is spec 2 of 4; the map's
decisions were split for implementability, not re-decided. Every decision below was settled in #120,
#121, #125 and #131.
Blocked by: #134 (spec 1 of this series) (the admin dashboard — it creates the Series detail page these
controls live on, the Forced Poll column the Cover replacement rides, and the orphan filter the
Remove control is reached from).
Problem Statement
The dashboard can now find broken Series. It cannot fix one.
Four concrete things I cannot do as owner:
A Cover that no longer matches the Site. The store only ever fills an empty cover address, so
once a Series has a Cover there is no path that replaces it. Worse, the address is derived from the
source URL, so a Site that re-arts a Series behind the same URL is invisible twice over: the file is
not written, the row is not updated, and the client holds the old bytes for a week under an immutable
cache header. Every Reader sees stale artwork and nothing in the system can tell.
A Latest Chapter I cannot make right. Where a Series' page still reads, a forced check fixes the
number — the Poll is the oracle. But where the page cannot be read at all, the stored number may have
come from a single Reader's report, nothing will ever confirm or contradict it, and no filter can find
it moving because it never moves.
A series URL that points at nothing. The column is write-once: it is set when the row is created
and no Poll and no Reader PUT ever changes it. So a Series whose address went bad stays bad for ever,
and the row is unrepairable by any client action.
Rows nobody holds. Removing a Bookmark leaves the Series behind and nothing deletes one, so the
row outlives every relationship to it: no Reader can reach it, no Poll visits it, and it still owns a
Cover file on a small disk. The dashboard now shows me these; it gives me no way to be rid of one.
And when I look at a number, I cannot tell where it came from — my own hand, a Reader's report, or a
machine read.
Solution
Four interventions on the Series detail page, and one derived line that explains a number's origin.
Cover replacement is not its own control. Asking for a Forced Poll is asking to accept the page
as it now stands, so a forced pass replaces the Cover while an ordinary pass keeps filling only a
blank one. Underneath, new Cover writes are addressed by the SHA-256 of the bytes rather than of the
source URL, which is what makes a re-art visible at all — and makes "the Site is serving the same
artwork" an honest no-op the page can report instead of a double silent failure.
A Latest Chapter correction with no authority. One numeric input. The value is overwritten by the
next successful Poll and by any Reader's report, and that is correct: the Poll is the oracle, and a
page read is stronger evidence than a typed number. No pin, no floor. It exists only for the Series
whose page cannot be read — where the page can be read, a forced check is the answer.
A series URL repair, owner-typed and gated. Validated by the same gate the poller already uses
before it fetches anything, because a client-supplied URL is exactly what that gate exists for. The
repair fetches nothing; the owner presses Check now afterwards.
Orphan removal, one row at a time. A plain delete, with the existing foreign key as the entire
guard — its refusal means "a Reader bookmarked it again". Cover bytes are reclaimed only when no
other Series points at them, by one guarded helper shared with the Cover replacement path.
Provenance is derived, never recorded. One line on the detail page naming the actor class behind
the current number: a Correction, a Sighting, or a machine read. No history table, and none is coming.
User Stories
artwork, so that Readers stop seeing a picture the Site has replaced.
not have to reason about which of two buttons re-reads the page.
does not move artwork under a Reader for no visible reason or spend a Cover fetch per Series per
cycle.
that I learn the Site has not re-arted rather than watching a button do nothing.
uses, so that the two Sites whose covers are gated keep working exactly as they already do.
sidecar is asleep, so that there is no new failure mode and no new notification to learn.
like a destruction — the Cover that disagrees with the Site is the confusing state, not the change.
fix is actually visible to me.
that the number Readers see is not left at whatever one Reader's browser happened to report.
machine remains the authority and my typo cannot outlive the Site becoming readable again.
exactly one rule — a page read beats a typed number — and not two.
number the new-chapter accent compares against.
reaches a row every Reader reads.
value, so that I do not mistake a stopgap for a pin.
correction that just landed from one that has stood unchallenged for a month.
never reads as history.
marker alone, so that the common case does not silently erase the fact that the value is mine.
that the next Poll does not credit or blame that Reader for my number.
something a machine earned and my typo is not an unappealable penalty.
is no announcement to read and the new-chapter accent simply follows the number.
being permanently unpollable.
be talked into probing arbitrary hosts from its own network position.
address it only stored.
with a visible result.
job, so that I do not sit typing the same fix into two hundred forms.
shared row onto a different work that the same gate would happily accept.
accumulating and stop holding Cover bytes on a small disk.
am never offered a control the database will refuse.
that the refusal reads as "somebody bookmarked it again" rather than as an error.
removing one Series cannot blank another's artwork.
failure leaves a repairable broken image rather than an unnamed file nothing can reach.
the recovery path is a control I already have.
not claim byte deletion the guard may refuse or claim an effect on Readers who do not exist.
list cannot be a single mis-click.
still claiming one orphan after the last one is gone does not need a page reload to clear.
am not left refreshing a page whose subject no longer exists.
without weighing it as data loss.
can judge whether to trust it before I act on it.
table, so that the dashboard never becomes a record of which Reader read what and when.
Implementation Decisions
Cover replacement rides the Forced Poll
Poll keeps fill-if-blank unchanged. The pass already holds the Cover it read when it decides whether
to fill a blank one, so a forced pass costs no extra page read.
series.force_cover_atcolumn — two columns, two verbs, two controls and anotherfield on the admin projection, for a rare action. Rejected: a synchronous fetch inside the admin
request — it contradicts the standing decision that intervention reaches the poller through the
database, and it would block an owner request on the browser sidecar.
re-reading chapters. The two are one act — read this Series' page now and accept what it says.
publishes today, and Acquisition no longer claims to be the only read that establishes a Cover.
The Cover address becomes a hash of the bytes
Three facts make a same-address replace impossible: the file is installed with a hard link that
ignores an existing target, the covers insert does nothing on conflict, and the public cover route is
served immutable for a week. So the address must change for a replacement to be visible — and
today it is the SHA-256 of the source URL.
untouched, and there is no migration. The only contract on the value is the 64-hex-character
pattern the store validates, so a mixed derivation is legal with no schema change and legacy rows
stay readable.
globbed by version — and would need a run-once Go step walking the whole cover tree at boot.
Rejected: a URL hash salted with a per-Series refetch generation (a column whose value means nothing
to a reader of the row).
only if the Site serves the same bytes. "Unchanged" becomes observable instead of a silent no-op,
and the page can say so.
CoverAddress(sourceURL)has no production caller — only tests andstore internals — so the "name a Cover before you have the bytes" property its doc comment defends
is not load-bearing. And the prefetch path's existing-cover shortcut stops matching byte-addressed
rows: delete the shortcut and let that legacy heal path fetch, rather than adding a
source_urlcolumn to the covers table to keep an optimisation on a rare repair.
contradicts the URL-hash statement in the cover-address migration and in the store's own doc
comment, which the change must rewrite.
Store surface for the replacement
ReplaceSeriesCover(site, seriesID, sourceURL string, body []byte, contentType string) errorbesidethe existing
SetSeriesCover. Both go through the same cover installer and differ only in the updatepredicate: the existing method keeps its empty-address guard.
force boolparameter. Three existing callers would passfalsefor ever, and thefirst caller that passes
trueturns fill-if-empty into overwrite at the call site rather than inthe store. The existing test that asserts a Cover is not overwritten guards behaviour that should
stay unparameterised.
coveras well ascover_address, because a re-art may sit behind a newsource URL.
so the gated cover host and the browser-only Site keep going through the sidecar and everything else
goes over plain TLS — one routing rule for Acquisition, Poll and this action. No pre-flight
refusal when the sidecar is down: the Lane skips its pass, the request marker ages, and an ageing
marker is already the evidence that a Lane is stuck. The Lanes page already states the sidecar fact,
so no new notification is added.
page holds that, per the one-figure-in-one-place rule. Rejected: a "no browser" mark on every Series
row of a browser Site, which would put Lane state into the Series list.
Cover that disagrees with the Site is the state that confuses the Reader, so the change is the
remedy and not the risk. The confirm law covers actions that pull a Series out of a list, and this
pulls none.
Latest Chapter correction
The motivating case was not a defect. The lightnovelworld ticket that prompted this settled the
opposite: the chapter-list maximum is Latest Chapter, and capping it at the Site's own
newest-chapter banner is pre-rejected. A Reader below that maximum genuinely has unread chapters and
the accent is earned. So "fixes the class of that issue" is struck.
What survives is one class no machine can reach: a Reader-raised value on a Series whose page the Poll
cannot read. Where the page is readable, a Forced Poll is the remedy — the pass judges the
outstanding Sighting and rewrites from the Site — so a correction on a readable Series is not a
correction, it is a workaround for an adapter bug.
There are two overwriters, not one. The poller rewrites on any inequality with what the Site
publishes; and the series Upsert's conflict clause is unconditional last-write-wins from any
Reader's PUT, which both userscripts fire on every series-page visit, sending the site-read value up
or down and sending it even unchanged. So "sticky" was never one guard: it would be a pin column
plus a guard on the poller path and on the client write path every Reader hits.
Decisions:
the moment any machine write lands. A floor is rejected outright: it would have made the downward
direction — the one that was actually right in the motivating case — unrepairable. On the class this
control is for, nothing overwrites it anyway; that is what "the Poll cannot read the page" means.
else 400. The stored label is
"Chapter " + the formatted number. Two inputs are rejected: nothingdepends on matching a Site's typography, and a free-text label invents a failure mode where the
number is right and the panel renders the typo. Leaving the old label is worse still — the panel
would keep saying the old chapter after the number moved.
series.latest_corrected_at bigint NOT NULL DEFAULT 0meaning "the current value is ahuman's", written by the correction and zeroed by every machine write of the value:
CorrectLatestChapter(site, seriesID string, num float64, ts int64) errorsets the label,the number and the stamp;
, latest_corrected_at = 0in the same update — alreadyconditional in effect, since the pass returns early on equality and so only writes when the number
moved;
actually changing:
latest_corrected_at = CASE WHEN excluded.latest_chapter_num IS DISTINCT FROM series.latest_chapter_num THEN 0 ELSE latest_corrected_at END.Unconditional zeroing would be wrong for the common case: after a correction a Reader's cached row
holds the corrected number, so an ordinary Progress PUT resends it and would clear the stamp while
the value is still the owner's. One clause in an existing statement — no extra round trip, no
second update — but it touches the security-reviewed Upsert, so say so in that PR.
corrected_by(there is one owner), anda stamp cannot outlive the value it describes.
correction, if the Series carries a raising Reader, call
ClearSightingAttribution— neverRecordSightingOutcome(..., false). Marking the Reader is tempting (this is the one class where amark would otherwise never land) but recovery is twenty confirming Polls, and this control exists
precisely because no Poll can read the page — so a mark earned here is permanent in practice and an
owner's typo would be unappealable. Marks stay machine-earned. Leaving the attribution in place is
wrong outright: the next Poll would credit or blame that Reader for the owner's number.
Chapter, and the accent follows the number against each Reader's own progress, so a downward
correction can only extinguish an accent and an upward one lights it — which is what "the Site
published" means.
Series URL repair
SetSeriesURL(site, seriesID, url string) error; the handler validates with theexisting gate before storing. That gate is currently unexported in the poller, so it is
renamed to
latest.FetchableSeriesURLwith its in-package callers updated — one implementation,never a second gate. The web package already imports
latest.the row exists — for the owner only.
time and the store drops it, deliberately.
seriesis a shared row: one Reader could repoint aSeries every other Reader reads at a different work on the same host, and the gate would pass it,
because the gate stops SSRF, not mis-pointing.
every row of that Site at once, and a per-Series form is the wrong tool for it. That is a SQL
migration and is out of scope.
Orphan removal
detail page, and both render it only at a zero Reader count.
is one the owner learns to ignore, and the foreign-key refusal is for the race, not for everyday
feedback.
read individually is a different risk class, and the orphan set is short by construction (a Reader
has to drop the last Bookmark).
DELETE FROM series WHERE site = $1 AND series_id = $2. Theexisting bookmarks-to-series foreign key has deliberately no cascade, so it refuses the delete while
any Bookmark exists — which is the definition of orphan. No
NOT EXISTSpre-check: it wouldadd a read-then-write window and a second copy of the definition that can drift. A foreign-key
violation is translated at the handler into "a Reader has bookmarked this Series again" and the row
re-rendered with its new count.
or a Poll refetches the Cover. Removing an orphan discards a row nothing reads, not history.
the press would write the request, no Lane would ever consume it, and the row would age a pending
marker for ever — firing the stuck-Lane signal on a Series that is not stuck. A row at zero Readers
carries Remove only. Rejected: making a Forced Poll override the join. A Series with no Reader has
no consumer for the result.
Cover byte reclamation — one guarded helper, two callers
DELETE FROM covers WHERE address = $1 AND NOT EXISTS (SELECT 1 FROM series WHERE cover_address = $1),plus the unlink of the sharded path. Called at both causes: orphan removal, and the forced Cover
replace whose byte-derived address strands the old one the moment the art changes.
series.cover_addressis the only reference that keeps bytes alive;
series.coverholds the third-party source URL andreferences nothing.
demonicscans.org: twelve Series from the home page gave twelve distinct cover addresses and twelve
distinct byte digests, and a page for a Series that does not exist publishes no cover image at all,
so there is no shared placeholder. Rare, not impossible.
reason is that the covers row is the handle. Keep it until last and an interrupted reclamation is
discoverable in SQL — covers rows unreferenced by any series cover address — and re-running finishes
the job. Delete the row first and the leftover file is named by nothing: no query lists it and finding
it means walking the sharded tree and diffing against the database. That is dark garbage needing
manual deletion; the alternative is a repairable broken image.
is what makes a retry possible.
delete the covers row — the guard cannot pass while the series row still points at the address.
(same address, file re-linked) or new ones (new address, row repointed). The forced replace path has
no empty-address predicate, which is what makes this true; the earlier note that a non-blank cover
address is unrepairable described the routine paths only and is superseded.
between guard and unlink. Narrow, and its outcome is the repairable case.
whole-table predicate — the one whose being wrong is unrecoverable — to fix a problem two known
callsites prevent, and it needs a schedule this backend has no ticker for. Rejected: never
reclaiming; a Cover is 50–500 KB on a swapless 1974 MiB VPS while a series row is 200 bytes.
than a tree walk. This spec does not build it.
Latest Chapter provenance — derived, never recorded
Nothing records how the number came to hold its value, and nothing will. No history table, no
attribution column, no migration. Three actor classes, all derived from state this series already
decides:
has judged it; surfaced anonymously as the admin row's Sighting flag.
Acquisition does not survive as a distinct actor. Acquisition stamps the check timestamp exactly as
a Poll does, so an acquired value and a polled value are indistinguishable the moment the Acquisition
finishes. Accepted deliberately: the only case where the difference is actionable — acquired once and
never read again — is already the
uncheckedfilter, and telling the two apart would need a column,which is the thing this declines to add.
Consequence: the correction stamp is promoted from convenience to load-bearing. It is the only
evidence a Correction ever happened, so it must not be dropped, and its zeroing rules are what keep the
derivation honest.
Presentation: one line on the Series detail page naming the actor class beside the number.
Owner-only by route. No list-row field (a row has two lines), no new filter name, no landing figure —
an actor class is context for a Series you are already looking at, never a population to sweep.
The Cover half is closed by construction, and a superseded Cover is deliberately unrecoverable: new
bytes are addressed by their own hash, so a replacement is visible in the address itself, and the
unreferenced covers row is reclaimed. The Site is the only true source of what the art should be.
A history table was designed before it was dropped, and the design is recorded so it is not
re-proposed: one row per actual change (a no-change PUT and a confirming Poll writing nothing, roughly
52 rows per Series per year), actor class only, newest-twenty-per-Series pruned on insert in the lazy
style the sessions table already uses. Three costs killed it:
Reader id — the obvious next request, since a Sighting comes from someone — and it becomes a record
of which Series each Reader reads and when.
both the correction stamp and the newest row's actor class, and the two would eventually disagree.
re-establishes the number without reference to history; distrust of a Reader is already the
Sighting-disagreement counter, which is durable, judged by Polls, and self-healing over twenty
agreements. A trail's unique payload was "who to distrust", and that mechanism already exists and is
better.
Routes and presentation
POST /admin/series/{key}/latestnumPOST /admin/series/{key}/series-urlurllatest.FetchableSeriesURL, else 400POST /admin/series/{key}/removeAll join the admin route list behind the owner gate, and form bodies are capped the way the API path
caps them.
copy saying the next successful Poll overwrites the value. Gating on "looks unverifiable" would make
this wait on per-Series failure state for cosmetics, and every gate variant forbids the repair exactly
when it gets easy — a briefly-healed Site does not make a wrong stored number right.
and neither does), never ember, no danger accent. A
corrected <age> agomarker renders while thestamp is non-zero; it is transient by design and must not read like history.
only what is certain: "Removes this series and its stored cover. No Reader has it bookmarked; one
re-bookmarking it recreates the row." Byte deletion is not promised — the guard decides. The earlier
draft copy claiming it removes the Series "for every Reader" and deletes its Cover bytes is wrong
twice over: an orphan has no Readers, and the bytes may survive the guard.
line: it costs a query per confirm-row render and buys a fact the owner cannot act on. No
"unreclaimed covers: N" figure for a state that has never occurred and that the page cannot fix. A
failed unlink is logged, and the ordering above makes it findable.
re-rendering the
<N> series · <label>heading — the row and the count are one fact, and a headingstill reading 1 after the last orphan is gone is a lie the owner must reload to clear. The filter
select's option counts stay stale until the next navigation: eight snippets per delete to keep one
number honest is not worth it. The detail page redirects to the orphan list — the subject no longer
exists, a refresh would 404, and the list is where the owner was headed.
with Check now below; the provenance line sits beside the number.
Testing Decisions
What makes a good test here: it asserts an outcome the owner or a Reader can observe — a stored
value after a request, a rendered marker, a file's presence, a refusal's status code — and it fails on a
plausible bug. Nothing asserts a helper's internals or a log line.
The primary seam is unchanged and existing: the router over a real store and a throwaway Postgres,
driven with an owner session cookie. Every control is reached as a POST and asserted through the
response and the resulting database state. No new seam, no new fake.
Modules and coverage:
owner-gated by joining the route list; a non-numeric, zero or negative chapter is 400 and does not
reach the store; a URL failing the gate is 400 and does not reach the store; Remove is rendered
only at a zero Reader count; a removal that hits the foreign key renders "a Reader has bookmarked
this Series again" with the fresh count rather than a 500; the removal response carries the
re-rendered heading count; the detail page redirects after removal; the
corrected <age> agomarkerappears while the stamp is set and is gone after a machine write; the provenance line names each of
the three actor classes for the three states that produce them.
CorrectLatestChapterstamps and the poller's chapter setter zeroes; an Upsert with the samenumber keeps the stamp and one with a different number zeroes it (this pair is the whole point of
the conditional clause and is the most likely thing to get wrong); a correction clears the raising
Reader and leaves both Sighting counters untouched;
ReplaceSeriesCoveroverwrites a non-emptyaddress while the existing setter still refuses to (the existing no-overwrite test must keep
passing unchanged — it is the guard that this change did not leak into the routine path); the same
bytes yield the same address and different bytes a different one; the reclamation helper deletes the
covers row and the file when nothing points at the address, and deletes neither when a second
Series does; the delete of a series row is refused while a Bookmark exists and succeeds when none
does;
SetSeriesURLwrites where the Upsert would not.forced pass replaces an existing Cover and an unforced pass does not; a forced pass over identical
bytes is a no-op the caller can distinguish; a forced pass on a Series whose Cover file was unlinked
re-links or repoints the row; the Cover fetch routes through the same fetcher selection as the page
read.
actually gone, and the interrupted case (row present, file absent) is asserted to be findable by the
unreferenced-covers query and repaired by a forced pass.
Out of Scope
seriesis shared; the gate stops SSRF, not mis-pointing.once. That is a SQL migration, and the dashboard's library-wide job is finding broken rows.
force_cover_atcolumn, or a synchronous Cover fetch in therequest.
filter, whose first ground (nothing stores the previous number) stays true, and a flap detector,
which was reachable only under the history option and dies with it.
should be.
Further Notes
ADRs both stop at 0011 and everything the map specified is unbuilt. This spec's only schema work is
series.latest_corrected_at bigint NOT NULL DEFAULT 0; take the next free number after thedashboard spec's migrations.
the source URL. It is hard to reverse once data exists under both derivations, and it contradicts
statements the change must rewrite in the cover-address migration and the store's doc comment. The
reclamation rule needs no ADR — it is one helper and reversible.
takes whatever Cover the Site publishes today, Acquisition no longer claims to be the only read
that establishes a Cover, and Correction is defined. Nothing new is needed.
state that a later spec adds: an owner typing a number is not evidence the page became readable, and
only a successful Poll may reset it. Any new per-Series table must key on
(site, series_id)withON DELETE CASCADEtoseries, so orphan removal stays a single statement and the FK-as-orphan-testproperty survives — the cascade must reach poll state only, never
bookmarks, whose refusal isthe guard.
one implementation, no second gate), the security-reviewed series Upsert (one new conditional clause),
and the owner gate. Say which invariant you preserved in the PR and run the full backend test suite
before calling it done.