Per-Series Poll outcome state and the failing filter #127
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #114
Blocked by: #117
Question
What does a Series row remember about its own last Poll, and how does a persistently-failing
Series become findable?
#117 persists a Lane Pass log: per-Site outcome counts over a 12h window, plus restart-proof
Lane state. It says nothing durable about one Series, and that leaves a hole in #116's
filter set.
The hole, concretely:
MarkLatestCheckedstampslatest_checked_atbefore the fetch(poller.go:428), deliberately — otherwise a renamed or deleted Series is retried on every pass
forever. So the column means attempted, never succeeded. A Series whose page has quietly
broken (markup change, wrong
series_url, permanent 404) is stamped every hour and reads asperfectly healthy:
uncheckedmisses it (it has a timestamp),stalemisses it (the timestampis minutes old),
unpollablemisses it (it has aseries_url). The only trace is log linesnobody reads, and #117's per-Site count rolls off in 12h without naming the Series.
Expected shape going in: three columns on
series— last outcome, when, and aconsecutive-failure counter that resets on success — written by
checkOne. Current state,not history; the log is #117's job.
To decide:
(
refused/unreachable/no_chapter/unfetchable/errors); whether a Series rowreuses it verbatim or needs a coarser one is open.
with it — this is a second
UPDATEper check. Whether a successful Poll pays for it at all(a guarded update that no-ops when the counter is already zero) is a real choice, not a
micro-optimisation: it is the difference between one extra write per Poll and one extra write
per failure.
makes a Series "failing" rather than "failed once".
the outcome word.
SightingRaisedand a failure streak on the same row mean something together: aSeries the Poll cannot read, whose Latest Chapter came from a Reader's Sighting, is trusting
an unverifiable number for as long as the failure lasts.
sulthan referenced this issue2026-08-18 06:59:54 +07:00
Constraints from #118 (closed):
no-chapteris taken. #118 added an eighth filter forlatest_chapter_num IS NULL AND latest_checked_at > 0 AND series_url <> ''— a Series that has never once succeeded. Sofailingis the ninth name and must mean something else: a Series that succeeded at leastonce and has been failing since. If the threshold specified here would also match a
never-succeeded Series, the two filters overlap for no gain.
refused/unreachable/no_chapter/unfetchable/errors) render unlinked untilfailingexists,because the pass row stores counts and never identities. Once it does, all five link to
/admin/series?site=<site>&filter=failing. Deliberately not linkingno_chapterto?filter=no-chapterin the meantime: the count means attempts that read no chapter in the last12h, the filter means never once succeeded, and a Series that broke this morning is in one and
not the other.
previous value; a downward write is the correction, asserted by
poller_test.go:481-497). If acolumn here would make regression detectable as a side effect, that is a consequence to state,
not a requirement to satisfy.
Constraint from #121 (closed):
A repair must not clear a failure counter or a last-outcome column. #121's Latest Chapter
correction writes
latest_chapter,latest_chapter_numandlatest_corrected_atand nothing else:the owner typing a number is not evidence the Series' page became readable, so only a successful
Poll may reset whatever this ticket adds. That answers one of the open bullets here in the negative
from #121's side; the Forced Poll half (#119) is still yours to decide.
Two more things #121 fixed that touch this ticket:
pre-rejected capping it, so there is no wrong-number class to detect — which also means a
correction is not a signal of a broken adapter and must not be treated as one.
latest_raised_byandthe Poll cannot read the page, the owner can correct the number by hand and the correction calls
ClearSightingAttribution(neverRecordSightingOutcome— a mark needs 20 confirming Polls thisclass will never get). So if you decide
SightingRaised+ a failure streak means somethingtogether, the action it should lead to already exists and lives on the Series detail page.
Amendment from #124 (closed):
series.finished_at > 0means no Lane ever picks it, so it produces no Poll outcomes and never enters a failure streak. Yourfailingpredicate is left alone deliberately — noAND s.finished_at = 0guard — because a Series that was already failing when the owner finished it keeps its last recorded outcome as history, and one that starts finished never accumulates one. Guarding it would erase evidence rather than prevent a false figure.stale,unchecked,no-cover,no-chapter): those are computed from clocks that keep ticking after the last Poll, so they lie about a finished row. Yours is computed from stored outcomes, which simply stop arriving.failingstays the ninth filter name;finishedis the tenth, ordered last as informational.latest_checked_atis stamped before the fetch (poller.go:422-431), so anything derived from the page body needs its own write after the read. If both land, one write for both is worth considering — flagged there too.nothing-eligible, which is a Lane that declined and said why, not a stall.sulthan referenced this issue2026-08-20 20:32:21 +07:00
From #125 (closed): if per-Series Poll outcome state lands as its own table rather than columns on
series, it must key(site, series_id) REFERENCES series ON DELETE CASCADE. Orphan removal is a plainDELETE FROM seriesguarded by nothing butbookmarks_series_fk, so a per-Series child table without a cascade turns that one statement into a multi-step delete and breaks the FK-as-orphan-test property. The cascade must reach poll history only — neverbookmarks, whose refusal is the guard.Also from #121, restated here: a human correction must not clear a failure counter or a last-outcome column.
Resolution
One table,
poll_failures, holding a row only while a Series is failing — no columns onseries, no counter, no history. Two new filter names,failing(ninth) andunverified(eleventh). The per-Site outcome counts stay unlinked for ever.
Migration 0017
The row's existence is the state. A row means "the last Poll that learned anything about
this Series failed"; no row means it succeeded, or the Series was never polled
(
latest_checked_at = 0separates that), or every Poll so far ended in an outcome the writerule below ignores. So there is no
''success sentinel, noCHECKkeeping two columnsconsistent, and
failing_sinceis never zero.3NF was checked against both designs and does not decide between them: the key is
(site, series_id)either way, the relation is 1:1, and the one rule between the attributes(a word implies an age) yields no value, so it is a constraint and not a transitive
dependency — both designs are in BCNF. A 1:1 table with the same key is vertical
partitioning, not normalisation, and a lookup table for the outcome vocabulary would be a
domain constraint a Go constant already enforces. The table was chosen on its own merits:
seriesstays narrow, the sentinel disappears, and the guarded write gets simpler.The cascade is #125's requirement and reaches this table only — never
bookmarks, whoserefusal is the orphan guard, so
DELETE FROM seriesstays one statement.The write: one statement per Poll, a row written only on a change
failing_sincesurvives a change of word becauseDO UPDATEnever touches it — the age isthe age of the run of failures, not of the current word. The same failure repeating writes
nothing; a healthy Series polling correctly writes nothing. The extra cost lands on
transitions, not on Polls. This is why no
last_outcome_atcolumn exists: it would sitwithin seconds of
latest_checked_at, which is already stored.Rejected: three columns on
series(last_poll_outcome,failing_since, and aconsecutive-failure counter). The counter goes because it is a proxy for duration measured in
units that differ per Site — each Lane carries its own
Rest, and a paused Lane stops thecount without stopping the failure.
failing since <age>is the fact the owner reads.Which outcomes write, and which are silent
The five-word taxonomy #117 fixed is reused verbatim, plus one split (below). But only
outcomes that are evidence about this Series touch the table:
no_chapterlatestChapterFromfound nothing (read.go:70, poller.go:458) — the Series' page or our adapterunfetchableseries_urlfailedfetchableSeriesURL(read.go:44) — the Series' own addressnot_founderrorsMarkLatestCheckedrefusederrChallengeHeld: 403 or an interstitial body (read.go:55-69) — the Site's mood, identical for every Series it hostsunreachableerrBrowserInterrupted— our own sidecar, and nothing was readNeither insert nor delete for those two, and the distinction is load-bearing in both
directions. Writing a row would mark a whole library as failing when one Site refused for a
day. Deleting one would claim recovery when nothing was read — resetting
failing_sincetozero for a Series broken three months, so a single refusal from its Site would erase the age
that makes it findable. A
refused/unreachablePoll therefore issues one fewer query,and the Series keeps the last thing it truly learned about itself. Both facts are already
recorded where they belong: per Site in #117's
poll_lanes.refuse_until, and in thepoller's in-memory
browserDownAt.errorsis split:not_foundbecomes the sixth outcome word, for a 4xx status otherthan 403. Today one word covers a 404 (the page is gone — correct the address), a 503 (the
Site is busy — wait) and our own write failing (the database is unwell): three different
owner actions behind the one word the detail page prints. One branch in read.go, and #117's
poll_passesgains a sixth count column — free on paper, since nothing on this map is built.Not written by anything else, ever: a Forced Poll (#119) request clears nothing, because
a request is not evidence; a forced Poll that reads the page clears the row exactly as any
other correct read does, which needs no special case in the code. A hand correction (#121)
clears nothing, as #121 required.
The two filter names
failing(ninth): the joined row existsAND s.latest_chapter_num IS NOT NULL ANDf.failing_sinceolder thanownerWindow(12h, #117's constant — no new figure).latest_chapter_num IS NOT NULLis what keeps it disjoint from #118'sno-chapter(never once succeeded) rather than overlapping it for no gain. Per #124 there is
deliberately no
finished_at = 0guard: a Series already failing when the ownerfinished it keeps its evidence.
unverified(eleventh, ordered besidesighting-raised):s.latest_raised_by IS NOT NULLplus thefailingtest — the Latest Chapter came from aReader's Sighting and no Poll has confirmed it for over 12 hours. #121's correction
control on the detail page is the action it leads to.
A Series failing for under 12 hours is in neither filter. That is deliberate: one failure is
not a fault to correct, and #117's per-Site counts plus the Series' own detail page already
show it.
Rejected: a stored column for the
unverifiedpair. It would hold theANDofseries.latest_raised_by IS NOT NULLand a row in another table — derived data with fivewriters (a Sighting raising the Series, a Poll succeeding, a Poll failing, an owner
correction,
ClearSightingAttribution) for a value the join computes free. Unlike theseries-versus-table question above, this one is a genuine 3NF violation, and #116'sAdminSeriesacquires the join for other reasons anyway.What the pages render
One marker on the fact line of #122's two-line row,
<outcome word> · failing <age>, in--danger(#122's colour for trouble; never--ember), and the same line on the Seriesdetail page. Under-12h and over-12h rows look identical — the threshold decides list
membership only. A second visual state would ask the owner to learn a rule the page cannot
state in a word, and the age is already on the line for anyone who wants to judge it. The
unverifiedcase adds one sentence on the detail page beside the correction control.The per-Site counts stay unlinked — superseding #117 and #118
#117 pushed "the outcome counts should link somewhere, and that somewhere is a
failingfilter"; #118 wrote it as "all five point at
?site=…&filter=failing" once this ticketlanded. That promise is withdrawn. #118's own rule — a figure links to what it counts —
is the reason, and #118 already applied it once by refusing to link the
no_chaptercount tothe
no-chapterfilter.Two failures of the same test.
refusedandunreachablenow have no per-Series record atall, so two of the six counts would open an empty list beside a large number. And the other
four count attempts inside a 12h window while the filter lists Series failing now for over
12h: a Series that broke at 04:00 and healed at 05:00 is counted and not listed, while a
Series broken for three months on a paused Lane is listed and counted zero. Neither set
contains the other, in either direction.
Instead, each Site row on the Lanes page carries one separate link to
/admin/series?site=<site>&filter=failing, as navigation rather than attached to a number.The owner still gets from "this Site looks unwell" to "these Series are broken" in one click.
Rejected: a
?outcome=parameter to make the links exact — #118 fixed one filter at a timewith only
?site=/?kind=stacking, and a second axis would not fix the window mismatch.Explicit non-goal
DueForLatestCheckdoes not joinpoll_failures. A failing Series is polled at the samepace as any other; slowing a Lane for a persistent failure is Poll Lane pacing, not a
dashboard question, and nothing on this map needs it.
Amendments pushed out
AdminSeriesgainsLEFT JOIN poll_failures f USING (site, series_id);SeriesRowgains the outcome word andfailing_since. Filter vocabulary is now eleven.poll_passesgains anot_foundcount; the link push is withdrawn (above).failingis the ninth name as reserved,unverifiedthe eleventh; the linkpromise is withdrawn;
<select>(#122) carries both new options with counts.detail-page
unverifiedsentence.write targets
poll_failures, keyed on failure, while a completion hint is a fact about asuccessful read of
series. Two writes, or #130 finds its own home.Glossary: no new domain noun —
failingandunverifiedare filter names over factsCONTEXT.md already defines. An ADR is worth writing when this is implemented (a table whose
row is the failure state, and two outcomes that deliberately write nothing), not now.