Cross-Series read model and the privacy boundary #116
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of #114
Question
What does the admin Series list read, and what must it refuse to show?
There is no generic all-Series query today:
store.List/Getare per-readerID, andDueForLatestCheck(backend/internal/store/store.go) is the only cross-reader join. It alreadycomputes an anonymous
readerCount, which is the shape this view should follow.To decide:
seriesColumnsor needs its own projection. Only compile-time constants may be concatenated into query text.
needs a bound; whether that is a page size, a hard cap, or a search box is open.
is what
DueForLatestCheckandEligibleSeriesCountmean by eligible)?series.latest_raised_byis an FK toreadersandlatest_sighted_atsays when a Reader's report last stood in for a Poll (ADR-0011). That isper-Reader identity on a shared row, and the privacy boundary says Series-level facts plus
counts only. Does the Series view show that a Latest Chapter was Sighting-raised without
saying by whom, show the Reader, or omit it entirely — and if it is hidden, how does the owner
investigate a Series whose Latest Chapter looks wrong?
Added by #115
The read model has a second consumer: the landing page's stats block needs one grouped
aggregate over
series— per Site, and library-wide — counting no cover (cover_address = ''),never checked (
latest_checked_at = 0with aseries_url), stale over 24h, unpollable(
series_url = ''), and Sighting-raised (latest_sighted_at > latest_checked_at), plus themanga/novel split. One
COUNT(…) FILTER (WHERE …) … GROUP BY sitepass, not one query perfigure.
EligibleSeriesCountis the wrong shape for it: one Site per call, one number back.Filter state is URL-addressable (
/admin/series?filter=no-cover), so each filter this ticketsupports must be nameable in a query string.
Resolution
Two store methods, a dedicated row type that cannot carry Reader identity, one URL-addressable
filter enum, and a 12-hour staleness threshold. The privacy boundary is enforced by the
projection, not by a template.
Surface
Two methods, not one: the landing page reads only the aggregate and must not pay for 50 rows it
discards. The filter vocabulary is shared by being one set of named predicate constants, not by
being one method.
SeriesRow — the privacy boundary as a type
store.Seriesis not reused.seriesColumns(store.go:202) carriess.latest_raised_by, andscanSerieslands it onSeries.LatestRaisedBy, so reusing it would put Reader identity in theadmin handler and leave the boundary resting on a template that happens not to print a field.
SeriesRowhas no field for it:SightingRaisedanswers the attribution question: the owner learns that a reader's Sighting, nota Poll, set this Latest Chapter, and learns nothing about which Reader. It is computed in SQL so
no caller repeats the comparison.
latest_raised_bynever leaves the store package. Guard it theway api_test.go:597-600 guards
latest_checked_atagainstbookmarkColumns: a test asserting thenew column constant does not mention
latest_raised_by.Key()derivessite:series_idas everywhere else — the route key from #115.The row query
store.Deleteremoves a Bookmark and leaves theseriesrow, andnothing deletes one, so orphan Series accumulate. The Lane's
HAVING COUNT(*) > 0hides them;the dashboard's third job is Library-wide hygiene, so it must not.
reader_count = 0is theorphan marker, and it is an anonymous Series-level fact. Removal is its own ticket, #125.
COUNT, notCOUNT(*) FILTER (WHERE b.status <> 'finished'). Itcounts every Bookmark on the Series. This knowingly disagrees with the two Lane queries for as
long as the finished Lifecycle bucket exists; #124 removes that bucket, after which the two
definitions are the same set. Recording the divergence rather than hiding it: until #124 lands,
a Series every Reader finished shows a non-zero count and is never Polled.
latest_checked_at = 0, so ordering onthat column alone gives no stable page boundary and rows would repeat or vanish across pages.
query text. Site and Kind are
$Nparameters.?filter=no-covers.cover_address = ''uncheckeds.latest_checked_at = 0 AND s.series_url <> ''stales.latest_checked_at > 0 AND s.latest_checked_at <= $staleBeforeunpollables.series_url = ''sighting-raiseds.latest_sighted_at > s.latest_checked_atorphanHAVING COUNT(b.reader_id) = 0unpollablemeansseries_url = ''only. The second unpollable case — aseries_urlwhosehost fails the pin in
fetchableSeriesURL(poller.go:555) — is invisible to SQL, needs the Siteregistry in Go, and would break both the count and the pager if filtered after the read. It
belongs to #121, which repairs a wrong
series_url; it is a repair, not a hygiene count.An empty
series_urlis reachable and permanent: the column defaults to'',Upsertwritestitle/series_url/coveronly when the row is brand new (store.go:814, 858-869), and no Pollever writes it. So a PUT that omitted it creates a Series no client action can fix.
Paging and the total
50 rows per page,
?page=1-based. The total comes fromCOUNT(*) OVER ()in the same query:window functions run after
GROUP BY/HAVINGand beforeLIMIT, so the figure counts thefiltered groups, and one
WHEREclause cannot disagree with a second copy of itself.Safety, since the window count vanishes on an empty page: the handler treats zero rows with
Page > 1as an over-run and re-reads at page 1. That costs a second query only in the casethat would otherwise show a blank list with no pager.
Staleness
staleAfter = 12 * time.Hour, a constant in the web package, passed down asStaleBefore.Twelve hours, not 24: the owner checks once in the day and once at night, and each check should
cover the interval since the last one. This supersedes the "unchecked over 24h" figure in
#115's landing verdict line. It is a human threshold, deliberately not a multiple of
defaultRest, and the landing page reads the database, not the poller, so it cannot follow theLane's pace anyway.
Aggregate
SeriesStatsis one pass:GROUP BY s.sitewithCOUNT(*) FILTER (WHERE …)per hygiene classplus the manga/novel split. Library-wide totals are summed in Go over five Sites —
ROLLUPwouldadd a NULL-Site row every scanner has to special-case. The orphan count needs the Bookmark side,
so the query joins a grouped subquery (
LEFT JOIN (SELECT site, series_id FROM bookmarks GROUP BY site, series_id) b USING (site, series_id)) and countsFILTER (WHERE b.site IS NULL), ratherthan putting a subquery inside a
FILTER. Landing-page counts include orphans: they are exactlywhat needs attention.
Index
Migration
0012:CREATE INDEX IF NOT EXISTS series_latest_checked_at_idx ON series (latest_checked_at);serieshas only its primary key today (no migration creates an index). This one supports thedefault order and is a judgement, not a measurement — a
GROUP BYover a join may ignore it.Revisit with a timing on real data before adding a second.
Pushed onto other tickets
the stats block links, and how a filtered list presents itself.
series_urlrepair.From #119 (closed):
seriesgainsforce_poll_at bigint NOT NULL DEFAULT 0in migration 0013. SeriesRow projects it and exposes a derivedPollPending(force_poll_at > latest_checked_at) so the admin row can rendercheck requested <age> ago. Theuncheckedandstalefilters are unaffected — a Forced Poll never writeslatest_checked_at, which is exactly why the flag is its own column rather than a zeroed timestamp.Amendment from #117 (closed):
staleAfterbecomesownerWindow = 12 * time.Hour, one constant feeding both the stalenesscutoff specified here and #117's per-Site outcome window. Same 12h, same rationale (one look by
day, one by night, each covering the interval since the last); two differently-named 12h
constants on one page is how they drift apart.
StaleBeforeinSeriesFilteris unchanged —the store stays clockless.
latest_checked_atis stamped beforethe fetch (poller.go:428), so it means attempted. A Series broken for a month is
unchecked-clean,stale-clean andunpollable-clean. #127 owns the fix (per-Series outcomecolumns plus an eighth filter name).
sulthan referenced this issue2026-08-18 06:59:54 +07:00
Amendment from #118 (closed):
no-chapter, joins the enum as another compile-time predicateconstant:
s.latest_chapter_num IS NULL AND s.latest_checked_at > 0 AND s.series_url <> ''— aSeries the poller has attempted and never once read a chapter number from. Disjoint from
uncheckedby construction (latest_checked_at = 0vs> 0), so the two counts neverdouble-report a row.
SeriesStatsgains the matchingCOUNT(*) FILTER (WHERE ...)column, so the landing block canshow and link it. One more
FILTERin the pass that already exists — no new query shape.labels beside them.
sighting-raisedfilter carries more weight than this ticket gave it: #118 rejected a"Latest Chapter went backwards" filter, leaving
sighting-raisedas the only lens on a LatestChapter that may be wrong.
Amendment from #124 (closed):
ReaderCountdivergence you recorded is resolved. #124 deletes the finished Lifecycle bucket and both Lane queries dropHAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0, so your plainCOUNTand the set the Lanes poll are now the same set, exactly as you predicted.SeriesRowgainsFinishedAt int64(projected the way #119's amendment addedforce_poll_at), so the list row can show the state and the detail page can offer the undo.finished→s.finished_at > 0, andSeriesStatsgains the matchingCOUNT(*) FILTER (WHERE finished_at > 0).AND s.finished_at = 0:stale,unchecked,no-cover,no-chapter. A finished Series stops being checked, so without the guard it ages intostalefor ever.unpollable,orphanandsighting-raisedare untouched — each is still a real repair on a finished row.bookmarkColumnsgains a derived reader-facingfinished bool(s.finished_at > 0), a plain label with no filtering or ordering effect.finished_atitself never appears in a client-visible write path, andUpsert's explicitINSERT INTO series (...)column list is what enforces that — the same mechanism that already protectscover.Amendment from #127 (closed):
AdminSeriesgainsLEFT JOIN poll_failures f USING (site, series_id)(migration 0017), andSeriesRowgains two fields: the outcome word andfailing_since. Nothing onserieswas added — the row's existence inpoll_failuresis the failure state, so there is no success sentinel to project.no-chapter(eighth), #124finished(tenth), and #127 addsfailing(ninth) andunverified(eleventh).failing= the joined row existsAND s.latest_chapter_num IS NOT NULL AND f.failing_sinceolder thanownerWindow;unverified=s.latest_raised_by IS NOT NULLplus thefailingtest.unverifiedfree, and it is why #127 rejected storing that pair as a column: with the two facts in two tables, a stored copy would need five writers for a value the join computes.