Spec: admin dashboard - pages, Lane observability, poll pass log, and per-Series intervention #134

Closed
opened 2026-08-21 15:47:44 +07:00 by sulthan · 1 comment
Owner

Spec derived from the wayfinder map #114, which is fully charted. This is spec 1 of 4; the map's
decisions were split for implementability, not re-decided. Nothing here is new — every decision
below was settled in #115, #116, #117, #118, #119, #122 and #123, and this document is the
handoff-ready synthesis.

Blocks: nothing. Blocked by: nothing.
Later specs in the series build on this one: per-Series intervention, the Finished Series cutover,
and per-Series Poll knowledge plus owner notification.

Problem Statement

I own this deployment and I cannot tell whether it is working.

The admin page shows a Reader roster and a Poll Lane block, and the Lane block says "No data yet"
for up to an hour after every deploy, because Lane state lives in the poller's memory and dies with
the process. So the one page whose job is proving the poller is alive cannot answer the question
after the most common event in the system's life.

Worse, the fault that actually costs me something is invisible from every other surface. When a
Poll Lane stops working, the web UI still loads, every Bookmark still opens, Progress still syncs,
and Latest Chapter quietly stops moving. No ember lights, and a Lane that is stuck looks exactly
like a week when the Sites published nothing. There is no reason to open the admin page, which is
precisely why the fault survives.

I also cannot see my library as a whole. Every read of a Series is scoped to one Reader, so there
is no way to ask "how many Series have no Cover", "which ones has the poller never read a chapter
from", or "which ones does nobody hold any more". Those rows accumulate: removing a Bookmark
leaves the Series behind, and nothing ever deletes one.

And I cannot ask for anything. If one Series' Latest Chapter looks stuck I wait for its turn in the
Lane's hour. If a Site is being hammered or is misbehaving I have no way to stop its Lane short of
a redeploy with the whole poller switched off.

Solution

A four-page owner surface, plus the Lane state made durable underneath it.

Observability. The landing page leads with one line — a verdict, how many Series are waiting,
how many have not been checked in twelve hours — and then a stats block where every figure is also
the link to the list of what it counts. The Lanes page shows one row per Site: what its last pass
found waiting, how many it read, its pace, and, when it did no work, the reason it declined. That
reason is the whole difference between a Lane resting and a Lane stuck, and it is recorded rather
than guessed.

Library-wide hygiene. A filterable, bookmarkable Series list across every Reader's library at
once, with eight named hygiene filters. Each figure on the landing page is a door into the list it
counts. A zero prints as a digit and is not a link, because following it lands nowhere.

Intervention, through the database. Two controls that write a row the poller notices on its
next pass, and never command the poller: Check now on a Series, and Pause on a Lane with a
mandatory expiry. Both survive a restart, because both are facts about a Series or a Site rather
than about the running process, and both mean the whole surface stays testable with no poller
running at all.

Privacy. Every figure the owner sees is a Series-level fact plus an anonymous Reader count. The
owner never learns which Reader reads what, and the boundary is enforced by the shape of the type
the store returns, not by a template that happens not to print a field.

User Stories

  1. As the owner, I want the admin surface split into pages I can bookmark, so that I can send
    myself straight to the Series list rather than scrolling one long page.
  2. As the owner, I want one nav row across Overview, Lanes, Readers and Series, so that I always
    know where the rest of the surface is.
  3. As the owner, I want a single verdict line at the top of the landing page, so that I can decide
    in one glance whether to keep reading my library or start investigating.
  4. As the owner, I want the verdict line to say how many lanes need attention rather than just
    "healthy", so that I know the size of the problem before I click.
  5. As the owner, I want a virgin database to say that no Lane has reported yet rather than drawing
    confident zeroes, so that "nothing has happened" is never rendered as "everything is fine".
  6. As the owner, I want Lane state to survive a restart, so that the page still answers the
    question thirty seconds after a deploy.
  7. As the owner, I want to know what a Lane's last pass actually did, so that I can tell a Lane
    that read nothing because nothing was due from one that read nothing because it is broken.
  8. As the owner, I want a Lane that declined to work to tell me why it declined, so that a paused
    Lane, a refusing Site, a sleeping browser and a failed query are not all rendered as the same
    silence.
  9. As the owner, I want the one true stall — Series waiting, none read, no reason given — to be
    distinguishable from the eight other ways a pass can end early, so that I only investigate real
    faults.
  10. As the owner, I want named Poll outcome counts per Site rather than one "failures" number, so
    that a Site refusing me and my adapter reading no chapter are not the same fact.
  11. As the owner, I want a zero outcome count to read as "none observed", so that the page never
    claims a clean bill of health it cannot actually measure.
  12. As the owner, I want a Site's refusal to outlive a restart, so that restarting the backend does
    not immediately re-probe a Site that just told us to back off.
  13. As the owner, I want the browser sidecar's reachability to be re-probed after a restart, so that
    a Chrome I woke up is noticed rather than remembered as dead.
  14. As the owner, I want to see how far back the pass log reaches, so that I can ask what the poller
    was doing at four in the morning and not only what it is doing now.
  15. As the owner, I want the pass log pruned automatically, so that a table nobody reads does not
    grow on a small VPS.
  16. As the owner, I want a per-Site table on the landing page showing library shape, so that I can
    see which Site holds most of my collection.
  17. As the owner, I want a filterable list of every Series across every Reader, so that I can find
    broken rows without asking about one Reader at a time.
  18. As the owner, I want the filter state in the URL, so that I can bookmark "the Series with no
    cover" and come back to it.
  19. As the owner, I want to filter to Series with no stored Cover, so that I can find the rows that
    render a placeholder in everyone's library.
  20. As the owner, I want to filter to Series the poller has never checked, so that I can find rows
    that were created and then forgotten.
  21. As the owner, I want to filter to Series not checked in twelve hours, so that I can catch a Lane
    that has been quietly skipping work.
  22. As the owner, I want to filter to Series with no page to fetch, so that I can find the rows a
    PUT created without a series URL and that no client action can ever repair.
  23. As the owner, I want to filter to Series whose Latest Chapter came from a Reader's report rather
    than from a Poll, so that I have a worklist of numbers nothing has verified.
  24. As the owner, I want to filter to Series no Reader holds any more, so that I can see the rows
    that outlived every relationship to them.
  25. As the owner, I want to filter to Series the poller has attempted and never once read a chapter
    from, so that I can find an adapter that has never worked for a Site rather than one that broke
    yesterday.
  26. As the owner, I want every figure in the stats block to be the link to the list it counts, so
    that a count and its entry point are one control.
  27. As the owner, I want a zero to print as a digit and not as a link, so that I never click through
    to an empty list.
  28. As the owner, I want the filtered list's heading to state the count and the filter, so that the
    number the landing page promised and the number the list shows come from one query.
  29. As the owner, I want an empty filtered list to name the filter it is empty for, so that an empty
    hygiene list reads as good news rather than as a broken page.
  30. As the owner, I want to narrow the list by Site and by Library on top of a hygiene filter, so
    that I can act on one Site at a time without losing the filter.
  31. As the owner, I want the list paged with a stable order, so that rows do not repeat or vanish as
    I move between pages.
  32. As the owner, I want a per-Series page keyed by the same composite the rest of the system uses,
    so that its address is derivable from a row I am already looking at.
  33. As the owner, I want to ask for one Series to be checked now, so that I do not wait an hour to
    find out whether its page still reads.
  34. As the owner, I want Check now to tell me how long ago I asked and give me no estimate, so
    that the page does not pretend to know when a sleeping browser will wake.
  35. As the owner, I want an unserved request to keep ageing rather than expiring, so that an old
    pending marker is itself the evidence that a Lane is stuck.
  36. As the owner, I want a forced check to jump the ordinary waiting rules, so that asking by hand
    actually means something.
  37. As the owner, I want a forced check to never override a Site that is actively refusing us, so
    that my impatience cannot make a refusal worse.
  38. As the owner, I want a forced check to wake a sleeping browser, so that the wake thresholds
    exist to stop the machine waking itself, not to stop me.
  39. As the owner, I want the Check now control hidden on a Series with no page to fetch and on one
    no Reader holds, so that I am not offered a button that can never do anything.
  40. As the owner, I want to pause one Site's Lane for a bounded time, so that I can stop asking a
    Site that is having a bad day without stopping the other five.
  41. As the owner, I want a pause to require an expiry, so that I cannot leave a silent outage behind
    on the one surface whose job is proving the poller is alive.
  42. As the owner, I want a pause to survive a restart, so that it is a fact about the Site rather
    than about the process.
  43. As the owner, I want a paused Lane to read as paused rather than as stalled, so that my own act
    is not reported back to me as a fault.
  44. As the owner, I want a Reader's first bookmark of a Series on a paused Site to still read the
    page, so that pausing a Lane never breaks somebody adding a Series.
  45. As the owner, I want a resumed Lane to find its full queue waiting, so that a pause delays work
    rather than discarding it.
  46. As the owner, I want each action to answer with freshly rendered markup, so that the figures I
    see after a press describe the state after the press.
  47. As the owner, I want only the Lanes block to refresh on a timer, so that a half-open confirm row
    is never eaten by an auto-swap and the list does not re-sort between my press and my confirm.
  48. As a Reader, I want none of this to appear anywhere in my library, so that the owner's
    operations surface is not something I have to understand.
  49. As a Reader, I want the owner to be unable to see which Series I read or where I am in them, so
    that a dashboard for keeping the poller healthy is not a reading log.
  50. As the owner, I want to be told which Series a Reader's report raised without being told which
    Reader, so that I can act on a suspicious number without acquiring information I said I would
    not hold.
  51. As the owner, I want the surface to be readable on a phone, so that I can check the poller from
    the same device I read on.
  52. As the owner, I want the admin surface to look like the rest of the application, so that it does
    not read as a bolted-on tool.
  53. As the owner, I want no ember accent anywhere on the admin surface, so that the one accent that
    means "new chapter" keeps meaning only that.

Implementation Decisions

Access model and routes

  • requireOwner stays the entire access model: single owner, no roles, no second admin, no per-user
    grants. Every new route joins adminRoutes, so the owner gate and the route list cannot drift and
    AdminPatterns keeps covering all of them by construction.
  • Five pages plus one fragment plus the action verbs:
Route Kind Holds
GET /admin page verdict line, stats block, per-Site table (library shape only), nav row
GET /admin/lanes page per-Lane detail, pause controls; the only timer on the surface
GET /admin/readers page the existing roster and its two actions
GET /admin/series page filterable Series list; filter state in the query string
GET /admin/series/{key} page per-Series detail; key is site:series_id
GET /ui/admin/lanes fragment the self-refreshing Lanes block
POST /admin/series/{key}/poll action request a Forced Poll; answers with the swapped row
POST /admin/lanes/{site}/pause action duration from the form; answers with the swapped block
POST /admin/lanes/{site}/resume action answers with the swapped block
  • The two existing Reader actions keep their current top-level addresses. Renaming them for symmetry
    edits working templates for no gain.
  • Series detail is keyed by the composite site:series_id in one path segment, matching the shape
    the wire and store.Get already use. A surrogate id would need a column and a migration to save
    nothing; : is legal unescaped in a path segment and no observed series id carries /.
  • Bookmarkable pages under /admin/..., self-refreshing fragments under /ui/admin/..., actions
    answering with the swapped fragment — the shape renderRoster already uses.
  • The only entry point into admin stays the owner-gated link in the application chrome. No
    per-Series admin link from the library cards: that is a second gated branch in the reading
    templates for a hop the Series list already provides.

The page split rule

The landing page aggregates over Sites; the Lanes page shows per-Lane detail. No figure appears on
both.
(This replaces the earlier "landing from the database, Lanes from memory" rule, whose
premise was the in-memory state this spec deletes.)

Lane state moves into the database

  • New table for the pass log, one row per Lane Pass, append-only, roughly 120 rows a day across six
    Sites:
CREATE TABLE poll_passes (
  site        text    NOT NULL,
  ran_at      bigint  NOT NULL,  -- unix ms, from the poller's clock
  skip        text    NOT NULL,  -- '' = the pass ran; else why it returned early
  due         int     NOT NULL,
  checked     int     NOT NULL,
  gap_ms      bigint  NOT NULL,
  clamped     boolean NOT NULL,
  refused     int     NOT NULL,
  unreachable int     NOT NULL,
  no_chapter  int     NOT NULL,
  unfetchable int     NOT NULL,
  errors      int     NOT NULL,
  PRIMARY KEY (site, ran_at)
);

(site, ran_at) is the whole index budget: one goroutine per Lane writes sequentially, so the
pair is unique without a surrogate id, and it serves both reads — latest row per Site, and a
per-Site window sum. Retention scans, which at roughly 1.7k live rows is cheaper than a second
index.

  • New per-Site Lane state row, poll_lanes(site primary key, paused_until, refuse_until), both
    defaulting to zero. This is one row read at the top of a pass serving two gates.
  • Refusal becomes durable (poll_lanes.refuse_until) and the poller's in-memory refusal map is
    deleted. A refusal is the Site's mood and outlives our process. This changes today's behaviour,
    where a restart forgets a refusing Site and immediately re-probes it — deliberately.
  • The browser-down timestamp stays in memory. The sidecar being unreachable is a fact about our
    own reach, and a restart re-probing Chrome is correct behaviour rather than lost state.
  • Deleted: the poller's laneStates map, recordLaneState, LaneState, Status,
    LaneStatus(), the whole status file, and web.LaneReporter. With the store as the source, an
    admin test inserts a pass row rather than constructing a fake reporter. Browser configuration
    becomes "is BROWSER_WS_URL set" read in the web layer's config — strictly more accurate, since
    it describes the deployment rather than whether one goroutine happened to construct a fetcher.
    Browser reachability is derived: any browser Site whose latest pass carries unreachable > 0
    inside the refusal backoff of now, which needs that backoff constant exported from latest.

The skip enum — one value per return path

The Lane pass has nine exits and today a page sees only "due, nothing checked", which reads as a
stall in eight cases that are not one. One text column, one value per return path:

skip meaning
'' the pass reached the loop
paused the pause row was read at the top
refusing refusal backoff
sidecar-down a sibling browser Lane lost Chrome
no-fetcher browser Site, no browser configured, no fallback
due-query the due query failed
asleep under both browser wake thresholds
eligible-count the eligible count failed
nothing-eligible nothing eligible; sleeps a full rest

The stall rule follows from it: due > 0 AND checked = 0 AND skip = '' is the only true stall.
Everything else is a Lane that declined to work and said why.

Outcome counts

The classification the Series read already makes is counted on the pass row: refused (a challenge
held), unreachable (the browser interrupted), no_chapter (200, real HTML, no chapter found),
unfetchable (the host pin refused the stored URL, or no fetcher), errors (everything else —
non-200, transport, a failed check stamp).

  • no_chapter is its own count and is never folded into refused: it is what a broken adapter looks
    like when it breaks loudly.
  • A success count is derived, never stored: checked - (refused + no_chapter + unfetchable + errors). unreachable is excluded from that arithmetic because the sidecar-loss path returns
    before the checked counter increments. A stored success column would be a fifth way to get that
    wrong.
  • The page never prints the word "failures" bare. Named counts only, and a zero renders as none
    observed
    — the wording is the honesty, because the failure kind that would hurt most (an adapter
    parsing a wrong number successfully) is not in this taxonomy and never can be.
  • The five counts render unlinked: the pass row holds counts and never identities, so there is no
    list of the four refused Series to point at. A later spec revisits the navigation, not the counts.

Carry-forward

The existing rule moves verbatim: a pass that returned before computing its figures carries the
previous pass's due, gap, clamped and checked forward — exactly when the pass's own gap is zero
— so no row states a zero it did not measure and the read stays a single DISTINCT ON. With skip
recorded, a genuine zero beside skip = 'due-query' now reads correctly rather than as a
measurement, which is why the rule is kept rather than widened.

Rejected: nullable columns plus a read-side "last non-null per Site", and writing no row at all for
a skipped pass (a Lane refusing for a day would look like a Lane that stopped existing).

Store surface for Lane state

type LanePass struct {
	Site                 string
	RanAt                int64
	Skip                 string
	Due, Checked         int
	GapMS                int64
	Clamped              bool
	Refused, Unreachable, NoChapter, Unfetchable, Errors int
	PausedUntil, RefuseUntil int64 // joined from poll_lanes on read; not pass columns
}

func (s *Store) RecordLanePass(p LanePass, retainBefore int64) error
func (s *Store) LatestLanePass(site string) (LanePass, bool, error)
func (s *Store) LatestLanePasses() ([]LanePass, error)
func (s *Store) LanePassOutcomes(since int64) ([]SiteOutcomes, error)
func (s *Store) SetLaneRefusal(site string, until int64) error
func (s *Store) PauseLane(site string, until int64) error   // rejects a non-future expiry
func (s *Store) ResumeLane(site string) error               // zeroes paused_until; keeps the row
func (s *Store) PausedLanes() ([]LanePause, error)
func (s *Store) LanePausedUntil(site string) (int64, error)
func (s *Store) ForceSeriesPoll(site, seriesID string, at int64) error
  • RecordLanePass inserts and prunes in the same call — delete-on-insert, so the Lane goroutine
    is the pruner and no ticker enters a backend that has none (sessions already expire lazily at
    lookup for the same reason). retainBefore is caller-supplied, keeping the store clockless as the
    due query already is.
  • LatestLanePasses is DISTINCT ON (site) … ORDER BY site, ran_at DESC left-joined to the Lane
    row: one query for the Lanes page and the same rows the landing verdict sums.
  • LatestLanePass(site) is the carry-forward read — the recorder needs one Site, not six.
  • Retention is 14 days and is not the display window. The window is what the owner is shown;
    retention is how far back a question can reach. The two must not be collapsed.

Forced Poll

  • One column, series.force_poll_at bigint NOT NULL DEFAULT 0, unix ms, zero meaning never asked.
    Writing it again re-stamps the request time; the write is idempotent.
  • Pending is derived, never stored: force_poll_at > latest_checked_at. It clears itself with no
    second write and no sweeper, because the check stamp is written before the fetch — so the first
    attempt ends the pending state whatever the attempt returns. That stamp-before-fetch order is
    load-bearing
    ; changing it silently makes forced requests sticky.
  • No expiry. A request the Lane never reaches keeps ageing in the UI rather than vanishing.
  • The due query gains the flag in three places, and force_poll_at joins its GROUP BY list:
WHERE  s.site = $1
  AND  s.series_url <> ''
  AND  (s.latest_checked_at <= $2::bigint OR s.force_poll_at > s.latest_checked_at)
HAVING (COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
        OR s.force_poll_at > s.latest_checked_at)
   AND (COUNT(*) > 1
        OR s.latest_sighted_at <= $2::bigint
        OR s.latest_checked_at <= $3::bigint
        OR s.force_poll_at > s.latest_checked_at)
ORDER BY (s.force_poll_at > s.latest_checked_at) DESC, reader_count DESC, s.latest_checked_at ASC

Note: the HAVING … status <> 'finished' clause is today's Lifecycle test and is deleted by
the Finished Series spec later in this series. Implement it as it stands here; that spec replaces
it with a series.finished_at test rather than amending it.

  • A Forced Poll overrides the rest cutoff, the Sighting-deferral clause, and the finished-only
    bucket. It never overrides an empty series URL (nothing to fetch), the Bookmarks join (a
    Series no Reader holds has no consumer for the result), the Lane's refusal backoff (hand-forcing a
    request at a Site that is actively refusing is the one move that makes it worse), the sidecar-down
    skip, or the Lane's gap.
  • One pass-level gate it does open: the browser wake thresholds. A forced Series wakes a sleeping
    Chrome — those thresholds exist to stop the machine waking itself for one unattended check, and a
    human asking is not that. If the home machine is off, nothing happens and the request ages
    visibly.
  • Rejected: zeroing latest_checked_at as the force signal (it corrupts the never-checked and stale
    counts the landing page exists to show, and makes a pending marker impossible).

Paused Lane

  • The pause row is read at the top of a pass, ahead of the refusal check. A paused Lane records its
    pass with skip = 'paused' and sleeps until the expiry. Its Series stay due and unstamped — the
    identical state a missing browser leaves them in, so nothing new has to handle it and the queue is
    intact when the pause lifts.
  • Expiry is mandatory, offered as 1h / 6h / 24h.
  • No global runtime pause. The deploy-time poll switch stays the only whole-poller stop; the case
    for a runtime one is the case where you already have a shell.
  • Acquisition is unaffected. A Reader's first bookmark of a Series on a paused Site still reads
    the page. Pause governs the Lane only.
  • Rejected: an indefinite pause, and a site = '*' pseudo-row for a global one (a second meaning for
    the primary key of a six-row table).

The cross-Series read model

Two store methods over a dedicated row type. The privacy boundary is the projection, not a template.

type SeriesFilter struct {
	Site        string // "" = every Site
	Kind        string // "" = both Libraries
	Filter      string // one of the eight names below; unknown filters nothing
	StaleBefore int64  // caller-supplied epoch-ms cutoff; the store stays clockless
	Page        int    // 1-based
}

func (s *Store) AdminSeries(f SeriesFilter) (rows []SeriesRow, total int, err error)
func (s *Store) SeriesStats(staleBefore int64) ([]SiteStats, error)

type SeriesRow struct {
	Site, SeriesID   string
	Title, Kind      string
	LatestChapter    string
	LatestChapterNum *float64
	LatestCheckedAt  int64
	ForcePollAt      int64
	PollPending      bool // force_poll_at > latest_checked_at
	SightingRaised   bool // latest_sighted_at > latest_checked_at
	HasCover         bool // cover_address <> ''
	Pollable         bool // series_url <> ''
	ReaderCount      int
}
  • Two methods, not one. The landing page reads only the aggregate and must not pay for fifty rows
    it discards. The filter vocabulary is shared by being one set of named predicate constants, not by
    being one method.
  • store.Series is deliberately not reused. Its column list carries the Reader id that raised a
    Sighting, so reusing it would put Reader identity in the admin handler and leave the boundary
    resting on a template that happens not to print a field. SeriesRow has no field for it, and
    latest_raised_by never leaves the store package. SightingRaised answers the attribution
    question — the owner learns a Reader's report set this number, and nothing about which Reader — and
    is computed in SQL so no caller repeats the comparison.
  • The row query:
SELECT <adminSeriesColumns>, COUNT(b.reader_id) AS reader_count, COUNT(*) OVER () AS total
  FROM series s
  LEFT JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id
 WHERE ($1::text = '' OR s.site = $1) AND ($2::text = '' OR s.kind = $2) [AND <predicate>]
 GROUP BY <the projected series columns>
[HAVING COUNT(b.reader_id) = 0]
 ORDER BY s.latest_checked_at ASC, s.site, s.series_id
 LIMIT 50 OFFSET $n
  • LEFT JOIN, not JOIN. Removing a Bookmark leaves the Series row and nothing deletes one, so
    orphans accumulate; the Lane's join hides them and the dashboard's third job is hygiene, so it must
    not. reader_count = 0 is the orphan marker, and it is an anonymous Series-level fact.
  • Reader count is a plain COUNT, every Bookmark on the Series. This knowingly disagrees with the
    two Lane queries for as long as the finished Lifecycle bucket exists; the Finished Series spec
    removes that bucket, after which the two definitions are the same set. Until then, a Series every
    Reader finished shows a non-zero count and is never Polled. Recorded rather than hidden.
  • The tie-break is mandatory. Every unpollable Series has a zero check timestamp, so ordering on
    that column alone gives no stable page boundary and rows would repeat or vanish across pages.
  • Predicates are compile-time constants selected by the filter name; the name never reaches query
    text. Site and Kind are bound parameters. Only compile-time constants may be concatenated into
    query text — that rule is not relaxed here.
  • 50 rows a page, ?page= 1-based. The total comes from COUNT(*) OVER () in the same query: window
    functions run after grouping and before the limit, so the figure counts the filtered groups and one
    where-clause cannot disagree with a second copy of itself. Because that count vanishes on an empty
    page, the handler treats zero rows with page > 1 as an over-run and re-reads at page 1.
  • SeriesStats is one grouped pass — COUNT(…) FILTER (WHERE …) per class grouped by Site, plus the
    Library split — not one query per figure. Library-wide totals are summed in Go over six Sites;
    ROLLUP would add a null-Site row every scanner has to special-case. The orphan count needs the
    Bookmark side, so the query left-joins a grouped subquery and counts where that side is null rather
    than putting a subquery inside a FILTER. Landing counts include orphans: they are exactly what
    needs attention. The existing eligible-count method is the wrong shape for this — one Site per call,
    one number back.
  • One index: series (latest_checked_at). The table has only its primary key today. This supports the
    default order and is a judgement, not a measurement — a grouped query over a join may ignore it.
    Re-time on real data before adding a second.

The filter vocabulary — eight names

?filter= predicate label
unpollable s.series_url = '' No series URL
no-chapter s.latest_chapter_num IS NULL AND s.latest_checked_at > 0 AND s.series_url <> '' Never read a chapter
orphan HAVING COUNT(b.reader_id) = 0 No Readers
unchecked s.latest_checked_at = 0 AND s.series_url <> '' Never checked
stale s.latest_checked_at > 0 AND s.latest_checked_at <= $staleBefore Not checked in 12h
no-cover s.cover_address = '' No cover
sighting-raised s.latest_sighted_at > s.latest_checked_at Latest from a Reader
unknown or absent none All series
  • Ordered permanent-and-fixable first. unpollable and orphan are labelled as the repair they need
    rather than as the SQL they are: the owner arrives to act, not to admire a predicate.
  • unpollable means an empty series URL only. The second unpollable case — a URL whose host fails
    the fetch gate — is invisible to SQL, needs the Site registry in Go, and would break both the count
    and the pager if filtered after the read. It is a repair, not a hygiene count, and belongs to the
    intervention spec. Note the empty case is reachable and permanent: the column defaults to empty, the
    Upsert writes the series URL only when the row is brand new, and no Poll ever writes it, so a PUT
    that omitted it creates a Series no client action can fix.
  • no-chapter and unchecked are disjoint by construction (zero versus non-zero check stamp), so the
    two counts never double-report a row. no-chapter is named to match the pass-level no_chapter
    count deliberately: same observation, one durable on the row, the other counted over a window.
  • Rejected outright: a "Latest Chapter went backwards" filter. Not implementable — the row holds
    only the current number, the poller writes downward unconditionally on any inequality, and a
    Reader's PUT can lower it too, so detection needs the fact captured at write time in a new column.
    And not wanted — a downward write is the correction, not the fault (the poller tests seed 400 and
    assert 296 after a poll), so the filter would flag precisely the Series that just healed. Worse, it
    misses the case that actually costs something: an adapter reading a stable wrong number every hour
    never moves, so nothing ever fires. sighting-raised stays the only honest suspicion lens, and hand
    repair is the intervention spec's job.

The landing page

  • Verdict line: <verdict> · N series waiting · M unchecked over 12h, fully database-computed.
    N sums due over the latest pass per Site; M counts Series with a series URL whose check stamp
    is older than the window. The verdict reads "All lanes healthy" when nothing needs attention, else
    " lanes need attention"; with no pass rows at all it says no Lane has reported rather than
    "healthy". That state does not disappear — it narrows from after every deploy to a virgin
    database
    .
  • One window constant, ownerWindow = 12 * time.Hour, in the web package, feeding both the
    staleness cutoff and the outcome window. Twelve hours because the owner looks once by day and once
    by night and each look should cover the interval since the last. It is a human threshold,
    deliberately not a multiple of the Lane's rest, and the page reads the database rather than the
    poller so it cannot follow the Lane's pace anyway. Two differently-named twelve-hour constants on
    one page is how they drift apart.
  • Stats block: library size and the manga/novel split, the roster count, the eight hygiene counts,
    library-wide and per Site.
  • Every figure is an entry point, not only the hygiene ones: total to the unfiltered list, the
    Library split to ?kind=, a per-Site row label to ?site=, the roster count to the Readers page,
    each hygiene count to ?filter=, each per-Site hygiene count to both. A zero renders the digit
    and is not a link
    — the figure stays, because a measured zero is a real fact, but no anchor,
    because following it lands on an empty list.
  • The per-Site table on the landing page carries library shape only. The five Poll outcome sums
    live on the Lanes page; on the landing they were nine columns of zeros burying the four columns that
    move.
  • No aggregate problem count. The classes overlap — one orphaned Series with no URL and no cover
    is three counts and one row — so a sum over-reports while a distinct count is a number nothing can
    be done about. The stats block is the whole hygiene surface and every figure on it is individually
    actionable.

Refresh

Only the Lanes block refreshes, on the existing 30s timer, now a six-row primary-key read rather than
a map copy. Three reasons the other pages have none: the Lane rest is an hour, so the check stamp
moves at that granularity and a 30s timer would re-run a cross-Series join roughly 120 times an hour
to redraw the same rows; an auto-swap on an action surface eats a half-open confirm row or re-sorts
the list between the press and the confirm; and a Forced Poll is asynchronous by construction, so a
timer would show "nothing yet" for up to an hour. A manual refresh button stays available later as one
attribute; it is not bought now.

Visual surface

The artifact to port is the Claude Design project BookmarkManager Web UI
(969ac210-fe02-4c01-ae1b-9a271dcc779a), files admin.css and admin-overview.html,
admin-lanes.html, admin-readers.html, admin-series.html, admin-series-detail.html. The
three-variant sketch on branch prototype/admin-surface-122 is superseded exploration — variant B
won; port from the design project, not from that branch.

  • One new token, --measure-wide: 1080px, admin only; the 760px reading measure is unchanged and
    admin is the only consumer of the wide one.
  • Row shape: aligned borderless columns. One CSS grid per table, a hairline header of mono
    uppercase labels, hairline row rules, no vertical rules, no card backgrounds, no corners.
    Gutters are cell padding, never a column gap — a gap slices the row rule into segments and reads as
    a ragged edge. Figures are tabular mono in mixed case at roughly zero tracking; uppercase at wide
    tracking is for labels and marks only. Titles stay in the display face.
  • The Series list is the two-line form: a subgrid row with the title on its own full-width line
    and the facts on the line under it, so a long title never breaks column rhythm. Separation is
    banding rather than hairlines, so a title and its facts read as one record; banding is a class,
    not an nth-of-type, because collapsed confirm rows are row siblings and would throw the
    alternation off. Lanes and the landing per-Site table are single-line rows; their numerics are
    centred rather than right-flushed, because at column width a right-flushed figure loses its header.
  • Notes chips cap at two plus a faint +N tail rather than stacking.
  • Actions are right-aligned 12px ghosts, hovering in the patina accent. Check now is
    unconfirmed, and its ageing pending marker renders once, on the title line flush right above
    the actions — under the action it added a second line to every row. Destructive actions use the
    danger ghost and open an in-place confirm row: a full-width grid row on the danger wash with
    Keep / Remove, shipping hidden and unmounted, or every eligible row prints an empty striped
    band.
  • No ember anywhere on admin. Patina is the accent; danger is for lane trouble and destruction; a
    deliberate pause stays patina. Both colour branches are touched together.
  • Nav row of four display-face tabs with the active tab underlined; Library and Log out group as a
    right-aligned cluster in the topbar. Section heads are mono uppercase led by a 34px patina tick, not
    a full-width hairline — stacked rules competed with the tables' own rules. The verdict line is set
    in the mono data face at 15px, not the display face: it is three counts, not a page title.
  • Stats block is an auto-fit grid at minmax(232px, 1fr), one rule on the block, zeros in the muted
    colour and unlinked.
  • Lanes columns: Site / Due / Checked / Gap / Last pass / outcomes-and-state / controls. Browser
    reachability is a status line pinned to the section heading, not a paragraph. A trouble state
    renders danger; a pause renders patina as paused · resumes in 3h. The pause select and Pause
    live in one bar and Resume replaces them in the same slot.
  • Series detail: back ghost, 28px display title, mono key line site:series_id · site · kind, a 160px
    hatched cover, an uppercase mono meta row carrying the marks, then a two-column grid reserved for
    the intervention forms the next spec adds, with Check now below.
  • Readers keeps its existing shape, re-set to the table's type tiers, with a reserved right-aligned
    action cluster so a chip cannot make one row taller.
  • Phone: Lanes stacks at ≤1019px (its phrase run cannot hold a fractional track sooner), every
    other table and the detail grid at ≤899px; stacked rows flex-wrap with the title full-width and
    actions pushed right.

The filter control

A <select> whose option labels carry their counts (No cover (3)), not a row of eight plain
links — eight hygiene labels do not read as a chip row. Site is a second select; Library is a
three-way segmented row marked active in patina. Filters stay one-at-a-time and URL-addressable.

Frontend dependencies

Nothing new. The vendored htmx is 2.0.4 and every primitive this surface needs is in that build
and already used in tree: hx-get plus hx-push-url for URL-addressable filters (the reading tabs
already prove it, and the docs' requirement that a pushed URL render a full page is satisfied by these
handlers), hx-trigger="keyup changed delay:500ms" for a debounced search, hx-target plus
hx-swap="outerHTML" or an out-of-band swap for a single-row re-render, hx-trigger="every 30s"
scoped to its own fragment for the Lanes block, and either hx-confirm or the existing inline confirm
row for gating. htmx has no paging attribute: load-more is a beforeend swap and numbered pages are
the tab pattern plus a pushed URL — same primitives, different swap and URL semantics, so paging is an
IA decision rather than a capability limit.

The existing client-side filter script is not reusable here. It is a pure title filter over an
already-rendered list, it issues no requests, it changes no URL, and it is not loaded on the admin page
at all. Client-side filter state cannot be bookmarkable without hand-written history calls, which is
exactly what these routes must avoid — so filtering is server-side. Research note:
docs/research/htmx-filterable-admin-series-list.md on branch research/htmx-series-list; delete the
branch once this spec is implemented.

Testing Decisions

What makes a good test here: it asserts observable behaviour at a seam the owner or a Reader can
actually reach — a rendered page, a store method's returned rows, a poller pass's effect on the
database — and it fails on a plausible bug rather than on a refactor. No test asserts a template's
internal structure, a private helper's shape, or a log line's wording unless that wording is the
contract (the none observed zero is).

The primary seam is the router, and it is an existing one: the package-level web tests build a real
router over a throwaway Postgres and drive it with an owner session cookie. Every admin page, filter,
action, empty state and gate is asserted through a request and its rendered response. This spec
reduces the seam count
: deleting LaneReporter deletes the fakeLanes fake, and a Lanes-page test
inserts a pass row instead of constructing one.

Modules and what is tested at each:

  • Router / web (existing seam, prior art: the admin roster, owner-gate and Lane-status tests):
    every new route is owner-gated (the existing pattern-driven gate test covers it by construction once
    the routes join the route list); the verdict line's three states, including no-passes-yet; each of
    the eight filters returns the rows it names and no others; a zero figure renders unlinked; the
    heading's count agrees with the row count; each empty state names its filter; ?site= and ?kind=
    stack on a filter without dropping it; page 2 of a one-page result re-reads at page 1; Check now is
    absent on a Series with no URL and on one with no Readers; a pause renders as paused rather than as
    stalled; a Lane with a skip value renders its reason rather than a stall; the outcome counts render
    named with none observed at zero.
  • Store (existing seam, prior art: the existing store tests over a throwaway Postgres with real
    migrations): each filter predicate against seeded rows, including the orphan case that only a left
    join can see and the disjointness of no-chapter and unchecked; the paging tie-break with several
    rows sharing a zero check stamp; the window total agreeing with the filtered group count; the
    aggregate agreeing with the row query for the same filter; RecordLanePass pruning on insert;
    LatestLanePasses returning one row per Site; the pause upsert rejecting a non-future expiry and
    ResumeLane zeroing rather than deleting the row.
  • The privacy boundary gets its own test, modelled on the existing test that guards the Bookmark
    column list: assert the admin column constant does not mention latest_raised_by, and that
    SeriesRow has no field for it. This is the one place a template-only guarantee would rot silently.
  • Poller (existing seam, prior art: the round-at-a-time pass tests with a fake fetcher and an
    injected clock): a pass records exactly one row with the right skip for each of the nine return
    paths; carry-forward fires exactly when the pass's own gap is zero; a paused Lane makes no fetch and
    leaves its Series due and unstamped; a durable refusal is honoured across a fresh poller; a forced
    Series is fetched ahead of the rest cutoff, the Sighting deferral and the finished bucket, and is
    not fetched through a refusal backoff, an empty URL or the Bookmarks join; a forced Series wakes
    a sleeping browser Lane; the pending flag self-clears on the check stamp with no second write.

No new fake, no new interface, no framework. The only test-visible seam this spec adds is the store
itself, which the poller already holds.

Out of Scope

  • Multiple admins, a role column, or per-user admin grants. The single-owner model is the
    boundary.
  • Inspecting an individual Reader's library or reading progress. Ruled out by the privacy
    boundary; only anonymous counts cross into admin views.
  • Metrics, charts, or a timeseries store. A swapless 1974 MiB VPS running Postgres argues against
    it, and no decision on this map needs a graph.
  • Per-Series history of any kind. The pass log is per Lane. A per-Series check log was rejected on
    cost and, more importantly, as insufficient: a pass that checked nothing writes nothing, so "the
    Lane ran, everything was rested, all healthy" — the primary health signal — would be invisible.
  • A free-text last-error column. It becomes the thing that gets read instead of the log, and it
    cannot be aggregated.
  • An admin audit trail. Different provenance (machine observation versus human action), different
    retention instinct, and a Forced Poll is already self-evidencing — the unserved request ages
    visibly. A 14-day pass log would silently expire the one thing worth looking back on.
  • A global runtime poller pause. The deploy-time switch stays the only whole-poller stop.
  • Bulk rewrite of series URLs across a whole Site. A host change invalidates every row of a Site
    at once; that is a SQL migration, and the dashboard's library-wide job is finding broken rows, not
    batch-repairing them.
  • Per-Series correction, Cover replacement, orphan removal, the Finished Series, per-Series failure
    state, the completion hint, and outbound notification.
    All specified, all in the three later specs
    in this series.

Further Notes

  • Migration numbers are claimed in landing order, not in map order. Measured 2026-08-21: the tree's
    migrations and ADRs both stop at 0011, and everything the map specified (the map called them
    0012–0019) is unbuilt. This spec's schema work is: the series (latest_checked_at) index;
    series.force_poll_at; poll_lanes(site, paused_until, refuse_until); and poll_passes. Take the
    next free numbers in whatever order they land — migrations are globbed by version, so contiguity at
    merge time is what matters, not agreement with the map's paper numbering.
  • Two ADRs are worth writing with the implementation, not before: persisted Lane state plus the
    deletion of its in-memory twin, and the through-the-database command seam with its queue-jump rules.
  • Glossary: CONTEXT.md already names Lane Pass, Forced Poll, Paused Lane, Stall, Correction and
    Orphan Series — those edits landed while charting. Nothing new is needed here.
  • Two figures in the earlier charting were superseded and the final values are above: staleness is
    12h, not 24h, and the display window and retention are 12h and 14 days, two windows on
    purpose.
  • The stall test in this spec is amended by the notification spec later in the series: it gains
    AND refused = 0, because the twice-refused break inside the pass loop is not an early return and
    so writes due > 0, checked = 0, skip = ''. Implement the test as stated here; that spec carries
    the amendment with its own tests.
  • Security-critical surfaces touched: the owner gate (every new route must join the route list rather
    than checking inside itself), and the store's query construction (only compile-time constants may be
    concatenated). Form bodies on the action routes are capped the way the API path caps them. Say which
    invariant you preserved in the PR and run the full backend test suite before calling it done.
Spec derived from the wayfinder map #114, which is fully charted. This is spec 1 of 4; the map's decisions were split for implementability, not re-decided. Nothing here is new — every decision below was settled in #115, #116, #117, #118, #119, #122 and #123, and this document is the handoff-ready synthesis. Blocks: nothing. Blocked by: nothing. Later specs in the series build on this one: per-Series intervention, the Finished Series cutover, and per-Series Poll knowledge plus owner notification. ## Problem Statement I own this deployment and I cannot tell whether it is working. The admin page shows a Reader roster and a Poll Lane block, and the Lane block says "No data yet" for up to an hour after every deploy, because Lane state lives in the poller's memory and dies with the process. So the one page whose job is proving the poller is alive cannot answer the question after the most common event in the system's life. Worse, the fault that actually costs me something is invisible from every other surface. When a Poll Lane stops working, the web UI still loads, every Bookmark still opens, Progress still syncs, and Latest Chapter quietly stops moving. No ember lights, and a Lane that is stuck looks exactly like a week when the Sites published nothing. There is no reason to open the admin page, which is precisely why the fault survives. I also cannot see my library as a whole. Every read of a Series is scoped to one Reader, so there is no way to ask "how many Series have no Cover", "which ones has the poller never read a chapter from", or "which ones does nobody hold any more". Those rows accumulate: removing a Bookmark leaves the Series behind, and nothing ever deletes one. And I cannot ask for anything. If one Series' Latest Chapter looks stuck I wait for its turn in the Lane's hour. If a Site is being hammered or is misbehaving I have no way to stop its Lane short of a redeploy with the whole poller switched off. ## Solution A four-page owner surface, plus the Lane state made durable underneath it. **Observability.** The landing page leads with one line — a verdict, how many Series are waiting, how many have not been checked in twelve hours — and then a stats block where every figure is also the link to the list of what it counts. The Lanes page shows one row per Site: what its last pass found waiting, how many it read, its pace, and, when it did no work, the reason it declined. That reason is the whole difference between a Lane resting and a Lane stuck, and it is recorded rather than guessed. **Library-wide hygiene.** A filterable, bookmarkable Series list across every Reader's library at once, with eight named hygiene filters. Each figure on the landing page is a door into the list it counts. A zero prints as a digit and is not a link, because following it lands nowhere. **Intervention, through the database.** Two controls that write a row the poller notices on its next pass, and never command the poller: *Check now* on a Series, and *Pause* on a Lane with a mandatory expiry. Both survive a restart, because both are facts about a Series or a Site rather than about the running process, and both mean the whole surface stays testable with no poller running at all. **Privacy.** Every figure the owner sees is a Series-level fact plus an anonymous Reader count. The owner never learns which Reader reads what, and the boundary is enforced by the shape of the type the store returns, not by a template that happens not to print a field. ## User Stories 1. As the owner, I want the admin surface split into pages I can bookmark, so that I can send myself straight to the Series list rather than scrolling one long page. 2. As the owner, I want one nav row across Overview, Lanes, Readers and Series, so that I always know where the rest of the surface is. 3. As the owner, I want a single verdict line at the top of the landing page, so that I can decide in one glance whether to keep reading my library or start investigating. 4. As the owner, I want the verdict line to say how many lanes need attention rather than just "healthy", so that I know the size of the problem before I click. 5. As the owner, I want a virgin database to say that no Lane has reported yet rather than drawing confident zeroes, so that "nothing has happened" is never rendered as "everything is fine". 6. As the owner, I want Lane state to survive a restart, so that the page still answers the question thirty seconds after a deploy. 7. As the owner, I want to know what a Lane's last pass actually did, so that I can tell a Lane that read nothing because nothing was due from one that read nothing because it is broken. 8. As the owner, I want a Lane that declined to work to tell me why it declined, so that a paused Lane, a refusing Site, a sleeping browser and a failed query are not all rendered as the same silence. 9. As the owner, I want the one true stall — Series waiting, none read, no reason given — to be distinguishable from the eight other ways a pass can end early, so that I only investigate real faults. 10. As the owner, I want named Poll outcome counts per Site rather than one "failures" number, so that a Site refusing me and my adapter reading no chapter are not the same fact. 11. As the owner, I want a zero outcome count to read as "none observed", so that the page never claims a clean bill of health it cannot actually measure. 12. As the owner, I want a Site's refusal to outlive a restart, so that restarting the backend does not immediately re-probe a Site that just told us to back off. 13. As the owner, I want the browser sidecar's reachability to be re-probed after a restart, so that a Chrome I woke up is noticed rather than remembered as dead. 14. As the owner, I want to see how far back the pass log reaches, so that I can ask what the poller was doing at four in the morning and not only what it is doing now. 15. As the owner, I want the pass log pruned automatically, so that a table nobody reads does not grow on a small VPS. 16. As the owner, I want a per-Site table on the landing page showing library shape, so that I can see which Site holds most of my collection. 17. As the owner, I want a filterable list of every Series across every Reader, so that I can find broken rows without asking about one Reader at a time. 18. As the owner, I want the filter state in the URL, so that I can bookmark "the Series with no cover" and come back to it. 19. As the owner, I want to filter to Series with no stored Cover, so that I can find the rows that render a placeholder in everyone's library. 20. As the owner, I want to filter to Series the poller has never checked, so that I can find rows that were created and then forgotten. 21. As the owner, I want to filter to Series not checked in twelve hours, so that I can catch a Lane that has been quietly skipping work. 22. As the owner, I want to filter to Series with no page to fetch, so that I can find the rows a PUT created without a series URL and that no client action can ever repair. 23. As the owner, I want to filter to Series whose Latest Chapter came from a Reader's report rather than from a Poll, so that I have a worklist of numbers nothing has verified. 24. As the owner, I want to filter to Series no Reader holds any more, so that I can see the rows that outlived every relationship to them. 25. As the owner, I want to filter to Series the poller has attempted and never once read a chapter from, so that I can find an adapter that has never worked for a Site rather than one that broke yesterday. 26. As the owner, I want every figure in the stats block to be the link to the list it counts, so that a count and its entry point are one control. 27. As the owner, I want a zero to print as a digit and not as a link, so that I never click through to an empty list. 28. As the owner, I want the filtered list's heading to state the count and the filter, so that the number the landing page promised and the number the list shows come from one query. 29. As the owner, I want an empty filtered list to name the filter it is empty for, so that an empty hygiene list reads as good news rather than as a broken page. 30. As the owner, I want to narrow the list by Site and by Library on top of a hygiene filter, so that I can act on one Site at a time without losing the filter. 31. As the owner, I want the list paged with a stable order, so that rows do not repeat or vanish as I move between pages. 32. As the owner, I want a per-Series page keyed by the same composite the rest of the system uses, so that its address is derivable from a row I am already looking at. 33. As the owner, I want to ask for one Series to be checked now, so that I do not wait an hour to find out whether its page still reads. 34. As the owner, I want *Check now* to tell me how long ago I asked and give me no estimate, so that the page does not pretend to know when a sleeping browser will wake. 35. As the owner, I want an unserved request to keep ageing rather than expiring, so that an old pending marker is itself the evidence that a Lane is stuck. 36. As the owner, I want a forced check to jump the ordinary waiting rules, so that asking by hand actually means something. 37. As the owner, I want a forced check to never override a Site that is actively refusing us, so that my impatience cannot make a refusal worse. 38. As the owner, I want a forced check to wake a sleeping browser, so that the wake thresholds exist to stop the machine waking itself, not to stop me. 39. As the owner, I want the *Check now* control hidden on a Series with no page to fetch and on one no Reader holds, so that I am not offered a button that can never do anything. 40. As the owner, I want to pause one Site's Lane for a bounded time, so that I can stop asking a Site that is having a bad day without stopping the other five. 41. As the owner, I want a pause to require an expiry, so that I cannot leave a silent outage behind on the one surface whose job is proving the poller is alive. 42. As the owner, I want a pause to survive a restart, so that it is a fact about the Site rather than about the process. 43. As the owner, I want a paused Lane to read as paused rather than as stalled, so that my own act is not reported back to me as a fault. 44. As the owner, I want a Reader's first bookmark of a Series on a paused Site to still read the page, so that pausing a Lane never breaks somebody adding a Series. 45. As the owner, I want a resumed Lane to find its full queue waiting, so that a pause delays work rather than discarding it. 46. As the owner, I want each action to answer with freshly rendered markup, so that the figures I see after a press describe the state after the press. 47. As the owner, I want only the Lanes block to refresh on a timer, so that a half-open confirm row is never eaten by an auto-swap and the list does not re-sort between my press and my confirm. 48. As a Reader, I want none of this to appear anywhere in my library, so that the owner's operations surface is not something I have to understand. 49. As a Reader, I want the owner to be unable to see which Series I read or where I am in them, so that a dashboard for keeping the poller healthy is not a reading log. 50. As the owner, I want to be told which Series a Reader's report raised without being told which Reader, so that I can act on a suspicious number without acquiring information I said I would not hold. 51. As the owner, I want the surface to be readable on a phone, so that I can check the poller from the same device I read on. 52. As the owner, I want the admin surface to look like the rest of the application, so that it does not read as a bolted-on tool. 53. As the owner, I want no ember accent anywhere on the admin surface, so that the one accent that means "new chapter" keeps meaning only that. ## Implementation Decisions ### Access model and routes - `requireOwner` stays the entire access model: single owner, no roles, no second admin, no per-user grants. Every new route joins `adminRoutes`, so the owner gate and the route list cannot drift and `AdminPatterns` keeps covering all of them by construction. - Five pages plus one fragment plus the action verbs: | Route | Kind | Holds | |---|---|---| | `GET /admin` | page | verdict line, stats block, per-Site table (library shape only), nav row | | `GET /admin/lanes` | page | per-Lane detail, pause controls; the only timer on the surface | | `GET /admin/readers` | page | the existing roster and its two actions | | `GET /admin/series` | page | filterable Series list; filter state in the query string | | `GET /admin/series/{key}` | page | per-Series detail; `key` is `site:series_id` | | `GET /ui/admin/lanes` | fragment | the self-refreshing Lanes block | | `POST /admin/series/{key}/poll` | action | request a Forced Poll; answers with the swapped row | | `POST /admin/lanes/{site}/pause` | action | duration from the form; answers with the swapped block | | `POST /admin/lanes/{site}/resume` | action | answers with the swapped block | - The two existing Reader actions keep their current top-level addresses. Renaming them for symmetry edits working templates for no gain. - Series detail is keyed by the composite `site:series_id` in one path segment, matching the shape the wire and `store.Get` already use. A surrogate id would need a column and a migration to save nothing; `:` is legal unescaped in a path segment and no observed series id carries `/`. - Bookmarkable pages under `/admin/...`, self-refreshing fragments under `/ui/admin/...`, actions answering with the swapped fragment — the shape `renderRoster` already uses. - The only entry point into admin stays the owner-gated link in the application chrome. No per-Series admin link from the library cards: that is a second gated branch in the reading templates for a hop the Series list already provides. ### The page split rule **The landing page aggregates over Sites; the Lanes page shows per-Lane detail. No figure appears on both.** (This replaces the earlier "landing from the database, Lanes from memory" rule, whose premise was the in-memory state this spec deletes.) ### Lane state moves into the database - New table for the pass log, one row per Lane Pass, append-only, roughly 120 rows a day across six Sites: ```sql CREATE TABLE poll_passes ( site text NOT NULL, ran_at bigint NOT NULL, -- unix ms, from the poller's clock skip text NOT NULL, -- '' = the pass ran; else why it returned early due int NOT NULL, checked int NOT NULL, gap_ms bigint NOT NULL, clamped boolean NOT NULL, refused int NOT NULL, unreachable int NOT NULL, no_chapter int NOT NULL, unfetchable int NOT NULL, errors int NOT NULL, PRIMARY KEY (site, ran_at) ); ``` `(site, ran_at)` is the whole index budget: one goroutine per Lane writes sequentially, so the pair is unique without a surrogate id, and it serves both reads — latest row per Site, and a per-Site window sum. Retention scans, which at roughly 1.7k live rows is cheaper than a second index. - New per-Site Lane state row, `poll_lanes(site primary key, paused_until, refuse_until)`, both defaulting to zero. This is one row read at the top of a pass serving two gates. - **Refusal becomes durable** (`poll_lanes.refuse_until`) and the poller's in-memory refusal map is deleted. A refusal is the *Site's* mood and outlives our process. This changes today's behaviour, where a restart forgets a refusing Site and immediately re-probes it — deliberately. - **The browser-down timestamp stays in memory.** The sidecar being unreachable is a fact about our own reach, and a restart re-probing Chrome is correct behaviour rather than lost state. - **Deleted**: the poller's `laneStates` map, `recordLaneState`, `LaneState`, `Status`, `LaneStatus()`, the whole status file, and `web.LaneReporter`. With the store as the source, an admin test inserts a pass row rather than constructing a fake reporter. Browser configuration becomes "is `BROWSER_WS_URL` set" read in the web layer's config — strictly more accurate, since it describes the deployment rather than whether one goroutine happened to construct a fetcher. Browser reachability is derived: any browser Site whose latest pass carries `unreachable > 0` inside the refusal backoff of now, which needs that backoff constant exported from `latest`. ### The skip enum — one value per return path The Lane pass has nine exits and today a page sees only "due, nothing checked", which reads as a stall in eight cases that are not one. One text column, one value per *return* path: | `skip` | meaning | |---|---| | `''` | the pass reached the loop | | `paused` | the pause row was read at the top | | `refusing` | refusal backoff | | `sidecar-down` | a sibling browser Lane lost Chrome | | `no-fetcher` | browser Site, no browser configured, no fallback | | `due-query` | the due query failed | | `asleep` | under both browser wake thresholds | | `eligible-count` | the eligible count failed | | `nothing-eligible` | nothing eligible; sleeps a full rest | **The stall rule follows from it: `due > 0 AND checked = 0 AND skip = ''` is the only true stall.** Everything else is a Lane that declined to work and said why. ### Outcome counts The classification the Series read already makes is counted on the pass row: `refused` (a challenge held), `unreachable` (the browser interrupted), `no_chapter` (200, real HTML, no chapter found), `unfetchable` (the host pin refused the stored URL, or no fetcher), `errors` (everything else — non-200, transport, a failed check stamp). - `no_chapter` is its own count and is never folded into `refused`: it is what a broken adapter looks like when it breaks loudly. - A success count is **derived, never stored**: `checked - (refused + no_chapter + unfetchable + errors)`. `unreachable` is excluded from that arithmetic because the sidecar-loss path returns before the checked counter increments. A stored success column would be a fifth way to get that wrong. - **The page never prints the word "failures" bare.** Named counts only, and a zero renders as *none observed* — the wording is the honesty, because the failure kind that would hurt most (an adapter parsing a *wrong* number successfully) is not in this taxonomy and never can be. - The five counts render **unlinked**: the pass row holds counts and never identities, so there is no list of the four refused Series to point at. A later spec revisits the navigation, not the counts. ### Carry-forward The existing rule moves verbatim: a pass that returned before computing its figures carries the previous pass's due, gap, clamped and checked forward — **exactly when the pass's own gap is zero** — so no row states a zero it did not measure and the read stays a single `DISTINCT ON`. With `skip` recorded, a genuine zero beside `skip = 'due-query'` now reads correctly rather than as a measurement, which is why the rule is kept rather than widened. Rejected: nullable columns plus a read-side "last non-null per Site", and writing no row at all for a skipped pass (a Lane refusing for a day would look like a Lane that stopped existing). ### Store surface for Lane state ```go type LanePass struct { Site string RanAt int64 Skip string Due, Checked int GapMS int64 Clamped bool Refused, Unreachable, NoChapter, Unfetchable, Errors int PausedUntil, RefuseUntil int64 // joined from poll_lanes on read; not pass columns } func (s *Store) RecordLanePass(p LanePass, retainBefore int64) error func (s *Store) LatestLanePass(site string) (LanePass, bool, error) func (s *Store) LatestLanePasses() ([]LanePass, error) func (s *Store) LanePassOutcomes(since int64) ([]SiteOutcomes, error) func (s *Store) SetLaneRefusal(site string, until int64) error func (s *Store) PauseLane(site string, until int64) error // rejects a non-future expiry func (s *Store) ResumeLane(site string) error // zeroes paused_until; keeps the row func (s *Store) PausedLanes() ([]LanePause, error) func (s *Store) LanePausedUntil(site string) (int64, error) func (s *Store) ForceSeriesPoll(site, seriesID string, at int64) error ``` - `RecordLanePass` inserts and prunes in the same call — **delete-on-insert**, so the Lane goroutine is the pruner and no ticker enters a backend that has none (sessions already expire lazily at lookup for the same reason). `retainBefore` is caller-supplied, keeping the store clockless as the due query already is. - `LatestLanePasses` is `DISTINCT ON (site) … ORDER BY site, ran_at DESC` left-joined to the Lane row: one query for the Lanes page and the same rows the landing verdict sums. - `LatestLanePass(site)` is the carry-forward read — the recorder needs one Site, not six. - **Retention is 14 days and is not the display window.** The window is what the owner is shown; retention is how far back a question can reach. The two must not be collapsed. ### Forced Poll - One column, `series.force_poll_at bigint NOT NULL DEFAULT 0`, unix ms, zero meaning never asked. Writing it again re-stamps the request time; the write is idempotent. - **Pending is derived, never stored**: `force_poll_at > latest_checked_at`. It clears itself with no second write and no sweeper, because the check stamp is written *before* the fetch — so the first attempt ends the pending state whatever the attempt returns. **That stamp-before-fetch order is load-bearing**; changing it silently makes forced requests sticky. - **No expiry.** A request the Lane never reaches keeps ageing in the UI rather than vanishing. - The due query gains the flag in three places, and `force_poll_at` joins its `GROUP BY` list: ``` WHERE s.site = $1 AND s.series_url <> '' AND (s.latest_checked_at <= $2::bigint OR s.force_poll_at > s.latest_checked_at) HAVING (COUNT(*) FILTER (WHERE b.status <> 'finished') > 0 OR s.force_poll_at > s.latest_checked_at) AND (COUNT(*) > 1 OR s.latest_sighted_at <= $2::bigint OR s.latest_checked_at <= $3::bigint OR s.force_poll_at > s.latest_checked_at) ORDER BY (s.force_poll_at > s.latest_checked_at) DESC, reader_count DESC, s.latest_checked_at ASC ``` Note: the `HAVING … status <> 'finished'` clause is today's Lifecycle test and is **deleted** by the Finished Series spec later in this series. Implement it as it stands here; that spec replaces it with a `series.finished_at` test rather than amending it. - **A Forced Poll overrides** the rest cutoff, the Sighting-deferral clause, and the finished-only bucket. **It never overrides** an empty series URL (nothing to fetch), the Bookmarks join (a Series no Reader holds has no consumer for the result), the Lane's refusal backoff (hand-forcing a request at a Site that is actively refusing is the one move that makes it worse), the sidecar-down skip, or the Lane's gap. - One pass-level gate it *does* open: the browser wake thresholds. A forced Series wakes a sleeping Chrome — those thresholds exist to stop the machine waking itself for one unattended check, and a human asking is not that. If the home machine is off, nothing happens and the request ages visibly. - Rejected: zeroing `latest_checked_at` as the force signal (it corrupts the never-checked and stale counts the landing page exists to show, and makes a pending marker impossible). ### Paused Lane - The pause row is read at the top of a pass, ahead of the refusal check. A paused Lane records its pass with `skip = 'paused'` and sleeps until the expiry. Its Series stay due and unstamped — the identical state a missing browser leaves them in, so nothing new has to handle it and the queue is intact when the pause lifts. - **Expiry is mandatory**, offered as 1h / 6h / 24h. - **No global runtime pause.** The deploy-time poll switch stays the only whole-poller stop; the case for a runtime one is the case where you already have a shell. - **Acquisition is unaffected.** A Reader's first bookmark of a Series on a paused Site still reads the page. Pause governs the Lane only. - Rejected: an indefinite pause, and a `site = '*'` pseudo-row for a global one (a second meaning for the primary key of a six-row table). ### The cross-Series read model Two store methods over a dedicated row type. The privacy boundary is the projection, not a template. ```go type SeriesFilter struct { Site string // "" = every Site Kind string // "" = both Libraries Filter string // one of the eight names below; unknown filters nothing StaleBefore int64 // caller-supplied epoch-ms cutoff; the store stays clockless Page int // 1-based } func (s *Store) AdminSeries(f SeriesFilter) (rows []SeriesRow, total int, err error) func (s *Store) SeriesStats(staleBefore int64) ([]SiteStats, error) type SeriesRow struct { Site, SeriesID string Title, Kind string LatestChapter string LatestChapterNum *float64 LatestCheckedAt int64 ForcePollAt int64 PollPending bool // force_poll_at > latest_checked_at SightingRaised bool // latest_sighted_at > latest_checked_at HasCover bool // cover_address <> '' Pollable bool // series_url <> '' ReaderCount int } ``` - **Two methods, not one.** The landing page reads only the aggregate and must not pay for fifty rows it discards. The filter vocabulary is shared by being one set of named predicate constants, not by being one method. - **`store.Series` is deliberately not reused.** Its column list carries the Reader id that raised a Sighting, so reusing it would put Reader identity in the admin handler and leave the boundary resting on a template that happens not to print a field. `SeriesRow` has no field for it, and `latest_raised_by` never leaves the store package. `SightingRaised` answers the attribution question — the owner learns a Reader's report set this number, and nothing about which Reader — and is computed in SQL so no caller repeats the comparison. - The row query: ``` SELECT <adminSeriesColumns>, COUNT(b.reader_id) AS reader_count, COUNT(*) OVER () AS total FROM series s LEFT JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id WHERE ($1::text = '' OR s.site = $1) AND ($2::text = '' OR s.kind = $2) [AND <predicate>] GROUP BY <the projected series columns> [HAVING COUNT(b.reader_id) = 0] ORDER BY s.latest_checked_at ASC, s.site, s.series_id LIMIT 50 OFFSET $n ``` - **LEFT JOIN, not JOIN.** Removing a Bookmark leaves the Series row and nothing deletes one, so orphans accumulate; the Lane's join hides them and the dashboard's third job is hygiene, so it must not. `reader_count = 0` *is* the orphan marker, and it is an anonymous Series-level fact. - **Reader count is a plain `COUNT`**, every Bookmark on the Series. This knowingly disagrees with the two Lane queries for as long as the finished Lifecycle bucket exists; the Finished Series spec removes that bucket, after which the two definitions are the same set. Until then, a Series every Reader finished shows a non-zero count and is never Polled. Recorded rather than hidden. - **The tie-break is mandatory.** Every unpollable Series has a zero check timestamp, so ordering on that column alone gives no stable page boundary and rows would repeat or vanish across pages. - **Predicates are compile-time constants** selected by the filter name; the name never reaches query text. Site and Kind are bound parameters. Only compile-time constants may be concatenated into query text — that rule is not relaxed here. - 50 rows a page, `?page=` 1-based. The total comes from `COUNT(*) OVER ()` in the same query: window functions run after grouping and before the limit, so the figure counts the filtered groups and one where-clause cannot disagree with a second copy of itself. Because that count vanishes on an empty page, the handler treats zero rows with `page > 1` as an over-run and re-reads at page 1. - `SeriesStats` is one grouped pass — `COUNT(…) FILTER (WHERE …)` per class grouped by Site, plus the Library split — not one query per figure. Library-wide totals are summed in Go over six Sites; `ROLLUP` would add a null-Site row every scanner has to special-case. The orphan count needs the Bookmark side, so the query left-joins a grouped subquery and counts where that side is null rather than putting a subquery inside a `FILTER`. Landing counts include orphans: they are exactly what needs attention. The existing eligible-count method is the wrong shape for this — one Site per call, one number back. - One index: `series (latest_checked_at)`. The table has only its primary key today. This supports the default order and is **a judgement, not a measurement** — a grouped query over a join may ignore it. Re-time on real data before adding a second. ### The filter vocabulary — eight names | `?filter=` | predicate | label | |---|---|---| | `unpollable` | `s.series_url = ''` | No series URL | | `no-chapter` | `s.latest_chapter_num IS NULL AND s.latest_checked_at > 0 AND s.series_url <> ''` | Never read a chapter | | `orphan` | `HAVING COUNT(b.reader_id) = 0` | No Readers | | `unchecked` | `s.latest_checked_at = 0 AND s.series_url <> ''` | Never checked | | `stale` | `s.latest_checked_at > 0 AND s.latest_checked_at <= $staleBefore` | Not checked in 12h | | `no-cover` | `s.cover_address = ''` | No cover | | `sighting-raised` | `s.latest_sighted_at > s.latest_checked_at` | Latest from a Reader | | unknown or absent | none | All series | - Ordered permanent-and-fixable first. `unpollable` and `orphan` are labelled as the repair they need rather than as the SQL they are: the owner arrives to act, not to admire a predicate. - `unpollable` means an **empty** series URL only. The second unpollable case — a URL whose host fails the fetch gate — is invisible to SQL, needs the Site registry in Go, and would break both the count and the pager if filtered after the read. It is a repair, not a hygiene count, and belongs to the intervention spec. Note the empty case is reachable and permanent: the column defaults to empty, the Upsert writes the series URL only when the row is brand new, and no Poll ever writes it, so a PUT that omitted it creates a Series no client action can fix. - `no-chapter` and `unchecked` are disjoint by construction (zero versus non-zero check stamp), so the two counts never double-report a row. `no-chapter` is named to match the pass-level `no_chapter` count deliberately: same observation, one durable on the row, the other counted over a window. - **Rejected outright: a "Latest Chapter went backwards" filter.** Not implementable — the row holds only the current number, the poller writes downward unconditionally on any inequality, and a Reader's PUT can lower it too, so detection needs the fact captured at write time in a new column. And not wanted — a downward write is the **correction**, not the fault (the poller tests seed 400 and assert 296 after a poll), so the filter would flag precisely the Series that just healed. Worse, it misses the case that actually costs something: an adapter reading a *stable* wrong number every hour never moves, so nothing ever fires. `sighting-raised` stays the only honest suspicion lens, and hand repair is the intervention spec's job. ### The landing page - Verdict line: `<verdict> · N series waiting · M unchecked over 12h`, fully database-computed. `N` sums `due` over the latest pass per Site; `M` counts Series with a series URL whose check stamp is older than the window. The verdict reads "All lanes healthy" when nothing needs attention, else "<k> lanes need attention"; with no pass rows at all it says no Lane has reported rather than "healthy". That state does not disappear — it narrows from *after every deploy* to *a virgin database*. - **One window constant, `ownerWindow = 12 * time.Hour`**, in the web package, feeding **both** the staleness cutoff and the outcome window. Twelve hours because the owner looks once by day and once by night and each look should cover the interval since the last. It is a human threshold, deliberately not a multiple of the Lane's rest, and the page reads the database rather than the poller so it cannot follow the Lane's pace anyway. Two differently-named twelve-hour constants on one page is how they drift apart. - Stats block: library size and the manga/novel split, the roster count, the eight hygiene counts, library-wide and per Site. - **Every figure is an entry point**, not only the hygiene ones: total to the unfiltered list, the Library split to `?kind=`, a per-Site row label to `?site=`, the roster count to the Readers page, each hygiene count to `?filter=`, each per-Site hygiene count to both. **A zero renders the digit and is not a link** — the figure stays, because a measured zero is a real fact, but no anchor, because following it lands on an empty list. - The per-Site table on the landing page carries **library shape only**. The five Poll outcome sums live on the Lanes page; on the landing they were nine columns of zeros burying the four columns that move. - **No aggregate problem count.** The classes overlap — one orphaned Series with no URL and no cover is three counts and one row — so a sum over-reports while a distinct count is a number nothing can be done about. The stats block is the whole hygiene surface and every figure on it is individually actionable. ### Refresh Only the Lanes block refreshes, on the existing 30s timer, now a six-row primary-key read rather than a map copy. Three reasons the other pages have none: the Lane rest is an hour, so the check stamp moves at that granularity and a 30s timer would re-run a cross-Series join roughly 120 times an hour to redraw the same rows; an auto-swap on an action surface eats a half-open confirm row or re-sorts the list between the press and the confirm; and a Forced Poll is asynchronous by construction, so a timer would show "nothing yet" for up to an hour. A manual refresh button stays available later as one attribute; it is not bought now. ### Visual surface The artifact to port is the Claude Design project **BookmarkManager Web UI** (`969ac210-fe02-4c01-ae1b-9a271dcc779a`), files `admin.css` and `admin-overview.html`, `admin-lanes.html`, `admin-readers.html`, `admin-series.html`, `admin-series-detail.html`. The three-variant sketch on branch `prototype/admin-surface-122` is **superseded exploration** — variant B won; port from the design project, not from that branch. - **One new token, `--measure-wide: 1080px`**, admin only; the 760px reading measure is unchanged and admin is the only consumer of the wide one. - **Row shape: aligned borderless columns.** One CSS grid per table, a hairline header of mono uppercase labels, hairline row rules, **no vertical rules, no card backgrounds, no corners**. Gutters are cell padding, never a column gap — a gap slices the row rule into segments and reads as a ragged edge. Figures are tabular mono in mixed case at roughly zero tracking; uppercase at wide tracking is for labels and marks only. Titles stay in the display face. - **The Series list is the two-line form**: a subgrid row with the title on its own full-width line and the facts on the line under it, so a long title never breaks column rhythm. Separation is **banding** rather than hairlines, so a title and its facts read as one record; banding is a class, not an `nth-of-type`, because collapsed confirm rows are row siblings and would throw the alternation off. Lanes and the landing per-Site table are single-line rows; their numerics are centred rather than right-flushed, because at column width a right-flushed figure loses its header. - Notes chips cap at **two plus a faint `+N` tail** rather than stacking. - **Actions are right-aligned 12px ghosts**, hovering in the patina accent. *Check now* is **unconfirmed**, and its ageing pending marker renders **once**, on the title line flush right above the actions — under the action it added a second line to every row. Destructive actions use the danger ghost and open an in-place confirm row: a full-width grid row on the danger wash with *Keep* / *Remove*, shipping hidden and unmounted, or every eligible row prints an empty striped band. - **No ember anywhere on admin.** Patina is the accent; danger is for lane trouble and destruction; a deliberate pause stays patina. Both colour branches are touched together. - Nav row of four display-face tabs with the active tab underlined; Library and Log out group as a right-aligned cluster in the topbar. Section heads are mono uppercase led by a 34px patina tick, not a full-width hairline — stacked rules competed with the tables' own rules. The verdict line is set in the mono data face at 15px, not the display face: it is three counts, not a page title. - Stats block is an auto-fit grid at `minmax(232px, 1fr)`, one rule on the block, zeros in the muted colour and unlinked. - Lanes columns: Site / Due / Checked / Gap / Last pass / outcomes-and-state / controls. Browser reachability is a status line pinned to the section heading, not a paragraph. A trouble state renders danger; a pause renders patina as `paused · resumes in 3h`. The pause select and *Pause* live in one bar and *Resume* replaces them in the same slot. - Series detail: back ghost, 28px display title, mono key line `site:series_id · site · kind`, a 160px hatched cover, an uppercase mono meta row carrying the marks, then a two-column grid reserved for the intervention forms the next spec adds, with *Check now* below. - Readers keeps its existing shape, re-set to the table's type tiers, with a reserved right-aligned action cluster so a chip cannot make one row taller. - **Phone**: Lanes stacks at ≤1019px (its phrase run cannot hold a fractional track sooner), every other table and the detail grid at ≤899px; stacked rows flex-wrap with the title full-width and actions pushed right. ### The filter control **A `<select>` whose option labels carry their counts** (`No cover (3)`), not a row of eight plain links — eight hygiene labels do not read as a chip row. Site is a second select; Library is a three-way segmented row marked active in patina. Filters stay one-at-a-time and URL-addressable. ### Frontend dependencies **Nothing new.** The vendored htmx is 2.0.4 and every primitive this surface needs is in that build and already used in tree: `hx-get` plus `hx-push-url` for URL-addressable filters (the reading tabs already prove it, and the docs' requirement that a pushed URL render a full page is satisfied by these handlers), `hx-trigger="keyup changed delay:500ms"` for a debounced search, `hx-target` plus `hx-swap="outerHTML"` or an out-of-band swap for a single-row re-render, `hx-trigger="every 30s"` scoped to its own fragment for the Lanes block, and either `hx-confirm` or the existing inline confirm row for gating. htmx has no paging attribute: load-more is a `beforeend` swap and numbered pages are the tab pattern plus a pushed URL — same primitives, different swap and URL semantics, so paging is an IA decision rather than a capability limit. **The existing client-side filter script is not reusable here.** It is a pure title filter over an already-rendered list, it issues no requests, it changes no URL, and it is not loaded on the admin page at all. Client-side filter state cannot be bookmarkable without hand-written history calls, which is exactly what these routes must avoid — so filtering is server-side. Research note: `docs/research/htmx-filterable-admin-series-list.md` on branch `research/htmx-series-list`; delete the branch once this spec is implemented. ## Testing Decisions **What makes a good test here**: it asserts observable behaviour at a seam the owner or a Reader can actually reach — a rendered page, a store method's returned rows, a poller pass's effect on the database — and it fails on a plausible bug rather than on a refactor. No test asserts a template's internal structure, a private helper's shape, or a log line's wording unless that wording *is* the contract (the *none observed* zero is). **The primary seam is the router**, and it is an existing one: the package-level web tests build a real router over a throwaway Postgres and drive it with an owner session cookie. Every admin page, filter, action, empty state and gate is asserted through a request and its rendered response. **This spec reduces the seam count**: deleting `LaneReporter` deletes the `fakeLanes` fake, and a Lanes-page test inserts a pass row instead of constructing one. Modules and what is tested at each: - **Router / web** (existing seam, prior art: the admin roster, owner-gate and Lane-status tests): every new route is owner-gated (the existing pattern-driven gate test covers it by construction once the routes join the route list); the verdict line's three states, including no-passes-yet; each of the eight filters returns the rows it names and no others; a zero figure renders unlinked; the heading's count agrees with the row count; each empty state names its filter; `?site=` and `?kind=` stack on a filter without dropping it; page 2 of a one-page result re-reads at page 1; *Check now* is absent on a Series with no URL and on one with no Readers; a pause renders as paused rather than as stalled; a Lane with a `skip` value renders its reason rather than a stall; the outcome counts render named with *none observed* at zero. - **Store** (existing seam, prior art: the existing store tests over a throwaway Postgres with real migrations): each filter predicate against seeded rows, including the orphan case that only a left join can see and the disjointness of `no-chapter` and `unchecked`; the paging tie-break with several rows sharing a zero check stamp; the window total agreeing with the filtered group count; the aggregate agreeing with the row query for the same filter; `RecordLanePass` pruning on insert; `LatestLanePasses` returning one row per Site; the pause upsert rejecting a non-future expiry and `ResumeLane` zeroing rather than deleting the row. - **The privacy boundary gets its own test**, modelled on the existing test that guards the Bookmark column list: assert the admin column constant does not mention `latest_raised_by`, and that `SeriesRow` has no field for it. This is the one place a template-only guarantee would rot silently. - **Poller** (existing seam, prior art: the round-at-a-time pass tests with a fake fetcher and an injected clock): a pass records exactly one row with the right `skip` for each of the nine return paths; carry-forward fires exactly when the pass's own gap is zero; a paused Lane makes no fetch and leaves its Series due and unstamped; a durable refusal is honoured across a fresh poller; a forced Series is fetched ahead of the rest cutoff, the Sighting deferral and the finished bucket, and is **not** fetched through a refusal backoff, an empty URL or the Bookmarks join; a forced Series wakes a sleeping browser Lane; the pending flag self-clears on the check stamp with no second write. No new fake, no new interface, no framework. The only test-visible seam this spec adds is the store itself, which the poller already holds. ## Out of Scope - **Multiple admins, a role column, or per-user admin grants.** The single-owner model is the boundary. - **Inspecting an individual Reader's library or reading progress.** Ruled out by the privacy boundary; only anonymous counts cross into admin views. - **Metrics, charts, or a timeseries store.** A swapless 1974 MiB VPS running Postgres argues against it, and no decision on this map needs a graph. - **Per-Series history of any kind.** The pass log is per Lane. A per-Series check log was rejected on cost and, more importantly, as insufficient: a pass that checked nothing writes nothing, so "the Lane ran, everything was rested, all healthy" — the primary health signal — would be invisible. - **A free-text last-error column.** It becomes the thing that gets read instead of the log, and it cannot be aggregated. - **An admin audit trail.** Different provenance (machine observation versus human action), different retention instinct, and a Forced Poll is already self-evidencing — the unserved request ages visibly. A 14-day pass log would silently expire the one thing worth looking back on. - **A global runtime poller pause.** The deploy-time switch stays the only whole-poller stop. - **Bulk rewrite of series URLs across a whole Site.** A host change invalidates every row of a Site at once; that is a SQL migration, and the dashboard's library-wide job is *finding* broken rows, not batch-repairing them. - **Per-Series correction, Cover replacement, orphan removal, the Finished Series, per-Series failure state, the completion hint, and outbound notification.** All specified, all in the three later specs in this series. ## Further Notes - **Migration numbers are claimed in landing order, not in map order.** Measured 2026-08-21: the tree's migrations and ADRs both stop at 0011, and everything the map specified (the map called them 0012–0019) is unbuilt. This spec's schema work is: the `series (latest_checked_at)` index; `series.force_poll_at`; `poll_lanes(site, paused_until, refuse_until)`; and `poll_passes`. Take the next free numbers in whatever order they land — migrations are globbed by version, so contiguity at merge time is what matters, not agreement with the map's paper numbering. - **Two ADRs are worth writing with the implementation**, not before: persisted Lane state plus the deletion of its in-memory twin, and the through-the-database command seam with its queue-jump rules. - **Glossary**: `CONTEXT.md` already names Lane Pass, Forced Poll, Paused Lane, Stall, Correction and Orphan Series — those edits landed while charting. Nothing new is needed here. - Two figures in the earlier charting were superseded and the final values are above: staleness is **12h, not 24h**, and the display window and retention are **12h and 14 days**, two windows on purpose. - The stall test in this spec is amended by the notification spec later in the series: it gains `AND refused = 0`, because the twice-refused break inside the pass loop is not an early return and so writes `due > 0, checked = 0, skip = ''`. Implement the test as stated here; that spec carries the amendment with its own tests. - Security-critical surfaces touched: the owner gate (every new route must join the route list rather than checking inside itself), and the store's query construction (only compile-time constants may be concatenated). Form bodies on the action routes are capped the way the API path caps them. Say which invariant you preserved in the PR and run the full backend test suite before calling it done.
sulthan added the ready-for-agent label 2026-08-21 15:47:44 +07:00
Author
Owner

Broken out into ten tickets, all labelled ready-for-agent, blocking edges as body lines:

# Ticket Blocked by
#138 Split the admin surface into four bookmarkable pages with a nav row —
#139 Durable Lane state: poll_passes and poll_lanes, plus their store surface —
#140 Cross-Series admin read model, with the privacy boundary in the projection —
#141 The poller records one pass row per exit, with a skip reason and outcome counts #139
#142 Series list page: eight hygiene filters, Site and Library narrowing, paging #138, #140
#143 Overview page: a verdict line, and a stats block where every figure is a door #138, #139, #140
#144 Per-Series detail page, keyed by the composite the rest of the system uses #138, #140
#145 Lanes page reads the database, and the in-memory Lane state is deleted #138, #139, #141
#146 Forced Poll: ask for one Series to be checked now #141, #140, #142, #144
#147 Pause and resume one Site's Lane, with a mandatory expiry #141, #145

#138, #139 and #140 have no blockers and can run in parallel. #139 → #141 → #145 is expand–migrate–contract on the in-memory Lane state: the durable table lands unread, the poller dual-writes, then the twin is deleted atomically in one ticket, because a half-deleted twin means two sources of truth in tree. #140 lands series.force_poll_at unused for the same reason; #146 wires it. The visual port is folded into each page's ticket with the shared admin.css foundation in #138, rather than being a horizontal CSS ticket.

Two findings against the code while slicing, both about the skip enum. runLanePass has ten return statements, not nine.

The refused-twice exit does not need the amendment this spec promises. Further Notes says the twice-refused break "writes due > 0, checked = 0, skip = ''" and that the notification spec will fix the stall test with AND refused = 0. The premise does not hold: st.Checked++ runs after the error switch in the pass loop, so a refused Series still counts as checked — it was attempted — and two refusals means checked = 2, never zero. The stall test cannot fire on that path. This spec's own outcome arithmetic agrees, since deriving success as checked - (refused + …) only works if checked counts attempts including refusals. Recommend dropping the AND refused = 0 amendment from the notification spec rather than implementing it.

The real gap is the mid-loop browser-unreachable exit, and it is left open deliberately. When checkOne reports Chrome unreachable the Lane marks the sidecar down and returns immediately, before the checked counter increments. If the first Series in the queue is the one that loses Chrome, that pass writes due > 0, checked = 0, and this spec gives that exit no skip value — so it records skip = '' and renders as the one true stall. A dead Chrome is the most routine event in this deployment (the sidecar is on-demand and the home machine sleeps), so the surface would report it as an unexplained fault: exactly the false positive the spec exists to eliminate. The sidecar-down value is spent on a different thing, a sibling browser Lane declining at the top of a pass.

Per decision, #141 implements the enum verbatim as specified and does not invent a tenth value, so this false positive ships and is settled by the later spec in the series. #141 says so explicitly, so nobody improvises. The cheap fix when that spec lands is to record this exit as sidecar-down too, with unreachable = 1 — same reason, same word, and the difference between "I lost Chrome" and "my sibling lost Chrome" is not one the owner would act on differently.

Broken out into ten tickets, all labelled `ready-for-agent`, blocking edges as body lines: | # | Ticket | Blocked by | |---|---|---| | #138 | Split the admin surface into four bookmarkable pages with a nav row | — | | #139 | Durable Lane state: `poll_passes` and `poll_lanes`, plus their store surface | — | | #140 | Cross-Series admin read model, with the privacy boundary in the projection | — | | #141 | The poller records one pass row per exit, with a skip reason and outcome counts | #139 | | #142 | Series list page: eight hygiene filters, Site and Library narrowing, paging | #138, #140 | | #143 | Overview page: a verdict line, and a stats block where every figure is a door | #138, #139, #140 | | #144 | Per-Series detail page, keyed by the composite the rest of the system uses | #138, #140 | | #145 | Lanes page reads the database, and the in-memory Lane state is deleted | #138, #139, #141 | | #146 | Forced Poll: ask for one Series to be checked now | #141, #140, #142, #144 | | #147 | Pause and resume one Site's Lane, with a mandatory expiry | #141, #145 | #138, #139 and #140 have no blockers and can run in parallel. #139 → #141 → #145 is expand–migrate–contract on the in-memory Lane state: the durable table lands unread, the poller dual-writes, then the twin is deleted atomically in one ticket, because a half-deleted twin means two sources of truth in tree. #140 lands `series.force_poll_at` unused for the same reason; #146 wires it. The visual port is folded into each page's ticket with the shared `admin.css` foundation in #138, rather than being a horizontal CSS ticket. Two findings against the code while slicing, both about the skip enum. `runLanePass` has **ten** `return` statements, not nine. **The refused-twice exit does not need the amendment this spec promises.** Further Notes says the twice-refused break "writes `due > 0, checked = 0, skip = ''`" and that the notification spec will fix the stall test with `AND refused = 0`. The premise does not hold: `st.Checked++` runs after the error switch in the pass loop, so a refused Series still counts as checked — it was attempted — and two refusals means `checked = 2`, never zero. The stall test cannot fire on that path. This spec's own outcome arithmetic agrees, since deriving success as `checked - (refused + …)` only works if `checked` counts attempts including refusals. **Recommend dropping the `AND refused = 0` amendment from the notification spec rather than implementing it.** **The real gap is the mid-loop browser-unreachable exit, and it is left open deliberately.** When `checkOne` reports Chrome unreachable the Lane marks the sidecar down and returns immediately, *before* the checked counter increments. If the first Series in the queue is the one that loses Chrome, that pass writes `due > 0, checked = 0`, and this spec gives that exit no skip value — so it records `skip = ''` and renders as the one true stall. A dead Chrome is the most routine event in this deployment (the sidecar is on-demand and the home machine sleeps), so the surface would report it as an unexplained fault: exactly the false positive the spec exists to eliminate. The `sidecar-down` value is spent on a different thing, a *sibling* browser Lane declining at the top of a pass. Per decision, #141 implements the enum verbatim as specified and does **not** invent a tenth value, so this false positive ships and is settled by the later spec in the series. #141 says so explicitly, so nobody improvises. The cheap fix when that spec lands is to record this exit as `sidecar-down` too, with `unreachable = 1` — same reason, same word, and the difference between "I lost Chrome" and "my sibling lost Chrome" is not one the owner would act on differently.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sulthan/mangaBookmark#134