sulthan/mangaBookmark
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Poll history: persist it, or stay a live-now view #117

New Issue
Closed
opened 2026-08-17 15:22:43 +07:00 by sulthan · 2 comments
sulthan commented 2026-08-17 15:22:43 +07:00
Owner
Copy Link
Copy Source

Part of #114

Question

Should the backend persist poll history, and if so in what shape?

Nothing is persisted today. Poller.laneStates holds the single most recent pass per Lane in
memory and loses it on restart — which is why the page reads "No data yet" seconds after a
deploy. The only per-poll DB writes are MarkLatestChecked, SetLatestChapter,
RecordSighting*, SetSeriesCover, and there is no per-Series failure counter column anywhere.

Expected shape going in: live-now plus a bounded last-N-runs, not a timeseries — a swapless
1974 MiB VPS running Postgres argues hard against volume.

To decide:

  • Whether history is per-Lane-pass or per-Series-check. Per-Series is what "this Series has been
    failing for a week" needs; per-Lane is far cheaper and answers "what has the poller been doing".
  • The table shape, retention policy, and who prunes (a sweep, a bounded insert, or a rolling
    window). Note sessions deliberately has no background sweep — expiry is lazy at lookup.
  • Whether failure is recorded at all, and how. A Site refusing, a browser sidecar asleep, and an
    adapter reading a wrong number are three different failures, and only the first two are visible
    to the Lane today.
  • What the dashboard reads from it, and whether the summary line changes once it exists.
  • Whether this subsumes the admin audit trail sitting in the map's fog, or leaves it separate.

Added by #115

Hard requirement from the landing page: per-Site failure counts over the last 24h, wanted in
the landing page's per-Site table. Impossible today, so until this ticket lands that column is
absent rather than zero.

Two consequences for the shape:

  • The count is per Site over a rolling 24h window, which is the narrowest thing that satisfies
    the dashboard — it does not by itself require per-Series history.
  • Only two of the three failure kinds are visible to the Lane (a Site refusing, the sidecar
    unreachable). An adapter reading a wrong number (#79) is not, so a rendered "0 failures" would
    lie in exactly the case that motivates the dashboard. Whatever this ticket persists must let
    the page distinguish "no failures seen" from "this kind of failure is not observable".
Part of #114 ## Question Should the backend persist poll history, and if so in what shape? Nothing is persisted today. `Poller.laneStates` holds the single most recent pass per Lane in memory and loses it on restart — which is why the page reads "No data yet" seconds after a deploy. The only per-poll DB writes are `MarkLatestChecked`, `SetLatestChapter`, `RecordSighting*`, `SetSeriesCover`, and there is no per-Series failure counter column anywhere. Expected shape going in: live-now plus a bounded last-N-runs, not a timeseries — a swapless 1974 MiB VPS running Postgres argues hard against volume. To decide: - Whether history is per-Lane-pass or per-Series-check. Per-Series is what "this Series has been failing for a week" needs; per-Lane is far cheaper and answers "what has the poller been doing". - The table shape, retention policy, and who prunes (a sweep, a bounded insert, or a rolling window). Note `sessions` deliberately has no background sweep — expiry is lazy at lookup. - Whether failure is recorded at all, and how. A Site refusing, a browser sidecar asleep, and an adapter reading a wrong number are three different failures, and only the first two are visible to the Lane today. - What the dashboard reads from it, and whether the summary line changes once it exists. - Whether this subsumes the admin audit trail sitting in the map's fog, or leaves it separate. ## Added by #115 Hard requirement from the landing page: **per-Site failure counts over the last 24h**, wanted in the landing page's per-Site table. Impossible today, so until this ticket lands that column is absent rather than zero. Two consequences for the shape: - The count is per Site over a rolling 24h window, which is the narrowest thing that satisfies the dashboard — it does not by itself require per-Series history. - Only two of the three failure kinds are visible to the Lane (a Site refusing, the sidecar unreachable). An adapter reading a wrong number (#79) is not, so a rendered "0 failures" would lie in exactly the case that motivates the dashboard. Whatever this ticket persists must let the page distinguish "no failures seen" from "this kind of failure is not observable".
sulthan added the wayfinder:grilling label 2026-08-17 15:22:43 +07:00
sulthan self-assigned this 2026-08-17 17:53:24 +07:00
sulthan removed their assignment 2026-08-17 19:00:27 +07:00
sulthan self-assigned this 2026-08-17 19:01:21 +07:00
sulthan commented 2026-08-17 19:33:34 +07:00
Author
Owner
Copy Link
Copy Source

Resolution

One append-only Lane Pass log, a generalised per-Site Lane row, and the deletion of the
poller's in-memory page state. Display window 12h; retention 14 days — two different
windows, deliberately.

Migration 0014

CREATE TABLE poll_passes (
  site        text    NOT NULL,
  ran_at      bigint  NOT NULL,  -- unix ms, from p.Now()
  skip        text    NOT NULL,  -- '' = the pass ran; else why it returned early
  due         int     NOT NULL,
  checked     int     NOT NULL,
  gap_ms      bigint  NOT NULL,
  clamped     boolean NOT NULL,
  refused     int     NOT NULL,
  unreachable int     NOT NULL,
  no_chapter  int     NOT NULL,
  unfetchable int     NOT NULL,
  errors      int     NOT NULL,
  PRIMARY KEY (site, ran_at)
);
ALTER TABLE poll_lanes ADD COLUMN refuse_until bigint NOT NULL DEFAULT 0;

(site, ran_at) is the whole index budget. One goroutine per Lane writes sequentially, so the
pair is unique without a surrogate id, and it serves both reads: latest row per Site, and a
per-Site window sum. Retention's WHERE ran_at < cutoff scans, which at ~1.7k live rows is
cheaper than a second index. Volume: 5 Sites x ~24 passes/day = ~120 rows/day.

Not persisted, because each is derivable and a second copy is a second thing to drift:
refusing (from poll_lanes.refuse_until — LaneStatus already overwrites the in-memory
field at snapshot time, status.go:53), browser (registry: isBrowserSite), and any ok
count (see below).

The skip enum — one value per return path

runLanePass has nine exits, and today a page cannot tell them apart: all it sees is Due > 0, Checked == 0, which reads as a stall in eight cases that are not one. One text column fixes
that and replaces the Asleep boolean:

skip return path
'' the pass reached the loop
paused #119's pause row read at the top
refusing refusal backoff (poller.go:229)
sidecar-down sibling browser Lane lost Chrome (poller.go:234)
no-fetcher browser Site, no BROWSER_WS_URL, no fallback (poller.go:245)
due-query DueForLatestCheck failed (poller.go:255)
asleep under both wake thresholds, ADR-0005 (poller.go:261)
eligible-count EligibleSeriesCount failed (poller.go:280)
nothing-eligible eligible == 0, sleeps a full rest (poller.go:290)

The stall rule follows from it: due > 0 AND checked = 0 AND skip = '' is the only true
stall. Everything else is a Lane that declined to work and said why.

Outcome counts — the classification already exists

readSeriesPage classifies six outcomes and checkOne throws all of them at log.Printf.
The pass row counts them:

column source
refused errChallengeHeld — 403 cf-mitigated or an interstitial body (read.go:55-69)
unreachable errBrowserInterrupted
no_chapter !facts.HasLatest — "no chapter links in N bytes" (poller.go:458)
unfetchable errNotFetchable (host pin) or errNoFetcher
errors everything else: non-200, transport, MarkLatestChecked failure

no_chapter is its own count and never folded into refused: it is what a broken adapter
looks like when it breaks loudly, and #115's honesty requirement is precisely that the page not
present adapter breakage as a quiet zero.

A success count is derived, never stored: ok = checked - (refused + no_chapter + unfetchable + errors). unreachable is excluded from that arithmetic because the sidecar-loss
path returns before st.Checked++ (poller.go:321-324), so the Series that lost Chrome was never
counted as checked. A stored ok column would be a fifth way to get that wrong.

The page never renders the word "failures" bare. Named counts, and a zero renders as
none observed — the wording is the honesty, because the one failure kind that matters most
(#79: an adapter parsing a wrong number successfully) is not in this taxonomy and never can be.

Carry-forward stays on the write

recordLaneState (status.go:75-77) carries the previous pass's due/gap/clamped/checked
forward when the pass returned before computing them, so the page never states a zero it
did not measure. That rule moves verbatim — carry forward exactly when the pass's own gap is
0 (today: the refusing and sidecar-down exits) — so every row is self-describing and the
read is one DISTINCT ON. Not preserved-by-accident: with skip recorded, a genuine zero next
to skip = 'due-query' now reads correctly instead of as a measurement, which is why the rule
is kept rather than widened.

Rejected: NULL columns plus a read-side "last non-NULL per Site" (two-part read, COALESCE
gymnastics for a value the writer already holds), and writing no row for a skipped pass (a Lane
refusing for a day would look like a Lane that stopped existing).

Store surface

type LanePass struct {
	Site        string
	RanAt       int64
	Skip        string
	Due, Checked int
	GapMS       int64
	Clamped     bool
	Refused, Unreachable, NoChapter, Unfetchable, Errors int
	PausedUntil, RefuseUntil int64 // joined from poll_lanes on read; not pass columns
}

func (s *Store) RecordLanePass(p LanePass, retainBefore int64) error
func (s *Store) LatestLanePass(site string) (LanePass, bool, error)
func (s *Store) LatestLanePasses() ([]LanePass, error)
func (s *Store) LanePassOutcomes(since int64) ([]SiteOutcomes, error)
func (s *Store) SetLaneRefusal(site string, until int64) error
  • RecordLanePass inserts and prunes in the same call — delete-on-insert, so the Lane
    goroutine is the pruner and no time.Ticker enters a backend that has none (sessions
    expires lazily at lookup for the same reason). retainBefore is caller-supplied, keeping the
    store clockless like DueForLatestCheck.
  • LatestLanePasses is DISTINCT ON (site) ... ORDER BY site, ran_at DESC, LEFT JOINed to
    poll_lanes — one query for the Lanes page, and the same rows the landing verdict sums.
  • LatestLanePass(site) is the carry-forward read: 5 reads an hour, and the recorder needs one
    Site, not five.
  • LanePassOutcomes(since) is GROUP BY site with SUM per outcome column over the window.

What gets deleted

  • Poller.laneStates, recordLaneState, LaneState, Status, LaneStatus() — status.go goes
    away; the pass row is the record.
  • web.LaneReporter. This overrides #114's Notes and #119's answer, both of which kept it
    as one read-only method so the admin page stayed testable with no poller running. That goal is
    better served by no interface at all: with the store as the source, an admin test inserts a
    pass row and asserts the render, with no fake reporter to construct. The Note's letter goes,
    its reason survives.
  • Poller.refuseUntil (the map). Refusal becomes poll_lanes.refuse_until, read at the top of
    a pass alongside #119's pause read — one row, one query, two gates.

Poller.browserDownAt stays in memory, and the distinction is the point: a refusal is the
Site's mood and outlives our process, while the sidecar being unreachable is a fact about
our own reach — a restart re-probing Chrome is correct behaviour, not lost state. Note the
consequence: refuse_until becoming durable changes today's behaviour, where a restart forgets
a refusing Site and immediately re-probes it. That is the same argument #119 made for
paused_until.

Status.BrowserConfigured came from p.BrowserFetch != nil, a Poller field. It becomes
BROWSER_WS_URL != "" in the web layer's config — strictly more accurate, since it describes
the deployment rather than whether one goroutine happened to construct a fetcher. Browser
reachability is derived: any browser Site whose latest pass carries unreachable > 0 inside
refuseBackoff of now. That needs the constant exported as latest.RefuseBackoff (15m,
sites.go:406) — one exported constant instead of one interface.

One window constant, 12h

ownerWindow = 12 * time.Hour in the web package, feeding both the staleness cutoff and the
outcome window. #116 superseded #115's 24h figure for staleness only, so the failure window was
still 24h on paper; the rationale it gave (the owner looks once by day and once by night, and
each look should cover the interval since the last) governs both figures identically, and two
different windows on one page is a comprehension bug. This retires #116's staleAfter name —
a rename on paper, since nothing is built.

Retention is not this constant: 14 days, and the two must not be collapsed. The window is
what the owner is shown; retention is how far back a question can reach.

What the pages read

#115's split rule — "landing from the database, Lanes page from memory" — is void, because
its premise was the memory this ticket deletes. Replacement, same guarantee on a new axis:
landing shows aggregates over Sites, the Lanes page shows per-Lane detail; no figure appears
on both.

  • Landing: the verdict line, now fully database-computed — <verdict> · N series waiting · M unchecked over 12h, with N = SUM(due) over the latest pass per Site. The per-Site table
    gains the five outcome sums over 12h from LanePassOutcomes. The "no Lane has reported yet"
    state does not disappear — it narrows from after every deploy to a virgin database.
  • Lanes page: one row per Lane with due / checked / gap / clamped / skip / paused_until / refuse_until, plus #119's Pause and Resume controls. Keeps the only timer on the surface
    (#115), still 30s — now a 5-row primary-key read instead of a map copy, which is what
    overrides #115's "the timer is free because it touches no database" argument: Postgres holds
    that page in cache, and one source of truth is worth more than the read.

Amendments to closed tickets

  • #119 — poll_lanes was specified as "a row exists only while paused; resume deletes it".
    It becomes the per-Site Lane state row: paused_until and refuse_until both default 0,
    ResumeLane zeroes paused_until instead of deleting the row, and PausedLanes()
    filters paused_until > now. A refusing-but-unpaused Lane needs a row, so the delete-on-resume
    invariant cannot survive. skip = 'paused' is this ticket's record of #119's pause exit.
  • #116 — staleAfter becomes ownerWindow; see the new ticket below for the hole its filter
    set leaves.
  • #114 Notes — the LaneReporter stays one read-only method bullet is rewritten to say the
    admin page reads Lane state from the database.

Pushed onto other tickets

  • New ticket: per-Series Poll outcome state. This ticket answers "what has the poller been
    doing" per Site, and nothing durable about one Series. latest_checked_at is stamped
    before the fetch (poller.go:428), so it means attempted, never succeeded — a Series
    broken for a month is unchecked-clean, stale-clean and unpollable-clean in every filter
    #116 defined, while its per-Site count rolls off in 12h. That is a hole in the filter set, not
    a nice-to-have.
  • #118 — the Lanes page and the landing per-Site table are now filter-count consumers too;
    the outcome counts should link somewhere, and after the new ticket lands that somewhere is a
    failing filter.
  • #122 — prototypes a Lane row carrying a skip explanation and five named outcome counts
    with a none observed empty state, not a single "errors" number.

Rejected

  • Latest pass upserted onto poll_lanes (5 rows forever, no retention, no pruning). Fails
    only the 12h window — which is exactly the question "does that window earn a table", answered
    yes: without it the dashboard can say a Lane is broken now but not that it was broken at 04:00.
  • One row per Series check. Heaviest (a Series can produce 24 rows/day at defaultRest = 1h), and still insufficient: a pass that checked nothing writes nothing, so "the Lane ran,
    everything was rested, all healthy" — the primary health signal — is invisible. The per-Series
    question it would answer is answered better by three columns on series.
  • A free-text last-error column. It becomes the thing that gets read instead of the log, and
    it cannot be aggregated.
  • Folding the admin audit trail in here (#119 rejected poll_commands for the same reason,
    hanging it on this ticket). Not subsumed: different provenance (machine observation vs human
    action), different retention instinct, and a Forced Poll is already self-evidencing —
    force_poll_at > latest_checked_at is the record, ageing visibly while unserved. A 14-day
    pass log would silently expire the one thing worth looking back on.

Glossary: CONTEXT.md gains Lane Pass. An ADR is worth writing when this is implemented
(persisted Lane state plus the deletion of the in-memory twin), not now — the map produces the spec.

## Resolution One append-only Lane Pass log, a generalised per-Site Lane row, and the deletion of the poller's in-memory page state. Display window **12h**; retention **14 days** — two different windows, deliberately. ### Migration 0014 ```sql CREATE TABLE poll_passes ( site text NOT NULL, ran_at bigint NOT NULL, -- unix ms, from p.Now() skip text NOT NULL, -- '' = the pass ran; else why it returned early due int NOT NULL, checked int NOT NULL, gap_ms bigint NOT NULL, clamped boolean NOT NULL, refused int NOT NULL, unreachable int NOT NULL, no_chapter int NOT NULL, unfetchable int NOT NULL, errors int NOT NULL, PRIMARY KEY (site, ran_at) ); ALTER TABLE poll_lanes ADD COLUMN refuse_until bigint NOT NULL DEFAULT 0; ``` `(site, ran_at)` is the whole index budget. One goroutine per Lane writes sequentially, so the pair is unique without a surrogate id, and it serves both reads: latest row per Site, and a per-Site window sum. Retention's `WHERE ran_at < cutoff` scans, which at ~1.7k live rows is cheaper than a second index. Volume: 5 Sites x ~24 passes/day = **~120 rows/day**. Not persisted, because each is derivable and a second copy is a second thing to drift: `refusing` (from `poll_lanes.refuse_until` — `LaneStatus` already overwrites the in-memory field at snapshot time, status.go:53), `browser` (registry: `isBrowserSite`), and any `ok` count (see below). ### The skip enum — one value per return path `runLanePass` has nine exits, and today a page cannot tell them apart: all it sees is `Due > 0, Checked == 0`, which reads as a stall in eight cases that are not one. One text column fixes that and replaces the `Asleep` boolean: | `skip` | return path | |---|---| | `''` | the pass reached the loop | | `paused` | #119's pause row read at the top | | `refusing` | refusal backoff (poller.go:229) | | `sidecar-down` | sibling browser Lane lost Chrome (poller.go:234) | | `no-fetcher` | browser Site, no `BROWSER_WS_URL`, no fallback (poller.go:245) | | `due-query` | `DueForLatestCheck` failed (poller.go:255) | | `asleep` | under both wake thresholds, ADR-0005 (poller.go:261) | | `eligible-count` | `EligibleSeriesCount` failed (poller.go:280) | | `nothing-eligible` | `eligible == 0`, sleeps a full rest (poller.go:290) | **The stall rule follows from it**: `due > 0 AND checked = 0 AND skip = ''` is the only true stall. Everything else is a Lane that declined to work and said why. ### Outcome counts — the classification already exists `readSeriesPage` classifies six outcomes and `checkOne` throws all of them at `log.Printf`. The pass row counts them: | column | source | |---|---| | `refused` | `errChallengeHeld` — 403 `cf-mitigated` or an interstitial body (read.go:55-69) | | `unreachable` | `errBrowserInterrupted` | | `no_chapter` | `!facts.HasLatest` — "no chapter links in N bytes" (poller.go:458) | | `unfetchable` | `errNotFetchable` (host pin) or `errNoFetcher` | | `errors` | everything else: non-200, transport, `MarkLatestChecked` failure | `no_chapter` is its **own** count and never folded into `refused`: it is what a broken adapter looks like when it breaks loudly, and #115's honesty requirement is precisely that the page not present adapter breakage as a quiet zero. A success count is **derived, never stored**: `ok = checked - (refused + no_chapter + unfetchable + errors)`. `unreachable` is excluded from that arithmetic because the sidecar-loss path returns before `st.Checked++` (poller.go:321-324), so the Series that lost Chrome was never counted as checked. A stored `ok` column would be a fifth way to get that wrong. **The page never renders the word "failures" bare.** Named counts, and a zero renders as *none observed* — the wording is the honesty, because the one failure kind that matters most (#79: an adapter parsing a *wrong* number successfully) is not in this taxonomy and never can be. ### Carry-forward stays on the write `recordLaneState` (status.go:75-77) carries the previous pass's `due/gap/clamped/checked` forward when the pass returned before computing them, so the page never states a zero it did not measure. That rule moves verbatim — **carry forward exactly when the pass's own gap is 0** (today: the `refusing` and `sidecar-down` exits) — so every row is self-describing and the read is one `DISTINCT ON`. Not preserved-by-accident: with `skip` recorded, a genuine zero next to `skip = 'due-query'` now reads correctly instead of as a measurement, which is why the rule is kept rather than widened. Rejected: NULL columns plus a read-side "last non-NULL per Site" (two-part read, `COALESCE` gymnastics for a value the writer already holds), and writing no row for a skipped pass (a Lane refusing for a day would look like a Lane that stopped existing). ### Store surface ```go type LanePass struct { Site string RanAt int64 Skip string Due, Checked int GapMS int64 Clamped bool Refused, Unreachable, NoChapter, Unfetchable, Errors int PausedUntil, RefuseUntil int64 // joined from poll_lanes on read; not pass columns } func (s *Store) RecordLanePass(p LanePass, retainBefore int64) error func (s *Store) LatestLanePass(site string) (LanePass, bool, error) func (s *Store) LatestLanePasses() ([]LanePass, error) func (s *Store) LanePassOutcomes(since int64) ([]SiteOutcomes, error) func (s *Store) SetLaneRefusal(site string, until int64) error ``` - `RecordLanePass` inserts and prunes in the same call — **delete-on-insert**, so the Lane goroutine is the pruner and no `time.Ticker` enters a backend that has none (`sessions` expires lazily at lookup for the same reason). `retainBefore` is caller-supplied, keeping the store clockless like `DueForLatestCheck`. - `LatestLanePasses` is `DISTINCT ON (site) ... ORDER BY site, ran_at DESC`, LEFT JOINed to `poll_lanes` — one query for the Lanes page, and the same rows the landing verdict sums. - `LatestLanePass(site)` is the carry-forward read: 5 reads an hour, and the recorder needs one Site, not five. - `LanePassOutcomes(since)` is `GROUP BY site` with `SUM` per outcome column over the window. ### What gets deleted - `Poller.laneStates`, `recordLaneState`, `LaneState`, `Status`, `LaneStatus()` — status.go goes away; the pass row is the record. - `web.LaneReporter`. **This overrides #114's Notes and #119's answer**, both of which kept it as one read-only method so the admin page stayed testable with no poller running. That goal is better served by no interface at all: with the store as the source, an admin test inserts a pass row and asserts the render, with no fake reporter to construct. The Note's letter goes, its reason survives. - `Poller.refuseUntil` (the map). Refusal becomes `poll_lanes.refuse_until`, read at the top of a pass alongside #119's pause read — one row, one query, two gates. `Poller.browserDownAt` **stays in memory**, and the distinction is the point: a refusal is the *Site's* mood and outlives our process, while the sidecar being unreachable is a fact about *our own reach* — a restart re-probing Chrome is correct behaviour, not lost state. Note the consequence: `refuse_until` becoming durable changes today's behaviour, where a restart forgets a refusing Site and immediately re-probes it. That is the same argument #119 made for `paused_until`. `Status.BrowserConfigured` came from `p.BrowserFetch != nil`, a Poller field. It becomes `BROWSER_WS_URL != ""` in the web layer's config — strictly more accurate, since it describes the deployment rather than whether one goroutine happened to construct a fetcher. Browser reachability is **derived**: any browser Site whose latest pass carries `unreachable > 0` inside `refuseBackoff` of now. That needs the constant exported as `latest.RefuseBackoff` (15m, sites.go:406) — one exported constant instead of one interface. ### One window constant, 12h `ownerWindow = 12 * time.Hour` in the web package, feeding **both** the staleness cutoff and the outcome window. #116 superseded #115's 24h figure for staleness only, so the failure window was still 24h on paper; the rationale it gave (the owner looks once by day and once by night, and each look should cover the interval since the last) governs both figures identically, and two different windows on one page is a comprehension bug. This retires #116's `staleAfter` name — a rename on paper, since nothing is built. Retention is **not** this constant: 14 days, and the two must not be collapsed. The window is what the owner is shown; retention is how far back a question can reach. ### What the pages read #115's split rule — "landing from the database, Lanes page from memory" — is **void**, because its premise was the memory this ticket deletes. Replacement, same guarantee on a new axis: **landing shows aggregates over Sites, the Lanes page shows per-Lane detail; no figure appears on both.** - **Landing**: the verdict line, now fully database-computed — `<verdict> · N series waiting · M unchecked over 12h`, with `N = SUM(due)` over the latest pass per Site. The per-Site table gains the five outcome sums over 12h from `LanePassOutcomes`. The "no Lane has reported yet" state does not disappear — it narrows from *after every deploy* to *a virgin database*. - **Lanes page**: one row per Lane with `due / checked / gap / clamped / skip / paused_until / refuse_until`, plus #119's Pause and Resume controls. Keeps the only timer on the surface (#115), still 30s — now a 5-row primary-key read instead of a map copy, which is what overrides #115's "the timer is free because it touches no database" argument: Postgres holds that page in cache, and one source of truth is worth more than the read. ### Amendments to closed tickets - **#119** — `poll_lanes` was specified as "a row exists only while paused; resume deletes it". It becomes the per-Site Lane state row: `paused_until` and `refuse_until` both default 0, `ResumeLane` **zeroes** `paused_until` instead of deleting the row, and `PausedLanes()` filters `paused_until > now`. A refusing-but-unpaused Lane needs a row, so the delete-on-resume invariant cannot survive. `skip = 'paused'` is this ticket's record of #119's pause exit. - **#116** — `staleAfter` becomes `ownerWindow`; see the new ticket below for the hole its filter set leaves. - **#114 Notes** — the `LaneReporter stays one read-only method` bullet is rewritten to say the admin page reads Lane state from the database. ### Pushed onto other tickets - **New ticket: per-Series Poll outcome state.** This ticket answers "what has the poller been doing" per Site, and nothing durable about *one* Series. `latest_checked_at` is stamped **before** the fetch (poller.go:428), so it means *attempted*, never *succeeded* — a Series broken for a month is `unchecked`-clean, `stale`-clean and `unpollable`-clean in every filter #116 defined, while its per-Site count rolls off in 12h. That is a hole in the filter set, not a nice-to-have. - **#118** — the Lanes page and the landing per-Site table are now filter-count consumers too; the outcome counts should link somewhere, and after the new ticket lands that somewhere is a `failing` filter. - **#122** — prototypes a Lane row carrying a `skip` explanation and five named outcome counts with a *none observed* empty state, not a single "errors" number. ### Rejected - **Latest pass upserted onto `poll_lanes`** (5 rows forever, no retention, no pruning). Fails only the 12h window — which is exactly the question "does that window earn a table", answered yes: without it the dashboard can say a Lane is broken *now* but not that it was broken at 04:00. - **One row per Series check.** Heaviest (a Series can produce 24 rows/day at `defaultRest = 1h`), and *still* insufficient: a pass that checked nothing writes nothing, so "the Lane ran, everything was rested, all healthy" — the primary health signal — is invisible. The per-Series question it would answer is answered better by three columns on `series`. - **A free-text last-error column.** It becomes the thing that gets read instead of the log, and it cannot be aggregated. - **Folding the admin audit trail in here** (#119 rejected `poll_commands` for the same reason, hanging it on this ticket). Not subsumed: different provenance (machine observation vs human action), different retention instinct, and a Forced Poll is already self-evidencing — `force_poll_at > latest_checked_at` *is* the record, ageing visibly while unserved. A 14-day pass log would silently expire the one thing worth looking back on. Glossary: `CONTEXT.md` gains **Lane Pass**. An ADR is worth writing when this is implemented (persisted Lane state plus the deletion of the in-memory twin), not now — the map produces the spec.
sulthan commented 2026-08-20 21:18:19 +07:00
Author
Owner
Copy Link
Copy Source

Amendments from #127 (closed):

  • poll_passes gains a sixth outcome count, not_found — a 4xx status other than 403. Your five-word taxonomy is otherwise reused verbatim. The reason for the split is that errors today covers a 404 (the page is gone, the owner must correct the address), a 5xx (wait) and a failed MarkLatestChecked (our database is unwell) — three different owner actions behind the one word a page prints. One branch in read.go, free on paper since nothing here is built.
  • The link push is withdrawn. You pushed "the outcome counts should link somewhere, and after the new ticket lands that somewhere is a failing filter"; #118 wrote it as all five pointing at ?site=…&filter=failing. #127 keeps them unlinked permanently. Two reasons: refused and unreachable deliberately write no per-Series state at all, so those two counts would open an empty list beside a large number; and the remaining four count attempts in a 12h window while the filter lists Series failing now for over 12h — a Series that broke at 04:00 and healed at 05:00 is counted and not listed, one broken three months on a paused Lane is listed and counted zero. Neither set contains the other. Each Site row on the Lanes page instead carries one separate navigation link to /admin/series?site=<site>&filter=failing.
  • Your two in-memory/durable split holds and is reused as the write rule: poll_lanes.refuse_until remains the whole record of a refusing Site and browserDownAt the whole record of a lost sidecar, so a refused or unreachable Poll issues no statement against the new poll_failures table.
  • ownerWindow (12h) is reused for the failing threshold — no new constant. Retention is not involved: poll_failures holds current state and is deleted on the next correct read.
Amendments from #127 (closed): - **`poll_passes` gains a sixth outcome count, `not_found`** — a 4xx status other than 403. Your five-word taxonomy is otherwise reused verbatim. The reason for the split is that `errors` today covers a 404 (the page is gone, the owner must correct the address), a 5xx (wait) and a failed `MarkLatestChecked` (our database is unwell) — three different owner actions behind the one word a page prints. One branch in read.go, free on paper since nothing here is built. - **The link push is withdrawn.** You pushed "the outcome counts should link somewhere, and after the new ticket lands that somewhere is a `failing` filter"; #118 wrote it as all five pointing at `?site=…&filter=failing`. #127 keeps them **unlinked permanently**. Two reasons: `refused` and `unreachable` deliberately write no per-Series state at all, so those two counts would open an empty list beside a large number; and the remaining four count *attempts in a 12h window* while the filter lists *Series failing now for over 12h* — a Series that broke at 04:00 and healed at 05:00 is counted and not listed, one broken three months on a paused Lane is listed and counted zero. Neither set contains the other. Each Site row on the Lanes page instead carries one separate navigation link to `/admin/series?site=<site>&filter=failing`. - Your two in-memory/durable split holds and is reused as the write rule: `poll_lanes.refuse_until` remains the whole record of a refusing Site and `browserDownAt` the whole record of a lost sidecar, so a `refused` or `unreachable` Poll issues **no statement** against the new `poll_failures` table. - `ownerWindow` (12h) is reused for the `failing` threshold — no new constant. Retention is not involved: `poll_failures` holds current state and is deleted on the next correct read.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something is broken
 enhancement
New feature or improvement
 needs-info
Waiting on reporter for more information
 needs-triage
Maintainer needs to evaluate
 ready-for-agent
Fully specified, ready for an AFK agent
 ready-for-human
Needs human implementation
 wayfinder:grilling
wayfinder:map
wayfinder:prototype
wayfinder:research
wontfix
Will not be actioned
No Label wayfinder:grilling
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
sulthan
No Assignees sulthan
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sulthan/mangaBookmark#117
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?