The poller records one pass row per exit, with a skip reason and outcome counts #141

Closed
opened 2026-08-21 16:39:20 +07:00 by sulthan · 1 comment
Owner

Parent

Spec #134.

What to build

When a Poll Lane stops working the web UI still loads, every Bookmark still opens, Progress still syncs, and Latest Chapter quietly stops moving. A Lane that is stuck looks exactly like a week when the Sites published nothing. Today a page sees only "due, nothing checked", which reads as a stall in eight cases that are not one. This ticket makes every way a pass can end write exactly one durable row saying what happened.

One text column, one value per return path: empty for a pass that reached its loop, then paused (the pause row was read at the top), refusing (refusal backoff), sidecar-down (a sibling browser Lane lost Chrome), no-fetcher (browser Site, no browser configured, no fallback), due-query (the due query failed), asleep (under both browser wake thresholds), eligible-count (the eligible count failed), nothing-eligible (nothing eligible; sleeps a full rest). The stall rule follows from it: due series, none checked, and no skip reason is the only true stall. Everything else is a Lane that declined to work and said why.

Five outcome counts from the classification the Series read already makes: refused (a challenge held), unreachable (the browser interrupted), no chapter (200, real HTML, no chapter found), unfetchable (the host pin refused the stored URL, or no fetcher), errors (everything else -- non-200, transport, a failed check stamp). No chapter is its own count and is never folded into refused: it is what a broken adapter looks like when it breaks loudly. A success count is derived, never stored -- checked minus the four, with unreachable excluded from that arithmetic because the sidecar-loss path returns before the checked counter increments. A stored success column would be a fifth way to get that wrong.

Carry-forward moves verbatim: a pass that returned before computing its figures carries the previous pass's due, gap, clamped and checked forward, exactly when the pass's own gap is zero, so no row states a zero it did not measure and the read stays a single latest-per-Site query. With a skip recorded, a genuine zero beside a failed due query now reads correctly rather than as a measurement, which is why the rule is kept rather than widened. Rejected: nullable columns with a read-side "last non-null per Site", and writing no row at all for a skipped pass (a Lane refusing for a day would look like a Lane that stopped existing).

Refusal becomes durable. The refusal gate at the top of a pass reads the Lane row rather than memory, and a refusal is recorded through the store. This deliberately changes today's behaviour, where a restart forgets a refusing Site and immediately re-probes it: a refusal is the Site's mood and outlives our process. The in-memory refusal map keeps being written for now, purely so the existing Lane status block still renders; its deletion belongs to the ticket that deletes the whole in-memory twin. The browser-down timestamp stays in memory on purpose -- the sidecar being unreachable is a fact about our own reach, and a restart re-probing Chrome is correct behaviour rather than lost state.

One deliberate omission: the mid-loop browser-unreachable return is left exactly as the spec has it, writing an empty skip. Do not invent a tenth enum value or amend the stall rule here; the consequence is recorded on #134 for the later spec in the series to settle.

Acceptance criteria

  • A pass records exactly one row with the right skip value for each of the nine documented return paths, driven at the existing round-at-a-time seam with a fake fetcher and an injected clock
  • Carry-forward fires exactly when the pass's own gap is zero, and not otherwise
  • The five outcome counts match the classification over a seeded mix of refusals, missing chapters, unfetchable URLs and transport errors
  • A durable refusal is honoured by a freshly constructed poller -- a restart does not re-probe a Site inside its backoff
  • The retention cutoff passed to the recorder is 14 days behind the poller's clock
  • No success count is stored anywhere
  • The existing Lane status block still renders unchanged; no reading-surface behaviour changes
  • go test ./... green

Blocked by

  • #139 — Durable Lane state: poll_passes and poll_lanes, plus their store surface
## Parent Spec #134. ## What to build When a Poll Lane stops working the web UI still loads, every Bookmark still opens, Progress still syncs, and Latest Chapter quietly stops moving. A Lane that is stuck looks exactly like a week when the Sites published nothing. Today a page sees only "due, nothing checked", which reads as a stall in eight cases that are not one. This ticket makes every way a pass can end write exactly one durable row saying what happened. One text column, one value per *return* path: empty for a pass that reached its loop, then `paused` (the pause row was read at the top), `refusing` (refusal backoff), `sidecar-down` (a sibling browser Lane lost Chrome), `no-fetcher` (browser Site, no browser configured, no fallback), `due-query` (the due query failed), `asleep` (under both browser wake thresholds), `eligible-count` (the eligible count failed), `nothing-eligible` (nothing eligible; sleeps a full rest). **The stall rule follows from it: due series, none checked, and no skip reason is the only true stall.** Everything else is a Lane that declined to work and said why. Five outcome counts from the classification the Series read already makes: refused (a challenge held), unreachable (the browser interrupted), no chapter (200, real HTML, no chapter found), unfetchable (the host pin refused the stored URL, or no fetcher), errors (everything else -- non-200, transport, a failed check stamp). No chapter is its own count and is never folded into refused: it is what a broken adapter looks like when it breaks loudly. **A success count is derived, never stored** -- checked minus the four, with unreachable excluded from that arithmetic because the sidecar-loss path returns before the checked counter increments. A stored success column would be a fifth way to get that wrong. Carry-forward moves verbatim: a pass that returned before computing its figures carries the previous pass's due, gap, clamped and checked forward, **exactly when the pass's own gap is zero**, so no row states a zero it did not measure and the read stays a single latest-per-Site query. With a skip recorded, a genuine zero beside a failed due query now reads correctly rather than as a measurement, which is why the rule is kept rather than widened. Rejected: nullable columns with a read-side "last non-null per Site", and writing no row at all for a skipped pass (a Lane refusing for a day would look like a Lane that stopped existing). **Refusal becomes durable.** The refusal gate at the top of a pass reads the Lane row rather than memory, and a refusal is recorded through the store. This deliberately changes today's behaviour, where a restart forgets a refusing Site and immediately re-probes it: a refusal is the *Site's* mood and outlives our process. The in-memory refusal map keeps being written for now, purely so the existing Lane status block still renders; its deletion belongs to the ticket that deletes the whole in-memory twin. The browser-down timestamp **stays in memory** on purpose -- the sidecar being unreachable is a fact about our own reach, and a restart re-probing Chrome is correct behaviour rather than lost state. One deliberate omission: the mid-loop browser-unreachable return is left exactly as the spec has it, writing an empty skip. Do **not** invent a tenth enum value or amend the stall rule here; the consequence is recorded on #134 for the later spec in the series to settle. ## Acceptance criteria - [ ] A pass records exactly one row with the right skip value for each of the nine documented return paths, driven at the existing round-at-a-time seam with a fake fetcher and an injected clock - [ ] Carry-forward fires exactly when the pass's own gap is zero, and not otherwise - [ ] The five outcome counts match the classification over a seeded mix of refusals, missing chapters, unfetchable URLs and transport errors - [ ] A durable refusal is honoured by a freshly constructed poller -- a restart does not re-probe a Site inside its backoff - [ ] The retention cutoff passed to the recorder is 14 days behind the poller's clock - [ ] No success count is stored anywhere - [ ] The existing Lane status block still renders unchanged; no reading-surface behaviour changes - [ ] `go test ./...` green ## Blocked by - #139 — Durable Lane state: poll_passes and poll_lanes, plus their store surface
sulthan added the ready-for-agent label 2026-08-21 16:39:20 +07:00
Author
Owner

Implemented on spec-134 (commit 503fb49).

  • One durable poll_passes row per exit: the nine documented skip values, empty for a pass that reached the loop; the mid-loop browser-unreachable exit writes an empty skip by design (no tenth value invented).
  • Five outcome counts from the read's own classification (refused, unreachable, no-chapter, unfetchable, errors); success is derived, never stored.
  • Carry-forward fires exactly when the pass's own gap is zero.
  • Durable refusal: top-of-pass gate reads the poll_lanes row; a freshly constructed poller does not re-probe a Site inside its backoff. In-memory twin kept for the Lane status block, per #134.
  • Pause gate read at the top of a pass, ahead of the refusal check.
  • Retention 14 days behind the poller's clock.

All nine paths + counts + carry-forward + restart-refusal + retention driven at the round seam with a fake fetcher and injected clock; go test ./... green. Two-axis review: no blocking findings.

Implemented on `spec-134` (commit `503fb49`). - One durable `poll_passes` row per exit: the nine documented skip values, empty for a pass that reached the loop; the mid-loop browser-unreachable exit writes an empty skip by design (no tenth value invented). - Five outcome counts from the read's own classification (refused, unreachable, no-chapter, unfetchable, errors); success is derived, never stored. - Carry-forward fires exactly when the pass's own gap is zero. - Durable refusal: top-of-pass gate reads the `poll_lanes` row; a freshly constructed poller does not re-probe a Site inside its backoff. In-memory twin kept for the Lane status block, per #134. - Pause gate read at the top of a pass, ahead of the refusal check. - Retention 14 days behind the poller's clock. All nine paths + counts + carry-forward + restart-refusal + retention driven at the round seam with a fake fetcher and injected clock; `go test ./...` green. Two-axis review: no blocking findings.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sulthan/mangaBookmark#141