Durable Lane state: poll_passes and poll_lanes, plus their store surface #139

New Issue

2026-08-21T16:39:20+07:00

sulthan commented

2026-08-21 16:39:20 +07:00

Parent

Spec #134.

What to build

Lane observability stops dying with the process. Today Lane state lives in the poller's memory, so the one page whose job is proving the poller is alive says "no data yet" for up to an hour after every deploy. This ticket lands the durable home for that state and the store surface over it. Nothing reads it yet -- the poller starts writing it and the page starts reading it in later tickets -- so this one is verified at the store seam.

Two tables. A pass log, one row per Lane Pass, append-only, roughly 120 rows a day across six Sites: the Site, the run stamp in unix ms from the poller's clock, a skip text column (empty meaning the pass reached its loop, otherwise why it returned early), the pass figures (due, checked, gap, clamped) and five named outcome counts (refused, unreachable, no chapter, unfetchable, errors). (site, ran_at) is the whole index budget: one goroutine per Lane writes sequentially so the pair is unique without a surrogate id, and it serves both reads -- latest row per Site, and a per-Site window sum. Retention scans, which at roughly 1.7k live rows is cheaper than a second index.

A per-Site Lane state row holding paused_until and refuse_until, both defaulting to zero -- one row read at the top of a pass serving two gates.

Store methods, over a LanePass row type that carries the pause and refusal stamps joined from the Lane row on read rather than as pass columns:

RecordLanePass inserts and prunes in the same call -- delete-on-insert, so the Lane goroutine is the pruner and no ticker enters a backend that has none (sessions already expire lazily at lookup for the same reason). The retention cutoff is caller-supplied, keeping the store clockless as the due query already is. Retention is 14 days and is not the display window -- the window is what the owner is shown, retention is how far back a question can reach, and the two must not be collapsed.
LatestLanePass(site) -- the carry-forward read; the recorder needs one Site, not six.
LatestLanePasses() -- DISTINCT ON (site) newest-first, left-joined to the Lane row: one query serving both the Lanes page and the landing verdict's sum.
LanePassOutcomes(since) -- per-Site outcome sums over a window.
SetLaneRefusal, PauseLane (rejects a non-future expiry), ResumeLane (zeroes the pause stamp and keeps the row), PausedLanes, LanePausedUntil.

Migrations take the next free numbers. Measured 2026-08-21: the tree's migrations stop at 0011 and everything the parent map specified is unbuilt. Migrations are globbed by version, so contiguity at merge time is what matters, not agreement with any paper numbering.

Acceptance criteria

RecordLanePass prunes rows older than the caller's cutoff in the same call that inserts
LatestLanePasses returns exactly one row per Site, newest per Site, with the pause and refusal stamps joined in
LanePassOutcomes sums the five counts per Site over the given window and ignores rows outside it
PauseLane rejects a non-future expiry; ResumeLane zeroes the pause stamp rather than deleting the row
The store stays clockless: every cutoff and stamp arrives from the caller
Every query is parameterized and only compile-time constants reach query text
Store tests run against the throwaway Postgres with the real migrations, per the existing store test pattern
go test ./... green

Blocked by

None — can start immediately.

## Parent Spec #134. ## What to build Lane observability stops dying with the process. Today Lane state lives in the poller's memory, so the one page whose job is proving the poller is alive says "no data yet" for up to an hour after every deploy. This ticket lands the durable home for that state and the store surface over it. Nothing reads it yet -- the poller starts writing it and the page starts reading it in later tickets -- so this one is verified at the store seam. Two tables. A pass log, one row per Lane Pass, append-only, roughly 120 rows a day across six Sites: the Site, the run stamp in unix ms from the poller's clock, a `skip` text column (empty meaning the pass reached its loop, otherwise why it returned early), the pass figures (due, checked, gap, clamped) and five named outcome counts (refused, unreachable, no chapter, unfetchable, errors). `(site, ran_at)` is the whole index budget: one goroutine per Lane writes sequentially so the pair is unique without a surrogate id, and it serves both reads -- latest row per Site, and a per-Site window sum. Retention scans, which at roughly 1.7k live rows is cheaper than a second index. A per-Site Lane state row holding `paused_until` and `refuse_until`, both defaulting to zero -- one row read at the top of a pass serving two gates. Store methods, over a `LanePass` row type that carries the pause and refusal stamps joined from the Lane row on read rather than as pass columns: - `RecordLanePass` inserts and prunes in the same call -- delete-on-insert, so the Lane goroutine is the pruner and no ticker enters a backend that has none (sessions already expire lazily at lookup for the same reason). The retention cutoff is caller-supplied, keeping the store clockless as the due query already is. **Retention is 14 days and is not the display window** -- the window is what the owner is shown, retention is how far back a question can reach, and the two must not be collapsed. - `LatestLanePass(site)` -- the carry-forward read; the recorder needs one Site, not six. - `LatestLanePasses()` -- `DISTINCT ON (site)` newest-first, left-joined to the Lane row: one query serving both the Lanes page and the landing verdict's sum. - `LanePassOutcomes(since)` -- per-Site outcome sums over a window. - `SetLaneRefusal`, `PauseLane` (rejects a non-future expiry), `ResumeLane` (zeroes the pause stamp and keeps the row), `PausedLanes`, `LanePausedUntil`. Migrations take the next free numbers. Measured 2026-08-21: the tree's migrations stop at 0011 and everything the parent map specified is unbuilt. Migrations are globbed by version, so contiguity at merge time is what matters, not agreement with any paper numbering. ## Acceptance criteria - [x] `RecordLanePass` prunes rows older than the caller's cutoff in the same call that inserts - [x] `LatestLanePasses` returns exactly one row per Site, newest per Site, with the pause and refusal stamps joined in - [x] `LanePassOutcomes` sums the five counts per Site over the given window and ignores rows outside it - [x] `PauseLane` rejects a non-future expiry; `ResumeLane` zeroes the pause stamp rather than deleting the row - [x] The store stays clockless: every cutoff and stamp arrives from the caller - [x] Every query is parameterized and only compile-time constants reach query text - [x] Store tests run against the throwaway Postgres with the real migrations, per the existing store test pattern - [x] `go test ./...` green ## Blocked by None — can start immediately.

sulthan added the ready-for-agent label 2026-08-21 16:39:20 +07:00

sulthan referenced this issue

2026-08-21 16:39:20 +07:00

The poller records one pass row per exit, with a skip reason and outcome counts #141

sulthan referenced this issue

2026-08-21 16:39:22 +07:00

Overview page: a verdict line, and a stats block where every figure is a door #143

sulthan referenced this issue

2026-08-21 16:39:24 +07:00

Lanes page reads the database, and the in-memory Lane state is deleted #145

sulthan referenced this issue

2026-08-21 16:39:47 +07:00

Spec: admin dashboard - pages, Lane observability, poll pass log, and per-Series intervention #134

sulthan commented

2026-08-21 17:37:35 +07:00

Acceptance check complete against commit fd1131d.

RecordLanePass inserts and prunes older rows in the same transaction.
LatestLanePasses returns one newest row per Site via DISTINCT ON and left-joins current pause/refusal stamps.
LanePassOutcomes sums all five named outcomes per Site from the caller-supplied window boundary.
PauseLane rejects zero/negative expiry sentinels; the caller supplies and validates the wall-clock future timestamp. ResumeLane clears paused_until while retaining the poll_lanes row and any refusal stamp.
Store methods remain clockless: cutoffs and timestamps are caller arguments; queries bind request values and only concatenate compile-time column lists.
Store tests use the real migrations and throwaway Postgres.
Verification: go test ./... — 454 passed in 10 packages.

Ready to close.

Acceptance check complete against commit fd1131d. - RecordLanePass inserts and prunes older rows in the same transaction. - LatestLanePasses returns one newest row per Site via DISTINCT ON and left-joins current pause/refusal stamps. - LanePassOutcomes sums all five named outcomes per Site from the caller-supplied window boundary. - PauseLane rejects zero/negative expiry sentinels; the caller supplies and validates the wall-clock future timestamp. ResumeLane clears paused_until while retaining the poll_lanes row and any refusal stamp. - Store methods remain clockless: cutoffs and timestamps are caller arguments; queries bind request values and only concatenate compile-time column lists. - Store tests use the real migrations and throwaway Postgres. - Verification: go test ./... — 454 passed in 10 packages. Ready to close.

sulthan closed this issue

2026-08-21 17:37:43 +07:00

sulthan referenced this issue

2026-08-22 07:37:07 +07:00

Admin dashboard: pages, Lane observability, poll pass log, per-Series intervention (#134) #148

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sulthan/mangaBookmark#139