mangaBookmark/backend/AGENTS.md

Scope: `backend/`.

Each entry names the code that holds the truth — read that for *what it does*.
The prose here is only what code cannot tell you: rationale, rejected
alternatives, dated measurements, and invariants a plausible refactor would
silently break.

### Layout

`backend/main.go` → `newRouter` is the composition root, the only place
packages are wired. Packages under `backend/internal/`: `store`, `latest`,
`session`, `httpmw`, `api`, `userscript`, `web`, `token`, `pgtest`. Root-level
`*_test.go` exercise the full router; unit tests live beside their package.

Not visible from any single file: stdlib `net/http` with no framework,
Postgres over `jackc/pgx/v5`, `CGO_ENABLED=0` static binary into a distroless
image, TLS terminated by the reverse proxy so the service listens plain `:8080`.

### Schema — `internal/store/migrations/*.sql`, run by `store.migrate`

- Migration files are **append-only**. Editing an applied one changes nothing
  on a database that already recorded its version in `schema_migrations`, so
  the fix silently applies to new deployments only.
- No column probing, no data-fixup migrations. Both were SQLite-era machinery
  and were removed deliberately — don't reintroduce either.

### Tests need Docker — `internal/pgtest`

`pgtest.Main` from `TestMain` starts one `postgres:17-alpine` per test binary;
`pgtest.URL` hands each test its own database. A package whose tests touch the
store must have that `TestMain` or it has no database at all.

### Reader-owned store — `internal/store`, `internal/token`

Four tables; shape is in the migrations, behaviour in `Store`'s methods.

- **Credentials are derived, never stored.** `token.Token(TOKEN_KEY, discord_id, epoch)`
  is an HMAC; only its SHA-256 reaches `readers.token_sha256`. So install URLs
  can be rebuilt after any restart, and a database leak yields nothing usable.
- **The owner's epoch-0 hash is refreshed at startup only while the row has
  never been rotated.** Drop that condition and a restart resurrects a
  rotated-away credential.
- `Store.EnsureReader` never rewrites an existing row's hash — a returning
  Reader's login must not invalidate their installed scripts.
- **Every read and write is scoped to the acting Reader**, resolved from the
  presented credential by `httpmw.Auth` and carried in the request context.
  There is no unauthenticated-by-Reader route and no global token.
- **`series` holds what readers share, `bookmarks` only what differs.** A
  bookmark key is `(reader_id, site, series_id)` with no surrogate id; the wire
  `key` is derived as `site:series_id` on read.
- `Store.Upsert` splits one flat body across both tables and enforces the
  ownership rule: client `title`/`series_url`/`cover` are written **only when
  the series row is new**, so one reader cannot retitle a shared series.
- Sync is last-write-wins and the wire format stays flat — clients depend on
  both; neither is an implementation detail to tidy up.

### Web UI — `internal/web`

Routes, templates and assets are all in that package; `AdminPatterns()` and
`adminRoutes()` enumerate the privileged ones.

- **`backend/Dockerfile` must copy the whole `internal/` tree**, not just
  `*.go`: templates and static assets are `go:embed`-ed from
  `internal/web/`.
- **Guild membership *is* registration.** `discordCallback` gates on membership
  (plus `DISCORD_REQUIRED_ROLE` when set) and only then calls
  `Store.EnsureReader`, so a refusal creates nothing.
- Sessions are rows, not signatures: the cookie carries an opaque id and
  expiry is checked on lookup, which is what makes deleting the row an instant
  revocation.
- UI mutations go through `Store.Get` + `Store.Upsert` so the `updated_at` rule
  below stays in exactly one place.
- `listView.Fresh` exists because a Reader with no bookmarks at all needs
  install links, not an empty-filter message.
- **Design-tool caveat:** `detect.mjs backend/internal/web/templates` reports a
  **false clean**. Templates link `/static/style.css` root-absolutely (correct —
  it is served from `/`), but the detector resolves hrefs with
  `path.resolve(fileDir, href)`, which drops the directory on a leading `/` and
  skips the file silently; a relative href doesn't help either, since a
  template's directory isn't its served path. Always pass
  `backend/internal/web/static` too. The one finding there, `overused-font` on
  "Instrument Serif", is a deliberate identity choice, not debt.

### Confirm gating — `internal/web/static/filter.js`, `toggleConfirmRow(key, kind)`

Every action that pulls a series out of the list (`archive|finish|remove`) opens
its own `.confirm-row`; restore fires instantly because it is the reversal.
Remove wears the ember wash, the two reversible ones wear `.calm` grey.
**`--ember` is reserved for the new-chapter signal** — the busy bar and inline
errors must use `--mute`, or the one colour that means "something to read"
stops meaning it.

### Latest-chapter poller — `internal/latest`, Site registry in `sites.go`

One goroutine per Site (a Poll Lane) re-checks that Site's bookmarked series
from the backend's own network position, so `latest_chapter` stays fresh while
nobody is browsing. The userscript's `reportLatestChapter` is a second,
parallel signal — it PUTs every read, unchanged numbers included, because an
unchanged read is exactly the Sighting worth deferring a Poll on.

- **Pace lives in the Site registry, not config.** Two clocks: per-series rest
  (`series.latest_checked_at`, enforced in `Store.DueForLatestCheck`'s WHERE)
  and per-Lane gap (`effectiveGap`). The five env knobs that used to size one
  shared pace are gone; don't add them back.
- **The poller walks Series, not Bookmarks** — a series several readers hold is
  fetched once per cycle, and the due queue orders `reader_count DESC,
  latest_checked_at ASC` so the widely-read ones win contention.
- **The series row is stamped *before* the fetch**, so a permanently broken
  series waits out its rest instead of being retried every tick.
- `Store.SetLatestChapter` is a single-column UPDATE, deliberately not a
  read-modify-write of the bookmark: it therefore cannot revert read progress
  or move `updated_at`. The old stale-re-read race died with the Get+Upsert
  flow — don't restore one here.

**Sightings** (`Store.RecordSighting`, the due query's HAVING clause,
`latest.checkOne`) let a Reader's own page read defer a Poll.

- Recorded by the PUT handler **before** the Upsert, because the raise test
  needs the row as it stands.
- A Series is deferred only while it has exactly one Bookmark, was sighted
  within one Rest, and is under `sightingCeilingRests` since its last Poll — so
  a shared Series is never deferred and nothing goes six hours unpolled
  whatever arrives.
- A *higher* report clears the attribution rather than crediting it: the value
  the Poll then stores is its own, so a later retraction isn't the Reader's
  fault.
- `store.SightingDisagreementLimit` contradictions stop a Reader deferring —
  their reports still write the Latest Chapter — and
  `store.SightingAgreementsToClear` agreements forgive them, as does the
  owner's clear-marks control.
- Deferral is recomputed from live facts each round, so nothing needs
  invalidating when a Series gains a second Bookmark. The one input read
  earlier is the Reader's marks, so crossing or clearing a threshold takes
  effect from their next Sighting and the standing already bought lasts out its
  rest.

**Refusals and browser loss are Lane-local.** Two `errChallengeHeld` in a pass
stop that Site for `refuseBackoff` while other Lanes continue. An
`errBrowserInterrupted` (remote Chrome restarted) sets a shared Poller flag so
the *other* browser Lanes skip their passes for the same window — otherwise a
restarting Chrome stamps one Series per Lane per pass, burning rests on
failures. The flag decays and they probe again.

- **`isInterstitial` matches the orchestration path
  `/cdn-cgi/challenge-platform/h/`, never the bare prefix.** Cloudflare injects
  `/cdn-cgi/challenge-platform/scripts/jsd/main.js` into ordinary 200 pages
  once a zone turns JS detections on, which demonic did on 2026-08-16: the
  prefix match read every real demonic page as a refusal and parked the Lane in
  backoff while plain TLS was returning full series pages.
- Fetches use `bogdanfinn/tls-client` with a Chrome profile as defence in depth
  against fingerprint blocking; any failure logs and skips.
- kagane, comix and novelfull sit behind Cloudflare JS challenges the TLS
  client can't clear, so they go over CDP (`BROWSER_WS_URL`). kagane and comix
  are simply not polled when it's unset — a plain fetch would only retrieve a
  challenge page — while novelfull still attempts plain TLS, because its
  challenge is a live time-varying fact and its cover bytes never need a browser.
- **comix's browser read is an in-tab `fetch()` of the Series URL, not a DOM
  render.** It is an SPA: rendering cost ~65 requests for the same
  server-rendered HTML one fetch returns (measured 2026-08-12).
- Browser Lanes wake Chrome only when 5+ Series are due or one has waited 15m,
  and cover work runs in the background so a slow CDN can't eat a Lane's gap.

### Covers — `Store.OnSeriesCreated`, `latest.Acquirer`, `latest.CoverBytesFetcher`, `Store.SetSeriesCover`

Acquired once when the first Bookmark of a Series is created, then served from
our own origin by the public `GET /covers/{addr}`.

- Acquisition runs in a goroutine: the Reader's PUT must neither block on a
  Site nor fail with one. Every failure is logged and dropped, leaving the
  Bookmark intact.
- The wire `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once
  bytes exist and `""` before — **never an address that 404s**. Absolute
  because the userscript renders it on a Site's origin.
- `GET /covers/{addr}` is public and uncredentialed by design: no cookie or
  token of ours may travel to a Site's origin.
- A client-sent `cover` is decoded and discarded, permanently — wire
  compatibility, not an oversight.
- **One route serves all six Sites.** No proxy, no per-Site rewrite, no second
  place that decides a renderable address: the wire `cover` is it. Templates
  render `.Cover` and nothing else. The old kagane-only serving path
  (`/img/kagane/{id}` plus a template rewrite) is gone; don't reintroduce a
  per-Site route because one Site's CDN misbehaves.
- The only Site names left in cover code are in `browserOnlyCoverURL`
  (`internal/latest`): kagane answers a plain fetch with a challenge *and*
  `cross-origin-resource-policy: same-origin`, and `static.comix.to` answers
  with the same challenge its pages serve. Every other Site's CDN answers plain
  TLS.
- **comix cover bytes must arrive by direct navigation, not an in-page fetch:**
  its Series page sets `cross-origin-embedder-policy: require-corp`, which
  fails a page-context fetch of `static.comix.to`.
- With no browser configured, kagane and comix Covers are simply absent;
  novelfull still gets one whenever its page answers a plain request.

### `updated_at` drives list order — `Store.Upsert`

The server applies its own timestamp only when the row is new or
`last_chapter_num` changes, else it keeps the stored value. **Favouriting a
series, or a newly published chapter arriving, must not reorder the list** —
only real reading progress moves a row. Consequently `PUT` returns the row **as
stored** and clients must adopt that response rather than their own payload.

### Lifecycle buckets — `status` on each bookmark

`reading` | `archived` | `finished`, orthogonal to `favorite`. Archived and
finished appear only in their own tab, never in All, Updated, Favourites or the
recent strip. The poller keeps checking archived series and skips finished ones.

- `finished` is settable only from the web UI; `PUT /bookmarks/{key}` rejects
  it with 400.
- **An empty incoming status means "keep the stored one"**, and it is resolved
  on the `VALUES` side of `Store.Upsert`, not in the conflict clause:
  `excluded.*` is the post-evaluation row, so a default applied there would
  wipe the bucket on every PUT from a client predating the column.

### Config — `Config` / `loadConfig` / `loadLatestPoll` in `backend/main.go`

That function is the complete list of env vars, their defaults, and which are
required. What it can't tell you:

- `PUBLIC_BASE_URL` must be an absolute origin because every Cover URL on the
  wire is built from it and the userscript renders on a Site's origin.
- `BROWSER_WS_URL` **must be a tailnet IP, never a hostname** — Chrome's
  DevTools handler 500s `/json/version` for any Host that isn't an IP or
  `localhost`. Unset (the default) disables browser polling.
- `USERSCRIPT_PATH` / `NOVEL_USERSCRIPT_PATH` are bindmounted files; the
  `__API_TOKEN__` placeholder inside them is substituted with the requesting
  Reader's credential at serve time.
- Pace is per Site in the registry, not env. The
  `_COOLDOWN`/`_BROWSER_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` knobs are
  gone on purpose.
- The 1h rest for browser Sites is safe on documented grounds: a challenged
  page costs seconds of a serialized single-tab browser, free-plan zones carry
  no bot score and no published per-IP rate input, and `cf_clearance` expires
  in 30 minutes, so every cadence at or above 1h re-solves anyway.

### Userscript install & rotation — `internal/userscript`, `internal/token`

Session-gated `GET /install/{manga,novel}-bookmark.user.js` renders the
bindmounted script with the acting Reader's derived credential substituted in,
so the credential never appears in page markup, the address bar, or a redirect.
`?download=1` adds `Content-Disposition: attachment` for mobile Violentmonkey,
which ignores a `.user.js` navigation. `POST /rotate-token` is an atomic epoch
bump plus hash rewrite and invalidates every installed copy — the panel must
keep warning to reinstall on all devices.

### Owner-only admin — `internal/web/admin.go`

- **Every route reaching past the acting Reader is listed in `adminRoutes()`
  and wrapped in `requireOwner` at registration** — add it there, not as a
  check inside a handler; `web.AdminPatterns()` is what the gate test walks. A
  non-owner gets 404, never 403.
- The one owner comparison left outside the gate is in `index`
  (`view.Owner = readerID == h.store.OwnerID()`): it gates a link, not an
  endpoint, so it is a rendering decision a registration-time wrapper cannot
  express. Do not "unify" it into the gate.
- Lane figures come through the `web.LaneReporter` seam
  (`latest.Poller.LaneStatus`), never a table. `main.newRouter` takes the
  reporter as an interface and converts a nil `*Poller` to a nil interface — a
  typed nil would make the page claim a poller exists.
- A pass that returns before computing figures (refusal backoff, sidecar down)
  carries the previous pass's numbers forward rather than recording zeroes.
- **`Checked` next to `Due` is what separates a stopped Lane from a quiet one**,
  so neither may be dropped from the row.
- Due-without-Checked is **not** by itself a stall: a browser Lane under both
  wake thresholds sets `LaneState.Asleep` and renders "browser asleep", and
  never counts toward `Attention`. That is the commonest healthy state for
  kagane, comix and novelfull, so spending the stall mark on it would train the
  owner to ignore the mark that matters.