diff --git a/AGENTS.md b/AGENTS.md index 3ad738b..9402151 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ # AGENTS.md -Guidance for OpenCode (and Claude Code) working in this repo. +Repo-wide guidance for coding agents. ## What this is @@ -13,15 +13,18 @@ One backend, one `bookmarks` table: a `kind` column (`manga`|`novel`) splits the ## Hard constraints (drive design — don't violate) +Nothing below is derivable from reading the code — it is why the code looks the +way it does, plus dated measurements against services we don't control. + Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free where plain web APIs suffice — keeps portability across engines: - **Avoid `GM_*` unless needed.** Prefer page `localStorage` over `GM_setValue`/`GM_getValue`, on-page UI over `GM_registerMenuCommand`, plain `fetch()` over `GM_xmlhttpRequest` for cross-origin. - Cross-origin `fetch()` work **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block). - Every site is its **own origin with its own `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync required, not optional. - Userscript run in **isolated world**, so embedded API token safe from site's JS. -- Cloudflare's block on manga sites is **per-zone configuration plus request fingerprint, not IP reputation — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — a Site can turn its protection on overnight, which is exactly what comix.to did on 2026-08-12. An earlier version of this line blamed "Cloudflare's bot scoring"; that was wrong. The 1-99 bot score is Enterprise Bot Management only and does not exist for a free-plan zone, and no per-IP request rate is documented as an input to challenge issuance — `docs/research/cloudflare-bot-scoring-and-poll-cadence.md`. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test. -- **kagane.to, comix.to and novelfull.com are the exception to the above** — all three sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`). When that's unset, kagane and comix are skipped entirely (a plain fetch would only retrieve a challenge page) while novelfull pages are still attempted over plain TLS — its challenge is a live time-varying fact and its cover bytes never need the browser. comix turned hostile on 2026-08-12 (#98): its cover host `static.comix.to` is gated too, so its cover bytes go through the browser as well, and its page is read as an in-tab `fetch()` of the series URL rather than a rendered DOM — comix is an SPA, and rendering costs ~65 requests for the same server-rendered HTML one fetch returns. The three other sites poll fine over plain TLS. +- Cloudflare's block on manga sites is **per-zone configuration plus request fingerprint, not IP reputation — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — a Site can turn its protection on overnight, which is exactly what comix.to did on 2026-08-12. An earlier version of this line blamed "Cloudflare's bot scoring"; that was wrong. The 1-99 bot score is Enterprise Bot Management only and does not exist for a free-plan zone, and no per-IP request rate is documented as an input to challenge issuance. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test. +- **kagane.to, comix.to and novelfull.com are the exception to the above** — all three sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`). When that's unset, kagane and comix are skipped entirely (a plain fetch would only retrieve a challenge page) while novelfull pages are still attempted over plain TLS — its challenge is a live time-varying fact and its cover bytes never need the browser. comix turned hostile on 2026-08-12: its cover host `static.comix.to` is gated too, so its cover bytes go through the browser as well, and its page is read as an in-tab `fetch()` of the series URL rather than a rendered DOM — comix is an SPA, and rendering costs ~65 requests for the same server-rendered HTML one fetch returns. The three other sites poll fine over plain TLS. - **The CDP browser must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a non-UTC clock zone cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, ignoring the file's contents, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead. -- **The browser is not in the API stack and must not be put back.** It's its own compose unit (`chrome/docker-compose.yml`) on a second machine, reached over the tailnet — it held 471 MiB on a 1974 MiB swapless VPS, and a residential egress avoids the cloud-hosting-IP signature Bot Fight Mode documentedly challenges (ADR-0006; not a better "score" — free-plan zones have no score). Consequences that constrain code: `BROWSER_WS_URL` must be a tailnet **IP** (a MagicDNS name 500s at `/json/version`, same trap as the old Docker service name); the CDP port binds to the tailnet address only, since CDP authenticates nothing and that host has a real LAN; and the browser is on-demand (ADR-0005), so an unreachable or asleep one must degrade exactly as an unset `BROWSER_WS_URL` — plain-TLS libraries unaffected, kagane/comix logged and skipped, stored covers still served. Never add `chromedp.NoModifyURL`: discovery per fetch is what makes a restarted Chrome invisible. +- **The browser is not in the API stack and must not be put back.** It's its own compose unit (`chrome/docker-compose.yml`) on a second machine, reached over the tailnet — it held 471 MiB on a 1974 MiB swapless VPS, and a residential egress avoids the cloud-hosting-IP signature Bot Fight Mode documentedly challenges (not a better "score" — free-plan zones have no score). Consequences that constrain code: `BROWSER_WS_URL` must be a tailnet **IP** (a MagicDNS name 500s at `/json/version`, same trap as the old Docker service name); the CDP port binds to the tailnet address only, since CDP authenticates nothing and that host has a real LAN; and the browser is on-demand, so an unreachable or asleep one must degrade exactly as an unset `BROWSER_WS_URL` — plain-TLS libraries unaffected, kagane/comix logged and skipped, stored covers still served. Never add `chromedp.NoModifyURL`: discovery per fetch is what makes a restarted Chrome invisible. - **UTC is the tell, not a country mismatch.** A UTC clock is the datacenter default, and the challenge refuses it; any real zone clears. Measured 2026-08-08, identical container, one Indonesian egress IP: UTC never cleared in 60s (twice), while `Asia/Jakarta` **and** `America/New_York` both cleared in 4s. An earlier note here claimed the zone had to match the egress IP's country — that was wrong, inferred from the host clock (`Asia/Bangkok`) rather than the measured egress. A second earlier claim, that Cloudflare "scores" a UTC clock, was also wrong: the measurement is real but the mechanism is not documented anywhere — Cloudflare publishes no timezone signal, and free-plan zones carry no score at all. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one. - **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives. @@ -36,7 +39,7 @@ Two Violentmonkey userscripts (isolated world, per-site adapters, localStorage c on-demand Chrome, separate machine (chrome/) ``` -Two deployable units on two machines: the API stack (`docker-compose.yml` + `docker-compose.prod.yml`, on the VPS) and the browser (`chrome/docker-compose.yml`, on the home machine). They share nothing but `BROWSER_WS_URL` and update independently. Backend-specific architecture (packages, endpoints, poller, config env vars) lives in `backend/AGENTS.md`. Userscript-specific structure (adapters, retry queue, UI, live URL shapes) lives in `userscript/AGENTS.md`. Deploy order `DEPLOY.md` (§7 for the browser), redeploy `REDEPLOY.md` (§8 for the browser). +Two deployable units on two machines: the API stack (`docker-compose.yml` + `docker-compose.prod.yml`, on the VPS) and the browser (`chrome/docker-compose.yml`, on the home machine). They share nothing but `BROWSER_WS_URL` and update independently. Backend-specific detail lives in `backend/AGENTS.md`, userscript-specific detail in `userscript/AGENTS.md`. ## Commands @@ -64,28 +67,29 @@ Smoke test: `curl` endpoints with `Authorization: Bearer `; confirm `OPTI ## Forge: Gitea, not GitHub -`origin` is self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` don't work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones: +`origin` is self-hosted Gitea instance (`gitea.violetcrown.my.id`, repo `sulthan/mangaBookmark`), so **`gh` don't work here — use `tea` (Gitea CLI) for anything past plain git.** `tea` infers the repo from `origin`; auth lives in `tea login`, not a `GH_TOKEN` env var. It prints rendered boxes rather than plain text, so pass `-o json` when parsing; a PR URL lands on the last line. -- Open PR: `tea pr create --head --base main --title "..." --description "..."` -- List / view / check out: `tea pr list`, `tea pr `, `tea pr checkout ` -- Issues: `tea issue create`, `tea issue list` -- Auth lives in `tea login`, not `GH_TOKEN` env var. - -`tea` print output as rendered boxes rather than plain text; PR URL lands on last line. +- PRs: `tea pr create --head --base main --title "..." --description "..."`, `tea pr list`, `tea pr `, `tea pr checkout `. +- Issues: `tea issue create --title "..." --description "..."` (`--labels`, `--assignees` optional), `tea issue --comments`, `tea issue list --state open|closed|all -o json`, `tea issue close `. +- Comments: `tea comment "..."` — `tea issue close` takes no `--comment` flag. +- Labels: `tea issue edit --add-labels "..."` / `--remove-labels "..."`. Gitea will **not** auto-create a label, so `tea labels create --name "..." --color "#rrggbb"` first. +- Triage vocabulary is `needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`. +- Gitea shares one index space across issues and PRs, so a bare `#42` may be either — try `tea pr 42`, fall back to `tea issue 42`. ## Design system -Web UI + userscript panel follow **Cinder**, rules in `docs/design-system.md` -— source of truth Claude Design project `BookmarkManager Web UI` -(`969ac210-fe02-4c01-ae1b-9a271dcc779a`). Read it before touching -`backend/internal/web/static/style.css`, `backend/internal/web/templates/*`, or userscript -`TEMPLATE`/`CSS`. Core law: **ember means new chapter only** — no other -state (busy, error, destruction) may use `--ember`; destruction gets -`--danger`. No cards/corners/shadows, one `--measure: 760px` column, tokens -only (never hardcode hex outside `:root`), both colour branches touched -together. Any move that pulls series out of list (archive/finish/remove) -must be confirm-gated via its own `.confirm-row`; only restore fires -instantly. +Web UI + userscript panel follow **Cinder**. Tokens are the `:root` block in +`backend/internal/web/static/style.css`; that file, `backend/internal/web/templates/*`, +and the userscript `TEMPLATE`/`CSS` are the only places it is expressed. +Source of truth for the visual language is the Claude Design project +`BookmarkManager Web UI` (`969ac210-fe02-4c01-ae1b-9a271dcc779a`). + +Core law: **ember means new chapter only** — no other state (busy, error, +destruction) may use `--ember`; destruction gets `--danger`. No +cards/corners/shadows, one `--measure: 760px` column, tokens only (never +hardcode hex outside `:root`), both colour branches touched together. Any move +that pulls a series out of the list (archive/finish/remove) must be +confirm-gated via its own `.confirm-row`; only restore fires instantly. ## Security invariants @@ -127,13 +131,19 @@ Review gate: auth, CORS, session, crypto, and the fetch gate are security-critic ## Comments Comment only if code alone can't carry info. Cost per read — must earn spot. +Wrong comment worse than none: it misleads readers and measurably degrades +LLM performance on the file. Missing comment costs little. Bias to fewer. -Write for: -- Why not what. Tradeoffs, non-obvious decisions. -- Load-bearing detail looking incidental — say so if "simplify" breaks it. +Docstring on public/exported surface — exception, near-always worth it. +Contract only: what it takes, returns, throws, mutates; units; pre/post +conditions. Not a restatement of the body. Skip on private/obvious. + +Inline — write for: +- Why not what. Tradeoffs, non-obvious decisions, rejected alternatives. +- heavy detail looking incidental — say so if "simplify" breaks it. - Non-local consequence, invisible from function alone. -- Wire format / encoding / interface contract — save callers re-deriving. -- Gotcha/workaround, with ref if exists. +- Wire format / encoding / ordering / invariant — save callers re-deriving. +- Gotcha/workaround, with ref (issue, RFC, vendor bug) if exists. - Domain/business rule not derivable from code. Skip: @@ -142,24 +152,49 @@ Skip: - Banners, dividers, `// helpers`. - Change narration (`// fix bug`, `// as requested`, `// new impl`) — git's job. - Commented-out code — delete. -- TODO without concrete action. +- TODO without concrete action + owner. +- Narrating the plan you just reasoned through. Plan in prose or in your head; +ship the code, not the transcript. +- Anything restating a name that could be fixed by renaming instead. -Style: one dense comment over function beats one per line inside. Tight, no worked example unless bug subtle. Wrong comment worse than none — update/delete on change. Default fewer — sparse+high-signal beats comprehensive. +Staleness filter: if the comment describes something likely to change +independently of this line, it will rot and start lying. Either anchor it to +something stable, assert it in a test, or leave it out. -Test: "competent reader get this from code in few sec?" Yes → skip. Needs detour through another file/spec/git-blame → write it. +Style: one dense comment over a function beats one per line inside. Tight; no +worked example unless the bug is subtle. On edit, update or delete stale +comments in the code you touch — silence beats a lie. -## Agent skills +Test: "competent reader get this from code in a few sec?" Yes → skip. +Needs detour through another file/spec/git-blame/external doc → write it. -`AGENTS.md` is the single source of truth for agent guidance; every `CLAUDE.md` in this repo is a symlink to the `AGENTS.md` beside it. Edit `AGENTS.md`. +## Writing an AGENTS.md -### Issue tracker +`AGENTS.md` is the single source of truth for agent guidance; every `CLAUDE.md` +in this repo is a symlink to the `AGENTS.md` beside it. Edit `AGENTS.md`. -Issues live as Gitea issues on `gitea.violetcrown.my.id` (`sulthan/mangaBookmark`), driven by the `tea` CLI — not `gh`. See `docs/agents/issue-tracker.md`. +**Cite code, never docs, issues, or plans.** A spec, ADR, plan file, or Gitea +issue records what was true when it was written and then goes stale silently; +an agent that follows the pointer reads a decision that may already have been +reversed. Code is the only source true at read time — cite a package, file, +symbol, env var, or route. The sole non-code exception is a sibling +`AGENTS.md`. If a doc holds a fact an agent needs, restate the fact here rather +than linking to it. -### Triage labels +**State a fact in prose only if the code cannot answer it.** Split by +derivability: -Default five-role vocabulary, label strings unchanged (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`). See `docs/agents/triage-labels.md`. +- *Structure* — packages, routes, env vars, columns, struct fields. Rots fast, + cheap to re-read. **Name the symbol, write nothing else.** +- *Mechanism* — what a function does, how a flow proceeds. **Name the symbol + plus at most one line of orientation.** +- *Rationale* — why it is this way, what a "simplify" would break, what was + tried and rejected. Not in the code and cannot be re-derived. **Write it out.** +- *Measurement* — an observation against something we don't control. **Write it + out with the date**; a dated fact is honest, an undated one pretends to be + permanent. -### Domain docs - -Single-context: one root `CONTEXT.md` plus `docs/adr/`, both created lazily. See `docs/agents/domain.md`. +Restating mechanism in prose is how these files rot: the code changes, the +paragraph doesn't, and the next agent trusts the paragraph. A pointer degrades +more honestly — and every symbol you name must actually exist, since a dead +pointer is a bug, not a stale sentence. diff --git a/backend/AGENTS.md b/backend/AGENTS.md index a03c7a2..2fd68d6 100644 --- a/backend/AGENTS.md +++ b/backend/AGENTS.md @@ -1,274 +1,268 @@ -Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system. +Scope: `backend/`. -- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + Postgres over `jackc/pgx/v5` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`. - Single binary, split into packages under `backend/internal/`: `store` - (Bookmark type, Postgres persistence, migration runner), `latest` (background - poller, site parsers, TLS fetcher), `session` (cookie signing, login - rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON - bookmark handlers), `userscript` (userscript-serving handler), `web` - (browser UI handler + `templates/` + `static/`, `go:embed`-ed). - `backend/main.go` is the composition root — the only place that wires - packages together into `newRouter`. Root-level `*_test.go` hold - integration tests that exercise the full router; unit tests for a - package live beside it under `internal/`. -- **Schema is migration-owned.** `internal/store/migrations/*.sql` is - `go:embed`-ed and applied on every start by `store.migrate`: one numbered - file per change, one transaction each, versions recorded in - `schema_migrations`. Files are **append-only** — editing an applied one - changes nothing on a database that already ran it. No column probing, no - data-fixup migrations: both were SQLite-era machinery and are gone. -- **Tests need Docker.** `internal/pgtest` starts one `postgres:17-alpine` - container per test binary (`TestMain` -> `pgtest.Main`) and hands each test - its own database (`pgtest.URL(t)`). A package whose tests touch the store - must have that `TestMain`. -- **Reader-owned store, four tables.** `readers` is keyed by Discord user ID - and carries the SHA-256 of the Reader's userscript credential plus a - `token_epoch` (issue #24). Credentials are derived, never stored: `token.Token(TOKEN_KEY, discord_id, epoch)` (HMAC, `internal/token`), and only its SHA-256 sits in `readers.token_sha256`, so install URLs can be rebuilt after any restart while a database leak yields nothing but hashes. The seed creates the **owner** row at startup; its epoch-0 hash is refreshed on every start **only while the row has never been rotated**, so a restart can never resurrect a rotated-away credential. Every other row is created by that Reader's own first login (`Store.EnsureReader`, idempotent on `discord_id`, and it never rewrites an existing row's hash). Rotation is `Store.RotateToken` (epoch bump + hash rewrite in one transaction), driven by the web UI. - `series` keyed `(site, series_id)` - (`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the - shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`), - Latest Chapter, `latest_checked_at`, and the Sighting pair - `latest_sighted_at`/`latest_raised_by` (issue #103) — and `bookmarks` holds only what - differs between readers: progress, favourite, lifecycle bucket, - `updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no - surrogate id; the wire `key` is derived as `site:series_id` on read — and - every store read/write is scoped to the reader it names. Auth resolves the - acting Reader from the presented credential (`httpmw.Auth`) and nothing - else — there is no unauthenticated-by-Reader route and no global token; the - reader id travels in the request context. Sync **last-write-wins**; the wire format - stays flat (ADR-0004). `Store.Upsert` decomposes one flat body across two - tables and enforces the ownership rule: client `title`/`series_url`/`cover` - are written only when the series row is new (ADR-0003). -- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). -- **Web UI:** same binary serve the browser UI on a second - hostname — `GET /` (list, or login page when no session), - `GET /auth/discord` + `GET /auth/discord/callback` (Discord OAuth, - ADR-0002), `POST /logout`, `GET /static/*`, htmx fragment endpoints - under `/ui/*`. Templates + assets `go:embed`-ed under - `backend/internal/web/`, so `backend/Dockerfile` must copy the whole - `internal/` tree, not just `*.go`. Sessions are rows in the `sessions` - table: the cookie carries only an opaque id, looked up (and expiry- - checked) on every request, and deleting the row revokes the session. - Guild membership *is* registration (issue #27): `discordCallback` gates on - membership (and `DISCORD_REQUIRED_ROLE` when set) and then calls - `Store.EnsureReader`, so a refusal creates nothing and a returning Reader - reuses their row. The owner is the only Reader with administrative reach: - `POST /readers/{id}/revoke` (404 for anyone else) drops that Reader's - sessions, and the `readers` panel renders only on the owner's page. - A Reader with no bookmarks at all sees `listView.Fresh`, whose empty state - offers both install links instead of describing a filter. - UI mutations read-modify-write - through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one - place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`. - **Design-tool caveat:** templates link `/static/style.css` root-absolutely - (correct — served from `/`), but impeccable detector resolves - stylesheet href with `path.resolve(fileDir, href)`, drops directory - on leading `/` and silently skip file. Relative href don't help - either: template's directory isn't its served path. So - `detect.mjs backend/internal/web/templates` reports **false clean** — - always pass `backend/internal/web/static` too. One finding there, - `overused-font` on "Instrument Serif", deliberate identity choice, not debt. -- **Every action that moves series out of list is confirm-gated.** - Archive, finish, remove each open own `.confirm-row` disclosure - (`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈ - `archive|finish|remove`); restore fire instantly since it's the reversal. - Remove's row wear ember wash, two reversible ones wear `.calm` grey. - `--ember` stay reserved for new-chapter signal: busy bar and inline - error use `--mute`. -- **Latest-chapter poller:** one goroutine per Site (a Poll Lane, issue #100), - each re-checking that Site's bookmarked series' newest published chapter from - backend's own network access, so `latest_chapter` stays fresh when the user - isn't browsing. Second, parallel signal — the userscript keeps its own - `maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` schedule, and its - `reportLatestChapter` PUTs every read, unchanged numbers included, because an - unchanged read is exactly the Sighting worth deferring a Poll on (#103). - Two independent clocks: per-series rest (`series.latest_checked_at`, - enforced by `Store.DueForLatestCheck`'s WHERE clause — `now - Rest`) and - per-Lane gap (the Lane sleeping between fetches, `effectiveGap`). Both live - in the Site registry (`internal/latest/sites.go`), not config: the five env - knobs that used to size a shared pace are gone. - The poller walks **Series, not Bookmarks** — a series referenced by several - bookmarks is fetched once per cycle, and the due queue orders - `reader_count DESC, latest_checked_at ASC` (ADR-0003). Series row stamped - *before* fetch so broken series wait out the rest instead of retrying - every tick; found chapter written straight to the series row via - `Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list - order — is never touched. - **Sightings** (issue #103, ADR-0011) let a Reader's own page read defer a - Poll: `Store.RecordSighting` — called by the PUT handler *before* the Upsert, - because the raise test needs the row as it stands — stamps - `series.latest_sighted_at` and, when the report raises the stored number, - names its Reader in `series.latest_raised_by`. The due query's HAVING clause - is where deferral lives: a Series is skipped only while it has exactly one - Bookmark, was sighted within one Rest, and is under the ceiling - (`sightingCeilingRests`, six of that Site's rests) since its last Poll. So a - shared Series is never deferred, and no Series goes six hours unpolled - whatever arrives. `checkOne` judges the named Reader off the comparison it - already makes: a lower number is a contradiction (logged with the Reader and - both numbers), the same number an agreement, a higher number the Site - publishing and neither — that last one clears the attribution instead, since - the value the Poll then stores is its own and a later retraction is not the - Reader's fault. Three contradictions - (`store.SightingDisagreementLimit`) stop that Reader deferring — their - reports still write the Latest Chapter — and twenty consecutive agreements - (`store.SightingAgreementsToClear`) forgive them, as does the owner's - clear-marks control. Deferral is recomputed from live facts every round, so - nothing needs invalidating when a Series gains a second Bookmark; the one - input read earlier is the Reader's marks, checked when the Sighting is - recorded, so crossing the threshold or being cleared takes effect from that - Reader's next Sighting and the standing already bought lasts out its rest. - Refusals and browser loss are Lane-local: two `errChallengeHeld` in one pass - stop that Site for `refuseBackoff` (15m) while other Lanes continue; an - `errBrowserInterrupted` (remote Chrome restart) sets a shared Poller flag - that makes the other browser Lanes skip their passes for the same 15m, so a - restarting Chrome doesn't stamp one Series per Lane per pass — after the - window the flag decays and they probe again. Browser Lanes wake Chrome only - when 5+ Series are due or one has waited 15m (ADR-0005 on-demand browser), - and cover work (both healing a stored source URL and filling a blank from - the series page) runs in the background so a slow CDN can't consume a - Lane's gap. - A refusal is only ever the challenge *page*: `isInterstitial` matches the - orchestration path `/cdn-cgi/challenge-platform/h/`, never the bare prefix. - Cloudflare injects `/cdn-cgi/challenge-platform/scripts/jsd/main.js` into - ordinary 200 pages once a zone turns JS detections on, which demonic did on - 2026-08-16 — the prefix match then read every real demonic page as a refusal - and parked that Lane in 15m backoff while plain TLS was returning the full - series page. - Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth - against fingerprint-based blocking; any failure log and skip. kagane, comix - and novelfull sit behind Cloudflare JavaScript challenges the TLS client - can't clear, so they are fetched over CDP via `BROWSER_WS_URL`; kagane and - comix are simply not polled when that's unset, while novelfull falls back to - a plain-TLS attempt — its challenge is a live time-varying fact, and its - cover bytes never need the browser. comix's browser read is an in-tab - `fetch()` of the Series URL, not a DOM render: it is an SPA, so rendering - costs ~65 requests for the same server-rendered HTML one fetch returns - (measured 2026-08-12, issue #98). See - `docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`. - The poller's series write is a single-column UPDATE - (`Store.SetLatestChapter`), not a read-modify-write of the whole bookmark: - it cannot revert read progress or move `updated_at`, so the old - stale-re-read race is gone with the Get+Upsert flow. -- **Covers are acquired at creation, then served from our own origin - (ADR-0007):** the first Bookmark of a Series fires `Store.OnSeriesCreated`, - which `latest.Acquirer` turns into one series-page fetch yielding both the - Latest Chapter and the cover URL; the bytes then go through - `latest.CoverBytesFetcher` into `Store.SetSeriesCover`. It runs in a - goroutine — the Reader's PUT must neither block on a Site nor fail with one - — and every failure is logged and dropped, leaving the Bookmark intact. The - wire's `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once - bytes exist and `""` before, never an address that 404s. `GET /covers/{addr}` - is public and uncredentialed: the userscript renders it on a Site's origin, - where no cookie or token of ours travels. A client-sent `cover` is decoded - and discarded, permanently (ADR-0004 compatibility). - Browser-backed Sites join the same pipeline (issue #62, extended to comix by - #98): kagane and comix pages *and* cover bytes go through the browser sidecar - (nothing falls back to a plain fetch, which would only retrieve a challenge - page), while novelfull needs the browser only for its HTML — the cover URL - comes out of the browser-fetched page and the bytes go over plain TLS. With - no browser configured, kagane and comix Covers are simply absent; novelfull - still gets one — at creation and on the poll — when its page body happens to - answer a plain request (the challenge is a live time-varying fact). comix - cover bytes must arrive by direct navigation, not an in-page fetch: its - Series page sets `cross-origin-embedder-policy: require-corp`, which fails a - page-context fetch of `static.comix.to`. The old kagane-only - serving path (`/img/kagane/{id}`, template rewrite, `CoverFetcher`) is gone - (issue #63): the one public route serves every Site. -- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4. -- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` | - `finished`, orthogonal to `favorite`. Archived and finished appear only in - own tab — not in All, Updated, Favourites, or recent strip. Poller keeps - checking archived series and skip finished ones. `finished` settable - only from web UI; `PUT /bookmarks/{key}` reject it with 400. - **Empty incoming status means "keep stored one"** — resolved on the - `VALUES` side of `Store.Upsert`, not conflict clause, since - `excluded.*` is post-evaluation row and default applied there would - wipe bucket on every PUT from client that predates column. See - `docs/superpowers/specs/2026-07-27-status-buckets-design.md`. -- **Config via env:** `TOKEN_KEY` (derives every Reader's userscript credential; - required), `OWNER_DISCORD_ID` (seeds the owner Reader — the administrator and - the owner of every pre-registration bookmark; required), - `ALLOWED_ORIGINS` (comma list), - `DATABASE_URL` (Postgres connection URL, required — no default), - `COVER_DIR` (required filesystem volume for content-addressed Cover bytes), - `PUBLIC_BASE_URL` (required origin this deployment answers on, trailing - slash trimmed; every Cover URL on the wire is built from it, absolute - because the userscript renders on a Site's origin — ADR-0007), - `PORT` (default `8080`), `DISCORD_CLIENT_ID`/`_CLIENT_SECRET`/`_GUILD_ID`/ - `_REDIRECT_URI` (required; Discord OAuth for the browser UI), - `DISCORD_REQUIRED_ROLE` (optional role gate, empty by default), - `DISCORD_API_BASE` (default `https://discord.com/api/v10`), - `LATEST_CHAPTER_POLL_ENABLED` (background latest-chapter poller kill - switch, default on). Pace is per Site in the registry (issue #100): every - Site rests an hour and gaps ten seconds, a Site with more eligible Series - than 360 tightens its own gap toward the 1s floor, and browser Lanes wake - Chrome only on demand (ADR-0005). The `_COOLDOWN`/`_BROWSER_COOLDOWN`/ - `_INTERVAL`/`_BATCH`/`_STAGGER` knobs that used to size a shared pace are - gone. The 1h rest for browser Sites is safe on documented grounds: a - challenged page costs seconds of a serialized single-tab browser, free-plan - zones have no bot score and no published per-IP rate input, and - `cf_clearance` expires in 30 minutes so every cadence at or above 1h - re-solves anyway — - `docs/research/cloudflare-bot-scoring-and-poll-cadence.md`. - `USERSCRIPT_PATH` and `NOVEL_USERSCRIPT_PATH` (files served at - `/u/{token}/manga-bookmark.user.js` and `/u/{token}/novel-bookmark.user.js`, - defaults `/userscript/manga-bookmark.user.js` and - `/userscript/novel-bookmark.user.js`, both supplied by bindmount; the - `__API_TOKEN__` placeholder inside them is substituted with the requesting - Reader's credential at serve time). - `BROWSER_WS_URL` (CDP endpoint of the browser, which runs on a **separate - machine** and is reached over the tailnet — ADR-0006, `chrome/docker-compose.yml`. - Used by the poller for kagane, comix and novelfull page fetches and by the - cover pipeline for kagane's and comix's image bytes (the browser is the only - route that clears the challenge those two serve their covers behind); unset — - the default — disables browser polling and leaves kagane and comix Covers - blank until stored bytes - exist. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s - `/json/version` for any Host that isn't an IP or `localhost`). -- **No per-Site cover path (issue #63):** every Cover — all six Sites — is - served by the one public `GET /covers/{addr}` route from content-addressed - bytes. There is no proxy, no per-Site rewrite, no second place that decides - a Cover's renderable address: the wire `cover` is it. The only place a Site - name still appears in cover code is the extraction module (`latest`), where - kagane's and comix's image URLs are claimed by `browserOnlyCoverURL` — kagane - answers a plain fetch with a challenge and +Each entry names the code that holds the truth — read that for *what it does*. +The prose here is only what code cannot tell you: rationale, rejected +alternatives, dated measurements, and invariants a plausible refactor would +silently break. + +### Layout + +`backend/main.go` → `newRouter` is the composition root, the only place +packages are wired. Packages under `backend/internal/`: `store`, `latest`, +`session`, `httpmw`, `api`, `userscript`, `web`, `token`, `pgtest`. Root-level +`*_test.go` exercise the full router; unit tests live beside their package. + +Not visible from any single file: stdlib `net/http` with no framework, +Postgres over `jackc/pgx/v5`, `CGO_ENABLED=0` static binary into a distroless +image, TLS terminated by the reverse proxy so the service listens plain `:8080`. + +### Schema — `internal/store/migrations/*.sql`, run by `store.migrate` + +- Migration files are **append-only**. Editing an applied one changes nothing + on a database that already recorded its version in `schema_migrations`, so + the fix silently applies to new deployments only. +- No column probing, no data-fixup migrations. Both were SQLite-era machinery + and were removed deliberately — don't reintroduce either. + +### Tests need Docker — `internal/pgtest` + +`pgtest.Main` from `TestMain` starts one `postgres:17-alpine` per test binary; +`pgtest.URL` hands each test its own database. A package whose tests touch the +store must have that `TestMain` or it has no database at all. + +### Reader-owned store — `internal/store`, `internal/token` + +Four tables; shape is in the migrations, behaviour in `Store`'s methods. + +- **Credentials are derived, never stored.** `token.Token(TOKEN_KEY, discord_id, epoch)` + is an HMAC; only its SHA-256 reaches `readers.token_sha256`. So install URLs + can be rebuilt after any restart, and a database leak yields nothing usable. +- **The owner's epoch-0 hash is refreshed at startup only while the row has + never been rotated.** Drop that condition and a restart resurrects a + rotated-away credential. +- `Store.EnsureReader` never rewrites an existing row's hash — a returning + Reader's login must not invalidate their installed scripts. +- **Every read and write is scoped to the acting Reader**, resolved from the + presented credential by `httpmw.Auth` and carried in the request context. + There is no unauthenticated-by-Reader route and no global token. +- **`series` holds what readers share, `bookmarks` only what differs.** A + bookmark key is `(reader_id, site, series_id)` with no surrogate id; the wire + `key` is derived as `site:series_id` on read. +- `Store.Upsert` splits one flat body across both tables and enforces the + ownership rule: client `title`/`series_url`/`cover` are written **only when + the series row is new**, so one reader cannot retitle a shared series. +- Sync is last-write-wins and the wire format stays flat — clients depend on + both; neither is an implementation detail to tidy up. + +### Web UI — `internal/web` + +Routes, templates and assets are all in that package; `AdminPatterns()` and +`adminRoutes()` enumerate the privileged ones. + +- **`backend/Dockerfile` must copy the whole `internal/` tree**, not just + `*.go`: templates and static assets are `go:embed`-ed from + `internal/web/`. +- **Guild membership *is* registration.** `discordCallback` gates on membership + (plus `DISCORD_REQUIRED_ROLE` when set) and only then calls + `Store.EnsureReader`, so a refusal creates nothing. +- Sessions are rows, not signatures: the cookie carries an opaque id and + expiry is checked on lookup, which is what makes deleting the row an instant + revocation. +- UI mutations go through `Store.Get` + `Store.Upsert` so the `updated_at` rule + below stays in exactly one place. +- `listView.Fresh` exists because a Reader with no bookmarks at all needs + install links, not an empty-filter message. +- **Design-tool caveat:** `detect.mjs backend/internal/web/templates` reports a + **false clean**. Templates link `/static/style.css` root-absolutely (correct — + it is served from `/`), but the detector resolves hrefs with + `path.resolve(fileDir, href)`, which drops the directory on a leading `/` and + skips the file silently; a relative href doesn't help either, since a + template's directory isn't its served path. Always pass + `backend/internal/web/static` too. The one finding there, `overused-font` on + "Instrument Serif", is a deliberate identity choice, not debt. + +### Confirm gating — `internal/web/static/filter.js`, `toggleConfirmRow(key, kind)` + +Every action that pulls a series out of the list (`archive|finish|remove`) opens +its own `.confirm-row`; restore fires instantly because it is the reversal. +Remove wears the ember wash, the two reversible ones wear `.calm` grey. +**`--ember` is reserved for the new-chapter signal** — the busy bar and inline +errors must use `--mute`, or the one colour that means "something to read" +stops meaning it. + +### Latest-chapter poller — `internal/latest`, Site registry in `sites.go` + +One goroutine per Site (a Poll Lane) re-checks that Site's bookmarked series +from the backend's own network position, so `latest_chapter` stays fresh while +nobody is browsing. The userscript's `reportLatestChapter` is a second, +parallel signal — it PUTs every read, unchanged numbers included, because an +unchanged read is exactly the Sighting worth deferring a Poll on. + +- **Pace lives in the Site registry, not config.** Two clocks: per-series rest + (`series.latest_checked_at`, enforced in `Store.DueForLatestCheck`'s WHERE) + and per-Lane gap (`effectiveGap`). The five env knobs that used to size one + shared pace are gone; don't add them back. +- **The poller walks Series, not Bookmarks** — a series several readers hold is + fetched once per cycle, and the due queue orders `reader_count DESC, + latest_checked_at ASC` so the widely-read ones win contention. +- **The series row is stamped *before* the fetch**, so a permanently broken + series waits out its rest instead of being retried every tick. +- `Store.SetLatestChapter` is a single-column UPDATE, deliberately not a + read-modify-write of the bookmark: it therefore cannot revert read progress + or move `updated_at`. The old stale-re-read race died with the Get+Upsert + flow — don't restore one here. + +**Sightings** (`Store.RecordSighting`, the due query's HAVING clause, +`latest.checkOne`) let a Reader's own page read defer a Poll. + +- Recorded by the PUT handler **before** the Upsert, because the raise test + needs the row as it stands. +- A Series is deferred only while it has exactly one Bookmark, was sighted + within one Rest, and is under `sightingCeilingRests` since its last Poll — so + a shared Series is never deferred and nothing goes six hours unpolled + whatever arrives. +- A *higher* report clears the attribution rather than crediting it: the value + the Poll then stores is its own, so a later retraction isn't the Reader's + fault. +- `store.SightingDisagreementLimit` contradictions stop a Reader deferring — + their reports still write the Latest Chapter — and + `store.SightingAgreementsToClear` agreements forgive them, as does the + owner's clear-marks control. +- Deferral is recomputed from live facts each round, so nothing needs + invalidating when a Series gains a second Bookmark. The one input read + earlier is the Reader's marks, so crossing or clearing a threshold takes + effect from their next Sighting and the standing already bought lasts out its + rest. + +**Refusals and browser loss are Lane-local.** Two `errChallengeHeld` in a pass +stop that Site for `refuseBackoff` while other Lanes continue. An +`errBrowserInterrupted` (remote Chrome restarted) sets a shared Poller flag so +the *other* browser Lanes skip their passes for the same window — otherwise a +restarting Chrome stamps one Series per Lane per pass, burning rests on +failures. The flag decays and they probe again. + +- **`isInterstitial` matches the orchestration path + `/cdn-cgi/challenge-platform/h/`, never the bare prefix.** Cloudflare injects + `/cdn-cgi/challenge-platform/scripts/jsd/main.js` into ordinary 200 pages + once a zone turns JS detections on, which demonic did on 2026-08-16: the + prefix match read every real demonic page as a refusal and parked the Lane in + backoff while plain TLS was returning full series pages. +- Fetches use `bogdanfinn/tls-client` with a Chrome profile as defence in depth + against fingerprint blocking; any failure logs and skips. +- kagane, comix and novelfull sit behind Cloudflare JS challenges the TLS + client can't clear, so they go over CDP (`BROWSER_WS_URL`). kagane and comix + are simply not polled when it's unset — a plain fetch would only retrieve a + challenge page — while novelfull still attempts plain TLS, because its + challenge is a live time-varying fact and its cover bytes never need a browser. +- **comix's browser read is an in-tab `fetch()` of the Series URL, not a DOM + render.** It is an SPA: rendering cost ~65 requests for the same + server-rendered HTML one fetch returns (measured 2026-08-12). +- Browser Lanes wake Chrome only when 5+ Series are due or one has waited 15m, + and cover work runs in the background so a slow CDN can't eat a Lane's gap. + +### Covers — `Store.OnSeriesCreated`, `latest.Acquirer`, `latest.CoverBytesFetcher`, `Store.SetSeriesCover` + +Acquired once when the first Bookmark of a Series is created, then served from +our own origin by the public `GET /covers/{addr}`. + +- Acquisition runs in a goroutine: the Reader's PUT must neither block on a + Site nor fail with one. Every failure is logged and dropped, leaving the + Bookmark intact. +- The wire `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once + bytes exist and `""` before — **never an address that 404s**. Absolute + because the userscript renders it on a Site's origin. +- `GET /covers/{addr}` is public and uncredentialed by design: no cookie or + token of ours may travel to a Site's origin. +- A client-sent `cover` is decoded and discarded, permanently — wire + compatibility, not an oversight. +- **One route serves all six Sites.** No proxy, no per-Site rewrite, no second + place that decides a renderable address: the wire `cover` is it. Templates + render `.Cover` and nothing else. The old kagane-only serving path + (`/img/kagane/{id}` plus a template rewrite) is gone; don't reintroduce a + per-Site route because one Site's CDN misbehaves. +- The only Site names left in cover code are in `browserOnlyCoverURL` + (`internal/latest`): kagane answers a plain fetch with a challenge *and* `cross-origin-resource-policy: same-origin`, and `static.comix.to` answers - one with the same Cloudflare challenge its pages serve; - every other Site's CDN answers plain TLS. Templates render `.Cover` — the - wire value — never anything else. -- **Web UI also owns:** session-gated `GET /install/{manga,novel}-bookmark.user.js` - (renders the bindmounted script with the acting Reader's derived credential - substituted in — the credential never appears in page markup, the address - bar, or a redirect; `?download=1` adds `Content-Disposition: attachment` for - mobile Violentmonkey, which ignores a `.user.js` navigation) and - `POST /rotate-token` (atomic epoch bump + hash - rewrite; invalidates every installed copy, so the panel warns to reinstall - on all devices). -- **Owner-only admin page (`internal/web/admin.go`, issue #102):** `GET /admin` - carries the Reader roster (sessions, Sighting counters, `POST - /readers/{id}/revoke` and `POST /readers/{id}/clear-marks`) and Poll Lane - status (`GET /ui/admin/lanes`, self-refreshing every 30s). Every route that - reaches past the acting Reader is listed in `adminRoutes()` and wrapped in - `requireOwner` at registration — add a route there, not a check inside a - handler; `web.AdminPatterns()` is what the gate test walks. A non-owner gets - 404, never 403. Lane figures come from the running poller through the - `web.LaneReporter` seam (`latest.Poller.LaneStatus`), never from a table: a - nil reporter or a Lane that has not finished a pass renders "no data yet" - rather than zeroes. `main.newRouter` takes the reporter as an interface and - converts a nil `*Poller` to a nil interface — a typed nil would make the page - claim a poller exists. - The one owner comparison left outside `requireOwner` is in `index` + with the same challenge its pages serve. Every other Site's CDN answers plain + TLS. +- **comix cover bytes must arrive by direct navigation, not an in-page fetch:** + its Series page sets `cross-origin-embedder-policy: require-corp`, which + fails a page-context fetch of `static.comix.to`. +- With no browser configured, kagane and comix Covers are simply absent; + novelfull still gets one whenever its page answers a plain request. + +### `updated_at` drives list order — `Store.Upsert` + +The server applies its own timestamp only when the row is new or +`last_chapter_num` changes, else it keeps the stored value. **Favouriting a +series, or a newly published chapter arriving, must not reorder the list** — +only real reading progress moves a row. Consequently `PUT` returns the row **as +stored** and clients must adopt that response rather than their own payload. + +### Lifecycle buckets — `status` on each bookmark + +`reading` | `archived` | `finished`, orthogonal to `favorite`. Archived and +finished appear only in their own tab, never in All, Updated, Favourites or the +recent strip. The poller keeps checking archived series and skips finished ones. + +- `finished` is settable only from the web UI; `PUT /bookmarks/{key}` rejects + it with 400. +- **An empty incoming status means "keep the stored one"**, and it is resolved + on the `VALUES` side of `Store.Upsert`, not in the conflict clause: + `excluded.*` is the post-evaluation row, so a default applied there would + wipe the bucket on every PUT from a client predating the column. + +### Config — `Config` / `loadConfig` / `loadLatestPoll` in `backend/main.go` + +That function is the complete list of env vars, their defaults, and which are +required. What it can't tell you: + +- `PUBLIC_BASE_URL` must be an absolute origin because every Cover URL on the + wire is built from it and the userscript renders on a Site's origin. +- `BROWSER_WS_URL` **must be a tailnet IP, never a hostname** — Chrome's + DevTools handler 500s `/json/version` for any Host that isn't an IP or + `localhost`. Unset (the default) disables browser polling. +- `USERSCRIPT_PATH` / `NOVEL_USERSCRIPT_PATH` are bindmounted files; the + `__API_TOKEN__` placeholder inside them is substituted with the requesting + Reader's credential at serve time. +- Pace is per Site in the registry, not env. The + `_COOLDOWN`/`_BROWSER_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` knobs are + gone on purpose. +- The 1h rest for browser Sites is safe on documented grounds: a challenged + page costs seconds of a serialized single-tab browser, free-plan zones carry + no bot score and no published per-IP rate input, and `cf_clearance` expires + in 30 minutes, so every cadence at or above 1h re-solves anyway. + +### Userscript install & rotation — `internal/userscript`, `internal/token` + +Session-gated `GET /install/{manga,novel}-bookmark.user.js` renders the +bindmounted script with the acting Reader's derived credential substituted in, +so the credential never appears in page markup, the address bar, or a redirect. +`?download=1` adds `Content-Disposition: attachment` for mobile Violentmonkey, +which ignores a `.user.js` navigation. `POST /rotate-token` is an atomic epoch +bump plus hash rewrite and invalidates every installed copy — the panel must +keep warning to reinstall on all devices. + +### Owner-only admin — `internal/web/admin.go` + +- **Every route reaching past the acting Reader is listed in `adminRoutes()` + and wrapped in `requireOwner` at registration** — add it there, not as a + check inside a handler; `web.AdminPatterns()` is what the gate test walks. A + non-owner gets 404, never 403. +- The one owner comparison left outside the gate is in `index` (`view.Owner = readerID == h.store.OwnerID()`): it gates a link, not an endpoint, so it is a rendering decision a registration-time wrapper cannot - express — do not "unify" it into the gate. - A Lane pass that returns before computing its figures (refusal backoff, - sidecar down) carries the previous pass's due count and gap forward rather - than recording zeroes; a Lane that has never reached a pace renders no gap at - all. `Checked` next to `Due` is what separates a stopped Lane from a quiet - one, so neither figure may be dropped from the row. - Due-without-Checked is *not* by itself a stall: a browser Lane under both - wake thresholds sets `LaneState.Asleep` at the on-demand gate and renders - "browser asleep" instead of "not checking", and never counts toward - `Attention`. That is the commonest healthy state for kagane, comix and - novelfull — one due Series, nothing checked — so spending the stall mark on - it would train the owner to ignore the mark that matters. + express. Do not "unify" it into the gate. +- Lane figures come through the `web.LaneReporter` seam + (`latest.Poller.LaneStatus`), never a table. `main.newRouter` takes the + reporter as an interface and converts a nil `*Poller` to a nil interface — a + typed nil would make the page claim a poller exists. +- A pass that returns before computing figures (refusal backoff, sidecar down) + carries the previous pass's numbers forward rather than recording zeroes. +- **`Checked` next to `Due` is what separates a stopped Lane from a quiet one**, + so neither may be dropped from the row. +- Due-without-Checked is **not** by itself a stall: a browser Lane under both + wake thresholds sets `LaneState.Asleep` and renders "browser asleep", and + never counts toward `Attention`. That is the commonest healthy state for + kagane, comix and novelfull, so spending the stall mark on it would train the + owner to ignore the mark that matters. diff --git a/userscript/AGENTS.md b/userscript/AGENTS.md index fe495d3..73accef 100644 --- a/userscript/AGENTS.md +++ b/userscript/AGENTS.md @@ -1,93 +1,103 @@ -Guidance for OpenCode (and Claude Code) working under `userscript/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system. +Scope: `userscript/`. -### Userscript structure (single IIFE, `manga-bookmark.user.js`) +Each entry names the code that holds the truth. The prose is only what the code +cannot tell you: rationale, invariants a refactor would break, and dated +observations about sites we don't control. -1. **Site adapters** — one per host, `detect(location, document)` return page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title` from **`og:title`** (or the page heading where a site ships no og: tags), not CSS classes. **No adapter reads a cover**: the backend acquires, stores and serves every Cover from its own origin (ADR-0007), the wire's `cover` is already an address on our origin, and `apiPut` strips any `cover` off an outgoing body. -2. **API client** — `apiGet/apiPut/apiDelete` with bearer header; `localStorage` key `bmgr:manga:cache` for instant render + offline fallback. -3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value. -4. **Retry queue** — every write go through `pushBookmark`/`pushDelete`, so - failed mutation park in `localStorage` (`bmgr:manga:queue`) and replayed on - next navigation, reconnect, or `refresh()`. Entries are markers - (`{key, op, sendStatus, attempts}`), never payloads — body read from - cache at send time, so one entry per key give ordering and coalescing for - free. `sendStatus` is **sticky**: while archive pending, later writes to - that key keep carrying bucket, which stop successful - in-between write from silently un-archiving series. `refresh()` drains - before it fetches and overlays anything still pending, so list never - flaps. 400 drops entry, 401 abort pass and keep queue, and - transient failures retry to cap of 10. Latest-chapter writes deliberately - stay out of queue. See - `docs/superpowers/specs/2026-07-27-offline-retry-queue-design.md`. -5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS - (critical on mobile). Three tabs (All / Favourites / Archived) and row of - link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG - block next to `API_BASE`. FAB is `7 × 44` edge tab whose *hit* area - widened to `28 × 72` by invisible `#hit` child; `#fab` must keep - `touch-action: none` and must **not** regain `overflow: hidden`. Since - `touch-action` resolved at gesture start, strip can't be both - browser-scrolled and script-dragged, so `makeDraggable` splits by intent: swipe - from `#hit` scrolls via `window.scrollBy`, hold of `ARM_MS` arms - reposition drag, visible sliver drags with no hold. See - `docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`. -6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fire without reload. Demonic uses classic reloads (initial `document-idle` run suffice). +### Structure — single IIFE, `manga-bookmark.user.js` -### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust) +Six parts, in file order: site adapters, API client, progress logic, retry +queue, UI, SPA navigation. -- **asurascans.com**: series `/comics/` (slug carries trailing - site-wide build-hash suffix, e.g. `-059befe1`, that **rotates on every - redeploy**), chapter `/comics//chapter/`. `seriesId` must strip - hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in userscript, - `asuraBuildHash` in backend); URLs keep full slug — stale-hash - URLs 302 to current ones. Astro-rendered; chapter links present in raw - server HTML. -- **demonicscans.org**: series `/manga/` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title//chapter//` (older `chaptered.php?manga=&chapter=` form still exists as redirect, what series-page chapter-list anchors link through). - Encodings (incl. triple-encoded punctuation like `%25252D`) identical - on /manga/ and /title/ pages, so decode-once seriesIds match — verified - 2026-07-28. -- **comix.to**: series `/title/-`, chapter - `/title/-/-chapter-`. Only the leading `` is - identity — the slug re-renders when a series is renamed (`comixSeriesId`). - An SPA that **never rewrites `og:title`**: the server-rendered head keeps - whatever document loaded first, so on a cold load `og:title` is the homepage's - "Comix — Read Comics online for free" and after an in-page hop it is the - *previous* series' name. `document.title` is the one thing client routing does - update, so titles come from there, with the chapter page's `" · Ch."` tail - stripped. It publishes no `og:image` either, which is one of the reasons cover +**Site adapters** — one per host, `detect(location, document)` returning page +`type` + IDs. + +- Identify type and IDs from **URL regex**, which is the most stable surface a + site exposes; take `title` from **`og:title`** (or the page heading where a + site ships no og: tags), never CSS classes. +- **No adapter reads a cover.** The backend acquires, stores and serves every + Cover from its own origin, the wire `cover` is already an address there, and + `apiPut` strips any `cover` off an outgoing body. + +**Progress logic** — auto-upsert `last_chapter` only when +`chapterNum >= stored last_chapter_num`; unparseable sets the current value. +Re-reading an old chapter must not regress progress. A manual panel override +forces any value. + +**Retry queue** — every write goes through `pushBookmark`/`pushDelete`. + +- Entries are markers (`{key, op, sendStatus, attempts}`), **never payloads**: + the body is read from cache at send time, so one entry per key gives ordering + and coalescing for free. +- `sendStatus` is **sticky** — while an archive is pending, later writes to that + key keep carrying the bucket. Without it a successful in-between write + silently un-archives the series. +- `refresh()` drains before it fetches and overlays anything still pending, so + the list never flaps. +- 400 drops the entry, 401 aborts the pass and keeps the queue, transient + failures retry to a cap. Latest-chapter writes deliberately stay out of the + queue. + +**UI** — rendered inside a **Shadow DOM** root to isolate it from site CSS, +which is critical on mobile. + +- The FAB's *hit* area is widened by an invisible `#hit` child. `#fab` must keep + `touch-action: none` and must **not** regain `overflow: hidden`. +- `touch-action` is resolved at gesture start, so the strip cannot be both + browser-scrolled and script-dragged. `makeDraggable` therefore splits by + intent: a swipe from `#hit` scrolls via `window.scrollBy`, a hold of `ARM_MS` + arms a reposition drag, and the visible sliver drags with no hold. + +**SPA navigation** — Asura is Astro and client-routes on comic/chapter pages, so +`history.pushState`/`replaceState` are patched and `popstate` listened to, and +`detect()` re-runs on URL change. Demonic uses classic reloads, where the +initial `document-idle` run suffices. + +### Live URL shapes + +Encoded in the adapters; the notes below are the parts a reader of the regex +would get wrong. **Verified 2026-07-26 unless dated otherwise — sites drift, so +re-check against a live page before trusting any of it.** + +- **asurascans.com** — the series slug carries a site-wide build-hash suffix + (e.g. `-059befe1`) that **rotates on every redeploy**, so `seriesId` must + strip it (`stripBuildHash` here, `asuraBuildHash` in the backend) while URLs + keep the full slug — stale-hash URLs 302 to current ones. +- **demonicscans.org** — slugs may URL-encode punctuation, and the older + `chaptered.php?manga=&chapter=` form still exists as a redirect, which + is what series-page chapter-list anchors link through. Encodings (including + triple-encoded punctuation like `%25252D`) are identical on `/manga/` and + `/title/` pages, so decode-once seriesIds match (verified 2026-07-28). +- **comix.to** — only the leading `` is identity; the slug re-renders when a + series is renamed (`comixSeriesId`). It is an SPA that **never rewrites + `og:title`**: the server-rendered head keeps whatever document loaded first, + so on a cold load `og:title` is the homepage's name and after an in-page hop + it is the *previous* series'. `document.title` is the one thing client routing + updates, hence titles come from there with the chapter page's `" · Ch."` + tail stripped. It publishes no `og:image` either, one of the reasons cover acquisition moved to the backend. -- **kagane.to**: series `/series/`, reader - `/series//reader/`. Reader URLs carry no chapter number, so - the number comes out of `og:title`. Two shapes exist: `" - Chapter - [ - Episode ]"` and, for volume-numbered series, `" - Volume - Chapter "` with no episode name — both must yield a bare series title, or - the volume tail lands in the bookmark's title. - Its covers are challenge- and CORP-protected, so nothing outside kagane.to can - load one directly; the panel renders the backend's own cover address like every - other Site. Behind a Cloudflare JS challenge, so the backend polls it - through the headless browser. -- **novelfull.com** (novel script): series `/.html`, chapter - `//chapter-[-].html`. No `og:*` tags at all — title from - `h3.title` (series) or `a.truyen-title` (chapter); the script reads no cover. - Behind a Cloudflare JS challenge no TLS fingerprint - clears, so the backend polls it through the headless browser. -- **lightnovelworld.net** (novel script): series `/novel//`, chapter - `/-chapter-/` — flat, at the site root. The chapter path's slug is a - Chapter Slug, not an identity: the Series address is read off the page's +- **kagane.to** — reader URLs carry no chapter number, so the number comes out + of `og:title`. Two shapes exist, `" - Chapter [ - Episode ]"` + and `" - Volume Chapter "`; both must yield a bare series + title, or the volume tail lands in the bookmark's title. Its covers are + challenge- and CORP-protected, so nothing outside kagane.to can load one — + the panel renders the backend's cover address like every other Site. +- **novelfull.com** (novel script) — no `og:*` tags at all, so the title comes + from `h3.title` (series) or `a.truyen-title` (chapter). +- **lightnovelworld.net** (novel script) — chapter paths are flat at the site + root and their slug is a **Chapter Slug, not an identity**: a Series may + publish under several. The Series address is read off the page's `a[aria-label='All Chapter']` (fallback: the BreadcrumbList's second crumb), - and a Series may publish under several Chapter Slugs. A chapter page with no - pointer resolves to `other`, so no Bookmark is offered. `h1.entry-title` is - the clean title on a series page and ` Chapter <n>` on a chapter page. - Its series page lists every chapter with an - absolute href, so the backend polls it with the plain TLS client. - The client performs no latest-chapter scan for this Site: the Poll's - one-hour cooldown dominates the client's four-hour throttle, so a scan - would add no freshness, and the page's wpdiscuz thread is a public write - surface a scan would have to truncate at. `computeLatestChapter` yields - null here and `backgroundRefreshLatest` skips the Site before any fetch. + and a chapter page with no pointer resolves to `other` so no Bookmark is + offered. The client runs **no latest-chapter scan** for this Site — + `computeLatestChapter` yields null and `backgroundRefreshLatest` skips it + before any fetch — because the backend Poll's one-hour cooldown dominates the + client's four-hour throttle, so a scan would add no freshness while having to + truncate at the page's wpdiscuz thread, a public write surface. -### Second script: `novel-bookmark.user.js` +### Second script — `novel-bookmark.user.js` A copy of the manga script with two adapters, `LIBRARY = "novel"` and -`STORE_PREFIX = "bmgr:novel:"`. No migration loop (this script has no previous -installation to carry keys over from). Installed alongside the manga script; -both write to the same backend with the same `LIBRARY` column discriminating -them. +`STORE_PREFIX = "bmgr:novel:"`. No migration loop, because this script has no +previous installation to carry keys over from. Installed alongside the manga +script; both write to the same backend, discriminated by `LIBRARY`.