sulthan/mangaBookmark
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: make every AGENTS.md cite code, not docs or issues (#113)

Browse Source 
Every `AGENTS.md` now cites code and nothing else.

## Why

Two rot mechanisms, same symptom — an agent confidently follows a stale statement:

1. **Non-code citations.** A spec, ADR, plan file, or issue records what was true when it was written. Nothing updates it when the decision reverses.
2. **Prose restating mechanism.** The code changes, the paragraph doesn't, and the next reader trusts the paragraph.

Code is the only source true at read time.

## What changed

**All three files:** removed every ADR ref, spec/plan pointer (`docs/superpowers/specs/*`, `plans/*`, `docs/research/*`), `DEPLOY.md`/`REDEPLOY.md`, `docs/agents/*`, and issue number. Facts those links carried are restated inline — the `tea` command set and the five triage label strings now live in the root Forge section. `### Domain docs` is deleted: it pointed only at `CONTEXT.md` and `docs/adr/`, neither of which exists.

**`backend/` and `userscript/`:** rewritten around derivability.

| Class | In code? | Treatment |
|---|---|---|
| Structure — packages, routes, env vars, columns | yes | name the symbol, nothing else |
| Mechanism — what a function does | yes | symbol + one line |
| Rationale — why, what a "simplify" breaks | **no** | written out |
| Measurement — observation against a service we don't control | **no** | written out, dated |

`backend/AGENTS.md` 20578 → 15512 bytes, `userscript/AGENTS.md` 7129 → 5912. Root grows 16905 → 19292: the cost of inlining the `docs/agents/*` facts plus the new rule.

**Rule** recorded in root as `## Writing an AGENTS.md`. Sole non-code exception is a sibling `AGENTS.md`. Closing clause: every symbol named must exist, since a dead pointer is a bug rather than a stale sentence.

**Harness-agnostic:** dropped the `Guidance for OpenCode (and Claude Code)` openers for plain scope lines.

## Verification

Applied the new rule to itself — extracted all 118 backticked identifiers across the three files and checked each against every `.go`, `.js`, `.sql`, `.html` and `.css` source. Zero repo symbols missing; the 8 non-matches are external (`GM_setValue`, `navigator.webdriver`, `HeadlessChrome`, `curl`, …).

That check caught a claim that was **already lying** on `main`: the cover section said `CoverFetcher` was gone, but `NewCoverFetcher`, `TLSCoverFetcher` and `BrowserCoverFetcher` are all live in `internal/latest`. Now names only the genuinely dead `/img/kagane/{id}` route. Exactly the failure the rule exists to prevent.

No code touched — documentation only, nothing to test.

Reviewed-on: #113
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #113.
This commit is contained in:
Sulthan Zaki
2026-08-17 13:43:49 +07:00
committed by sulthan
parent 550b258c59
commit 766aa8f00d
3 changed files with 433 additions and 394 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/AGENTS.md
+264 -270
View File
@@ -1,274 +1,268 @@
Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system.
Scope: `backend/`.
- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + Postgres over `jackc/pgx/v5` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`.
Single binary, split into packages under `backend/internal/`: `store`
(Bookmark type, Postgres persistence, migration runner), `latest` (background
poller, site parsers, TLS fetcher), `session` (cookie signing, login
rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON
bookmark handlers), `userscript` (userscript-serving handler), `web`
(browser UI handler + `templates/` + `static/`, `go:embed`-ed).
`backend/main.go` is the composition root — the only place that wires
packages together into `newRouter`. Root-level `*_test.go` hold
integration tests that exercise the full router; unit tests for a
package live beside it under `internal/`.
- **Schema is migration-owned.** `internal/store/migrations/*.sql` is
`go:embed`-ed and applied on every start by `store.migrate`: one numbered
file per change, one transaction each, versions recorded in
`schema_migrations`. Files are **append-only** — editing an applied one
changes nothing on a database that already ran it. No column probing, no
data-fixup migrations: both were SQLite-era machinery and are gone.
- **Tests need Docker.** `internal/pgtest` starts one `postgres:17-alpine`
container per test binary (`TestMain` -> `pgtest.Main`) and hands each test
its own database (`pgtest.URL(t)`). A package whose tests touch the store
must have that `TestMain`.
- **Reader-owned store, four tables.** `readers` is keyed by Discord user ID
and carries the SHA-256 of the Reader's userscript credential plus a
`token_epoch` (issue #24). Credentials are derived, never stored: `token.Token(TOKEN_KEY, discord_id, epoch)` (HMAC, `internal/token`), and only its SHA-256 sits in `readers.token_sha256`, so install URLs can be rebuilt after any restart while a database leak yields nothing but hashes. The seed creates the **owner** row at startup; its epoch-0 hash is refreshed on every start **only while the row has never been rotated**, so a restart can never resurrect a rotated-away credential. Every other row is created by that Reader's own first login (`Store.EnsureReader`, idempotent on `discord_id`, and it never rewrites an existing row's hash). Rotation is `Store.RotateToken` (epoch bump + hash rewrite in one transaction), driven by the web UI.
`series` keyed `(site, series_id)`
(`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the
shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`),
Latest Chapter, `latest_checked_at`, and the Sighting pair
`latest_sighted_at`/`latest_raised_by` (issue #103) — and `bookmarks` holds only what
differs between readers: progress, favourite, lifecycle bucket,
`updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no
surrogate id; the wire `key` is derived as `site:series_id` on read — and
every store read/write is scoped to the reader it names. Auth resolves the
acting Reader from the presented credential (`httpmw.Auth`) and nothing
else — there is no unauthenticated-by-Reader route and no global token; the
reader id travels in the request context. Sync **last-write-wins**; the wire format
stays flat (ADR-0004). `Store.Upsert` decomposes one flat body across two
tables and enforces the ownership rule: client `title`/`series_url`/`cover`
are written only when the series row is new (ADR-0003).
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- **Web UI:** same binary serve the browser UI on a second
hostname — `GET /` (list, or login page when no session),
`GET /auth/discord` + `GET /auth/discord/callback` (Discord OAuth,
ADR-0002), `POST /logout`, `GET /static/*`, htmx fragment endpoints
under `/ui/*`. Templates + assets `go:embed`-ed under
`backend/internal/web/`, so `backend/Dockerfile` must copy the whole
`internal/` tree, not just `*.go`. Sessions are rows in the `sessions`
table: the cookie carries only an opaque id, looked up (and expiry-
checked) on every request, and deleting the row revokes the session.
Guild membership *is* registration (issue #27): `discordCallback` gates on
membership (and `DISCORD_REQUIRED_ROLE` when set) and then calls
`Store.EnsureReader`, so a refusal creates nothing and a returning Reader
reuses their row. The owner is the only Reader with administrative reach:
`POST /readers/{id}/revoke` (404 for anyone else) drops that Reader's
sessions, and the `readers` panel renders only on the owner's page.
A Reader with no bookmarks at all sees `listView.Fresh`, whose empty state
offers both install links instead of describing a filter.
UI mutations read-modify-write
through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one
place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`.
**Design-tool caveat:** templates link `/static/style.css` root-absolutely
(correct — served from `/`), but impeccable detector resolves
stylesheet href with `path.resolve(fileDir, href)`, drops directory
on leading `/` and silently skip file. Relative href don't help
either: template's directory isn't its served path. So
`detect.mjs backend/internal/web/templates` reports **false clean** —
always pass `backend/internal/web/static` too. One finding there,
`overused-font` on "Instrument Serif", deliberate identity choice, not debt.
- **Every action that moves series out of list is confirm-gated.**
Archive, finish, remove each open own `.confirm-row` disclosure
(`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈
`archive|finish|remove`); restore fire instantly since it's the reversal.
Remove's row wear ember wash, two reversible ones wear `.calm` grey.
`--ember` stay reserved for new-chapter signal: busy bar and inline
error use `--mute`.
- **Latest-chapter poller:** one goroutine per Site (a Poll Lane, issue #100),
each re-checking that Site's bookmarked series' newest published chapter from
backend's own network access, so `latest_chapter` stays fresh when the user
isn't browsing. Second, parallel signal — the userscript keeps its own
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` schedule, and its
`reportLatestChapter` PUTs every read, unchanged numbers included, because an
unchanged read is exactly the Sighting worth deferring a Poll on (#103).
Two independent clocks: per-series rest (`series.latest_checked_at`,
enforced by `Store.DueForLatestCheck`'s WHERE clause — `now - Rest`) and
per-Lane gap (the Lane sleeping between fetches, `effectiveGap`). Both live
in the Site registry (`internal/latest/sites.go`), not config: the five env
knobs that used to size a shared pace are gone.
The poller walks **Series, not Bookmarks** — a series referenced by several
bookmarks is fetched once per cycle, and the due queue orders
`reader_count DESC, latest_checked_at ASC` (ADR-0003). Series row stamped
*before* fetch so broken series wait out the rest instead of retrying
every tick; found chapter written straight to the series row via
`Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
order — is never touched.
**Sightings** (issue #103, ADR-0011) let a Reader's own page read defer a
Poll: `Store.RecordSighting` — called by the PUT handler *before* the Upsert,
because the raise test needs the row as it stands — stamps
`series.latest_sighted_at` and, when the report raises the stored number,
names its Reader in `series.latest_raised_by`. The due query's HAVING clause
is where deferral lives: a Series is skipped only while it has exactly one
Bookmark, was sighted within one Rest, and is under the ceiling
(`sightingCeilingRests`, six of that Site's rests) since its last Poll. So a
shared Series is never deferred, and no Series goes six hours unpolled
whatever arrives. `checkOne` judges the named Reader off the comparison it
already makes: a lower number is a contradiction (logged with the Reader and
both numbers), the same number an agreement, a higher number the Site
publishing and neither — that last one clears the attribution instead, since
the value the Poll then stores is its own and a later retraction is not the
Reader's fault. Three contradictions
(`store.SightingDisagreementLimit`) stop that Reader deferring — their
reports still write the Latest Chapter — and twenty consecutive agreements
(`store.SightingAgreementsToClear`) forgive them, as does the owner's
clear-marks control. Deferral is recomputed from live facts every round, so
nothing needs invalidating when a Series gains a second Bookmark; the one
input read earlier is the Reader's marks, checked when the Sighting is
recorded, so crossing the threshold or being cleared takes effect from that
Reader's next Sighting and the standing already bought lasts out its rest.
Refusals and browser loss are Lane-local: two `errChallengeHeld` in one pass
stop that Site for `refuseBackoff` (15m) while other Lanes continue; an
`errBrowserInterrupted` (remote Chrome restart) sets a shared Poller flag
that makes the other browser Lanes skip their passes for the same 15m, so a
restarting Chrome doesn't stamp one Series per Lane per pass — after the
window the flag decays and they probe again. Browser Lanes wake Chrome only
when 5+ Series are due or one has waited 15m (ADR-0005 on-demand browser),
and cover work (both healing a stored source URL and filling a blank from
the series page) runs in the background so a slow CDN can't consume a
Lane's gap.
A refusal is only ever the challenge *page*: `isInterstitial` matches the
orchestration path `/cdn-cgi/challenge-platform/h/`, never the bare prefix.
Cloudflare injects `/cdn-cgi/challenge-platform/scripts/jsd/main.js` into
ordinary 200 pages once a zone turns JS detections on, which demonic did on
2026-08-16 — the prefix match then read every real demonic page as a refusal
and parked that Lane in 15m backoff while plain TLS was returning the full
series page.
Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth
against fingerprint-based blocking; any failure log and skip. kagane, comix
and novelfull sit behind Cloudflare JavaScript challenges the TLS client
can't clear, so they are fetched over CDP via `BROWSER_WS_URL`; kagane and
comix are simply not polled when that's unset, while novelfull falls back to
a plain-TLS attempt — its challenge is a live time-varying fact, and its
cover bytes never need the browser. comix's browser read is an in-tab
`fetch()` of the Series URL, not a DOM render: it is an SPA, so rendering
costs ~65 requests for the same server-rendered HTML one fetch returns
(measured 2026-08-12, issue #98). See
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
The poller's series write is a single-column UPDATE
(`Store.SetLatestChapter`), not a read-modify-write of the whole bookmark:
it cannot revert read progress or move `updated_at`, so the old
stale-re-read race is gone with the Get+Upsert flow.
- **Covers are acquired at creation, then served from our own origin
(ADR-0007):** the first Bookmark of a Series fires `Store.OnSeriesCreated`,
which `latest.Acquirer` turns into one series-page fetch yielding both the
Latest Chapter and the cover URL; the bytes then go through
`latest.CoverBytesFetcher` into `Store.SetSeriesCover`. It runs in a
goroutine — the Reader's PUT must neither block on a Site nor fail with one
— and every failure is logged and dropped, leaving the Bookmark intact. The
wire's `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once
bytes exist and `""` before, never an address that 404s. `GET /covers/{addr}`
is public and uncredentialed: the userscript renders it on a Site's origin,
where no cookie or token of ours travels. A client-sent `cover` is decoded
and discarded, permanently (ADR-0004 compatibility).
Browser-backed Sites join the same pipeline (issue #62, extended to comix by
#98): kagane and comix pages *and* cover bytes go through the browser sidecar
(nothing falls back to a plain fetch, which would only retrieve a challenge
page), while novelfull needs the browser only for its HTML — the cover URL
comes out of the browser-fetched page and the bytes go over plain TLS. With
no browser configured, kagane and comix Covers are simply absent; novelfull
still gets one — at creation and on the poll — when its page body happens to
answer a plain request (the challenge is a live time-varying fact). comix
cover bytes must arrive by direct navigation, not an in-page fetch: its
Series page sets `cross-origin-embedder-policy: require-corp`, which fails a
page-context fetch of `static.comix.to`. The old kagane-only
serving path (`/img/kagane/{id}`, template rewrite, `CoverFetcher`) is gone
(issue #63): the one public route serves every Site.
- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
`finished`, orthogonal to `favorite`. Archived and finished appear only in
own tab — not in All, Updated, Favourites, or recent strip. Poller keeps
checking archived series and skip finished ones. `finished` settable
only from web UI; `PUT /bookmarks/{key}` reject it with 400.
**Empty incoming status means "keep stored one"** — resolved on the
`VALUES` side of `Store.Upsert`, not conflict clause, since
`excluded.*` is post-evaluation row and default applied there would
wipe bucket on every PUT from client that predates column. See
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
- **Config via env:** `TOKEN_KEY` (derives every Reader's userscript credential;
required), `OWNER_DISCORD_ID` (seeds the owner Reader — the administrator and
the owner of every pre-registration bookmark; required),
`ALLOWED_ORIGINS` (comma list),
`DATABASE_URL` (Postgres connection URL, required — no default),
`COVER_DIR` (required filesystem volume for content-addressed Cover bytes),
`PUBLIC_BASE_URL` (required origin this deployment answers on, trailing
slash trimmed; every Cover URL on the wire is built from it, absolute
because the userscript renders on a Site's origin — ADR-0007),
`PORT` (default `8080`), `DISCORD_CLIENT_ID`/`_CLIENT_SECRET`/`_GUILD_ID`/
`_REDIRECT_URI` (required; Discord OAuth for the browser UI),
`DISCORD_REQUIRED_ROLE` (optional role gate, empty by default),
`DISCORD_API_BASE` (default `https://discord.com/api/v10`),
`LATEST_CHAPTER_POLL_ENABLED` (background latest-chapter poller kill
switch, default on). Pace is per Site in the registry (issue #100): every
Site rests an hour and gaps ten seconds, a Site with more eligible Series
than 360 tightens its own gap toward the 1s floor, and browser Lanes wake
Chrome only on demand (ADR-0005). The `_COOLDOWN`/`_BROWSER_COOLDOWN`/
`_INTERVAL`/`_BATCH`/`_STAGGER` knobs that used to size a shared pace are
gone. The 1h rest for browser Sites is safe on documented grounds: a
challenged page costs seconds of a serialized single-tab browser, free-plan
zones have no bot score and no published per-IP rate input, and
`cf_clearance` expires in 30 minutes so every cadence at or above 1h
re-solves anyway —
`docs/research/cloudflare-bot-scoring-and-poll-cadence.md`.
`USERSCRIPT_PATH` and `NOVEL_USERSCRIPT_PATH` (files served at
`/u/{token}/manga-bookmark.user.js` and `/u/{token}/novel-bookmark.user.js`,
defaults `/userscript/manga-bookmark.user.js` and
`/userscript/novel-bookmark.user.js`, both supplied by bindmount; the
`__API_TOKEN__` placeholder inside them is substituted with the requesting
Reader's credential at serve time).
`BROWSER_WS_URL` (CDP endpoint of the browser, which runs on a **separate
machine** and is reached over the tailnet — ADR-0006, `chrome/docker-compose.yml`.
Used by the poller for kagane, comix and novelfull page fetches and by the
cover pipeline for kagane's and comix's image bytes (the browser is the only
route that clears the challenge those two serve their covers behind); unset —
the default — disables browser polling and leaves kagane and comix Covers
blank until stored bytes
exist. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s
`/json/version` for any Host that isn't an IP or `localhost`).
- **No per-Site cover path (issue #63):** every Cover — all six Sites — is
served by the one public `GET /covers/{addr}` route from content-addressed
bytes. There is no proxy, no per-Site rewrite, no second place that decides
a Cover's renderable address: the wire `cover` is it. The only place a Site
name still appears in cover code is the extraction module (`latest`), where
kagane's and comix's image URLs are claimed by `browserOnlyCoverURL` — kagane
answers a plain fetch with a challenge and
Each entry names the code that holds the truth — read that for *what it does*.
The prose here is only what code cannot tell you: rationale, rejected
alternatives, dated measurements, and invariants a plausible refactor would
silently break.
### Layout
`backend/main.go` → `newRouter` is the composition root, the only place
packages are wired. Packages under `backend/internal/`: `store`, `latest`,
`session`, `httpmw`, `api`, `userscript`, `web`, `token`, `pgtest`. Root-level
`*_test.go` exercise the full router; unit tests live beside their package.
Not visible from any single file: stdlib `net/http` with no framework,
Postgres over `jackc/pgx/v5`, `CGO_ENABLED=0` static binary into a distroless
image, TLS terminated by the reverse proxy so the service listens plain `:8080`.
### Schema — `internal/store/migrations/*.sql`, run by `store.migrate`
- Migration files are **append-only**. Editing an applied one changes nothing
on a database that already recorded its version in `schema_migrations`, so
the fix silently applies to new deployments only.
- No column probing, no data-fixup migrations. Both were SQLite-era machinery
and were removed deliberately — don't reintroduce either.
### Tests need Docker — `internal/pgtest`
`pgtest.Main` from `TestMain` starts one `postgres:17-alpine` per test binary;
`pgtest.URL` hands each test its own database. A package whose tests touch the
store must have that `TestMain` or it has no database at all.
### Reader-owned store — `internal/store`, `internal/token`
Four tables; shape is in the migrations, behaviour in `Store`'s methods.
- **Credentials are derived, never stored.** `token.Token(TOKEN_KEY, discord_id, epoch)`
is an HMAC; only its SHA-256 reaches `readers.token_sha256`. So install URLs
can be rebuilt after any restart, and a database leak yields nothing usable.
- **The owner's epoch-0 hash is refreshed at startup only while the row has
never been rotated.** Drop that condition and a restart resurrects a
rotated-away credential.
- `Store.EnsureReader` never rewrites an existing row's hash — a returning
Reader's login must not invalidate their installed scripts.
- **Every read and write is scoped to the acting Reader**, resolved from the
presented credential by `httpmw.Auth` and carried in the request context.
There is no unauthenticated-by-Reader route and no global token.
- **`series` holds what readers share, `bookmarks` only what differs.** A
bookmark key is `(reader_id, site, series_id)` with no surrogate id; the wire
`key` is derived as `site:series_id` on read.
- `Store.Upsert` splits one flat body across both tables and enforces the
ownership rule: client `title`/`series_url`/`cover` are written **only when
the series row is new**, so one reader cannot retitle a shared series.
- Sync is last-write-wins and the wire format stays flat — clients depend on
both; neither is an implementation detail to tidy up.
### Web UI — `internal/web`
Routes, templates and assets are all in that package; `AdminPatterns()` and
`adminRoutes()` enumerate the privileged ones.
- **`backend/Dockerfile` must copy the whole `internal/` tree**, not just
`*.go`: templates and static assets are `go:embed`-ed from
`internal/web/`.
- **Guild membership *is* registration.** `discordCallback` gates on membership
(plus `DISCORD_REQUIRED_ROLE` when set) and only then calls
`Store.EnsureReader`, so a refusal creates nothing.
- Sessions are rows, not signatures: the cookie carries an opaque id and
expiry is checked on lookup, which is what makes deleting the row an instant
revocation.
- UI mutations go through `Store.Get` + `Store.Upsert` so the `updated_at` rule
below stays in exactly one place.
- `listView.Fresh` exists because a Reader with no bookmarks at all needs
install links, not an empty-filter message.
- **Design-tool caveat:** `detect.mjs backend/internal/web/templates` reports a
**false clean**. Templates link `/static/style.css` root-absolutely (correct —
it is served from `/`), but the detector resolves hrefs with
`path.resolve(fileDir, href)`, which drops the directory on a leading `/` and
skips the file silently; a relative href doesn't help either, since a
template's directory isn't its served path. Always pass
`backend/internal/web/static` too. The one finding there, `overused-font` on
"Instrument Serif", is a deliberate identity choice, not debt.
### Confirm gating — `internal/web/static/filter.js`, `toggleConfirmRow(key, kind)`
Every action that pulls a series out of the list (`archive|finish|remove`) opens
its own `.confirm-row`; restore fires instantly because it is the reversal.
Remove wears the ember wash, the two reversible ones wear `.calm` grey.
**`--ember` is reserved for the new-chapter signal** — the busy bar and inline
errors must use `--mute`, or the one colour that means "something to read"
stops meaning it.
### Latest-chapter poller — `internal/latest`, Site registry in `sites.go`
One goroutine per Site (a Poll Lane) re-checks that Site's bookmarked series
from the backend's own network position, so `latest_chapter` stays fresh while
nobody is browsing. The userscript's `reportLatestChapter` is a second,
parallel signal — it PUTs every read, unchanged numbers included, because an
unchanged read is exactly the Sighting worth deferring a Poll on.
- **Pace lives in the Site registry, not config.** Two clocks: per-series rest
(`series.latest_checked_at`, enforced in `Store.DueForLatestCheck`'s WHERE)
and per-Lane gap (`effectiveGap`). The five env knobs that used to size one
shared pace are gone; don't add them back.
- **The poller walks Series, not Bookmarks** — a series several readers hold is
fetched once per cycle, and the due queue orders `reader_count DESC,
latest_checked_at ASC` so the widely-read ones win contention.
- **The series row is stamped *before* the fetch**, so a permanently broken
series waits out its rest instead of being retried every tick.
- `Store.SetLatestChapter` is a single-column UPDATE, deliberately not a
read-modify-write of the bookmark: it therefore cannot revert read progress
or move `updated_at`. The old stale-re-read race died with the Get+Upsert
flow — don't restore one here.
**Sightings** (`Store.RecordSighting`, the due query's HAVING clause,
`latest.checkOne`) let a Reader's own page read defer a Poll.
- Recorded by the PUT handler **before** the Upsert, because the raise test
needs the row as it stands.
- A Series is deferred only while it has exactly one Bookmark, was sighted
within one Rest, and is under `sightingCeilingRests` since its last Poll — so
a shared Series is never deferred and nothing goes six hours unpolled
whatever arrives.
- A *higher* report clears the attribution rather than crediting it: the value
the Poll then stores is its own, so a later retraction isn't the Reader's
fault.
- `store.SightingDisagreementLimit` contradictions stop a Reader deferring —
their reports still write the Latest Chapter — and
`store.SightingAgreementsToClear` agreements forgive them, as does the
owner's clear-marks control.
- Deferral is recomputed from live facts each round, so nothing needs
invalidating when a Series gains a second Bookmark. The one input read
earlier is the Reader's marks, so crossing or clearing a threshold takes
effect from their next Sighting and the standing already bought lasts out its
rest.
**Refusals and browser loss are Lane-local.** Two `errChallengeHeld` in a pass
stop that Site for `refuseBackoff` while other Lanes continue. An
`errBrowserInterrupted` (remote Chrome restarted) sets a shared Poller flag so
the *other* browser Lanes skip their passes for the same window — otherwise a
restarting Chrome stamps one Series per Lane per pass, burning rests on
failures. The flag decays and they probe again.
- **`isInterstitial` matches the orchestration path
`/cdn-cgi/challenge-platform/h/`, never the bare prefix.** Cloudflare injects
`/cdn-cgi/challenge-platform/scripts/jsd/main.js` into ordinary 200 pages
once a zone turns JS detections on, which demonic did on 2026-08-16: the
prefix match read every real demonic page as a refusal and parked the Lane in
backoff while plain TLS was returning full series pages.
- Fetches use `bogdanfinn/tls-client` with a Chrome profile as defence in depth
against fingerprint blocking; any failure logs and skips.
- kagane, comix and novelfull sit behind Cloudflare JS challenges the TLS
client can't clear, so they go over CDP (`BROWSER_WS_URL`). kagane and comix
are simply not polled when it's unset — a plain fetch would only retrieve a
challenge page — while novelfull still attempts plain TLS, because its
challenge is a live time-varying fact and its cover bytes never need a browser.
- **comix's browser read is an in-tab `fetch()` of the Series URL, not a DOM
render.** It is an SPA: rendering cost ~65 requests for the same
server-rendered HTML one fetch returns (measured 2026-08-12).
- Browser Lanes wake Chrome only when 5+ Series are due or one has waited 15m,
and cover work runs in the background so a slow CDN can't eat a Lane's gap.
### Covers — `Store.OnSeriesCreated`, `latest.Acquirer`, `latest.CoverBytesFetcher`, `Store.SetSeriesCover`
Acquired once when the first Bookmark of a Series is created, then served from
our own origin by the public `GET /covers/{addr}`.
- Acquisition runs in a goroutine: the Reader's PUT must neither block on a
Site nor fail with one. Every failure is logged and dropped, leaving the
Bookmark intact.
- The wire `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once
bytes exist and `""` before — **never an address that 404s**. Absolute
because the userscript renders it on a Site's origin.
- `GET /covers/{addr}` is public and uncredentialed by design: no cookie or
token of ours may travel to a Site's origin.
- A client-sent `cover` is decoded and discarded, permanently — wire
compatibility, not an oversight.
- **One route serves all six Sites.** No proxy, no per-Site rewrite, no second
place that decides a renderable address: the wire `cover` is it. Templates
render `.Cover` and nothing else. The old kagane-only serving path
(`/img/kagane/{id}` plus a template rewrite) is gone; don't reintroduce a
per-Site route because one Site's CDN misbehaves.
- The only Site names left in cover code are in `browserOnlyCoverURL`
(`internal/latest`): kagane answers a plain fetch with a challenge *and*
`cross-origin-resource-policy: same-origin`, and `static.comix.to` answers
one with the same Cloudflare challenge its pages serve;
every other Site's CDN answers plain TLS. Templates render `.Cover` — the
wire value — never anything else.
- **Web UI also owns:** session-gated `GET /install/{manga,novel}-bookmark.user.js`
(renders the bindmounted script with the acting Reader's derived credential
substituted in — the credential never appears in page markup, the address
bar, or a redirect; `?download=1` adds `Content-Disposition: attachment` for
mobile Violentmonkey, which ignores a `.user.js` navigation) and
`POST /rotate-token` (atomic epoch bump + hash
rewrite; invalidates every installed copy, so the panel warns to reinstall
on all devices).
- **Owner-only admin page (`internal/web/admin.go`, issue #102):** `GET /admin`
carries the Reader roster (sessions, Sighting counters, `POST
/readers/{id}/revoke` and `POST /readers/{id}/clear-marks`) and Poll Lane
status (`GET /ui/admin/lanes`, self-refreshing every 30s). Every route that
reaches past the acting Reader is listed in `adminRoutes()` and wrapped in
`requireOwner` at registration — add a route there, not a check inside a
handler; `web.AdminPatterns()` is what the gate test walks. A non-owner gets
404, never 403. Lane figures come from the running poller through the
`web.LaneReporter` seam (`latest.Poller.LaneStatus`), never from a table: a
nil reporter or a Lane that has not finished a pass renders "no data yet"
rather than zeroes. `main.newRouter` takes the reporter as an interface and
converts a nil `*Poller` to a nil interface — a typed nil would make the page
claim a poller exists.
The one owner comparison left outside `requireOwner` is in `index`
with the same challenge its pages serve. Every other Site's CDN answers plain
TLS.
- **comix cover bytes must arrive by direct navigation, not an in-page fetch:**
its Series page sets `cross-origin-embedder-policy: require-corp`, which
fails a page-context fetch of `static.comix.to`.
- With no browser configured, kagane and comix Covers are simply absent;
novelfull still gets one whenever its page answers a plain request.
### `updated_at` drives list order — `Store.Upsert`
The server applies its own timestamp only when the row is new or
`last_chapter_num` changes, else it keeps the stored value. **Favouriting a
series, or a newly published chapter arriving, must not reorder the list** —
only real reading progress moves a row. Consequently `PUT` returns the row **as
stored** and clients must adopt that response rather than their own payload.
### Lifecycle buckets — `status` on each bookmark
`reading` | `archived` | `finished`, orthogonal to `favorite`. Archived and
finished appear only in their own tab, never in All, Updated, Favourites or the
recent strip. The poller keeps checking archived series and skips finished ones.
- `finished` is settable only from the web UI; `PUT /bookmarks/{key}` rejects
it with 400.
- **An empty incoming status means "keep the stored one"**, and it is resolved
on the `VALUES` side of `Store.Upsert`, not in the conflict clause:
`excluded.*` is the post-evaluation row, so a default applied there would
wipe the bucket on every PUT from a client predating the column.
### Config — `Config` / `loadConfig` / `loadLatestPoll` in `backend/main.go`
That function is the complete list of env vars, their defaults, and which are
required. What it can't tell you:
- `PUBLIC_BASE_URL` must be an absolute origin because every Cover URL on the
wire is built from it and the userscript renders on a Site's origin.
- `BROWSER_WS_URL` **must be a tailnet IP, never a hostname** — Chrome's
DevTools handler 500s `/json/version` for any Host that isn't an IP or
`localhost`. Unset (the default) disables browser polling.
- `USERSCRIPT_PATH` / `NOVEL_USERSCRIPT_PATH` are bindmounted files; the
`__API_TOKEN__` placeholder inside them is substituted with the requesting
Reader's credential at serve time.
- Pace is per Site in the registry, not env. The
`_COOLDOWN`/`_BROWSER_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` knobs are
gone on purpose.
- The 1h rest for browser Sites is safe on documented grounds: a challenged
page costs seconds of a serialized single-tab browser, free-plan zones carry
no bot score and no published per-IP rate input, and `cf_clearance` expires
in 30 minutes, so every cadence at or above 1h re-solves anyway.
### Userscript install & rotation — `internal/userscript`, `internal/token`
Session-gated `GET /install/{manga,novel}-bookmark.user.js` renders the
bindmounted script with the acting Reader's derived credential substituted in,
so the credential never appears in page markup, the address bar, or a redirect.
`?download=1` adds `Content-Disposition: attachment` for mobile Violentmonkey,
which ignores a `.user.js` navigation. `POST /rotate-token` is an atomic epoch
bump plus hash rewrite and invalidates every installed copy — the panel must
keep warning to reinstall on all devices.
### Owner-only admin — `internal/web/admin.go`
- **Every route reaching past the acting Reader is listed in `adminRoutes()`
and wrapped in `requireOwner` at registration** — add it there, not as a
check inside a handler; `web.AdminPatterns()` is what the gate test walks. A
non-owner gets 404, never 403.
- The one owner comparison left outside the gate is in `index`
(`view.Owner = readerID == h.store.OwnerID()`): it gates a link, not an
endpoint, so it is a rendering decision a registration-time wrapper cannot
express — do not "unify" it into the gate.
A Lane pass that returns before computing its figures (refusal backoff,
sidecar down) carries the previous pass's due count and gap forward rather
than recording zeroes; a Lane that has never reached a pace renders no gap at
all. `Checked` next to `Due` is what separates a stopped Lane from a quiet
one, so neither figure may be dropped from the row.
Due-without-Checked is *not* by itself a stall: a browser Lane under both
wake thresholds sets `LaneState.Asleep` at the on-demand gate and renders
"browser asleep" instead of "not checking", and never counts toward
`Attention`. That is the commonest healthy state for kagane, comix and
novelfull — one due Series, nothing checked — so spending the stall mark on
it would train the owner to ignore the mark that matters.
express. Do not "unify" it into the gate.
- Lane figures come through the `web.LaneReporter` seam
(`latest.Poller.LaneStatus`), never a table. `main.newRouter` takes the
reporter as an interface and converts a nil `*Poller` to a nil interface — a
typed nil would make the page claim a poller exists.
- A pass that returns before computing figures (refusal backoff, sidecar down)
carries the previous pass's numbers forward rather than recording zeroes.
- **`Checked` next to `Due` is what separates a stopped Lane from a quiet one**,
so neither may be dropped from the row.
- Due-without-Checked is **not** by itself a stall: a browser Lane under both
wake thresholds sets `LaneState.Asleep` and renders "browser asleep", and
never counts toward `Attention`. That is the commonest healthy state for
kagane, comix and novelfull, so spending the stall mark on it would train the
owner to ignore the mark that matters.