Compare commits

..

1 Commits

Author SHA1 Message Date
sulthan d168cf1ad0 chore: remove graphify
Measured on this repo, graphify cost more than it returned. `graphify query`
answers with a keyword-seeded BFS neighbourhood, not a location: asking where
CORS origin reflection lives returned 73 nodes, mostly api_test.go helpers
plus an unrelated golang-performance doc section matched on the word
"reflection", and never named httpmw/middleware.go:135 or main.go:121 — which
grep gives in 40ms. `graphify explain` on a known symbol is sound but
duplicates what the LSP already answers against live files.

Against that, the PreToolUse hooks injected a "run graphify query first"
paragraph on every grep and every source read, the map cost 685k input tokens
across five builds, and graph.json plus graph.html carried 3.4MB through 11
commits of churn. AGENTS.md is the better orientation artifact: it holds the
CDP and clock findings, the adapter list, and the security invariants, none of
which an AST graph can derive.

Removes the committed map, the AGENTS.md rules block, the .gitignore entries,
both hooks, and the stale `graphify query` instruction in the
implement-tickets skill. The CLI, its skill directory, and the post-commit /
post-checkout git hooks were uninstalled outside the tree.
2026-08-17 11:17:51 +07:00
8 changed files with 402 additions and 885 deletions
-75
View File
@@ -1,75 +0,0 @@
---
name: gitea
description: Use for every forge operation in this repo — read, create, comment on, label, close, or search an issue; create, review, merge, or check out a PR; and whenever `gh`, `issue://`, or `pr://` fails or a ticket number is ambiguous. This repo's forge is self-hosted Gitea driven by `tea`, not GitHub.
---
# Gitea, not GitHub
`origin` is the self-hosted Gitea instance `gitea.violetcrown.my.id`, repo
`sulthan/mangaBookmark`. Everything past plain git goes through
[`tea`](https://gitea.com/gitea/tea) (0.14.2 on this machine).
**`gh` is not installed**, so the harness's `issue://<n>` and `pr://<n>` URIs
error out (`GitHub CLI (gh) is not installed`, measured 2026-08-17). There is no
fallback to add — read tickets with `tea`.
`tea` infers the repo from `origin`; auth lives in `tea login`, never a
`GH_TOKEN`. Your Gitea username comes from `tea login list` — `tea` has no `@me`.
Flags are the environment's job: run `tea <command> --help` rather than trusting
a remembered flag. This file carries only what `--help` will not tell you.
## Commands
| Job | Command |
|---|---|
| Read | `tea issue <n> --comments` / `tea pr <n> --comments` — `--comments` is not optional |
| List | `tea issue list --state open\|closed\|all -o json --fields index,title,body,labels,state,author` |
| Search | `tea issue list -k "<keyword>" -L "<label>" -A "<author>"` (`-K all` also searches PRs) |
| Create | `tea issue create -t "..." -d "..."` (`-L`, `-a` optional) |
| Comment | `tea comment <n> "..."` |
| Label | `tea issue edit <n> --add-labels "..."` / `--remove-labels "..."` |
| Close | `tea issue close <n>` / `tea pr close <n>` |
| PR | `tea pr create --head <branch> --base main -t "..." -d "..."`, `tea pr checkout <n>`, `tea pr review <n>`, `tea pr merge <n>` |
## Traps
- **A bare read is a truncated read.** Without `--comments`, `tea issue <n>` and
`tea pr <n>` print the opening body and drop every comment silently — no
prompt, no marker, no hint that more exists (measured 2026-08-17: issue #123
prints 40 lines bare, 132 with `--comments`). The comments are where the
decisions live and the body is usually the stalest part of the ticket, so
**every read that exists to understand an issue or PR passes `--comments`**,
and understanding means body plus all comments plus whatever ticket they point
at. Comment count is `tea issue list --fields index,comments`, so a read that
shows fewer than that is incomplete. A PR's review comments are a second
stream: `tea pr review-comments <n>`.
- **One index space for issues and PRs.** A bare `#42` may be either: try
`tea pr 42`, fall back to `tea issue 42`. Say which one you found.
- **Output is rendered boxes**, not plain text. Anything you parse needs
`-o json`, plus `--fields` to keep the payload small. `tea pr create` prints
the PR URL on its last line.
- **`close` takes no `--comment`.** Comment with `tea comment <n>`, then close.
- **Gitea will not auto-create a label.** `tea labels list` first; missing one
gets `tea labels create --name "..." --color "#rrggbb"` before the `edit`.
- **Multi-line bodies go through a heredoc**, never inline escapes:
```bash
tea issue create -t "Title" -d "$(cat <<'EOF'
body line one
- acceptance criterion
EOF
)"
```
- **No sub-issue and no dependency command.** Gitea's API has issue
dependencies, `tea` does not expose them, so parentage and blocking live as
body lines — the shapes are in `docs/agents/issue-tracker.md`.
## Conventions this repo layers on top
Ticket bodies, wayfinding issues, and the PR-as-request-surface flag:
`docs/agents/issue-tracker.md`. Triage label strings: `docs/agents/triage-labels.md`.
A label named there still has to exist in the tracker before `--add-labels`.
Finish a forge action by stating the number you touched and its state after —
"commented and closed #71" — so the write is checkable without a second query.
+1 -1
View File
@@ -11,7 +11,7 @@ to the tracker. You do not write the implementation — every line of ticket cod
is written by a `ticket-implementer` subagent in its own git worktree. Reach for is written by a `ticket-implementer` subagent in its own git worktree. Reach for
the editor yourself only for a merge conflict resolution. the editor yourself only for a merge conflict resolution.
Ticket source and tracker conventions: `docs/agents/issue-tracker.md`. `tea` usage: skill `gitea`. Ticket source and `tea` usage: `docs/agents/issue-tracker.md`.
## 1. Collect the tickets ## 1. Collect the tickets
+41 -69
View File
@@ -1,6 +1,6 @@
# AGENTS.md # AGENTS.md
Repo-wide guidance for coding agents. Guidance for OpenCode (and Claude Code) working in this repo.
## What this is ## What this is
@@ -13,18 +13,15 @@ One backend, one `bookmarks` table: a `kind` column (`manga`|`novel`) splits the
## Hard constraints (drive design — don't violate) ## Hard constraints (drive design — don't violate)
Nothing below is derivable from reading the code — it is why the code looks the
way it does, plus dated measurements against services we don't control.
Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free where plain web APIs suffice — keeps portability across engines: Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free where plain web APIs suffice — keeps portability across engines:
- **Avoid `GM_*` unless needed.** Prefer page `localStorage` over `GM_setValue`/`GM_getValue`, on-page UI over `GM_registerMenuCommand`, plain `fetch()` over `GM_xmlhttpRequest` for cross-origin. - **Avoid `GM_*` unless needed.** Prefer page `localStorage` over `GM_setValue`/`GM_getValue`, on-page UI over `GM_registerMenuCommand`, plain `fetch()` over `GM_xmlhttpRequest` for cross-origin.
- Cross-origin `fetch()` work **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block). - Cross-origin `fetch()` work **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block).
- Every site is its **own origin with its own `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync required, not optional. - Every site is its **own origin with its own `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync required, not optional.
- Userscript run in **isolated world**, so embedded API token safe from site's JS. - Userscript run in **isolated world**, so embedded API token safe from site's JS.
- Cloudflare's block on manga sites is **per-zone configuration plus request fingerprint, not IP reputation — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — a Site can turn its protection on overnight, which is exactly what comix.to did on 2026-08-12. An earlier version of this line blamed "Cloudflare's bot scoring"; that was wrong. The 1-99 bot score is Enterprise Bot Management only and does not exist for a free-plan zone, and no per-IP request rate is documented as an input to challenge issuance. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test. - Cloudflare's block on manga sites is **per-zone configuration plus request fingerprint, not IP reputation — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — a Site can turn its protection on overnight, which is exactly what comix.to did on 2026-08-12. An earlier version of this line blamed "Cloudflare's bot scoring"; that was wrong. The 1-99 bot score is Enterprise Bot Management only and does not exist for a free-plan zone, and no per-IP request rate is documented as an input to challenge issuance — `docs/research/cloudflare-bot-scoring-and-poll-cadence.md`. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test.
- **kagane.to, comix.to and novelfull.com are the exception to the above** — all three sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`). When that's unset, kagane and comix are skipped entirely (a plain fetch would only retrieve a challenge page) while novelfull pages are still attempted over plain TLS — its challenge is a live time-varying fact and its cover bytes never need the browser. comix turned hostile on 2026-08-12: its cover host `static.comix.to` is gated too, so its cover bytes go through the browser as well, and its page is read as an in-tab `fetch()` of the series URL rather than a rendered DOM — comix is an SPA, and rendering costs ~65 requests for the same server-rendered HTML one fetch returns. The three other sites poll fine over plain TLS. - **kagane.to, comix.to and novelfull.com are the exception to the above** — all three sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`). When that's unset, kagane and comix are skipped entirely (a plain fetch would only retrieve a challenge page) while novelfull pages are still attempted over plain TLS — its challenge is a live time-varying fact and its cover bytes never need the browser. comix turned hostile on 2026-08-12 (#98): its cover host `static.comix.to` is gated too, so its cover bytes go through the browser as well, and its page is read as an in-tab `fetch()` of the series URL rather than a rendered DOM — comix is an SPA, and rendering costs ~65 requests for the same server-rendered HTML one fetch returns. The three other sites poll fine over plain TLS.
- **The CDP browser must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a non-UTC clock zone cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, ignoring the file's contents, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead. - **The CDP browser must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a non-UTC clock zone cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, ignoring the file's contents, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead.
- **The browser is not in the API stack and must not be put back.** It's its own compose unit (`chrome/docker-compose.yml`) on a second machine, reached over the tailnet — it held 471 MiB on a 1974 MiB swapless VPS, and a residential egress avoids the cloud-hosting-IP signature Bot Fight Mode documentedly challenges (not a better "score" — free-plan zones have no score). Consequences that constrain code: `BROWSER_WS_URL` must be a tailnet **IP** (a MagicDNS name 500s at `/json/version`, same trap as the old Docker service name); the CDP port binds to the tailnet address only, since CDP authenticates nothing and that host has a real LAN; and the browser is on-demand, so an unreachable or asleep one must degrade exactly as an unset `BROWSER_WS_URL` — plain-TLS libraries unaffected, kagane/comix logged and skipped, stored covers still served. Never add `chromedp.NoModifyURL`: discovery per fetch is what makes a restarted Chrome invisible. - **The browser is not in the API stack and must not be put back.** It's its own compose unit (`chrome/docker-compose.yml`) on a second machine, reached over the tailnet — it held 471 MiB on a 1974 MiB swapless VPS, and a residential egress avoids the cloud-hosting-IP signature Bot Fight Mode documentedly challenges (ADR-0006; not a better "score" — free-plan zones have no score). Consequences that constrain code: `BROWSER_WS_URL` must be a tailnet **IP** (a MagicDNS name 500s at `/json/version`, same trap as the old Docker service name); the CDP port binds to the tailnet address only, since CDP authenticates nothing and that host has a real LAN; and the browser is on-demand (ADR-0005), so an unreachable or asleep one must degrade exactly as an unset `BROWSER_WS_URL` — plain-TLS libraries unaffected, kagane/comix logged and skipped, stored covers still served. Never add `chromedp.NoModifyURL`: discovery per fetch is what makes a restarted Chrome invisible.
- **UTC is the tell, not a country mismatch.** A UTC clock is the datacenter default, and the challenge refuses it; any real zone clears. Measured 2026-08-08, identical container, one Indonesian egress IP: UTC never cleared in 60s (twice), while `Asia/Jakarta` **and** `America/New_York` both cleared in 4s. An earlier note here claimed the zone had to match the egress IP's country — that was wrong, inferred from the host clock (`Asia/Bangkok`) rather than the measured egress. A second earlier claim, that Cloudflare "scores" a UTC clock, was also wrong: the measurement is real but the mechanism is not documented anywhere — Cloudflare publishes no timezone signal, and free-plan zones carry no score at all. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one. - **UTC is the tell, not a country mismatch.** A UTC clock is the datacenter default, and the challenge refuses it; any real zone clears. Measured 2026-08-08, identical container, one Indonesian egress IP: UTC never cleared in 60s (twice), while `Asia/Jakarta` **and** `America/New_York` both cleared in 4s. An earlier note here claimed the zone had to match the egress IP's country — that was wrong, inferred from the host clock (`Asia/Bangkok`) rather than the measured egress. A second earlier claim, that Cloudflare "scores" a UTC clock, was also wrong: the measurement is real but the mechanism is not documented anywhere — Cloudflare publishes no timezone signal, and free-plan zones carry no score at all. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one.
- **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives. - **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives.
@@ -39,7 +36,7 @@ Two Violentmonkey userscripts (isolated world, per-site adapters, localStorage c
on-demand Chrome, separate machine (chrome/) on-demand Chrome, separate machine (chrome/)
``` ```
Two deployable units on two machines: the API stack (`docker-compose.yml` + `docker-compose.prod.yml`, on the VPS) and the browser (`chrome/docker-compose.yml`, on the home machine). They share nothing but `BROWSER_WS_URL` and update independently. Backend-specific detail lives in `backend/AGENTS.md`, userscript-specific detail in `userscript/AGENTS.md`. Two deployable units on two machines: the API stack (`docker-compose.yml` + `docker-compose.prod.yml`, on the VPS) and the browser (`chrome/docker-compose.yml`, on the home machine). They share nothing but `BROWSER_WS_URL` and update independently. Backend-specific architecture (packages, endpoints, poller, config env vars) lives in `backend/AGENTS.md`. Userscript-specific structure (adapters, retry queue, UI, live URL shapes) lives in `userscript/AGENTS.md`. Deploy order `DEPLOY.md` (§7 for the browser), redeploy `REDEPLOY.md` (§8 for the browser).
## Commands ## Commands
@@ -67,22 +64,28 @@ Smoke test: `curl` endpoints with `Authorization: Bearer <token>`; confirm `OPTI
## Forge: Gitea, not GitHub ## Forge: Gitea, not GitHub
`origin` is self-hosted Gitea (`gitea.violetcrown.my.id`, repo `sulthan/mangaBookmark`), so **`gh` don't work here and the `issue://`/`pr://` URIs error out — drive the forge with `tea`.** How to run it — commands, traps, JSON output: skill `gitea`. Tracker conventions (ticket bodies, wayfinding, PR-as-request-surface flag): `docs/agents/issue-tracker.md`. Triage label strings: `docs/agents/triage-labels.md`. `origin` is self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` don't work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones:
- Open PR: `tea pr create --head <branch> --base main --title "..." --description "..."`
- List / view / check out: `tea pr list`, `tea pr <n>`, `tea pr checkout <n>`
- Issues: `tea issue create`, `tea issue list`
- Auth lives in `tea login`, not `GH_TOKEN` env var.
`tea` print output as rendered boxes rather than plain text; PR URL lands on last line.
## Design system ## Design system
Web UI + userscript panel follow **Cinder**. Tokens are the `:root` block in Web UI + userscript panel follow **Cinder**, rules in `docs/design-system.md`
`backend/internal/web/static/style.css`; that file, `backend/internal/web/templates/*`, — source of truth Claude Design project `BookmarkManager Web UI`
and the userscript `TEMPLATE`/`CSS` are the only places it is expressed. (`969ac210-fe02-4c01-ae1b-9a271dcc779a`). Read it before touching
Source of truth for the visual language is the Claude Design project `backend/internal/web/static/style.css`, `backend/internal/web/templates/*`, or userscript
`BookmarkManager Web UI` (`969ac210-fe02-4c01-ae1b-9a271dcc779a`). `TEMPLATE`/`CSS`. Core law: **ember means new chapter only** — no other
state (busy, error, destruction) may use `--ember`; destruction gets
Core law: **ember means new chapter only** — no other state (busy, error, `--danger`. No cards/corners/shadows, one `--measure: 760px` column, tokens
destruction) may use `--ember`; destruction gets `--danger`. No only (never hardcode hex outside `:root`), both colour branches touched
cards/corners/shadows, one `--measure: 760px` column, tokens only (never together. Any move that pulls series out of list (archive/finish/remove)
hardcode hex outside `:root`), both colour branches touched together. Any move must be confirm-gated via its own `.confirm-row`; only restore fires
that pulls a series out of the list (archive/finish/remove) must be instantly.
confirm-gated via its own `.confirm-row`; only restore fires instantly.
## Security invariants ## Security invariants
@@ -124,19 +127,13 @@ Review gate: auth, CORS, session, crypto, and the fetch gate are security-critic
## Comments ## Comments
Comment only if code alone can't carry info. Cost per read — must earn spot. Comment only if code alone can't carry info. Cost per read — must earn spot.
Wrong comment worse than none: it misleads readers and measurably degrades
LLM performance on the file. Missing comment costs little. Bias to fewer.
Docstring on public/exported surface — exception, near-always worth it. Write for:
Contract only: what it takes, returns, throws, mutates; units; pre/post - Why not what. Tradeoffs, non-obvious decisions.
conditions. Not a restatement of the body. Skip on private/obvious. - Load-bearing detail looking incidental — say so if "simplify" breaks it.
Inline — write for:
- Why not what. Tradeoffs, non-obvious decisions, rejected alternatives.
- heavy detail looking incidental — say so if "simplify" breaks it.
- Non-local consequence, invisible from function alone. - Non-local consequence, invisible from function alone.
- Wire format / encoding / ordering / invariant — save callers re-deriving. - Wire format / encoding / interface contract — save callers re-deriving.
- Gotcha/workaround, with ref (issue, RFC, vendor bug) if exists. - Gotcha/workaround, with ref if exists.
- Domain/business rule not derivable from code. - Domain/business rule not derivable from code.
Skip: Skip:
@@ -145,49 +142,24 @@ Skip:
- Banners, dividers, `// helpers`. - Banners, dividers, `// helpers`.
- Change narration (`// fix bug`, `// as requested`, `// new impl`) — git's job. - Change narration (`// fix bug`, `// as requested`, `// new impl`) — git's job.
- Commented-out code — delete. - Commented-out code — delete.
- TODO without concrete action + owner. - TODO without concrete action.
- Narrating the plan you just reasoned through. Plan in prose or in your head;
ship the code, not the transcript.
- Anything restating a name that could be fixed by renaming instead.
Staleness filter: if the comment describes something likely to change Style: one dense comment over function beats one per line inside. Tight, no worked example unless bug subtle. Wrong comment worse than none — update/delete on change. Default fewer — sparse+high-signal beats comprehensive.
independently of this line, it will rot and start lying. Either anchor it to
something stable, assert it in a test, or leave it out.
Style: one dense comment over a function beats one per line inside. Tight; no Test: "competent reader get this from code in few sec?" Yes → skip. Needs detour through another file/spec/git-blame → write it.
worked example unless the bug is subtle. On edit, update or delete stale
comments in the code you touch — silence beats a lie.
Test: "competent reader get this from code in a few sec?" Yes → skip. ## Agent skills
Needs detour through another file/spec/git-blame/external doc → write it.
## Writing an AGENTS.md `AGENTS.md` is the single source of truth for agent guidance; every `CLAUDE.md` in this repo is a symlink to the `AGENTS.md` beside it. Edit `AGENTS.md`.
`AGENTS.md` is the single source of truth for agent guidance; every `CLAUDE.md` ### Issue tracker
in this repo is a symlink to the `AGENTS.md` beside it. Edit `AGENTS.md`.
**Cite code, never docs, issues, or plans.** A spec, ADR, plan file, or Gitea Issues live as Gitea issues on `gitea.violetcrown.my.id` (`sulthan/mangaBookmark`), driven by the `tea` CLI — not `gh`. See `docs/agents/issue-tracker.md`.
issue records what was true when it was written and then goes stale silently;
an agent that follows the pointer reads a decision that may already have been
reversed. Code is the only source true at read time — cite a package, file,
symbol, env var, or route. The sole non-code exception is a sibling
`AGENTS.md`. If a doc holds a fact an agent needs, restate the fact here rather
than linking to it.
**State a fact in prose only if the code cannot answer it.** Split by ### Triage labels
derivability:
- *Structure* — packages, routes, env vars, columns, struct fields. Rots fast, Default five-role vocabulary, label strings unchanged (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`). See `docs/agents/triage-labels.md`.
cheap to re-read. **Name the symbol, write nothing else.**
- *Mechanism* — what a function does, how a flow proceeds. **Name the symbol
plus at most one line of orientation.**
- *Rationale* — why it is this way, what a "simplify" would break, what was
tried and rejected. Not in the code and cannot be re-derived. **Write it out.**
- *Measurement* — an observation against something we don't control. **Write it
out with the date**; a dated fact is honest, an undated one pretends to be
permanent.
Restating mechanism in prose is how these files rot: the code changes, the ### Domain docs
paragraph doesn't, and the next agent trusts the paragraph. A pointer degrades
more honestly — and every symbol you name must actually exist, since a dead Single-context: one root `CONTEXT.md` plus `docs/adr/`, both created lazily. See `docs/agents/domain.md`.
pointer is a bug, not a stale sentence.
+270 -264
View File
@@ -1,268 +1,274 @@
Scope: `backend/`. Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system.
Each entry names the code that holds the truth — read that for *what it does*. - **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + Postgres over `jackc/pgx/v5` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`.
The prose here is only what code cannot tell you: rationale, rejected Single binary, split into packages under `backend/internal/`: `store`
alternatives, dated measurements, and invariants a plausible refactor would (Bookmark type, Postgres persistence, migration runner), `latest` (background
silently break. poller, site parsers, TLS fetcher), `session` (cookie signing, login
rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON
### Layout bookmark handlers), `userscript` (userscript-serving handler), `web`
(browser UI handler + `templates/` + `static/`, `go:embed`-ed).
`backend/main.go` → `newRouter` is the composition root, the only place `backend/main.go` is the composition root — the only place that wires
packages are wired. Packages under `backend/internal/`: `store`, `latest`, packages together into `newRouter`. Root-level `*_test.go` hold
`session`, `httpmw`, `api`, `userscript`, `web`, `token`, `pgtest`. Root-level integration tests that exercise the full router; unit tests for a
`*_test.go` exercise the full router; unit tests live beside their package. package live beside it under `internal/`.
- **Schema is migration-owned.** `internal/store/migrations/*.sql` is
Not visible from any single file: stdlib `net/http` with no framework, `go:embed`-ed and applied on every start by `store.migrate`: one numbered
Postgres over `jackc/pgx/v5`, `CGO_ENABLED=0` static binary into a distroless file per change, one transaction each, versions recorded in
image, TLS terminated by the reverse proxy so the service listens plain `:8080`. `schema_migrations`. Files are **append-only** — editing an applied one
changes nothing on a database that already ran it. No column probing, no
### Schema — `internal/store/migrations/*.sql`, run by `store.migrate` data-fixup migrations: both were SQLite-era machinery and are gone.
- **Tests need Docker.** `internal/pgtest` starts one `postgres:17-alpine`
- Migration files are **append-only**. Editing an applied one changes nothing container per test binary (`TestMain` -> `pgtest.Main`) and hands each test
on a database that already recorded its version in `schema_migrations`, so its own database (`pgtest.URL(t)`). A package whose tests touch the store
the fix silently applies to new deployments only. must have that `TestMain`.
- No column probing, no data-fixup migrations. Both were SQLite-era machinery - **Reader-owned store, four tables.** `readers` is keyed by Discord user ID
and were removed deliberately — don't reintroduce either. and carries the SHA-256 of the Reader's userscript credential plus a
`token_epoch` (issue #24). Credentials are derived, never stored: `token.Token(TOKEN_KEY, discord_id, epoch)` (HMAC, `internal/token`), and only its SHA-256 sits in `readers.token_sha256`, so install URLs can be rebuilt after any restart while a database leak yields nothing but hashes. The seed creates the **owner** row at startup; its epoch-0 hash is refreshed on every start **only while the row has never been rotated**, so a restart can never resurrect a rotated-away credential. Every other row is created by that Reader's own first login (`Store.EnsureReader`, idempotent on `discord_id`, and it never rewrites an existing row's hash). Rotation is `Store.RotateToken` (epoch bump + hash rewrite in one transaction), driven by the web UI.
### Tests need Docker — `internal/pgtest` `series` keyed `(site, series_id)`
(`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the
`pgtest.Main` from `TestMain` starts one `postgres:17-alpine` per test binary; shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`),
`pgtest.URL` hands each test its own database. A package whose tests touch the Latest Chapter, `latest_checked_at`, and the Sighting pair
store must have that `TestMain` or it has no database at all. `latest_sighted_at`/`latest_raised_by` (issue #103) — and `bookmarks` holds only what
differs between readers: progress, favourite, lifecycle bucket,
### Reader-owned store — `internal/store`, `internal/token` `updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no
surrogate id; the wire `key` is derived as `site:series_id` on read — and
Four tables; shape is in the migrations, behaviour in `Store`'s methods. every store read/write is scoped to the reader it names. Auth resolves the
acting Reader from the presented credential (`httpmw.Auth`) and nothing
- **Credentials are derived, never stored.** `token.Token(TOKEN_KEY, discord_id, epoch)` else — there is no unauthenticated-by-Reader route and no global token; the
is an HMAC; only its SHA-256 reaches `readers.token_sha256`. So install URLs reader id travels in the request context. Sync **last-write-wins**; the wire format
can be rebuilt after any restart, and a database leak yields nothing usable. stays flat (ADR-0004). `Store.Upsert` decomposes one flat body across two
- **The owner's epoch-0 hash is refreshed at startup only while the row has tables and enforces the ownership rule: client `title`/`series_url`/`cover`
never been rotated.** Drop that condition and a restart resurrects a are written only when the series row is new (ADR-0003).
rotated-away credential. - **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- `Store.EnsureReader` never rewrites an existing row's hash — a returning - **Web UI:** same binary serve the browser UI on a second
Reader's login must not invalidate their installed scripts. hostname — `GET /` (list, or login page when no session),
- **Every read and write is scoped to the acting Reader**, resolved from the `GET /auth/discord` + `GET /auth/discord/callback` (Discord OAuth,
presented credential by `httpmw.Auth` and carried in the request context. ADR-0002), `POST /logout`, `GET /static/*`, htmx fragment endpoints
There is no unauthenticated-by-Reader route and no global token. under `/ui/*`. Templates + assets `go:embed`-ed under
- **`series` holds what readers share, `bookmarks` only what differs.** A `backend/internal/web/`, so `backend/Dockerfile` must copy the whole
bookmark key is `(reader_id, site, series_id)` with no surrogate id; the wire `internal/` tree, not just `*.go`. Sessions are rows in the `sessions`
`key` is derived as `site:series_id` on read. table: the cookie carries only an opaque id, looked up (and expiry-
- `Store.Upsert` splits one flat body across both tables and enforces the checked) on every request, and deleting the row revokes the session.
ownership rule: client `title`/`series_url`/`cover` are written **only when Guild membership *is* registration (issue #27): `discordCallback` gates on
the series row is new**, so one reader cannot retitle a shared series. membership (and `DISCORD_REQUIRED_ROLE` when set) and then calls
- Sync is last-write-wins and the wire format stays flat — clients depend on `Store.EnsureReader`, so a refusal creates nothing and a returning Reader
both; neither is an implementation detail to tidy up. reuses their row. The owner is the only Reader with administrative reach:
`POST /readers/{id}/revoke` (404 for anyone else) drops that Reader's
### Web UI — `internal/web` sessions, and the `readers` panel renders only on the owner's page.
A Reader with no bookmarks at all sees `listView.Fresh`, whose empty state
Routes, templates and assets are all in that package; `AdminPatterns()` and offers both install links instead of describing a filter.
`adminRoutes()` enumerate the privileged ones. UI mutations read-modify-write
through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one
- **`backend/Dockerfile` must copy the whole `internal/` tree**, not just place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`.
`*.go`: templates and static assets are `go:embed`-ed from **Design-tool caveat:** templates link `/static/style.css` root-absolutely
`internal/web/`. (correct — served from `/`), but impeccable detector resolves
- **Guild membership *is* registration.** `discordCallback` gates on membership stylesheet href with `path.resolve(fileDir, href)`, drops directory
(plus `DISCORD_REQUIRED_ROLE` when set) and only then calls on leading `/` and silently skip file. Relative href don't help
`Store.EnsureReader`, so a refusal creates nothing. either: template's directory isn't its served path. So
- Sessions are rows, not signatures: the cookie carries an opaque id and `detect.mjs backend/internal/web/templates` reports **false clean** —
expiry is checked on lookup, which is what makes deleting the row an instant always pass `backend/internal/web/static` too. One finding there,
revocation. `overused-font` on "Instrument Serif", deliberate identity choice, not debt.
- UI mutations go through `Store.Get` + `Store.Upsert` so the `updated_at` rule - **Every action that moves series out of list is confirm-gated.**
below stays in exactly one place. Archive, finish, remove each open own `.confirm-row` disclosure
- `listView.Fresh` exists because a Reader with no bookmarks at all needs (`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈
install links, not an empty-filter message. `archive|finish|remove`); restore fire instantly since it's the reversal.
- **Design-tool caveat:** `detect.mjs backend/internal/web/templates` reports a Remove's row wear ember wash, two reversible ones wear `.calm` grey.
**false clean**. Templates link `/static/style.css` root-absolutely (correct — `--ember` stay reserved for new-chapter signal: busy bar and inline
it is served from `/`), but the detector resolves hrefs with error use `--mute`.
`path.resolve(fileDir, href)`, which drops the directory on a leading `/` and - **Latest-chapter poller:** one goroutine per Site (a Poll Lane, issue #100),
skips the file silently; a relative href doesn't help either, since a each re-checking that Site's bookmarked series' newest published chapter from
template's directory isn't its served path. Always pass backend's own network access, so `latest_chapter` stays fresh when the user
`backend/internal/web/static` too. The one finding there, `overused-font` on isn't browsing. Second, parallel signal — the userscript keeps its own
"Instrument Serif", is a deliberate identity choice, not debt. `maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` schedule, and its
`reportLatestChapter` PUTs every read, unchanged numbers included, because an
### Confirm gating — `internal/web/static/filter.js`, `toggleConfirmRow(key, kind)` unchanged read is exactly the Sighting worth deferring a Poll on (#103).
Two independent clocks: per-series rest (`series.latest_checked_at`,
Every action that pulls a series out of the list (`archive|finish|remove`) opens enforced by `Store.DueForLatestCheck`'s WHERE clause — `now - Rest`) and
its own `.confirm-row`; restore fires instantly because it is the reversal. per-Lane gap (the Lane sleeping between fetches, `effectiveGap`). Both live
Remove wears the ember wash, the two reversible ones wear `.calm` grey. in the Site registry (`internal/latest/sites.go`), not config: the five env
**`--ember` is reserved for the new-chapter signal** — the busy bar and inline knobs that used to size a shared pace are gone.
errors must use `--mute`, or the one colour that means "something to read" The poller walks **Series, not Bookmarks** — a series referenced by several
stops meaning it. bookmarks is fetched once per cycle, and the due queue orders
`reader_count DESC, latest_checked_at ASC` (ADR-0003). Series row stamped
### Latest-chapter poller — `internal/latest`, Site registry in `sites.go` *before* fetch so broken series wait out the rest instead of retrying
every tick; found chapter written straight to the series row via
One goroutine per Site (a Poll Lane) re-checks that Site's bookmarked series `Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
from the backend's own network position, so `latest_chapter` stays fresh while order — is never touched.
nobody is browsing. The userscript's `reportLatestChapter` is a second, **Sightings** (issue #103, ADR-0011) let a Reader's own page read defer a
parallel signal — it PUTs every read, unchanged numbers included, because an Poll: `Store.RecordSighting` — called by the PUT handler *before* the Upsert,
unchanged read is exactly the Sighting worth deferring a Poll on. because the raise test needs the row as it stands — stamps
`series.latest_sighted_at` and, when the report raises the stored number,
- **Pace lives in the Site registry, not config.** Two clocks: per-series rest names its Reader in `series.latest_raised_by`. The due query's HAVING clause
(`series.latest_checked_at`, enforced in `Store.DueForLatestCheck`'s WHERE) is where deferral lives: a Series is skipped only while it has exactly one
and per-Lane gap (`effectiveGap`). The five env knobs that used to size one Bookmark, was sighted within one Rest, and is under the ceiling
shared pace are gone; don't add them back. (`sightingCeilingRests`, six of that Site's rests) since its last Poll. So a
- **The poller walks Series, not Bookmarks** — a series several readers hold is shared Series is never deferred, and no Series goes six hours unpolled
fetched once per cycle, and the due queue orders `reader_count DESC, whatever arrives. `checkOne` judges the named Reader off the comparison it
latest_checked_at ASC` so the widely-read ones win contention. already makes: a lower number is a contradiction (logged with the Reader and
- **The series row is stamped *before* the fetch**, so a permanently broken both numbers), the same number an agreement, a higher number the Site
series waits out its rest instead of being retried every tick. publishing and neither — that last one clears the attribution instead, since
- `Store.SetLatestChapter` is a single-column UPDATE, deliberately not a the value the Poll then stores is its own and a later retraction is not the
read-modify-write of the bookmark: it therefore cannot revert read progress Reader's fault. Three contradictions
or move `updated_at`. The old stale-re-read race died with the Get+Upsert (`store.SightingDisagreementLimit`) stop that Reader deferring — their
flow — don't restore one here. reports still write the Latest Chapter — and twenty consecutive agreements
(`store.SightingAgreementsToClear`) forgive them, as does the owner's
**Sightings** (`Store.RecordSighting`, the due query's HAVING clause, clear-marks control. Deferral is recomputed from live facts every round, so
`latest.checkOne`) let a Reader's own page read defer a Poll. nothing needs invalidating when a Series gains a second Bookmark; the one
input read earlier is the Reader's marks, checked when the Sighting is
- Recorded by the PUT handler **before** the Upsert, because the raise test recorded, so crossing the threshold or being cleared takes effect from that
needs the row as it stands. Reader's next Sighting and the standing already bought lasts out its rest.
- A Series is deferred only while it has exactly one Bookmark, was sighted Refusals and browser loss are Lane-local: two `errChallengeHeld` in one pass
within one Rest, and is under `sightingCeilingRests` since its last Poll — so stop that Site for `refuseBackoff` (15m) while other Lanes continue; an
a shared Series is never deferred and nothing goes six hours unpolled `errBrowserInterrupted` (remote Chrome restart) sets a shared Poller flag
whatever arrives. that makes the other browser Lanes skip their passes for the same 15m, so a
- A *higher* report clears the attribution rather than crediting it: the value restarting Chrome doesn't stamp one Series per Lane per pass — after the
the Poll then stores is its own, so a later retraction isn't the Reader's window the flag decays and they probe again. Browser Lanes wake Chrome only
fault. when 5+ Series are due or one has waited 15m (ADR-0005 on-demand browser),
- `store.SightingDisagreementLimit` contradictions stop a Reader deferring — and cover work (both healing a stored source URL and filling a blank from
their reports still write the Latest Chapter — and the series page) runs in the background so a slow CDN can't consume a
`store.SightingAgreementsToClear` agreements forgive them, as does the Lane's gap.
owner's clear-marks control. A refusal is only ever the challenge *page*: `isInterstitial` matches the
- Deferral is recomputed from live facts each round, so nothing needs orchestration path `/cdn-cgi/challenge-platform/h/`, never the bare prefix.
invalidating when a Series gains a second Bookmark. The one input read Cloudflare injects `/cdn-cgi/challenge-platform/scripts/jsd/main.js` into
earlier is the Reader's marks, so crossing or clearing a threshold takes ordinary 200 pages once a zone turns JS detections on, which demonic did on
effect from their next Sighting and the standing already bought lasts out its 2026-08-16 — the prefix match then read every real demonic page as a refusal
rest. and parked that Lane in 15m backoff while plain TLS was returning the full
series page.
**Refusals and browser loss are Lane-local.** Two `errChallengeHeld` in a pass Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth
stop that Site for `refuseBackoff` while other Lanes continue. An against fingerprint-based blocking; any failure log and skip. kagane, comix
`errBrowserInterrupted` (remote Chrome restarted) sets a shared Poller flag so and novelfull sit behind Cloudflare JavaScript challenges the TLS client
the *other* browser Lanes skip their passes for the same window — otherwise a can't clear, so they are fetched over CDP via `BROWSER_WS_URL`; kagane and
restarting Chrome stamps one Series per Lane per pass, burning rests on comix are simply not polled when that's unset, while novelfull falls back to
failures. The flag decays and they probe again. a plain-TLS attempt — its challenge is a live time-varying fact, and its
cover bytes never need the browser. comix's browser read is an in-tab
- **`isInterstitial` matches the orchestration path `fetch()` of the Series URL, not a DOM render: it is an SPA, so rendering
`/cdn-cgi/challenge-platform/h/`, never the bare prefix.** Cloudflare injects costs ~65 requests for the same server-rendered HTML one fetch returns
`/cdn-cgi/challenge-platform/scripts/jsd/main.js` into ordinary 200 pages (measured 2026-08-12, issue #98). See
once a zone turns JS detections on, which demonic did on 2026-08-16: the `docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
prefix match read every real demonic page as a refusal and parked the Lane in The poller's series write is a single-column UPDATE
backoff while plain TLS was returning full series pages. (`Store.SetLatestChapter`), not a read-modify-write of the whole bookmark:
- Fetches use `bogdanfinn/tls-client` with a Chrome profile as defence in depth it cannot revert read progress or move `updated_at`, so the old
against fingerprint blocking; any failure logs and skips. stale-re-read race is gone with the Get+Upsert flow.
- kagane, comix and novelfull sit behind Cloudflare JS challenges the TLS - **Covers are acquired at creation, then served from our own origin
client can't clear, so they go over CDP (`BROWSER_WS_URL`). kagane and comix (ADR-0007):** the first Bookmark of a Series fires `Store.OnSeriesCreated`,
are simply not polled when it's unset — a plain fetch would only retrieve a which `latest.Acquirer` turns into one series-page fetch yielding both the
challenge page — while novelfull still attempts plain TLS, because its Latest Chapter and the cover URL; the bytes then go through
challenge is a live time-varying fact and its cover bytes never need a browser. `latest.CoverBytesFetcher` into `Store.SetSeriesCover`. It runs in a
- **comix's browser read is an in-tab `fetch()` of the Series URL, not a DOM goroutine — the Reader's PUT must neither block on a Site nor fail with one
render.** It is an SPA: rendering cost ~65 requests for the same — and every failure is logged and dropped, leaving the Bookmark intact. The
server-rendered HTML one fetch returns (measured 2026-08-12). wire's `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once
- Browser Lanes wake Chrome only when 5+ Series are due or one has waited 15m, bytes exist and `""` before, never an address that 404s. `GET /covers/{addr}`
and cover work runs in the background so a slow CDN can't eat a Lane's gap. is public and uncredentialed: the userscript renders it on a Site's origin,
where no cookie or token of ours travels. A client-sent `cover` is decoded
### Covers — `Store.OnSeriesCreated`, `latest.Acquirer`, `latest.CoverBytesFetcher`, `Store.SetSeriesCover` and discarded, permanently (ADR-0004 compatibility).
Browser-backed Sites join the same pipeline (issue #62, extended to comix by
Acquired once when the first Bookmark of a Series is created, then served from #98): kagane and comix pages *and* cover bytes go through the browser sidecar
our own origin by the public `GET /covers/{addr}`. (nothing falls back to a plain fetch, which would only retrieve a challenge
page), while novelfull needs the browser only for its HTML — the cover URL
- Acquisition runs in a goroutine: the Reader's PUT must neither block on a comes out of the browser-fetched page and the bytes go over plain TLS. With
Site nor fail with one. Every failure is logged and dropped, leaving the no browser configured, kagane and comix Covers are simply absent; novelfull
Bookmark intact. still gets one — at creation and on the poll — when its page body happens to
- The wire `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once answer a plain request (the challenge is a live time-varying fact). comix
bytes exist and `""` before — **never an address that 404s**. Absolute cover bytes must arrive by direct navigation, not an in-page fetch: its
because the userscript renders it on a Site's origin. Series page sets `cross-origin-embedder-policy: require-corp`, which fails a
- `GET /covers/{addr}` is public and uncredentialed by design: no cookie or page-context fetch of `static.comix.to`. The old kagane-only
token of ours may travel to a Site's origin. serving path (`/img/kagane/{id}`, template rewrite, `CoverFetcher`) is gone
- A client-sent `cover` is decoded and discarded, permanently — wire (issue #63): the one public route serves every Site.
compatibility, not an oversight. - **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
- **One route serves all six Sites.** No proxy, no per-Site rewrite, no second - **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
place that decides a renderable address: the wire `cover` is it. Templates `finished`, orthogonal to `favorite`. Archived and finished appear only in
render `.Cover` and nothing else. The old kagane-only serving path own tab — not in All, Updated, Favourites, or recent strip. Poller keeps
(`/img/kagane/{id}` plus a template rewrite) is gone; don't reintroduce a checking archived series and skip finished ones. `finished` settable
per-Site route because one Site's CDN misbehaves. only from web UI; `PUT /bookmarks/{key}` reject it with 400.
- The only Site names left in cover code are in `browserOnlyCoverURL` **Empty incoming status means "keep stored one"** — resolved on the
(`internal/latest`): kagane answers a plain fetch with a challenge *and* `VALUES` side of `Store.Upsert`, not conflict clause, since
`cross-origin-resource-policy: same-origin`, and `static.comix.to` answers `excluded.*` is post-evaluation row and default applied there would
with the same challenge its pages serve. Every other Site's CDN answers plain wipe bucket on every PUT from client that predates column. See
TLS. `docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
- **comix cover bytes must arrive by direct navigation, not an in-page fetch:** - **Config via env:** `TOKEN_KEY` (derives every Reader's userscript credential;
its Series page sets `cross-origin-embedder-policy: require-corp`, which required), `OWNER_DISCORD_ID` (seeds the owner Reader — the administrator and
fails a page-context fetch of `static.comix.to`. the owner of every pre-registration bookmark; required),
- With no browser configured, kagane and comix Covers are simply absent; `ALLOWED_ORIGINS` (comma list),
novelfull still gets one whenever its page answers a plain request. `DATABASE_URL` (Postgres connection URL, required — no default),
`COVER_DIR` (required filesystem volume for content-addressed Cover bytes),
### `updated_at` drives list order — `Store.Upsert` `PUBLIC_BASE_URL` (required origin this deployment answers on, trailing
slash trimmed; every Cover URL on the wire is built from it, absolute
The server applies its own timestamp only when the row is new or because the userscript renders on a Site's origin — ADR-0007),
`last_chapter_num` changes, else it keeps the stored value. **Favouriting a `PORT` (default `8080`), `DISCORD_CLIENT_ID`/`_CLIENT_SECRET`/`_GUILD_ID`/
series, or a newly published chapter arriving, must not reorder the list** — `_REDIRECT_URI` (required; Discord OAuth for the browser UI),
only real reading progress moves a row. Consequently `PUT` returns the row **as `DISCORD_REQUIRED_ROLE` (optional role gate, empty by default),
stored** and clients must adopt that response rather than their own payload. `DISCORD_API_BASE` (default `https://discord.com/api/v10`),
`LATEST_CHAPTER_POLL_ENABLED` (background latest-chapter poller kill
### Lifecycle buckets — `status` on each bookmark switch, default on). Pace is per Site in the registry (issue #100): every
Site rests an hour and gaps ten seconds, a Site with more eligible Series
`reading` | `archived` | `finished`, orthogonal to `favorite`. Archived and than 360 tightens its own gap toward the 1s floor, and browser Lanes wake
finished appear only in their own tab, never in All, Updated, Favourites or the Chrome only on demand (ADR-0005). The `_COOLDOWN`/`_BROWSER_COOLDOWN`/
recent strip. The poller keeps checking archived series and skips finished ones. `_INTERVAL`/`_BATCH`/`_STAGGER` knobs that used to size a shared pace are
gone. The 1h rest for browser Sites is safe on documented grounds: a
- `finished` is settable only from the web UI; `PUT /bookmarks/{key}` rejects challenged page costs seconds of a serialized single-tab browser, free-plan
it with 400. zones have no bot score and no published per-IP rate input, and
- **An empty incoming status means "keep the stored one"**, and it is resolved `cf_clearance` expires in 30 minutes so every cadence at or above 1h
on the `VALUES` side of `Store.Upsert`, not in the conflict clause: re-solves anyway —
`excluded.*` is the post-evaluation row, so a default applied there would `docs/research/cloudflare-bot-scoring-and-poll-cadence.md`.
wipe the bucket on every PUT from a client predating the column. `USERSCRIPT_PATH` and `NOVEL_USERSCRIPT_PATH` (files served at
`/u/{token}/manga-bookmark.user.js` and `/u/{token}/novel-bookmark.user.js`,
### Config — `Config` / `loadConfig` / `loadLatestPoll` in `backend/main.go` defaults `/userscript/manga-bookmark.user.js` and
`/userscript/novel-bookmark.user.js`, both supplied by bindmount; the
That function is the complete list of env vars, their defaults, and which are
required. What it can't tell you:
- `PUBLIC_BASE_URL` must be an absolute origin because every Cover URL on the
wire is built from it and the userscript renders on a Site's origin.
- `BROWSER_WS_URL` **must be a tailnet IP, never a hostname** — Chrome's
DevTools handler 500s `/json/version` for any Host that isn't an IP or
`localhost`. Unset (the default) disables browser polling.
- `USERSCRIPT_PATH` / `NOVEL_USERSCRIPT_PATH` are bindmounted files; the
`__API_TOKEN__` placeholder inside them is substituted with the requesting `__API_TOKEN__` placeholder inside them is substituted with the requesting
Reader's credential at serve time. Reader's credential at serve time).
- Pace is per Site in the registry, not env. The `BROWSER_WS_URL` (CDP endpoint of the browser, which runs on a **separate
`_COOLDOWN`/`_BROWSER_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` knobs are machine** and is reached over the tailnet — ADR-0006, `chrome/docker-compose.yml`.
gone on purpose. Used by the poller for kagane, comix and novelfull page fetches and by the
- The 1h rest for browser Sites is safe on documented grounds: a challenged cover pipeline for kagane's and comix's image bytes (the browser is the only
page costs seconds of a serialized single-tab browser, free-plan zones carry route that clears the challenge those two serve their covers behind); unset —
no bot score and no published per-IP rate input, and `cf_clearance` expires the default — disables browser polling and leaves kagane and comix Covers
in 30 minutes, so every cadence at or above 1h re-solves anyway. blank until stored bytes
exist. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s
### Userscript install & rotation — `internal/userscript`, `internal/token` `/json/version` for any Host that isn't an IP or `localhost`).
- **No per-Site cover path (issue #63):** every Cover — all six Sites — is
Session-gated `GET /install/{manga,novel}-bookmark.user.js` renders the served by the one public `GET /covers/{addr}` route from content-addressed
bindmounted script with the acting Reader's derived credential substituted in, bytes. There is no proxy, no per-Site rewrite, no second place that decides
so the credential never appears in page markup, the address bar, or a redirect. a Cover's renderable address: the wire `cover` is it. The only place a Site
`?download=1` adds `Content-Disposition: attachment` for mobile Violentmonkey, name still appears in cover code is the extraction module (`latest`), where
which ignores a `.user.js` navigation. `POST /rotate-token` is an atomic epoch kagane's and comix's image URLs are claimed by `browserOnlyCoverURL` — kagane
bump plus hash rewrite and invalidates every installed copy — the panel must answers a plain fetch with a challenge and
keep warning to reinstall on all devices. `cross-origin-resource-policy: same-origin`, and `static.comix.to` answers
one with the same Cloudflare challenge its pages serve;
### Owner-only admin — `internal/web/admin.go` every other Site's CDN answers plain TLS. Templates render `.Cover` — the
wire value — never anything else.
- **Every route reaching past the acting Reader is listed in `adminRoutes()` - **Web UI also owns:** session-gated `GET /install/{manga,novel}-bookmark.user.js`
and wrapped in `requireOwner` at registration** — add it there, not as a (renders the bindmounted script with the acting Reader's derived credential
check inside a handler; `web.AdminPatterns()` is what the gate test walks. A substituted in — the credential never appears in page markup, the address
non-owner gets 404, never 403. bar, or a redirect; `?download=1` adds `Content-Disposition: attachment` for
- The one owner comparison left outside the gate is in `index` mobile Violentmonkey, which ignores a `.user.js` navigation) and
`POST /rotate-token` (atomic epoch bump + hash
rewrite; invalidates every installed copy, so the panel warns to reinstall
on all devices).
- **Owner-only admin page (`internal/web/admin.go`, issue #102):** `GET /admin`
carries the Reader roster (sessions, Sighting counters, `POST
/readers/{id}/revoke` and `POST /readers/{id}/clear-marks`) and Poll Lane
status (`GET /ui/admin/lanes`, self-refreshing every 30s). Every route that
reaches past the acting Reader is listed in `adminRoutes()` and wrapped in
`requireOwner` at registration — add a route there, not a check inside a
handler; `web.AdminPatterns()` is what the gate test walks. A non-owner gets
404, never 403. Lane figures come from the running poller through the
`web.LaneReporter` seam (`latest.Poller.LaneStatus`), never from a table: a
nil reporter or a Lane that has not finished a pass renders "no data yet"
rather than zeroes. `main.newRouter` takes the reporter as an interface and
converts a nil `*Poller` to a nil interface — a typed nil would make the page
claim a poller exists.
The one owner comparison left outside `requireOwner` is in `index`
(`view.Owner = readerID == h.store.OwnerID()`): it gates a link, not an (`view.Owner = readerID == h.store.OwnerID()`): it gates a link, not an
endpoint, so it is a rendering decision a registration-time wrapper cannot endpoint, so it is a rendering decision a registration-time wrapper cannot
express. Do not "unify" it into the gate. express — do not "unify" it into the gate.
- Lane figures come through the `web.LaneReporter` seam A Lane pass that returns before computing its figures (refusal backoff,
(`latest.Poller.LaneStatus`), never a table. `main.newRouter` takes the sidecar down) carries the previous pass's due count and gap forward rather
reporter as an interface and converts a nil `*Poller` to a nil interface — a than recording zeroes; a Lane that has never reached a pace renders no gap at
typed nil would make the page claim a poller exists. all. `Checked` next to `Due` is what separates a stopped Lane from a quiet
- A pass that returns before computing figures (refusal backoff, sidecar down) one, so neither figure may be dropped from the row.
carries the previous pass's numbers forward rather than recording zeroes. Due-without-Checked is *not* by itself a stall: a browser Lane under both
- **`Checked` next to `Due` is what separates a stopped Lane from a quiet one**, wake thresholds sets `LaneState.Asleep` at the on-demand gate and renders
so neither may be dropped from the row. "browser asleep" instead of "not checking", and never counts toward
- Due-without-Checked is **not** by itself a stall: a browser Lane under both `Attention`. That is the commonest healthy state for kagane, comix and
wake thresholds sets `LaneState.Asleep` and renders "browser asleep", and novelfull — one due Series, nothing checked — so spending the stall mark on
never counts toward `Attention`. That is the commonest healthy state for it would train the owner to ignore the mark that matters.
kagane, comix and novelfull, so spending the stall mark on it would train the
owner to ignore the mark that matters.
+5 -14
View File
@@ -47,15 +47,6 @@ func fetchCoverBytes(ctx context.Context, cover string, browser BrowserCoverFetc
// inject it to exercise hostile DNS results without touching the live network. // inject it to exercise hostile DNS results without touching the live network.
type CoverResolver func(context.Context, string) ([]netip.Addr, error) type CoverResolver func(context.Context, string) ([]netip.Addr, error)
// maxCoverBytes caps one cover, separately from the series-page maxBodyBytes:
// a cover is a bounded binary asset, not a text page, and 4 MiB rejected 12%
// of asurascans covers measured 2026-08-17 (p90 4.52 MB, max 8.57 MB — two of
// the three over-cap files were JPEGs, not the animated GIF of issue #71).
// 10 MiB is ~18% headroom over that worst case and matches the GitHub and
// Discord image limits; see docs/research/gif-maximum-byte-size.md. GIF itself
// has no maximum size, so this number is policy, not format.
const maxCoverBytes = 10 << 20
// TLSCoverFetcher retrieves image bytes with the standard HTTPS client. Unlike // TLSCoverFetcher retrieves image bytes with the standard HTTPS client. Unlike
// TLSFetcher, it does not need a browser fingerprint: cover hosts are public // TLSFetcher, it does not need a browser fingerprint: cover hosts are public
// CDNs and the response is accepted only after the destination gate passes. // CDNs and the response is accepted only after the destination gate passes.
@@ -161,15 +152,15 @@ func (f *TLSCoverFetcher) Fetch(ctx context.Context, sourceURL string) ([]byte,
if !ok { if !ok {
return nil, "", fmt.Errorf("fetch cover: unsupported content type %q", raw) return nil, "", fmt.Errorf("fetch cover: unsupported content type %q", raw)
} }
if resp.ContentLength > maxCoverBytes { if resp.ContentLength > maxBodyBytes {
return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxCoverBytes) return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxBodyBytes)
} }
body, err := io.ReadAll(io.LimitReader(resp.Body, maxCoverBytes+1)) body, err := io.ReadAll(io.LimitReader(resp.Body, maxBodyBytes+1))
if err != nil { if err != nil {
return nil, "", fmt.Errorf("read cover: %w", err) return nil, "", fmt.Errorf("read cover: %w", err)
} }
if len(body) > maxCoverBytes { if len(body) > maxBodyBytes {
return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxCoverBytes) return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxBodyBytes)
} }
return body, contentType, nil return body, contentType, nil
} }
+1 -24
View File
@@ -171,7 +171,7 @@ func TestCoverFetcherRejectsOversizedBody(t *testing.T) {
var calls int var calls int
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) { client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
calls++ calls++
response := coverResponse(http.StatusOK, "image/webp", "", bytes.Repeat([]byte("x"), maxCoverBytes+1)) response := coverResponse(http.StatusOK, "image/webp", "", bytes.Repeat([]byte("x"), maxBodyBytes+1))
response.ContentLength = -1 response.ContentLength = -1
return response, nil return response, nil
})} })}
@@ -187,29 +187,6 @@ func TestCoverFetcherRejectsOversizedBody(t *testing.T) {
} }
} }
// Covers between the series-page cap and the cover cap must be accepted: the
// 4 MiB page cap rejected 12% of asurascans covers (issue #71).
func TestCoverFetcherAcceptsCoverOverPageCap(t *testing.T) {
body := bytes.Repeat([]byte("x"), maxBodyBytes+1)
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
return coverResponse(http.StatusOK, "image/gif", "", body), nil
})}
fetcher := newCoverFetcher(client, func(context.Context, string) ([]netip.Addr, error) {
return []netip.Addr{netip.MustParseAddr("198.51.100.10")}, nil
})
got, contentType, err := fetcher.Fetch(context.Background(), "https://cdn.example/big.gif")
if err != nil {
t.Fatalf("Fetch rejected a %d-byte cover: %v", len(body), err)
}
if len(got) != len(body) {
t.Fatalf("body = %d bytes, want %d", len(got), len(body))
}
if contentType != "image/gif" {
t.Fatalf("content type = %q, want image/gif", contentType)
}
}
func TestCoverFetcherRejectsNonImage(t *testing.T) { func TestCoverFetcherRejectsNonImage(t *testing.T) {
var calls int var calls int
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) { client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
-344
View File
@@ -1,344 +0,0 @@
# GIF — maximum byte size of a file
Research note for Gitea issue #71 (backend `maxBodyBytes` = 4 MiB rejects the
8,571,192-byte animated cover GIF at
`https://cdn.asurascans.com/asura-images/covers/a-dragonslayers-peerless-regression.gif`).
All facts fetched live on **2026-08-17**: the GIF89a spec at
`https://www.w3.org/Graphics/GIF/spec-gif89a.txt`, Go stdlib `image/gif`
sources at `/usr/local/go/src/image/gif/reader.go` (Go 1.26.5), Chromium
`blink/renderer/platform/image-decoders/` sources via
`chromium.googlesource.com`, Firefox `image/decoders/nsGIFDecoder2.cpp` via
`hg.mozilla.org`, and cover bytes probed with plain `curl` (desktop Chrome UA;
`HEAD`/ranged `GET`). **No Cloudflare challenge was encountered on any CDN
probe** — every request returned real headers, consistent with the AGENTS.md
note of 2026-07-26 that plain `curl` works against both scan sites from the
dev machine and the VPS.
Every claim carries the URL it came from, or a reproducible command.
Interpretation rather than observation is marked `[INFERENCE]`.
---
## 1. Summary answer table
| Question | Answer | Evidence |
|---|---|---|
| Does the GIF89a spec define a maximum file size? | **No.** There is no file-size field anywhere in the format; the only numeric ceilings are per-field (16-bit screen/image dimensions, 255-byte sub-blocks, 12-bit LZW codes). | §2 |
| Maximum logical screen | 65535 × 65535 pixels (unsigned 16-bit width/height). | §2.1 |
| Number of frames / image descriptors | Unbounded — "An unlimited number of images may be present per Data Stream." | §2.2 |
| Formal max byte size of any single GIF | None. Single-frame worst case ≈ **6.44 GB** (12-bit LZW, max canvas); animated GIFs are **unbounded** because frames are unbounded. | §3 |
| Does the backend's decoder (Go `image/gif`) bound size? | **No.** It reads 16-bit dimensions and allocates `width×height` bytes per frame; a 65535² frame forces a ~4 GiB allocation. No total-size or dimension guard. | §4.1 |
| Do browsers bound on-disk GIF size? | Chromium and Firefox: no on-wire size cap in their GIF readers; Chromium caps *decoded* memory at min(4 B × pixels, platform budget). | §4.3, §4.4 |
| Real cover sizes (asurascans, n=25) | min 190,410 B · median 1,275,082 B · p90 4,524,788 B · max 8,571,192 B · **3/25 > 4 MiB** (two JPEGs and the animated GIF) | §5 |
| Real cover sizes (demonicscans/readermc, n=78) | min 13,298 B · median 63,061 B · max 801,200 B · 0/78 > 4 MiB | §5 |
| Comparable service caps | GitHub: 10 MB for images/GIFs. Discord API: default 10 MiB per file. Wikimedia: 100 MiB upload / 5 GiB host. | §6 |
| Recommended cover cap for #71 | **10 MiB** (separate from the 4 MiB series-page cap). Covers 100% of the 103 observed covers; matches GitHub/Discord calibration; ≤ 20 MiB worst-case transient per concurrent fetch+serve on a 1974 MiB swapless VPS. | §7 |
---
## 2. What the GIF89a specification actually bounds
Source: `https://www.w3.org/Graphics/GIF/spec-gif89a.txt` (fetched 2026-08-17).
### 2.1 Fixed-width fields — the only hard ceilings
The format is a stream of fixed-width blocks; the numeric fields that *do*
have a ceiling are all 16-bit unsigned, little-endian ("multi-byte numeric
fields are ordered Least Significant Byte first", §4 of the spec):
- **Logical Screen Width / Height** — "Unsigned" 2-byte fields (§18, Logical
Screen Descriptor) → maximum **65535 × 65535** pixels.
- **Image Left / Top Position, Image Width / Height** — "Unsigned" 2-byte
fields (§20, Image Descriptor). Each image "must fit within the boundaries
of the Logical Screen" (§20a), so an image cannot exceed the 65535² canvas
even though its own fields would allow it.
- **Data sub-blocks** — "A data sub-block may contain from 0 to 255 data
bytes" (§15); each sub-block is preceded by a 1-byte size field and the
stream is terminated by a 0x00 Block Terminator (§16). This bounds a
*chunk*, not the stream.
- **Global/Local Color Tables** — optional, "3 x 2^(Size of Global Color
Table+1)" bytes with a 3-bit size field → at most 3 × 2⁸ = **768 bytes**
each (§19, §21).
- **LZW codes** — "The output codes are of variable length, starting at
<code size>+1 bits per code, **up to 12 bits per code**. This defines a
maximum code value of 4095 (0xFFF)" (Appendix F, COMPRESSION, rule 4).
- **Trailer** — a single byte, fixed value 0x3B, "indicating the end of the
GIF Data Stream" (§27).
### 2.2 What is unbounded
- **Number of images (frames).** §20a, verbatim: "This block is REQUIRED for
an image. Exactly one Image Descriptor must be present per image in the
Data Stream. **An unlimited number of images may be present per Data
Stream.**"
- **The Data Stream itself.** The grammar in Appendix B is
`<GIF Data Stream> ::= Header <Logical Screen> <Data>* Trailer`, and the
spec states "the entity Data … may be repeated any number of times,
including 0 times." There is **no field anywhere that carries a file size,
byte count, frame count, or total-length value**. §13 (Block Sizes) only
defines sizes *within* blocks.
### 2.3 Verdict
**The GIF89a specification defines no maximum file size.** The only hard
bounds are per-field: 65535×65535 pixels per screen/image, 255 bytes per
sub-block, 12 bits per LZW code, and one trailer byte. A compliant decoder
must process whatever stream the blocks describe. Any byte ceiling a
particular GIF actually hits is therefore *implicit* — 16-bit dimensions,
LZW code width, decoder memory, or an external policy — never something the
format itself enforces. `[INFERENCE]` This is why real-world GIFs cap out at
"a few GB at most" and every service that wants a bound has to impose one
itself (see §6; Wikimedia explicitly documents that a 4 GiB host limit was a
storage-representation artifact of 32-bit integers, `phab:T191805`, not a
format limit).
---
## 3. Theoretical worst case
### 3.1 Single frame, maximal canvas, 8-bit pixels
| Quantity | Value | Derivation |
|---|---|---|
| Max pixels | 4,294,836,225 | 65535 × 65535 |
| Raw 8-bit palette-index raster | 4,294,836,225 B ≈ **4.29 GB / 4.00 GiB** | 1 byte per pixel (Table Based Image Data, §22; Go's `image.Paletted` uses exactly 1 byte/pixel) |
| LZW worst case | ≈ **6.44 GB / 6.00 GiB** | codes ≤ 12 bits each (Appendix F), at most ~1 code per pixel for incompressible data → ≤ 12 bits/px = 1.5 B/px → 4,294,836,225 × 1.5 B |
| Sub-block overhead | ≈ +25.3 MB | every ≤255-byte chunk carries a 1-byte size field (§15): ⌈6,442,254,338 / 255⌉ ≈ 25,263,743 size bytes, + 1 block terminator |
| Fixed overhead | ≈ +1.6 KB | header 6 B (§17) + logical screen descriptor 7 B (§18) + global color table ≤ 768 B (§19) + image descriptor 10 B (§20) + local color table ≤ 768 B (§21) + LZW minimum code size 1 B (§22) |
So a **single maximal-frame GIF cannot exceed ≈ 6.47 GB on the wire**
(12-bit LZW bound), and LZW being lossless means the real byte count depends
entirely on image content — the same canvas can be a few KB (flat color) or
~6 GB (noise).
Two caveats, both marked `[INFERENCE]`:
- The "1.5 B/px" figure assumes ~one emitted code per pixel. An encoder is
permitted to emit a Clear code at any point (Appendix F: "The Clear code
can appear at any point in the image data stream"), so a
pathological-but-compliant encoder emitting clear+pixel per pixel reaches
~24 bits/px ≈ 12.9 GB for the max canvas. Real encoders do not do this;
12-bit/px is the practical bound.
- The spec's deferred-clear note (cover sheet) explicitly allows an encoder
to keep using a full table at 12-bit codes without clearing, so the 12-bit
cap holds for the whole stream, it cannot "grow" past 12 bits.
### 3.2 Animated GIFs: unbounded
Every frame is one Image Descriptor, each bounded by the 65535² canvas, but
the *count* of frames is unbounded (§2.2). Total bytes = sum over frames —
therefore **there is no finite maximum byte size for an animated GIF** in
the format. The only thing that stops a real one is decoder memory, a
service cap, or disk space. `[INFERENCE]` This is the category the issue #71
cover falls into: it is an animated GIF (NETSCAPE2.0 loop extension found at
offset 0x310 of the file, verified 2026-08-17 by a ranged GET), and its
8,571,192 bytes are ~2.04× the current 4 MiB backend cap.
---
## 4. Decoder-side real limits
### 4.1 Go `image/gif` (the backend's decoder path, stdlib)
Source: `/usr/local/go/src/image/gif/reader.go`, Go 1.26.5.
- Dimensions are read as little-endian uint16 — `left/top/width/height :=
int(d.tmp[N]) + int(d.tmp[N+1])<<8` (reader.go:490-493) — so the format
ceiling 65535 applies, and nothing smaller is enforced.
- The only geometric check is that each frame fits inside the logical
screen: `if left+width > d.width || top+height > d.height` →
`errors.New("gif: frame bounds larger than image bounds")` (reader.go:512-513).
- **There is no file-size, byte-count, frame-count, or pixel-count guard.**
Each frame allocates `image.NewPaletted(...)` (reader.go:515) — a
`[]byte` of width×height — so decoding one legal 65535² frame attempts a
**~4.29 GB allocation**. `DecodeAll` (reader.go:603-605) additionally
retains every frame's `Pix` slice for the lifetime of the returned `*GIF`.
- `[INFERENCE]` On the 1974 MiB swapless VPS (root AGENTS.md), decoding such
a file would OOM rather than error cleanly; nothing in stdlib protects
the process. This matters for §7: the backend stores cover bytes without
decoding them (see §5.3), so the fetch path never triggers this — but any
future "validate/re-encode server-side" scheme would.
- Grep for `MaxInt|limit|too large|bounds` in reader.go: the only hits are
the frame-bounds check above and the `tmp [1024]byte` scratch buffer
(reader.go:109); no size caps exist.
### 4.2 giflib / libgif
**Not verified from source.** On 2026-08-17 the giflib sources were not
reachable from this network: `github.com/giflib/giflib` returns 404 (repo
gone/moved), `gitlab.com/giflib/giflib/-/raw/...` answers a Cloudflare
"Just a moment…" challenge, and the SourceForge project download path
404s. No limit claim about giflib is made here. `[INFERENCE]` giflib is
widely known to be allocation-driven with no dimension cap, but that is not
checked against source and is not needed for issue #71 (the backend uses Go
stdlib, not giflib).
### 4.3 Chromium (browser behaviour, first-party source)
- `third_party/blink/renderer/platform/image-decoders/gif/gif_image_reader.cc`
(via `chromium.googlesource.com/chromium/src/+/main/...`, fetched
2026-08-17): **no GIF byte-size or dimension cap found** — grep for
`max|limit|too large|dimension|65535|overflow` matches only license text.
- The base `ImageDecoder` caps *decoded memory*, not transfer size:
`CalculateMaxDecodedBytes` computes `min(4 * num_pixels, platform_max_decoded_bytes)`
(8 bytes/pixel for high-bit-depth), and the header comment says "Ignoring
this limit can cause excessive memory use or even crashes on low-memory
devices"
(`image_decoder.cc:94-117`, `image_decoder.h:545-549`). The GIF reader
itself is untouched by this — it is a decoded-buffer budget.
- Practical consequence `[INFERENCE]`: a browser will happily download and
store a multi-GB GIF from its own cache perspective; Chromium only limits
what it *decodes* into pixels.
### 4.4 Firefox
`image/decoders/nsGIFDecoder2.cpp` (via `hg.mozilla.org/mozilla-central/
raw-file/tip/...`, fetched 2026-08-17): **no dimension or size limit**; the
only guards are on LZW code width (`MAX_BITS` = 12, "maximum codeword size
of 12 bits") and the decode stack. Nothing bounds the on-disk byte size.
### 4.5 Summary
No mainstream decoder enforces a byte-size ceiling; they stop at the 16-bit
dimension ceiling (Go, by construction) or at decoded-memory budgets
(Chromium) or nowhere (Firefox). A GIF's byte size is policed only by
*storage* policies — which is what §6 calibrates and §7 sets.
---
## 5. Practical distribution — what real manga covers weigh
Probed **2026-08-17** with `curl -sI` (HEAD) and ranged GETs, desktop Chrome
UA. No Cloudflare block on any request. Sample = covers *as the backend
would fetch them* (the `og:image`/page-listed cover URL), not thumbnails we
chose by hand.
### 5.1 Exact commands
```sh
# asurascans.com — harvest cover URLs from the homepage, then HEAD each
curl -s -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/126.0" https://asurascans.com/ -o home.html
grep -oE 'https://cdn\.asurascans\.com/asura-images/covers/[^"&\\< ]+\.(webp|gif|jpg|jpeg|png)' home.html \
| sort -u | grep -v '\-400\.' | head -25 > sample.txt # one full-res cover per series, no -400 thumbs
while read -r u; do curl -s -A "…Chrome/126.0" -I "$u" | tr -d '\r' \
| grep -iE '^content-length:'; done < sample.txt
# demonicscans.org — covers live on readermc.org (ADR-0007), URLs contain spaces/UTF-8
curl -s -A "…Chrome/126.0" https://demonicscans.org/ -o demonic.html
grep -oE 'src="https://readermc\.org/images/thumbnails/[^"]+"' demonic.html | tr -d 'src="' > demonic.txt
# …plus og:image from 5 manga pages (Catastrophic-Necromancer, Magic-Emperor, …)
# each URL percent-encoded per path segment (urllib.parse.quote, safe=':/') before HEAD
```
### 5.2 asurascans — 25 full-res covers (mixed formats)
Homepage fetched 200 (664,700 B). All 25 returned `200` with a real
`Content-Length`. Distribution:
| Statistic | Bytes |
|---|---|
| n | 25 |
| min | 190,410 |
| median | 1,275,082 |
| p90 | 4,524,788 |
| max | 8,571,192 |
| mean | 1,943,651 |
| **> 4 MiB (4,194,304)** | **3 (12%)** — `a-dragonslayers-peerless-regression.gif` 8,571,192 (the issue #71 cover, animated: NETSCAPE2.0 at 0x310, 550×733, 256 colors); `bad-born-blood.3008f6.webp` 4,524,788 `image/jpeg`; `ending-maker.cfbf53.webp` 4,619,303 `image/jpeg` |
Notes: the CDN serves `Content-Type` by stored bytes, not by URL extension
(the `.webp` URLs return `image/png`, `image/jpeg`, or `image/webp` — the
sample spans all four of `png/jpeg/webp/gif`). Two of the three over-cap
files are **not GIFs**, so the current 4 MiB cap already silently drops 12%
of asura covers of any format. p90 itself (4.52 MB) exceeds the cap.
### 5.3 demonicscans — 78 covers on readermc.org
78 unique cover URLs (73 from the homepage's `/images/thumbnails/` plus 5
`og:image` values from manga pages — demonicscans publishes the thumbnail
file as the full cover, so that is exactly what the backend would fetch).
**78/78 returned 200 with a real Content-Length** (spaces and UTF-8 in the
filenames were percent-encoded per path segment; the homepage's raw HTML
carries `’`-style mojibake for curly quotes, which was repaired by
latin-1→utf-8 re-encoding before probing).
| Statistic | Bytes |
|---|---|
| n | 78 |
| min | 13,298 |
| median | 63,061 |
| p90 | 206,994 |
| max | 801,200 |
| mean | 110,038 |
| > 4 MiB | 0 |
### 5.4 Reading
`[INFERENCE]` asurascans covers are the heavy tail (median 1.3 MB, top
decile > 4 MiB, occasional ~5–9 MB), demonicscans covers are tiny (all
< 0.8 MB). A cover cap must be chosen against the *asura* distribution —
the 8.57 MB animated GIF is not a freak one-off outlier; the 90th
percentile already crosses 4 MiB and two JPEGs sit between 4.5–4.7 MB.
---
## 6. Comparable documented byte caps (first-party docs only)
| Service | Cap | Source (fetched 2026-08-17) |
|---|---|---|
| GitHub (issues/PR comments) | **10 MB for images and gifs**; 25 MB other files; 10/100 MB video | `https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/attaching-files` — "The maximum file size is: 10MB for images and gifs … 25MB for all other files" |
| Discord (API uploads) | default **10 MiB per file**, higher with Nitro / boost tier | `https://discord.com/developers/docs/reference#uploading-files` — "The file upload size limit applies to each file in a request. The default limit is `10 MiB` for all users" (help-center article `support.discord.com/hc/en-us/articles/115002935588` exists but answered 403 from this network on the probe date, so its figures were not verified here) |
| Wikimedia Commons | **100 MiB** upload limit; hosting up to **5 GiB**; GIF thumbnails limited to **100 megapixels**; prior 4 GiB host cap was a 32-bit storage artifact (phab:T191805) | `https://commons.wikimedia.org/wiki/Commons:Maximum_file_size` |
| MDN | nothing — MDN documents no byte-size limit for images; browsers impose none (see §4.3–4.4) | `[INFERENCE]` from absence in the platform docs read in §4 |
Calibration takeaway: two major platforms independently land on **~10 MB**
as the ceiling for an uploadable image/GIF (GitHub exactly 10 MB, Discord
exactly 10 MiB), with Wikimedia the outlier at 100 MiB/5 GiB because it is a
media *archive*. A 10 MiB cover cap is therefore squarely inside industry
normal.
---
## 7. Recommendation for issue #71
**Raise the cover cap to 10 MiB (10,485,760 B) — as a separate constant, not
by moving the shared one.**
Why:
- **Fits the measured reality.** The largest observed cover is 8,571,192 B
(the issue's animated GIF) = 82% of 10 MiB; 10 MiB covers **100% of the
103 sampled covers** and the *entire* asura distribution, including its
heavy tail. 4 MiB rejects 12% of asura covers (two of them plain JPEGs).
- **Matches industry calibration** (§6): GitHub 10 MB images/GIFs, Discord
10 MiB default. A 10 MiB cap is a number every engineer recognizes, and
it leaves ~18% headroom over the current worst observed file.
- **Costs little on the target hardware.** The backend buffers cover bytes
whole during fetch (`backend/internal/latest/cover.go`: `ContentLength >
maxBodyBytes` rejection at :155, then `io.ReadAll(io.LimitReader(…,
maxBodyBytes+1))` at :158) and loads the full body per `GET /covers/…`
(`backend/internal/api/handlers.go`, `Cover` → `w.Write(body)`). Worst
case per concurrent fetch **+** serve is therefore 2 × cap = 20 MiB; even
ten of each concurrently is ~200 MiB of a 1974 MiB swapless VPS (~10%),
and the browser unit (471 MiB, root AGENTS.md) is no longer on that box.
The 4 MiB series-page cap is *not* the issue — measured pages run
100 KB–1.2 MB (`backend/internal/latest/fetch.go` comment) — so keep it.
- **The cap is a separate knob.** Today one `const maxBodyBytes = 4 << 20`
(`backend/internal/latest/fetch.go:17`) gates *both* series pages and
covers (`cover.go` references it). Raising it wholesale would loosen the
page-side memory guard for no benefit; a cover-specific constant (e.g.
`maxCoverBytes = 10 << 20`) keeps the two policies independent. The fetch
already double-checks `ContentLength` and the post-`LimitReader` length,
so a larger constant changes nothing else.
Alternatives and their costs:
| Option | Cost |
|---|---|
| Keep 4 MiB | 12% of asura covers (incl. non-GIF JPEGs) never stored — current bug, silent missing covers. |
| 16 MiB cap | 2× headroom over the observed max for future GIFs; +60% worst-case transient memory vs 10 MiB; diverges from the GitHub/Discord 10 MB calibration. |
| Server-side re-encode / downscale covers | Requires decoding → Go `image/gif` allocates width×height per frame with **no guard** (§4.1); a legal 65535² GIF forces a ~4.29 GB allocation on a 1974 MiB swapless box — OOM, not an error. Also mutates bytes, which the store treats as immutable/content-addressed (ADR-0007). Highest risk, no upside at this scale. |
| No cap | Unbounded transient memory and disk; rejected outright. |
Decision is the user's; on the evidence, **10 MiB for covers, 4 MiB for
pages** is the defensible middle.
+84 -94
View File
@@ -1,103 +1,93 @@
Scope: `userscript/`. Guidance for OpenCode (and Claude Code) working under `userscript/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system.
Each entry names the code that holds the truth. The prose is only what the code ### Userscript structure (single IIFE, `manga-bookmark.user.js`)
cannot tell you: rationale, invariants a refactor would break, and dated
observations about sites we don't control.
### Structure — single IIFE, `manga-bookmark.user.js` 1. **Site adapters** — one per host, `detect(location, document)` return page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title` from **`og:title`** (or the page heading where a site ships no og: tags), not CSS classes. **No adapter reads a cover**: the backend acquires, stores and serves every Cover from its own origin (ADR-0007), the wire's `cover` is already an address on our origin, and `apiPut` strips any `cover` off an outgoing body.
2. **API client** — `apiGet/apiPut/apiDelete` with bearer header; `localStorage` key `bmgr:manga:cache` for instant render + offline fallback.
3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value.
4. **Retry queue** — every write go through `pushBookmark`/`pushDelete`, so
failed mutation park in `localStorage` (`bmgr:manga:queue`) and replayed on
next navigation, reconnect, or `refresh()`. Entries are markers
(`{key, op, sendStatus, attempts}`), never payloads — body read from
cache at send time, so one entry per key give ordering and coalescing for
free. `sendStatus` is **sticky**: while archive pending, later writes to
that key keep carrying bucket, which stop successful
in-between write from silently un-archiving series. `refresh()` drains
before it fetches and overlays anything still pending, so list never
flaps. 400 drops entry, 401 abort pass and keep queue, and
transient failures retry to cap of 10. Latest-chapter writes deliberately
stay out of queue. See
`docs/superpowers/specs/2026-07-27-offline-retry-queue-design.md`.
5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS
(critical on mobile). Three tabs (All / Favourites / Archived) and row of
link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG
block next to `API_BASE`. FAB is `7 × 44` edge tab whose *hit* area
widened to `28 × 72` by invisible `#hit` child; `#fab` must keep
`touch-action: none` and must **not** regain `overflow: hidden`. Since
`touch-action` resolved at gesture start, strip can't be both
browser-scrolled and script-dragged, so `makeDraggable` splits by intent: swipe
from `#hit` scrolls via `window.scrollBy`, hold of `ARM_MS` arms
reposition drag, visible sliver drags with no hold. See
`docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`.
6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fire without reload. Demonic uses classic reloads (initial `document-idle` run suffice).
Six parts, in file order: site adapters, API client, progress logic, retry ### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust)
queue, UI, SPA navigation.
**Site adapters** — one per host, `detect(location, document)` returning page - **asurascans.com**: series `/comics/<slug>` (slug carries trailing
`type` + IDs. site-wide build-hash suffix, e.g. `-059befe1`, that **rotates on every
redeploy**), chapter `/comics/<slug>/chapter/<n>`. `seriesId` must strip
- Identify type and IDs from **URL regex**, which is the most stable surface a hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in userscript,
site exposes; take `title` from **`og:title`** (or the page heading where a `asuraBuildHash` in backend); URLs keep full slug — stale-hash
site ships no og: tags), never CSS classes. URLs 302 to current ones. Astro-rendered; chapter links present in raw
- **No adapter reads a cover.** The backend acquires, stores and serves every server HTML.
Cover from its own origin, the wire `cover` is already an address there, and - **demonicscans.org**: series `/manga/<slug>` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title/<slug>/chapter/<n>/<page>` (older `chaptered.php?manga=<id>&chapter=<n>` form still exists as redirect, what series-page chapter-list anchors link through).
`apiPut` strips any `cover` off an outgoing body. Encodings (incl. triple-encoded punctuation like `%25252D`) identical
on /manga/ and /title/ pages, so decode-once seriesIds match — verified
**Progress logic** — auto-upsert `last_chapter` only when 2026-07-28.
`chapterNum >= stored last_chapter_num`; unparseable sets the current value. - **comix.to**: series `/title/<id>-<slug>`, chapter
Re-reading an old chapter must not regress progress. A manual panel override `/title/<id>-<slug>/<uploadId>-chapter-<n>`. Only the leading `<id>` is
forces any value. identity — the slug re-renders when a series is renamed (`comixSeriesId`).
An SPA that **never rewrites `og:title`**: the server-rendered head keeps
**Retry queue** — every write goes through `pushBookmark`/`pushDelete`. whatever document loaded first, so on a cold load `og:title` is the homepage's
"Comix — Read Comics online for free" and after an in-page hop it is the
- Entries are markers (`{key, op, sendStatus, attempts}`), **never payloads**: *previous* series' name. `document.title` is the one thing client routing does
the body is read from cache at send time, so one entry per key gives ordering update, so titles come from there, with the chapter page's `" · Ch.<n>"` tail
and coalescing for free. stripped. It publishes no `og:image` either, which is one of the reasons cover
- `sendStatus` is **sticky** — while an archive is pending, later writes to that
key keep carrying the bucket. Without it a successful in-between write
silently un-archives the series.
- `refresh()` drains before it fetches and overlays anything still pending, so
the list never flaps.
- 400 drops the entry, 401 aborts the pass and keeps the queue, transient
failures retry to a cap. Latest-chapter writes deliberately stay out of the
queue.
**UI** — rendered inside a **Shadow DOM** root to isolate it from site CSS,
which is critical on mobile.
- The FAB's *hit* area is widened by an invisible `#hit` child. `#fab` must keep
`touch-action: none` and must **not** regain `overflow: hidden`.
- `touch-action` is resolved at gesture start, so the strip cannot be both
browser-scrolled and script-dragged. `makeDraggable` therefore splits by
intent: a swipe from `#hit` scrolls via `window.scrollBy`, a hold of `ARM_MS`
arms a reposition drag, and the visible sliver drags with no hold.
**SPA navigation** — Asura is Astro and client-routes on comic/chapter pages, so
`history.pushState`/`replaceState` are patched and `popstate` listened to, and
`detect()` re-runs on URL change. Demonic uses classic reloads, where the
initial `document-idle` run suffices.
### Live URL shapes
Encoded in the adapters; the notes below are the parts a reader of the regex
would get wrong. **Verified 2026-07-26 unless dated otherwise — sites drift, so
re-check against a live page before trusting any of it.**
- **asurascans.com** — the series slug carries a site-wide build-hash suffix
(e.g. `-059befe1`) that **rotates on every redeploy**, so `seriesId` must
strip it (`stripBuildHash` here, `asuraBuildHash` in the backend) while URLs
keep the full slug — stale-hash URLs 302 to current ones.
- **demonicscans.org** — slugs may URL-encode punctuation, and the older
`chaptered.php?manga=<id>&chapter=<n>` form still exists as a redirect, which
is what series-page chapter-list anchors link through. Encodings (including
triple-encoded punctuation like `%25252D`) are identical on `/manga/` and
`/title/` pages, so decode-once seriesIds match (verified 2026-07-28).
- **comix.to** — only the leading `<id>` is identity; the slug re-renders when a
series is renamed (`comixSeriesId`). It is an SPA that **never rewrites
`og:title`**: the server-rendered head keeps whatever document loaded first,
so on a cold load `og:title` is the homepage's name and after an in-page hop
it is the *previous* series'. `document.title` is the one thing client routing
updates, hence titles come from there with the chapter page's `" · Ch.<n>"`
tail stripped. It publishes no `og:image` either, one of the reasons cover
acquisition moved to the backend. acquisition moved to the backend.
- **kagane.to** — reader URLs carry no chapter number, so the number comes out - **kagane.to**: series `/series/<uuid>`, reader
of `og:title`. Two shapes exist, `"<Series> - Chapter <n>[ - Episode <n>]"` `/series/<uuid>/reader/<bookUuid>`. Reader URLs carry no chapter number, so
and `"<Series> - Volume <v> Chapter <n>"`; both must yield a bare series the number comes out of `og:title`. Two shapes exist: `"<Series> - Chapter
title, or the volume tail lands in the bookmark's title. Its covers are <n>[ - Episode <n>]"` and, for volume-numbered series, `"<Series> - Volume <v>
challenge- and CORP-protected, so nothing outside kagane.to can load one — Chapter <n>"` with no episode name — both must yield a bare series title, or
the panel renders the backend's cover address like every other Site. the volume tail lands in the bookmark's title.
- **novelfull.com** (novel script) — no `og:*` tags at all, so the title comes Its covers are challenge- and CORP-protected, so nothing outside kagane.to can
from `h3.title` (series) or `a.truyen-title` (chapter). load one directly; the panel renders the backend's own cover address like every
- **lightnovelworld.net** (novel script) — chapter paths are flat at the site other Site. Behind a Cloudflare JS challenge, so the backend polls it
root and their slug is a **Chapter Slug, not an identity**: a Series may through the headless browser.
publish under several. The Series address is read off the page's - **novelfull.com** (novel script): series `/<slug>.html`, chapter
`/<slug>/chapter-<n>[-<title-slug>].html`. No `og:*` tags at all — title from
`h3.title` (series) or `a.truyen-title` (chapter); the script reads no cover.
Behind a Cloudflare JS challenge no TLS fingerprint
clears, so the backend polls it through the headless browser.
- **lightnovelworld.net** (novel script): series `/novel/<slug>/`, chapter
`/<slug>-chapter-<n>/` — flat, at the site root. The chapter path's slug is a
Chapter Slug, not an identity: the Series address is read off the page's
`a[aria-label='All Chapter']` (fallback: the BreadcrumbList's second crumb), `a[aria-label='All Chapter']` (fallback: the BreadcrumbList's second crumb),
and a chapter page with no pointer resolves to `other` so no Bookmark is and a Series may publish under several Chapter Slugs. A chapter page with no
offered. The client runs **no latest-chapter scan** for this Site — pointer resolves to `other`, so no Bookmark is offered. `h1.entry-title` is
`computeLatestChapter` yields null and `backgroundRefreshLatest` skips it the clean title on a series page and `<Title> Chapter <n>` on a chapter page.
before any fetch — because the backend Poll's one-hour cooldown dominates the Its series page lists every chapter with an
client's four-hour throttle, so a scan would add no freshness while having to absolute href, so the backend polls it with the plain TLS client.
truncate at the page's wpdiscuz thread, a public write surface. The client performs no latest-chapter scan for this Site: the Poll's
one-hour cooldown dominates the client's four-hour throttle, so a scan
would add no freshness, and the page's wpdiscuz thread is a public write
surface a scan would have to truncate at. `computeLatestChapter` yields
null here and `backgroundRefreshLatest` skips the Site before any fetch.
### Second script — `novel-bookmark.user.js` ### Second script: `novel-bookmark.user.js`
A copy of the manga script with two adapters, `LIBRARY = "novel"` and A copy of the manga script with two adapters, `LIBRARY = "novel"` and
`STORE_PREFIX = "bmgr:novel:"`. No migration loop, because this script has no `STORE_PREFIX = "bmgr:novel:"`. No migration loop (this script has no previous
previous installation to carry keys over from. Installed alongside the manga installation to carry keys over from). Installed alongside the manga script;
script; both write to the same backend, discriminated by `LIBRARY`. both write to the same backend with the same `LIBRARY` column discriminating
them.