Compare commits

..

8 Commits

Author SHA1 Message Date
sulthan 5d330c2ff4 docs: make every AGENTS.md cite code, not docs or issues
A spec, ADR, plan file, or Gitea issue records what was true when it was
written and then goes stale silently, so an agent that follows the pointer
reads a decision that may already have been reversed. Code is the only
source true at read time.

Strip every non-code citation from the three AGENTS.md files (ADRs, spec
and plan files, docs/research, docs/agents/*, DEPLOY/REDEPLOY, and issue
numbers), restating inline any fact the linked doc actually carried: the
tea command set and triage label strings move into the root Forge section.
The Domain docs subsection goes entirely, as it pointed only at CONTEXT.md
and docs/adr/, neither of which exists.

Then rewrite the backend and userscript files around derivability, since
prose that restates mechanism rots the same way a doc link does. Structure
and mechanism now name a symbol and stop; rationale, rejected alternatives
and dated measurements stay written out, because code cannot carry them.
Record that split as a rule in the root file.

Verified by extracting all 118 backticked identifiers and checking each
against the Go, JS, SQL, HTML and CSS sources. That caught one claim that
was already lying: the old cover text said CoverFetcher was gone, but
NewCoverFetcher, TLSCoverFetcher and BrowserCoverFetcher are all live in
internal/latest, so the sentence now names only the dead /img/kagane route.

Also drop the "Guidance for OpenCode (and Claude Code)" openers, so the
files read the same under any harness.
2026-08-17 13:39:28 +07:00
sulthan 550b258c59 fix: give covers their own 10 MiB byte cap (#71) (#112)
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-17 12:06:14 +07:00
sulthan 3ac865cd08 chore: remove graphify (#111)
Removes the graphify integration. It was measured against this repo rather than assumed.

## Why

`graphify query` returns a keyword-seeded BFS neighbourhood, not a location. Asked where CORS origin reflection is implemented, it returned 73 nodes — mostly `api_test.go` helpers, plus a `Reflection and Type Assertions` section from `.agents/skills/golang-performance/references/cpu.md` matched on the word "reflection" — and never named `httpmw/middleware.go:135` or `main.go:121`. `grep` returned both in 39ms. Same shape asking how the poller skips kagane: 145 nodes, top hits `poller_test.go` helpers and two nodes named `T`.

`graphify explain "BrowserFetcher"` is sound (`browser.go L52`, 9 `EXTRACTED` edges), but that is what `lsp references` already answers, against live files instead of a snapshot.

Staleness was never the problem — `graph.json` rebuilt 5s after `f568fb5`, so the git hooks worked. Retrieval quality was.

## What it cost

- Two `PreToolUse` hooks injecting a "MANDATORY: run graphify query first" paragraph into context on **every** grep/find and every source-file read.
- 685k input tokens across 5 build runs (`cost.json`).
- 3.4MB of `graph.json` + `graph.html` tracked, across 11 commits of map-refresh churn.

`AGENTS.md` is the stronger orientation artifact for a repo this size: it carries the CDP constraints, the UTC-clock finding, the per-site adapter list, and the security invariants — none of which an AST graph derives. Graphify earns its keep on repos too large to grep coherently and without curated docs; not this one.

## Changes

- Delete the committed map (`graphify-out/`, -58k lines).
- Drop the `## graphify` rules block from `AGENTS.md` (`CLAUDE.md` is a symlink, so both).
- Drop the five `graphify-out/*` entries from `.gitignore`.
- Empty the two `PreToolUse` hooks in `.claude/settings.json`.
- Remove the stale `graphify query` instruction from `.claude/skills/implement-tickets/SKILL.md` — it pointed dispatched ticket-implementer agents at a binary that no longer exists.

Uninstalled outside the tree (not in this diff): the `graphifyy` CLI, `~/.claude/skills/graphify/`, the global `~/.claude/CLAUDE.md` block, the `Bash(graphify query *)` permission in the git-ignored `.claude/settings.local.json`, and the `post-commit` / `post-checkout` git hooks.

## Verification

`grep -ri graphify` over the worktree is clean; remaining hits are inside `.git/` (commit messages, two stale branch configs). No code touched — backend and userscript are untouched, so `go test ./...` is unaffected.

Reviewed-on: #111
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-17 11:43:15 +07:00
sulthan f568fb5e8c fix: an asleep browser Lane is not a stalled one on the admin page (#110)
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-16 22:04:18 +07:00
sulthan 20fff588cc fix: don't read Cloudflare's injected jsd script as a refusal (#109)
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-16 21:17:29 +07:00
sulthan ba679223b2 Sightings: a Reader report defers a Poll of a solitary Series (#103) (#108)
Closes #103.

A userscript PUT already carries the Latest Chapter the Reader's own browser read off the Series page. It may now stand in for a Poll, under one restriction and one ceiling:

- **Solitary Series only** — a Series two Readers share is Polled on schedule however recently it was sighted, so one Reader's mistake can never reach another's list.
- **One rest of standing**, and a **six-rest ceiling** (`sightingCeilingRests`, counted in the Site's own Rest): however many Sightings arrive, an unpolled Series is Polled.

Both live in the due query's HAVING clause (`Store.DueForLatestCheck`) — the same place the schedule has always been decided, so no timer and no second code path can disagree with it. No new query per scheduler round.

Judgement costs no extra request. `Poller.checkOne` already compares what the Site publishes against what is stored: a lower number contradicts the Sighting (Reader and both numbers logged), the same number confirms it, a higher number is the Site publishing and clears the attribution instead. Three contradictions stop that Reader deferring — their reports still write the Latest Chapter — and twenty consecutive confirmations forgive them, as does the owner's clear-marks control from #102.

One client change was required: both userscripts skipped the PUT when the number had not moved, so the case the whole mechanism exists for — visiting a Series with nothing new — never reached the backend. `reportLatestChapter` sends it, skipping only the local write and the re-render. A numberless PUT (favourite toggle, progress from a chapter page) is no Sighting and defers nothing.

Schema: migration `0011_series_sightings.sql` adds `series.latest_sighted_at` and `series.latest_raised_by`. Trust model, thresholds, and rejected alternatives with their citations: `docs/adr/0011-sighting-deferral-trust-model.md`.

Reviewed on both axes (spec against #103, standards against the repo's rules); the blocker — attribution surviving a Poll that overtook the report — is fixed and has a test that fails without the fix.

Verification: `go test ./...` green (needs Docker), `node --test userscript/test/*.test.js` 66 pass.
Reviewed-on: #108
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-16 20:10:59 +07:00
sulthan 1e6f1e985d Owner-only admin page: Reader roster plus Poll Lane status (#102) (#107)
Closes #102.

The only operational surface was /healthz and a fold-out roster inside the owner's own reading page. This adds /admin: an owner-only page carrying the Reader roster and one row per Poll Lane.

- **Poller seam.** `latest.Poller` records each Lane's last pass (`Site`, `Due`, `Checked`, `LastRun`, `Gap`, `Clamped`, `Browser`) and answers `LaneStatus()`; the page reads that snapshot, never a table. A pass that returns before computing its figures (refusal backoff, sidecar down) carries the previous pass's figures forward rather than recording zeroes, and a Lane that has never reached a pace renders no gap at all. Refusal and sidecar reachability are derived at snapshot time.
- **Owner gate at registration.** Every route reaching past the acting Reader lives in `adminRoutes()` and is wrapped in `requireOwner` when it is registered, so a missing gate is visible in the route list rather than hidden in a handler. `web.AdminPatterns()` is what the gate test walks, so a new route cannot be added without being tested. A non-owner gets 404, never 403.
- **Nil poller is a first-class state.** `main.newRouter` takes the reporter as an interface and converts a nil `*Poller` to a nil interface; no poller and no completed pass both render "No data yet" with the reason spelled out, rather than confident zeroes.
- **Roster moved** off the reading page onto /admin, with the Sighting counters and a confirm-gated `Clear marks` control. #103 fills those counters, so on delivery they read zero for everyone - deliberate ordering.
- **One accent, `--patina`** (verdigris, both colour branches): the far side of the wheel from ember's crimson and clear of the archive blue. Ember still means new chapter only; revocation still wears --danger.

Verification: `go vet ./...` and `go test ./...` green (Docker-backed); admin page screenshotted at 1100px and 390px in both colour schemes. Reviewed on both axes (spec, standards); findings on the accent hue, zero-figure honesty and three tests that could not fail are fixed in 58014eb.
Reviewed-on: #107
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-16 15:02:13 +07:00
sulthan 3303a55b20 feat: one Poll Lane per Site, replacing the shared pace (#100) (#106)
Closes #100.

Each Site runs its own Poll Lane: an independent goroutine with its own rest
and pace from the registry (`backend/internal/latest/sites.go`), replacing the
shared cooldown/interval/stagger/batch configuration. Rest (1h, all six Sites
including the browser trio) is enforced by the due query's WHERE clause; the
Lane sleeps its effective gap between fetches — the registry 10s, or
rest/eligible when a Site holds enough Series, floored at 1s with a
Site-naming warning when the floor engages.

Lane-local failure handling:
- Two challenge-held results stop that Site's Lane for 15m; the probes keep
  their stamp, untried Series stay due.
- A lost browser sets a shared Poller flag: the other browser Lanes skip
  their passes for the same 15m (no stamp-per-pass-per-Lane on a dead tab),
  then decay and probe again.
- Browser wake gate preserved (5 due, or one waiting 15m, ADR-0005); one tab
  shared by the three browser Sites; "browser lane behind by X" logged every
  pass.
- Cover work (healing a stored source URL and filling a blank from the series
  page) runs in the background so a slow CDN cannot consume a Lane's gap.

Removed: `LATEST_CHAPTER_POLL_{COOLDOWN,BROWSER_COOLDOWN,INTERVAL,BATCH,STAGGER}`
and the 6h browser rest. Only `LATEST_CHAPTER_POLL_ENABLED` remains; DEPLOY.md
documents the exact `.env` edit. ADR-0010 records the decisions.

Reviewed-on: #106
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-16 13:16:19 +07:00
43 changed files with 2890 additions and 55671 deletions
+1 -20
View File
@@ -1,24 +1,5 @@
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "CMD=$(python3 -c \"import json,sys; d=json.load(sys.stdin); print(d.get('tool_input',d).get('command',''))\" 2>/dev/null || true); case \"$CMD\" in *grep*|*rg\\ *|*ripgrep*|*find\\ *|*fd\\ *|*ack\\ *|*ag\\ *) [ -f graphify-out/graph.json ] && echo '{\"hookSpecificOutput\":{\"hookEventName\":\"PreToolUse\",\"additionalContext\":\"MANDATORY: graphify-out/graph.json exists. You MUST run `graphify query \\\"<question>\\\"` before grepping raw files. Only grep after graphify has oriented you, or to modify/debug specific lines.\"}}' || true ;; esac"
}
]
},
{
"matcher": "Read|Glob",
"hooks": [
{
"type": "command",
"command": "HIT=$(python3 -c \"import json,sys;d=json.load(sys.stdin);t=d.get('tool_input',d);exts=('.py','.js','.ts','.tsx','.jsx','.astro','.vue','.svelte','.go','.rs','.java','.rb','.c','.h','.cpp','.hpp','.cc','.cs','.kt','.swift','.php','.scala','.lua','.sh','.md','.rst','.txt','.mdx');vals=[str(t.get('file_path') or ''),str(t.get('pattern') or ''),str(t.get('path') or '')];j=' '.join(vals).lower().replace(chr(92),'/');tails=[('.'+x.rsplit('.',1)[-1]) for v in vals if v for x in [v.lower().replace(chr(92),'/').rsplit('/',1)[-1]] if '.' in x];sys.stdout.write('1' if 'graphify-out/' not in j and any(tl in exts for tl in tails) else '')\" 2>/dev/null || true); if [ \"$HIT\" = 1 ] && [ -f graphify-out/graph.json ]; then echo '{\"hookSpecificOutput\":{\"hookEventName\":\"PreToolUse\",\"additionalContext\":\"MANDATORY: graphify-out/graph.json exists. You MUST run graphify before reading source files. Use: `graphify query \\\"<question>\\\"` (scoped subgraph), `graphify explain \\\"<concept>\\\"`, or `graphify path \\\"<A>\\\" \\\"<B>\\\"`. Only read raw files after graphify has oriented you, or to modify/debug specific lines. This rule applies to subagents too \u2014 include it in every subagent prompt involving code exploration.\"}}'; fi || true"
}
]
}
]
"PreToolUse": []
}
}
+1 -2
View File
@@ -11,8 +11,7 @@ to the tracker. You do not write the implementation — every line of ticket cod
is written by a `ticket-implementer` subagent in its own git worktree. Reach for
the editor yourself only for a merge conflict resolution.
Ticket source and `tea` usage: `docs/agents/issue-tracker.md`. Codebase
questions: `graphify query "<question>"` before grepping.
Ticket source and `tea` usage: `docs/agents/issue-tracker.md`.
## 1. Collect the tickets
-5
View File
@@ -5,11 +5,6 @@
backend/server
backend/backend
.playwright-mcp/
# graphify map is committed; only regenerable/local parts are ignored
graphify-out/cost.json
graphify-out/cache/
graphify-out/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]/
graphify-out/.rebuild.lock
plans/
.scratch/
docs/superpowers/
+75 -50
View File
@@ -1,6 +1,6 @@
# AGENTS.md
Guidance for OpenCode (and Claude Code) working in this repo.
Repo-wide guidance for coding agents.
## What this is
@@ -13,15 +13,18 @@ One backend, one `bookmarks` table: a `kind` column (`manga`|`novel`) splits the
## Hard constraints (drive design — don't violate)
Nothing below is derivable from reading the code — it is why the code looks the
way it does, plus dated measurements against services we don't control.
Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free where plain web APIs suffice — keeps portability across engines:
- **Avoid `GM_*` unless needed.** Prefer page `localStorage` over `GM_setValue`/`GM_getValue`, on-page UI over `GM_registerMenuCommand`, plain `fetch()` over `GM_xmlhttpRequest` for cross-origin.
- Cross-origin `fetch()` work **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block).
- Every site is its **own origin with its own `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync required, not optional.
- Userscript run in **isolated world**, so embedded API token safe from site's JS.
- Cloudflare's block on manga sites is **per-zone configuration plus request fingerprint, not IP reputation — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — a Site can turn its protection on overnight, which is exactly what comix.to did on 2026-08-12. An earlier version of this line blamed "Cloudflare's bot scoring"; that was wrong. The 1-99 bot score is Enterprise Bot Management only and does not exist for a free-plan zone, and no per-IP request rate is documented as an input to challenge issuance — `docs/research/cloudflare-bot-scoring-and-poll-cadence.md`. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test.
- **kagane.to, comix.to and novelfull.com are the exception to the above** — all three sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`). When that's unset, kagane and comix are skipped entirely (a plain fetch would only retrieve a challenge page) while novelfull pages are still attempted over plain TLS — its challenge is a live time-varying fact and its cover bytes never need the browser. comix turned hostile on 2026-08-12 (#98): its cover host `static.comix.to` is gated too, so its cover bytes go through the browser as well, and its page is read as an in-tab `fetch()` of the series URL rather than a rendered DOM — comix is an SPA, and rendering costs ~65 requests for the same server-rendered HTML one fetch returns. The three other sites poll fine over plain TLS.
- Cloudflare's block on manga sites is **per-zone configuration plus request fingerprint, not IP reputation — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — a Site can turn its protection on overnight, which is exactly what comix.to did on 2026-08-12. An earlier version of this line blamed "Cloudflare's bot scoring"; that was wrong. The 1-99 bot score is Enterprise Bot Management only and does not exist for a free-plan zone, and no per-IP request rate is documented as an input to challenge issuance. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test.
- **kagane.to, comix.to and novelfull.com are the exception to the above** — all three sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`). When that's unset, kagane and comix are skipped entirely (a plain fetch would only retrieve a challenge page) while novelfull pages are still attempted over plain TLS — its challenge is a live time-varying fact and its cover bytes never need the browser. comix turned hostile on 2026-08-12: its cover host `static.comix.to` is gated too, so its cover bytes go through the browser as well, and its page is read as an in-tab `fetch()` of the series URL rather than a rendered DOM — comix is an SPA, and rendering costs ~65 requests for the same server-rendered HTML one fetch returns. The three other sites poll fine over plain TLS.
- **The CDP browser must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a non-UTC clock zone cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, ignoring the file's contents, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead.
- **The browser is not in the API stack and must not be put back.** It's its own compose unit (`chrome/docker-compose.yml`) on a second machine, reached over the tailnet — it held 471 MiB on a 1974 MiB swapless VPS, and a residential egress avoids the cloud-hosting-IP signature Bot Fight Mode documentedly challenges (ADR-0006; not a better "score" — free-plan zones have no score). Consequences that constrain code: `BROWSER_WS_URL` must be a tailnet **IP** (a MagicDNS name 500s at `/json/version`, same trap as the old Docker service name); the CDP port binds to the tailnet address only, since CDP authenticates nothing and that host has a real LAN; and the browser is on-demand (ADR-0005), so an unreachable or asleep one must degrade exactly as an unset `BROWSER_WS_URL` — plain-TLS libraries unaffected, kagane/comix logged and skipped, stored covers still served. Never add `chromedp.NoModifyURL`: discovery per fetch is what makes a restarted Chrome invisible.
- **The browser is not in the API stack and must not be put back.** It's its own compose unit (`chrome/docker-compose.yml`) on a second machine, reached over the tailnet — it held 471 MiB on a 1974 MiB swapless VPS, and a residential egress avoids the cloud-hosting-IP signature Bot Fight Mode documentedly challenges (not a better "score" — free-plan zones have no score). Consequences that constrain code: `BROWSER_WS_URL` must be a tailnet **IP** (a MagicDNS name 500s at `/json/version`, same trap as the old Docker service name); the CDP port binds to the tailnet address only, since CDP authenticates nothing and that host has a real LAN; and the browser is on-demand, so an unreachable or asleep one must degrade exactly as an unset `BROWSER_WS_URL` — plain-TLS libraries unaffected, kagane/comix logged and skipped, stored covers still served. Never add `chromedp.NoModifyURL`: discovery per fetch is what makes a restarted Chrome invisible.
- **UTC is the tell, not a country mismatch.** A UTC clock is the datacenter default, and the challenge refuses it; any real zone clears. Measured 2026-08-08, identical container, one Indonesian egress IP: UTC never cleared in 60s (twice), while `Asia/Jakarta` **and** `America/New_York` both cleared in 4s. An earlier note here claimed the zone had to match the egress IP's country — that was wrong, inferred from the host clock (`Asia/Bangkok`) rather than the measured egress. A second earlier claim, that Cloudflare "scores" a UTC clock, was also wrong: the measurement is real but the mechanism is not documented anywhere — Cloudflare publishes no timezone signal, and free-plan zones carry no score at all. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one.
- **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives.
@@ -36,7 +39,7 @@ Two Violentmonkey userscripts (isolated world, per-site adapters, localStorage c
on-demand Chrome, separate machine (chrome/)
```
Two deployable units on two machines: the API stack (`docker-compose.yml` + `docker-compose.prod.yml`, on the VPS) and the browser (`chrome/docker-compose.yml`, on the home machine). They share nothing but `BROWSER_WS_URL` and update independently. Backend-specific architecture (packages, endpoints, poller, config env vars) lives in `backend/AGENTS.md`. Userscript-specific structure (adapters, retry queue, UI, live URL shapes) lives in `userscript/AGENTS.md`. Deploy order `DEPLOY.md` (§7 for the browser), redeploy `REDEPLOY.md` (§8 for the browser).
Two deployable units on two machines: the API stack (`docker-compose.yml` + `docker-compose.prod.yml`, on the VPS) and the browser (`chrome/docker-compose.yml`, on the home machine). They share nothing but `BROWSER_WS_URL` and update independently. Backend-specific detail lives in `backend/AGENTS.md`, userscript-specific detail in `userscript/AGENTS.md`.
## Commands
@@ -64,28 +67,29 @@ Smoke test: `curl` endpoints with `Authorization: Bearer <token>`; confirm `OPTI
## Forge: Gitea, not GitHub
`origin` is self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` don't work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones:
`origin` is self-hosted Gitea instance (`gitea.violetcrown.my.id`, repo `sulthan/mangaBookmark`), so **`gh` don't work here — use `tea` (Gitea CLI) for anything past plain git.** `tea` infers the repo from `origin`; auth lives in `tea login`, not a `GH_TOKEN` env var. It prints rendered boxes rather than plain text, so pass `-o json` when parsing; a PR URL lands on the last line.
- Open PR: `tea pr create --head <branch> --base main --title "..." --description "..."`
- List / view / check out: `tea pr list`, `tea pr <n>`, `tea pr checkout <n>`
- Issues: `tea issue create`, `tea issue list`
- Auth lives in `tea login`, not `GH_TOKEN` env var.
`tea` print output as rendered boxes rather than plain text; PR URL lands on last line.
- PRs: `tea pr create --head <branch> --base main --title "..." --description "..."`, `tea pr list`, `tea pr <n>`, `tea pr checkout <n>`.
- Issues: `tea issue create --title "..." --description "..."` (`--labels`, `--assignees` optional), `tea issue <n> --comments`, `tea issue list --state open|closed|all -o json`, `tea issue close <n>`.
- Comments: `tea comment <n> "..."` — `tea issue close` takes no `--comment` flag.
- Labels: `tea issue edit <n> --add-labels "..."` / `--remove-labels "..."`. Gitea will **not** auto-create a label, so `tea labels create --name "..." --color "#rrggbb"` first.
- Triage vocabulary is `needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`.
- Gitea shares one index space across issues and PRs, so a bare `#42` may be either — try `tea pr 42`, fall back to `tea issue 42`.
## Design system
Web UI + userscript panel follow **Cinder**, rules in `docs/design-system.md`
— source of truth Claude Design project `BookmarkManager Web UI`
(`969ac210-fe02-4c01-ae1b-9a271dcc779a`). Read it before touching
`backend/internal/web/static/style.css`, `backend/internal/web/templates/*`, or userscript
`TEMPLATE`/`CSS`. Core law: **ember means new chapter only** — no other
state (busy, error, destruction) may use `--ember`; destruction gets
`--danger`. No cards/corners/shadows, one `--measure: 760px` column, tokens
only (never hardcode hex outside `:root`), both colour branches touched
together. Any move that pulls series out of list (archive/finish/remove)
must be confirm-gated via its own `.confirm-row`; only restore fires
instantly.
Web UI + userscript panel follow **Cinder**. Tokens are the `:root` block in
`backend/internal/web/static/style.css`; that file, `backend/internal/web/templates/*`,
and the userscript `TEMPLATE`/`CSS` are the only places it is expressed.
Source of truth for the visual language is the Claude Design project
`BookmarkManager Web UI` (`969ac210-fe02-4c01-ae1b-9a271dcc779a`).
Core law: **ember means new chapter only** — no other state (busy, error,
destruction) may use `--ember`; destruction gets `--danger`. No
cards/corners/shadows, one `--measure: 760px` column, tokens only (never
hardcode hex outside `:root`), both colour branches touched together. Any move
that pulls a series out of the list (archive/finish/remove) must be
confirm-gated via its own `.confirm-row`; only restore fires instantly.
## Security invariants
@@ -127,13 +131,19 @@ Review gate: auth, CORS, session, crypto, and the fetch gate are security-critic
## Comments
Comment only if code alone can't carry info. Cost per read — must earn spot.
Wrong comment worse than none: it misleads readers and measurably degrades
LLM performance on the file. Missing comment costs little. Bias to fewer.
Write for:
- Why not what. Tradeoffs, non-obvious decisions.
- Load-bearing detail looking incidental — say so if "simplify" breaks it.
Docstring on public/exported surface — exception, near-always worth it.
Contract only: what it takes, returns, throws, mutates; units; pre/post
conditions. Not a restatement of the body. Skip on private/obvious.
Inline — write for:
- Why not what. Tradeoffs, non-obvious decisions, rejected alternatives.
- heavy detail looking incidental — say so if "simplify" breaks it.
- Non-local consequence, invisible from function alone.
- Wire format / encoding / interface contract — save callers re-deriving.
- Gotcha/workaround, with ref if exists.
- Wire format / encoding / ordering / invariant — save callers re-deriving.
- Gotcha/workaround, with ref (issue, RFC, vendor bug) if exists.
- Domain/business rule not derivable from code.
Skip:
@@ -142,34 +152,49 @@ Skip:
- Banners, dividers, `// helpers`.
- Change narration (`// fix bug`, `// as requested`, `// new impl`) — git's job.
- Commented-out code — delete.
- TODO without concrete action.
- TODO without concrete action + owner.
- Narrating the plan you just reasoned through. Plan in prose or in your head;
ship the code, not the transcript.
- Anything restating a name that could be fixed by renaming instead.
Style: one dense comment over function beats one per line inside. Tight, no worked example unless bug subtle. Wrong comment worse than none — update/delete on change. Default fewer — sparse+high-signal beats comprehensive.
Staleness filter: if the comment describes something likely to change
independently of this line, it will rot and start lying. Either anchor it to
something stable, assert it in a test, or leave it out.
Test: "competent reader get this from code in few sec?" Yes → skip. Needs detour through another file/spec/git-blame → write it.
Style: one dense comment over a function beats one per line inside. Tight; no
worked example unless the bug is subtle. On edit, update or delete stale
comments in the code you touch — silence beats a lie.
## Agent skills
Test: "competent reader get this from code in a few sec?" Yes → skip.
Needs detour through another file/spec/git-blame/external doc → write it.
`AGENTS.md` is the single source of truth for agent guidance; every `CLAUDE.md` in this repo is a symlink to the `AGENTS.md` beside it. Edit `AGENTS.md`.
## Writing an AGENTS.md
### Issue tracker
`AGENTS.md` is the single source of truth for agent guidance; every `CLAUDE.md`
in this repo is a symlink to the `AGENTS.md` beside it. Edit `AGENTS.md`.
Issues live as Gitea issues on `gitea.violetcrown.my.id` (`sulthan/mangaBookmark`), driven by the `tea` CLI — not `gh`. See `docs/agents/issue-tracker.md`.
**Cite code, never docs, issues, or plans.** A spec, ADR, plan file, or Gitea
issue records what was true when it was written and then goes stale silently;
an agent that follows the pointer reads a decision that may already have been
reversed. Code is the only source true at read time — cite a package, file,
symbol, env var, or route. The sole non-code exception is a sibling
`AGENTS.md`. If a doc holds a fact an agent needs, restate the fact here rather
than linking to it.
### Triage labels
**State a fact in prose only if the code cannot answer it.** Split by
derivability:
Default five-role vocabulary, label strings unchanged (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`). See `docs/agents/triage-labels.md`.
- *Structure* — packages, routes, env vars, columns, struct fields. Rots fast,
cheap to re-read. **Name the symbol, write nothing else.**
- *Mechanism* — what a function does, how a flow proceeds. **Name the symbol
plus at most one line of orientation.**
- *Rationale* — why it is this way, what a "simplify" would break, what was
tried and rejected. Not in the code and cannot be re-derived. **Write it out.**
- *Measurement* — an observation against something we don't control. **Write it
out with the date**; a dated fact is honest, an undated one pretends to be
permanent.
### Domain docs
Single-context: one root `CONTEXT.md` plus `docs/adr/`, both created lazily. See `docs/agents/domain.md`.
## graphify
Project has knowledge graph at graphify-out/ with god nodes, community structure, cross-file relationships.
Rules:
- For codebase questions and exploration, always first run `graphify query "<question>"` when graphify-out/graph.json exists. Use `graphify path "<A>" "<B>"` for relationships and `graphify explain "<concept>"` for focused concepts. Return scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output.
- If graphify-out/wiki/index.md exists, use for broad navigation instead of raw source browsing.
- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain don't surface enough context.
- After modifying code, run `graphify update .` to keep graph current (AST-only, no API cost).
Restating mechanism in prose is how these files rot: the code changes, the
paragraph doesn't, and the next agent trusts the paragraph. A pointer degrades
more honestly — and every symbol you name must actually exist, since a dead
pointer is a bug, not a stale sentence.
+266 -214
View File
@@ -1,216 +1,268 @@
Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system.
Scope: `backend/`.
- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + Postgres over `jackc/pgx/v5` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`.
Single binary, split into packages under `backend/internal/`: `store`
(Bookmark type, Postgres persistence, migration runner), `latest` (background
poller, site parsers, TLS fetcher), `session` (cookie signing, login
rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON
bookmark handlers), `userscript` (userscript-serving handler), `web`
(browser UI handler + `templates/` + `static/`, `go:embed`-ed).
`backend/main.go` is the composition root — the only place that wires
packages together into `newRouter`. Root-level `*_test.go` hold
integration tests that exercise the full router; unit tests for a
package live beside it under `internal/`.
- **Schema is migration-owned.** `internal/store/migrations/*.sql` is
`go:embed`-ed and applied on every start by `store.migrate`: one numbered
file per change, one transaction each, versions recorded in
`schema_migrations`. Files are **append-only** — editing an applied one
changes nothing on a database that already ran it. No column probing, no
data-fixup migrations: both were SQLite-era machinery and are gone.
- **Tests need Docker.** `internal/pgtest` starts one `postgres:17-alpine`
container per test binary (`TestMain` -> `pgtest.Main`) and hands each test
its own database (`pgtest.URL(t)`). A package whose tests touch the store
must have that `TestMain`.
- **Reader-owned store, four tables.** `readers` is keyed by Discord user ID
and carries the SHA-256 of the Reader's userscript credential plus a
`token_epoch` (issue #24). Credentials are derived, never stored: `token.Token(TOKEN_KEY, discord_id, epoch)` (HMAC, `internal/token`), and only its SHA-256 sits in `readers.token_sha256`, so install URLs can be rebuilt after any restart while a database leak yields nothing but hashes. The seed creates the **owner** row at startup; its epoch-0 hash is refreshed on every start **only while the row has never been rotated**, so a restart can never resurrect a rotated-away credential. Every other row is created by that Reader's own first login (`Store.EnsureReader`, idempotent on `discord_id`, and it never rewrites an existing row's hash). Rotation is `Store.RotateToken` (epoch bump + hash rewrite in one transaction), driven by the web UI.
`series` keyed `(site, series_id)`
(`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the
shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`),
Latest Chapter, `latest_checked_at` — and `bookmarks` holds only what
differs between readers: progress, favourite, lifecycle bucket,
`updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no
surrogate id; the wire `key` is derived as `site:series_id` on read — and
every store read/write is scoped to the reader it names. Auth resolves the
acting Reader from the presented credential (`httpmw.Auth`) and nothing
else — there is no unauthenticated-by-Reader route and no global token; the
reader id travels in the request context. Sync **last-write-wins**; the wire format
stays flat (ADR-0004). `Store.Upsert` decomposes one flat body across two
tables and enforces the ownership rule: client `title`/`series_url`/`cover`
are written only when the series row is new (ADR-0003).
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- **Web UI:** same binary serve the browser UI on a second
hostname — `GET /` (list, or login page when no session),
`GET /auth/discord` + `GET /auth/discord/callback` (Discord OAuth,
ADR-0002), `POST /logout`, `GET /static/*`, htmx fragment endpoints
under `/ui/*`. Templates + assets `go:embed`-ed under
`backend/internal/web/`, so `backend/Dockerfile` must copy the whole
`internal/` tree, not just `*.go`. Sessions are rows in the `sessions`
table: the cookie carries only an opaque id, looked up (and expiry-
checked) on every request, and deleting the row revokes the session.
Guild membership *is* registration (issue #27): `discordCallback` gates on
membership (and `DISCORD_REQUIRED_ROLE` when set) and then calls
`Store.EnsureReader`, so a refusal creates nothing and a returning Reader
reuses their row. The owner is the only Reader with administrative reach:
`POST /readers/{id}/revoke` (404 for anyone else) drops that Reader's
sessions, and the `readers` panel renders only on the owner's page.
A Reader with no bookmarks at all sees `listView.Fresh`, whose empty state
offers both install links instead of describing a filter.
UI mutations read-modify-write
through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one
place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`.
**Design-tool caveat:** templates link `/static/style.css` root-absolutely
(correct — served from `/`), but impeccable detector resolves
stylesheet href with `path.resolve(fileDir, href)`, drops directory
on leading `/` and silently skip file. Relative href don't help
either: template's directory isn't its served path. So
`detect.mjs backend/internal/web/templates` reports **false clean** —
always pass `backend/internal/web/static` too. One finding there,
`overused-font` on "Instrument Serif", deliberate identity choice, not debt.
- **Every action that moves series out of list is confirm-gated.**
Archive, finish, remove each open own `.confirm-row` disclosure
(`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈
`archive|finish|remove`); restore fire instantly since it's the reversal.
Remove's row wear ember wash, two reversible ones wear `.calm` grey.
`--ember` stay reserved for new-chapter signal: busy bar and inline
error use `--mute`.
- **Latest-chapter poller:** one goroutine per Site (a Poll Lane, issue #100),
each re-checking that Site's bookmarked series' newest published chapter from
backend's own network access, so `latest_chapter` stays fresh when the user
isn't browsing. Second, parallel signal — the userscript keeps its own
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
Two independent clocks: per-series rest (`series.latest_checked_at`,
enforced by `Store.DueForLatestCheck`'s WHERE clause — `now - Rest`) and
per-Lane gap (the Lane sleeping between fetches, `effectiveGap`). Both live
in the Site registry (`internal/latest/sites.go`), not config: the five env
knobs that used to size a shared pace are gone.
The poller walks **Series, not Bookmarks** — a series referenced by several
bookmarks is fetched once per cycle, and the due queue orders
`reader_count DESC, latest_checked_at ASC` (ADR-0003). Series row stamped
*before* fetch so broken series wait out the rest instead of retrying
every tick; found chapter written straight to the series row via
`Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
order — is never touched.
Refusals and browser loss are Lane-local: two `errChallengeHeld` in one pass
stop that Site for `refuseBackoff` (15m) while other Lanes continue; an
`errBrowserInterrupted` (remote Chrome restart) sets a shared Poller flag
that makes the other browser Lanes skip their passes for the same 15m, so a
restarting Chrome doesn't stamp one Series per Lane per pass — after the
window the flag decays and they probe again. Browser Lanes wake Chrome only
when 5+ Series are due or one has waited 15m (ADR-0005 on-demand browser),
and cover work (both healing a stored source URL and filling a blank from
the series page) runs in the background so a slow CDN can't consume a
Lane's gap.
Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth
against fingerprint-based blocking; any failure log and skip. kagane, comix
and novelfull sit behind Cloudflare JavaScript challenges the TLS client
can't clear, so they are fetched over CDP via `BROWSER_WS_URL`; kagane and
comix are simply not polled when that's unset, while novelfull falls back to
a plain-TLS attempt — its challenge is a live time-varying fact, and its
cover bytes never need the browser. comix's browser read is an in-tab
`fetch()` of the Series URL, not a DOM render: it is an SPA, so rendering
costs ~65 requests for the same server-rendered HTML one fetch returns
(measured 2026-08-12, issue #98). See
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
The poller's series write is a single-column UPDATE
(`Store.SetLatestChapter`), not a read-modify-write of the whole bookmark:
it cannot revert read progress or move `updated_at`, so the old
stale-re-read race is gone with the Get+Upsert flow.
- **Covers are acquired at creation, then served from our own origin
(ADR-0007):** the first Bookmark of a Series fires `Store.OnSeriesCreated`,
which `latest.Acquirer` turns into one series-page fetch yielding both the
Latest Chapter and the cover URL; the bytes then go through
`latest.CoverBytesFetcher` into `Store.SetSeriesCover`. It runs in a
goroutine — the Reader's PUT must neither block on a Site nor fail with one
— and every failure is logged and dropped, leaving the Bookmark intact. The
wire's `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once
bytes exist and `""` before, never an address that 404s. `GET /covers/{addr}`
is public and uncredentialed: the userscript renders it on a Site's origin,
where no cookie or token of ours travels. A client-sent `cover` is decoded
and discarded, permanently (ADR-0004 compatibility).
Browser-backed Sites join the same pipeline (issue #62, extended to comix by
#98): kagane and comix pages *and* cover bytes go through the browser sidecar
(nothing falls back to a plain fetch, which would only retrieve a challenge
page), while novelfull needs the browser only for its HTML — the cover URL
comes out of the browser-fetched page and the bytes go over plain TLS. With
no browser configured, kagane and comix Covers are simply absent; novelfull
still gets one — at creation and on the poll — when its page body happens to
answer a plain request (the challenge is a live time-varying fact). comix
cover bytes must arrive by direct navigation, not an in-page fetch: its
Series page sets `cross-origin-embedder-policy: require-corp`, which fails a
page-context fetch of `static.comix.to`. The old kagane-only
serving path (`/img/kagane/{id}`, template rewrite, `CoverFetcher`) is gone
(issue #63): the one public route serves every Site.
- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
`finished`, orthogonal to `favorite`. Archived and finished appear only in
own tab — not in All, Updated, Favourites, or recent strip. Poller keeps
checking archived series and skip finished ones. `finished` settable
only from web UI; `PUT /bookmarks/{key}` reject it with 400.
**Empty incoming status means "keep stored one"** — resolved on the
`VALUES` side of `Store.Upsert`, not conflict clause, since
`excluded.*` is post-evaluation row and default applied there would
wipe bucket on every PUT from client that predates column. See
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
- **Config via env:** `TOKEN_KEY` (derives every Reader's userscript credential;
required), `OWNER_DISCORD_ID` (seeds the owner Reader — the administrator and
the owner of every pre-registration bookmark; required),
`ALLOWED_ORIGINS` (comma list),
`DATABASE_URL` (Postgres connection URL, required — no default),
`COVER_DIR` (required filesystem volume for content-addressed Cover bytes),
`PUBLIC_BASE_URL` (required origin this deployment answers on, trailing
slash trimmed; every Cover URL on the wire is built from it, absolute
because the userscript renders on a Site's origin — ADR-0007),
`PORT` (default `8080`), `DISCORD_CLIENT_ID`/`_CLIENT_SECRET`/`_GUILD_ID`/
`_REDIRECT_URI` (required; Discord OAuth for the browser UI),
`DISCORD_REQUIRED_ROLE` (optional role gate, empty by default),
`DISCORD_API_BASE` (default `https://discord.com/api/v10`),
`LATEST_CHAPTER_POLL_ENABLED` (background latest-chapter poller kill
switch, default on). Pace is per Site in the registry (issue #100): every
Site rests an hour and gaps ten seconds, a Site with more eligible Series
than 360 tightens its own gap toward the 1s floor, and browser Lanes wake
Chrome only on demand (ADR-0005). The `_COOLDOWN`/`_BROWSER_COOLDOWN`/
`_INTERVAL`/`_BATCH`/`_STAGGER` knobs that used to size a shared pace are
gone. The 1h rest for browser Sites is safe on documented grounds: a
challenged page costs seconds of a serialized single-tab browser, free-plan
zones have no bot score and no published per-IP rate input, and
`cf_clearance` expires in 30 minutes so every cadence at or above 1h
re-solves anyway —
`docs/research/cloudflare-bot-scoring-and-poll-cadence.md`.
`USERSCRIPT_PATH` and `NOVEL_USERSCRIPT_PATH` (files served at
`/u/{token}/manga-bookmark.user.js` and `/u/{token}/novel-bookmark.user.js`,
defaults `/userscript/manga-bookmark.user.js` and
`/userscript/novel-bookmark.user.js`, both supplied by bindmount; the
`__API_TOKEN__` placeholder inside them is substituted with the requesting
Reader's credential at serve time).
`BROWSER_WS_URL` (CDP endpoint of the browser, which runs on a **separate
machine** and is reached over the tailnet — ADR-0006, `chrome/docker-compose.yml`.
Used by the poller for kagane, comix and novelfull page fetches and by the
cover pipeline for kagane's and comix's image bytes (the browser is the only
route that clears the challenge those two serve their covers behind); unset —
the default — disables browser polling and leaves kagane and comix Covers
blank until stored bytes
exist. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s
`/json/version` for any Host that isn't an IP or `localhost`).
- **No per-Site cover path (issue #63):** every Cover — all six Sites — is
served by the one public `GET /covers/{addr}` route from content-addressed
bytes. There is no proxy, no per-Site rewrite, no second place that decides
a Cover's renderable address: the wire `cover` is it. The only place a Site
name still appears in cover code is the extraction module (`latest`), where
kagane's and comix's image URLs are claimed by `browserOnlyCoverURL` — kagane
answers a plain fetch with a challenge and
Each entry names the code that holds the truth — read that for *what it does*.
The prose here is only what code cannot tell you: rationale, rejected
alternatives, dated measurements, and invariants a plausible refactor would
silently break.
### Layout
`backend/main.go` → `newRouter` is the composition root, the only place
packages are wired. Packages under `backend/internal/`: `store`, `latest`,
`session`, `httpmw`, `api`, `userscript`, `web`, `token`, `pgtest`. Root-level
`*_test.go` exercise the full router; unit tests live beside their package.
Not visible from any single file: stdlib `net/http` with no framework,
Postgres over `jackc/pgx/v5`, `CGO_ENABLED=0` static binary into a distroless
image, TLS terminated by the reverse proxy so the service listens plain `:8080`.
### Schema — `internal/store/migrations/*.sql`, run by `store.migrate`
- Migration files are **append-only**. Editing an applied one changes nothing
on a database that already recorded its version in `schema_migrations`, so
the fix silently applies to new deployments only.
- No column probing, no data-fixup migrations. Both were SQLite-era machinery
and were removed deliberately — don't reintroduce either.
### Tests need Docker — `internal/pgtest`
`pgtest.Main` from `TestMain` starts one `postgres:17-alpine` per test binary;
`pgtest.URL` hands each test its own database. A package whose tests touch the
store must have that `TestMain` or it has no database at all.
### Reader-owned store — `internal/store`, `internal/token`
Four tables; shape is in the migrations, behaviour in `Store`'s methods.
- **Credentials are derived, never stored.** `token.Token(TOKEN_KEY, discord_id, epoch)`
is an HMAC; only its SHA-256 reaches `readers.token_sha256`. So install URLs
can be rebuilt after any restart, and a database leak yields nothing usable.
- **The owner's epoch-0 hash is refreshed at startup only while the row has
never been rotated.** Drop that condition and a restart resurrects a
rotated-away credential.
- `Store.EnsureReader` never rewrites an existing row's hash — a returning
Reader's login must not invalidate their installed scripts.
- **Every read and write is scoped to the acting Reader**, resolved from the
presented credential by `httpmw.Auth` and carried in the request context.
There is no unauthenticated-by-Reader route and no global token.
- **`series` holds what readers share, `bookmarks` only what differs.** A
bookmark key is `(reader_id, site, series_id)` with no surrogate id; the wire
`key` is derived as `site:series_id` on read.
- `Store.Upsert` splits one flat body across both tables and enforces the
ownership rule: client `title`/`series_url`/`cover` are written **only when
the series row is new**, so one reader cannot retitle a shared series.
- Sync is last-write-wins and the wire format stays flat — clients depend on
both; neither is an implementation detail to tidy up.
### Web UI — `internal/web`
Routes, templates and assets are all in that package; `AdminPatterns()` and
`adminRoutes()` enumerate the privileged ones.
- **`backend/Dockerfile` must copy the whole `internal/` tree**, not just
`*.go`: templates and static assets are `go:embed`-ed from
`internal/web/`.
- **Guild membership *is* registration.** `discordCallback` gates on membership
(plus `DISCORD_REQUIRED_ROLE` when set) and only then calls
`Store.EnsureReader`, so a refusal creates nothing.
- Sessions are rows, not signatures: the cookie carries an opaque id and
expiry is checked on lookup, which is what makes deleting the row an instant
revocation.
- UI mutations go through `Store.Get` + `Store.Upsert` so the `updated_at` rule
below stays in exactly one place.
- `listView.Fresh` exists because a Reader with no bookmarks at all needs
install links, not an empty-filter message.
- **Design-tool caveat:** `detect.mjs backend/internal/web/templates` reports a
**false clean**. Templates link `/static/style.css` root-absolutely (correct —
it is served from `/`), but the detector resolves hrefs with
`path.resolve(fileDir, href)`, which drops the directory on a leading `/` and
skips the file silently; a relative href doesn't help either, since a
template's directory isn't its served path. Always pass
`backend/internal/web/static` too. The one finding there, `overused-font` on
"Instrument Serif", is a deliberate identity choice, not debt.
### Confirm gating — `internal/web/static/filter.js`, `toggleConfirmRow(key, kind)`
Every action that pulls a series out of the list (`archive|finish|remove`) opens
its own `.confirm-row`; restore fires instantly because it is the reversal.
Remove wears the ember wash, the two reversible ones wear `.calm` grey.
**`--ember` is reserved for the new-chapter signal** — the busy bar and inline
errors must use `--mute`, or the one colour that means "something to read"
stops meaning it.
### Latest-chapter poller — `internal/latest`, Site registry in `sites.go`
One goroutine per Site (a Poll Lane) re-checks that Site's bookmarked series
from the backend's own network position, so `latest_chapter` stays fresh while
nobody is browsing. The userscript's `reportLatestChapter` is a second,
parallel signal — it PUTs every read, unchanged numbers included, because an
unchanged read is exactly the Sighting worth deferring a Poll on.
- **Pace lives in the Site registry, not config.** Two clocks: per-series rest
(`series.latest_checked_at`, enforced in `Store.DueForLatestCheck`'s WHERE)
and per-Lane gap (`effectiveGap`). The five env knobs that used to size one
shared pace are gone; don't add them back.
- **The poller walks Series, not Bookmarks** — a series several readers hold is
fetched once per cycle, and the due queue orders `reader_count DESC,
latest_checked_at ASC` so the widely-read ones win contention.
- **The series row is stamped *before* the fetch**, so a permanently broken
series waits out its rest instead of being retried every tick.
- `Store.SetLatestChapter` is a single-column UPDATE, deliberately not a
read-modify-write of the bookmark: it therefore cannot revert read progress
or move `updated_at`. The old stale-re-read race died with the Get+Upsert
flow — don't restore one here.
**Sightings** (`Store.RecordSighting`, the due query's HAVING clause,
`latest.checkOne`) let a Reader's own page read defer a Poll.
- Recorded by the PUT handler **before** the Upsert, because the raise test
needs the row as it stands.
- A Series is deferred only while it has exactly one Bookmark, was sighted
within one Rest, and is under `sightingCeilingRests` since its last Poll — so
a shared Series is never deferred and nothing goes six hours unpolled
whatever arrives.
- A *higher* report clears the attribution rather than crediting it: the value
the Poll then stores is its own, so a later retraction isn't the Reader's
fault.
- `store.SightingDisagreementLimit` contradictions stop a Reader deferring —
their reports still write the Latest Chapter — and
`store.SightingAgreementsToClear` agreements forgive them, as does the
owner's clear-marks control.
- Deferral is recomputed from live facts each round, so nothing needs
invalidating when a Series gains a second Bookmark. The one input read
earlier is the Reader's marks, so crossing or clearing a threshold takes
effect from their next Sighting and the standing already bought lasts out its
rest.
**Refusals and browser loss are Lane-local.** Two `errChallengeHeld` in a pass
stop that Site for `refuseBackoff` while other Lanes continue. An
`errBrowserInterrupted` (remote Chrome restarted) sets a shared Poller flag so
the *other* browser Lanes skip their passes for the same window — otherwise a
restarting Chrome stamps one Series per Lane per pass, burning rests on
failures. The flag decays and they probe again.
- **`isInterstitial` matches the orchestration path
`/cdn-cgi/challenge-platform/h/`, never the bare prefix.** Cloudflare injects
`/cdn-cgi/challenge-platform/scripts/jsd/main.js` into ordinary 200 pages
once a zone turns JS detections on, which demonic did on 2026-08-16: the
prefix match read every real demonic page as a refusal and parked the Lane in
backoff while plain TLS was returning full series pages.
- Fetches use `bogdanfinn/tls-client` with a Chrome profile as defence in depth
against fingerprint blocking; any failure logs and skips.
- kagane, comix and novelfull sit behind Cloudflare JS challenges the TLS
client can't clear, so they go over CDP (`BROWSER_WS_URL`). kagane and comix
are simply not polled when it's unset — a plain fetch would only retrieve a
challenge page — while novelfull still attempts plain TLS, because its
challenge is a live time-varying fact and its cover bytes never need a browser.
- **comix's browser read is an in-tab `fetch()` of the Series URL, not a DOM
render.** It is an SPA: rendering cost ~65 requests for the same
server-rendered HTML one fetch returns (measured 2026-08-12).
- Browser Lanes wake Chrome only when 5+ Series are due or one has waited 15m,
and cover work runs in the background so a slow CDN can't eat a Lane's gap.
### Covers — `Store.OnSeriesCreated`, `latest.Acquirer`, `latest.CoverBytesFetcher`, `Store.SetSeriesCover`
Acquired once when the first Bookmark of a Series is created, then served from
our own origin by the public `GET /covers/{addr}`.
- Acquisition runs in a goroutine: the Reader's PUT must neither block on a
Site nor fail with one. Every failure is logged and dropped, leaving the
Bookmark intact.
- The wire `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once
bytes exist and `""` before — **never an address that 404s**. Absolute
because the userscript renders it on a Site's origin.
- `GET /covers/{addr}` is public and uncredentialed by design: no cookie or
token of ours may travel to a Site's origin.
- A client-sent `cover` is decoded and discarded, permanently — wire
compatibility, not an oversight.
- **One route serves all six Sites.** No proxy, no per-Site rewrite, no second
place that decides a renderable address: the wire `cover` is it. Templates
render `.Cover` and nothing else. The old kagane-only serving path
(`/img/kagane/{id}` plus a template rewrite) is gone; don't reintroduce a
per-Site route because one Site's CDN misbehaves.
- The only Site names left in cover code are in `browserOnlyCoverURL`
(`internal/latest`): kagane answers a plain fetch with a challenge *and*
`cross-origin-resource-policy: same-origin`, and `static.comix.to` answers
one with the same Cloudflare challenge its pages serve;
every other Site's CDN answers plain TLS. Templates render `.Cover` — the
wire value — never anything else.
- **Web UI also owns:** session-gated `GET /install/{manga,novel}-bookmark.user.js`
(renders the bindmounted script with the acting Reader's derived credential
substituted in — the credential never appears in page markup, the address
bar, or a redirect; `?download=1` adds `Content-Disposition: attachment` for
mobile Violentmonkey, which ignores a `.user.js` navigation) and
`POST /rotate-token` (atomic epoch bump + hash
rewrite; invalidates every installed copy, so the panel warns to reinstall
on all devices).
Owner-only `POST /readers/{id}/revoke` (drops one Reader's session rows and
re-renders the `readers` panel; 404 for any non-owner) is the only route that
reaches across Readers.
with the same challenge its pages serve. Every other Site's CDN answers plain
TLS.
- **comix cover bytes must arrive by direct navigation, not an in-page fetch:**
its Series page sets `cross-origin-embedder-policy: require-corp`, which
fails a page-context fetch of `static.comix.to`.
- With no browser configured, kagane and comix Covers are simply absent;
novelfull still gets one whenever its page answers a plain request.
### `updated_at` drives list order — `Store.Upsert`
The server applies its own timestamp only when the row is new or
`last_chapter_num` changes, else it keeps the stored value. **Favouriting a
series, or a newly published chapter arriving, must not reorder the list** —
only real reading progress moves a row. Consequently `PUT` returns the row **as
stored** and clients must adopt that response rather than their own payload.
### Lifecycle buckets — `status` on each bookmark
`reading` | `archived` | `finished`, orthogonal to `favorite`. Archived and
finished appear only in their own tab, never in All, Updated, Favourites or the
recent strip. The poller keeps checking archived series and skips finished ones.
- `finished` is settable only from the web UI; `PUT /bookmarks/{key}` rejects
it with 400.
- **An empty incoming status means "keep the stored one"**, and it is resolved
on the `VALUES` side of `Store.Upsert`, not in the conflict clause:
`excluded.*` is the post-evaluation row, so a default applied there would
wipe the bucket on every PUT from a client predating the column.
### Config — `Config` / `loadConfig` / `loadLatestPoll` in `backend/main.go`
That function is the complete list of env vars, their defaults, and which are
required. What it can't tell you:
- `PUBLIC_BASE_URL` must be an absolute origin because every Cover URL on the
wire is built from it and the userscript renders on a Site's origin.
- `BROWSER_WS_URL` **must be a tailnet IP, never a hostname** — Chrome's
DevTools handler 500s `/json/version` for any Host that isn't an IP or
`localhost`. Unset (the default) disables browser polling.
- `USERSCRIPT_PATH` / `NOVEL_USERSCRIPT_PATH` are bindmounted files; the
`__API_TOKEN__` placeholder inside them is substituted with the requesting
Reader's credential at serve time.
- Pace is per Site in the registry, not env. The
`_COOLDOWN`/`_BROWSER_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` knobs are
gone on purpose.
- The 1h rest for browser Sites is safe on documented grounds: a challenged
page costs seconds of a serialized single-tab browser, free-plan zones carry
no bot score and no published per-IP rate input, and `cf_clearance` expires
in 30 minutes, so every cadence at or above 1h re-solves anyway.
### Userscript install & rotation — `internal/userscript`, `internal/token`
Session-gated `GET /install/{manga,novel}-bookmark.user.js` renders the
bindmounted script with the acting Reader's derived credential substituted in,
so the credential never appears in page markup, the address bar, or a redirect.
`?download=1` adds `Content-Disposition: attachment` for mobile Violentmonkey,
which ignores a `.user.js` navigation. `POST /rotate-token` is an atomic epoch
bump plus hash rewrite and invalidates every installed copy — the panel must
keep warning to reinstall on all devices.
### Owner-only admin — `internal/web/admin.go`
- **Every route reaching past the acting Reader is listed in `adminRoutes()`
and wrapped in `requireOwner` at registration** — add it there, not as a
check inside a handler; `web.AdminPatterns()` is what the gate test walks. A
non-owner gets 404, never 403.
- The one owner comparison left outside the gate is in `index`
(`view.Owner = readerID == h.store.OwnerID()`): it gates a link, not an
endpoint, so it is a rendering decision a registration-time wrapper cannot
express. Do not "unify" it into the gate.
- Lane figures come through the `web.LaneReporter` seam
(`latest.Poller.LaneStatus`), never a table. `main.newRouter` takes the
reporter as an interface and converts a nil `*Poller` to a nil interface — a
typed nil would make the page claim a poller exists.
- A pass that returns before computing figures (refusal backoff, sidecar down)
carries the previous pass's numbers forward rather than recording zeroes.
- **`Checked` next to `Due` is what separates a stopped Lane from a quiet one**,
so neither may be dropped from the row.
- Due-without-Checked is **not** by itself a stall: a browser Lane under both
wake thresholds sets `LaneState.Asleep` and renders "browser asleep", and
never counts toward `Attention`. That is the commonest healthy state for
kagane, comix and novelfull, so spending the stall mark on it would train the
owner to ignore the mark that matters.
+44 -4
View File
@@ -48,7 +48,7 @@ func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
func newTestServer(t *testing.T) http.Handler {
t.Helper()
return newRouter(newTestStore(t), testConfig())
return newRouter(newTestStore(t), testConfig(), nil)
}
func newTestStore(t *testing.T) *store.Store {
@@ -599,7 +599,7 @@ func TestLoadConfigDiscord(t *testing.T) {
// cooldown and the poller would re-fetch that series on every single tick.
func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) {
s := newTestStore(t)
srv := newRouter(s, testConfig())
srv := newRouter(s, testConfig(), nil)
seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", 777)
@@ -621,6 +621,46 @@ func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) {
}
}
// Sighting deferral (issue #103) only reaches production through the PUT
// handler: the store and poller can be right and the feature still dead if the
// handler never records the report. Asserted where a client can see it - the
// series stops being due the moment the PUT lands.
func TestPutRecordsASighting(t *testing.T) {
s := newTestStore(t)
srv := newRouter(s, testConfig(), nil)
now := time.Now().UnixMilli()
hour := time.Hour.Milliseconds()
seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", now-2*hour)
due, err := s.DueForLatestCheck("asura", now-hour, now-6*hour)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
if len(due) != 1 {
t.Fatalf("due before the PUT = %d series, want 1", len(due))
}
body := `{"key":"asura:x","site":"asura","series_id":"x",
"series_url":"https://asurascans.com/comics/x",
"last_chapter":"Chapter 5","last_chapter_num":5,
"latest_chapter":"Chapter 9","latest_chapter_num":9}`
req := httptest.NewRequest(http.MethodPut, "/bookmarks/asura:x", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
srv.ServeHTTP(rec, auth(req))
if rec.Code != http.StatusOK {
t.Fatalf("PUT status = %d, want 200 (body %s)", rec.Code, rec.Body.String())
}
due, err = s.DueForLatestCheck("asura", now-hour, now-6*hour)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
if len(due) != 0 {
t.Fatalf("due after the PUT = %d series, want 0: the handler recorded no Sighting", len(due))
}
}
// The userscript route is registered outside the web UI's Discord auth, so it
// must keep working whatever the web config — see internal/userscript for the
// handler's own behaviour. The credential in the path is the owner's derived
@@ -637,7 +677,7 @@ func TestUserscriptServedWithWebUIDisabled(t *testing.T) {
rr := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/u/"+ownerCredential()+"/manga-bookmark.user.js", nil)
newRouter(s, cfg).ServeHTTP(rr, req)
newRouter(s, cfg, nil).ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rr.Code)
}
@@ -659,7 +699,7 @@ func TestNovelUserscriptServed(t *testing.T) {
cfg := testConfig()
cfg.NovelUserscriptPath = novelPath
srv := newRouter(s, cfg)
srv := newRouter(s, cfg, nil)
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet,
+1 -1
View File
@@ -98,7 +98,7 @@ func TestPublicCoverNeverEchoesNonImage(t *testing.T) {
if _, err := db.Exec(`UPDATE covers SET content_type = 'text/html' WHERE address = $1`, address); err != nil {
t.Fatalf("poison row: %v", err)
}
rr := getCover(t, newRouter(st, testConfig()), "/covers/"+address, nil)
rr := getCover(t, newRouter(st, testConfig(), nil), "/covers/"+address, nil)
if rr.Code == http.StatusOK {
t.Fatalf("status = 200, want a refusal for a non-image row (body %q)", rr.Body.String())
}
+12 -1
View File
@@ -99,7 +99,18 @@ func (h *Handler) Put(w http.ResponseWriter, r *http.Request) {
// reading progress actually moved. Any client value is ignored.
b.UpdatedAt = time.Now().UnixMilli()
stored, err := h.Store.Upsert(httpmw.ReaderID(r), b)
// A userscript PUT is a Sighting: the Reader's browser was on the Series
// page and read its Latest Chapter (issue #103). Recorded before the
// Upsert, which is what makes the raise comparison possible, and never
// from the web UI's own read-modify-write — a Reader toggling a favourite
// has not looked at the Site and must not postpone a Poll. A failure here
// costs a deferral, not the write, so it is logged and dropped.
readerID := httpmw.ReaderID(r)
if err := h.Store.RecordSighting(readerID, b.Site, b.SeriesID, b.LatestChapterNum, b.UpdatedAt); err != nil {
log.Printf("record sighting: %v", err)
}
stored, err := h.Store.Upsert(readerID, b)
if err != nil {
log.Printf("upsert: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
+10 -3
View File
@@ -250,11 +250,18 @@ func browserConnectionLost(ctx context.Context) bool {
const challengePollInterval = 2 * time.Second
// isInterstitial reports whether html is Cloudflare's challenge page rather
// than the site's own. Matched on the challenge runtime's script path, which is
// stable across the interstitial's wording and locale — the visible "Just a
// than the site's own. Matched on the challenge orchestration path
// (/cdn-cgi/challenge-platform/h/<b|g|x>/orchestrate/...), which is stable
// across the interstitial's wording and locale — the visible "Just a
// moment..." title is neither.
//
// The bare "/cdn-cgi/challenge-platform/" prefix is NOT enough: Cloudflare
// injects /cdn-cgi/challenge-platform/scripts/jsd/main.js into ordinary 200
// pages when JS detections are on, so matching the prefix declared every real
// demonic page a refusal and parked that Lane in 15m backoff (observed
// 2026-08-16, demonic turned detections on).
func isInterstitial(html string) bool {
return strings.Contains(html, "/cdn-cgi/challenge-platform/")
return strings.Contains(html, "/cdn-cgi/challenge-platform/h/")
}
// run navigates to target and re-reads until done reports an answer, bounded by
+19
View File
@@ -117,6 +117,25 @@ func TestBrowserOnlyCoverURL(t *testing.T) {
})
}
}
// The jsd script is injected into ordinary 200 pages when a zone turns JS
// detections on; only the orchestration path means the page itself is the
// challenge. Conflating the two parked the demonic Lane in refusal backoff
// while every fetch was in fact the real series page (observed 2026-08-16).
func TestIsInterstitial(t *testing.T) {
if !isInterstitial(challengeFixture) {
t.Fatal("challenge page not detected as interstitial")
}
const jsdInjected = `<html><head><title>The Possessed Grappler</title>
<script src="/cdn-cgi/challenge-platform/scripts/jsd/main.js"></script></head>
<body><a href="/chaptered.php?manga=13721&chapter=22">Chapter 22</a></body></html>`
if isInterstitial(jsdInjected) {
t.Fatal("real page carrying the injected jsd script misread as interstitial")
}
if got, ok := demonicLatestChapter("", jsdInjected); !ok || got.Label != "Chapter 22" {
t.Fatalf("demonicLatestChapter = %+v, ok = %v, want Chapter 22", got, ok)
}
}
func TestClassifyBrowserInterruption(t *testing.T) {
if err := classifyBrowserError(context.Background(), true, context.Canceled); !errors.Is(err, errBrowserInterrupted) {
t.Fatalf("classifyBrowserError(context.Canceled) = %v, want browser interruption", err)
+14 -5
View File
@@ -47,6 +47,15 @@ func fetchCoverBytes(ctx context.Context, cover string, browser BrowserCoverFetc
// inject it to exercise hostile DNS results without touching the live network.
type CoverResolver func(context.Context, string) ([]netip.Addr, error)
// maxCoverBytes caps one cover, separately from the series-page maxBodyBytes:
// a cover is a bounded binary asset, not a text page, and 4 MiB rejected 12%
// of asurascans covers measured 2026-08-17 (p90 4.52 MB, max 8.57 MB — two of
// the three over-cap files were JPEGs, not the animated GIF of issue #71).
// 10 MiB is ~18% headroom over that worst case and matches the GitHub and
// Discord image limits; see docs/research/gif-maximum-byte-size.md. GIF itself
// has no maximum size, so this number is policy, not format.
const maxCoverBytes = 10 << 20
// TLSCoverFetcher retrieves image bytes with the standard HTTPS client. Unlike
// TLSFetcher, it does not need a browser fingerprint: cover hosts are public
// CDNs and the response is accepted only after the destination gate passes.
@@ -152,15 +161,15 @@ func (f *TLSCoverFetcher) Fetch(ctx context.Context, sourceURL string) ([]byte,
if !ok {
return nil, "", fmt.Errorf("fetch cover: unsupported content type %q", raw)
}
if resp.ContentLength > maxBodyBytes {
return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxBodyBytes)
if resp.ContentLength > maxCoverBytes {
return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxCoverBytes)
}
body, err := io.ReadAll(io.LimitReader(resp.Body, maxBodyBytes+1))
body, err := io.ReadAll(io.LimitReader(resp.Body, maxCoverBytes+1))
if err != nil {
return nil, "", fmt.Errorf("read cover: %w", err)
}
if len(body) > maxBodyBytes {
return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxBodyBytes)
if len(body) > maxCoverBytes {
return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxCoverBytes)
}
return body, contentType, nil
}
+24 -1
View File
@@ -171,7 +171,7 @@ func TestCoverFetcherRejectsOversizedBody(t *testing.T) {
var calls int
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
calls++
response := coverResponse(http.StatusOK, "image/webp", "", bytes.Repeat([]byte("x"), maxBodyBytes+1))
response := coverResponse(http.StatusOK, "image/webp", "", bytes.Repeat([]byte("x"), maxCoverBytes+1))
response.ContentLength = -1
return response, nil
})}
@@ -187,6 +187,29 @@ func TestCoverFetcherRejectsOversizedBody(t *testing.T) {
}
}
// Covers between the series-page cap and the cover cap must be accepted: the
// 4 MiB page cap rejected 12% of asurascans covers (issue #71).
func TestCoverFetcherAcceptsCoverOverPageCap(t *testing.T) {
body := bytes.Repeat([]byte("x"), maxBodyBytes+1)
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
return coverResponse(http.StatusOK, "image/gif", "", body), nil
})}
fetcher := newCoverFetcher(client, func(context.Context, string) ([]netip.Addr, error) {
return []netip.Addr{netip.MustParseAddr("198.51.100.10")}, nil
})
got, contentType, err := fetcher.Fetch(context.Background(), "https://cdn.example/big.gif")
if err != nil {
t.Fatalf("Fetch rejected a %d-byte cover: %v", len(body), err)
}
if len(got) != len(body) {
t.Fatalf("body = %d bytes, want %d", len(got), len(body))
}
if contentType != "image/gif" {
t.Fatalf("content type = %q, want image/gif", contentType)
}
}
func TestCoverFetcherRejectsNonImage(t *testing.T) {
var calls int
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
+63 -6
View File
@@ -57,6 +57,10 @@ type Poller struct {
mu sync.Mutex
refuseUntil map[string]time.Time
browserDownAt time.Time
// laneStates is the owner's page snapshot of each Lane's last pass
// (issue #102), keyed by Site. Guarded by mu; a Site appears only after
// its first pass, so a restart renders "no data yet" rather than zeroes.
laneStates map[string]LaneState
// coverWG tracks in-flight cover work. Covers heal in the background so a
// slow cover host cannot delay the next Series-page Poll; tests join it
// before asserting on cover fetches.
@@ -218,6 +222,10 @@ func (p *Poller) runOnce(ctx context.Context) {
// production Lane's rate limit; the deterministic test entry runs back to back.
func (p *Poller) runLanePass(ctx context.Context, name string, paced bool) time.Duration {
now := p.Now()
// Snapshot this pass for the owner's page (issue #102). Recorded on every
// return path, with the figures filled in where the pass computes them.
st := LaneState{Site: name, LastRun: now, Browser: isBrowserSite(name)}
defer func() { p.recordLaneState(st) }()
if until := p.refusalBackoff(name); now.Before(until) {
// Cooling down after a refusal: do not attempt this Site at all.
return until.Sub(now)
@@ -238,18 +246,24 @@ func (p *Poller) runLanePass(ctx context.Context, name string, paced bool) time.
// No fetcher at all right now (browser absent, no fallback): every
// Series stays unstamped and due, so a browser that appears after a
// restart finds its full queue waiting (issue #100).
st.Gap = defaultGap
return defaultGap
}
due, err := p.Store.DueForLatestCheck(name, now.Add(-s.Rest).UnixMilli())
due, err := p.Store.DueForLatestCheck(name, now.Add(-s.Rest).UnixMilli(),
now.Add(-sightingCeilingRests*s.Rest).UnixMilli())
if err != nil {
log.Printf("latest poll %s: due query: %v", name, err)
st.Gap = defaultGap
return defaultGap
}
st.Due = len(due)
if s.Browser != nil && f == p.BrowserFetch && !browserWakeDue(due, now, s.Rest) {
// Below both thresholds Chrome stays asleep (ADR-0005 on-demand
// browser): waking it for a single Poll would cost a challenge solve
// per request.
// per request. The Lane still paces at the default gap, which is what
// the owner's page must show rather than a zero.
st.Gap, st.Asleep = defaultGap, true
return defaultGap
}
if s.Browser != nil {
@@ -265,9 +279,11 @@ func (p *Poller) runLanePass(ctx context.Context, name string, paced bool) time.
eligible, err := p.Store.EligibleSeriesCount(name)
if err != nil {
log.Printf("latest poll %s: eligible count: %v", name, err)
st.Gap = defaultGap
return defaultGap
}
gap, clamped := effectiveGap(s, eligible)
st.Gap, st.Clamped = gap, clamped
if clamped {
log.Printf("latest poll %s: gap clamped to %s floor (eligible series=%d)", name, minGap, eligible)
}
@@ -278,7 +294,6 @@ func (p *Poller) runLanePass(ctx context.Context, name string, paced bool) time.
}
refusals := 0
checked := 0
for i, sr := range due {
if ctx.Err() != nil {
break
@@ -313,10 +328,10 @@ func (p *Poller) runLanePass(ctx context.Context, name string, paced bool) time.
} else {
refusals = 0
}
checked++
st.Checked++
}
if checked > 0 {
log.Printf("latest poll %s: due=%d checked=%d", name, len(due), checked)
if st.Checked > 0 {
log.Printf("latest poll %s: due=%d checked=%d", name, len(due), st.Checked)
}
if refusals >= 2 {
p.setRefusalBackoff(name, now.Add(refuseBackoff))
@@ -447,6 +462,11 @@ func (p *Poller) checkOne(ctx context.Context, sr store.Series) error {
return nil
}
// The Poll is the oracle for whatever Sighting last raised this Series
// (issue #103), and the judgement is free: the comparison below already
// exists, and no extra request is made to reach it.
p.judgeSighting(sr, facts.Latest.Num)
// Equality, not >, mirroring the userscript (L427): a site that retracts a
// chapter should correct the stored number downward. The comparison is
// against the due-query snapshot; a concurrent write in between only costs
@@ -467,6 +487,43 @@ func (p *Poller) checkOne(ctx context.Context, sr store.Series) error {
return nil
}
// judgeSighting settles the Sighting the Series' stored Latest Chapter is owed
// to, if any, against what the Site actually publishes. The asymmetry is the
// whole of the detection rule and is what keeps it free of false alarms: a Poll
// finding a *lower* number than stored means the Reader who raised it reported
// a chapter that does not exist, while a Poll finding a higher one is only the
// Site publishing since and means nothing about the report. Equality confirms
// the report, which is how an honest Reader earns back a mark.
//
// A Series with no attribution — the stored value is a Poll's own, or a
// previous Poll already judged the report — is nobody's to answer for.
func (p *Poller) judgeSighting(sr store.Series, found float64) {
if sr.LatestRaisedBy == nil || sr.LatestChapterNum == nil {
return
}
stored := *sr.LatestChapterNum
if found > stored {
// The report is neither confirmed nor contradicted, but it is answered:
// the value about to be stored is the Poll's own, so leaving the
// attribution would credit this Reader with the next Poll's agreement
// and blame them if the Site later retracts.
if err := p.Store.ClearSightingAttribution(sr.Site, sr.SeriesID, *sr.LatestRaisedBy); err != nil {
log.Printf("latest poll %q: clear sighting attribution: %v", sr.Key(), err)
}
return
}
if found < stored {
// Logged with both numbers and the Reader, because that is what tells a
// broken Site adapter (which marks every Reader of that Site at once)
// from one Reader deliberately lying.
log.Printf("latest poll %q: sighting contradicted: reader %d raised it to %v, site publishes %v",
sr.Key(), *sr.LatestRaisedBy, stored, found)
}
if err := p.Store.RecordSightingOutcome(sr.Site, sr.SeriesID, *sr.LatestRaisedBy, found == stored); err != nil {
log.Printf("latest poll %q: record sighting outcome: %v", sr.Key(), err)
}
}
// healCover runs prefetchCover in the background. Cover bytes come from a
// different host — often a CDN — and heal once in a Series's life, so they
// must not consume a Lane's gap: a large import with many blanks would
+109
View File
@@ -1388,6 +1388,11 @@ func TestBrowserLaneWakeThresholds(t *testing.T) {
if got := browser.callCount(); got != 0 {
t.Fatalf("browser fetches with 3 freshly-due series = %d, want 0 (Chrome stays asleep)", got)
}
// The owner's page reads this state off the snapshot, and Due-without-
// Checked has to be distinguishable there from a Lane that has stopped.
if lane := laneByName(t, p, "kagane"); !lane.Asleep || lane.Due != 3 || lane.Checked != 0 {
t.Fatalf("asleep kagane lane = %+v, want Asleep with 3 due and 0 checked", lane)
}
// 5 due crosses the count threshold.
for i := 3; i < 5; i++ {
seed(i)
@@ -1396,6 +1401,9 @@ func TestBrowserLaneWakeThresholds(t *testing.T) {
if got := browser.callCount(); got != 5 {
t.Fatalf("browser fetches with 5 due series = %d, want 5", got)
}
if lane := laneByName(t, p, "kagane"); lane.Asleep {
t.Fatalf("woken kagane lane still reports Asleep: %+v", lane)
}
// A single long-neglected series wakes the browser by age alone.
seedForCheck(t, s, "kagane:ancient", "https://kagane.to/series/ancient", 0)
p.runOnce(context.Background())
@@ -1404,6 +1412,19 @@ func TestBrowserLaneWakeThresholds(t *testing.T) {
}
}
// laneByName pulls one Lane out of the poller's snapshot, failing rather than
// returning a zero LaneState a caller would assert against by accident.
func laneByName(t *testing.T, p *Poller, site string) LaneState {
t.Helper()
for _, lane := range p.LaneStatus().Lanes {
if lane.Site == site {
return lane
}
}
t.Fatalf("no %q lane in the snapshot", site)
return LaneState{}
}
// When one browser Lane loses the sidecar, the round's remaining browser
// Lanes are skipped: every fetch would fail anyway, and their Series must not
// burn their stamps on a dead Chrome (issue #100).
@@ -1580,3 +1601,91 @@ func TestRunOnceClampWarningNamesTheSite(t *testing.T) {
t.Fatalf("clamp warning = %q, want it to name asura and 3601", got)
}
}
// The owner's admin page (issue #102) reads Lane state out of the poller.
// Before any pass the snapshot is empty — a restart must render "no data
// yet", not zeroes — and each pass records what it saw: the frozen clock,
// the due count and the pace, with refusal backoff derived at snapshot time.
func TestLaneStatus(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(5_000_000)
p := newTestPoller(t, s, &fakeFetcher{body: asuraSeriesFixture, status: 200}, now)
if st := p.LaneStatus(); len(st.Lanes) != 0 {
t.Fatalf("lanes before any pass = %d, want 0 (nothing has run)", len(st.Lanes))
} else if st.BrowserConfigured || st.BrowserReachable {
t.Fatalf("browser before any pass = configured=%v reachable=%v, want false without a browser fetcher", st.BrowserConfigured, st.BrowserReachable)
}
seedForCheck(t, s, "asura:chronicles", "https://asurascans.com/series/chronicles", 0)
// Two refusals put kagane's Lane into backoff; asura sits on its own Lane.
browser := &fakeFetcher{status: 403}
p.BrowserFetch = browser
for i := range 2 {
key := fmt.Sprintf("kagane:s%d", i)
seedForCheck(t, s, key, "https://kagane.to/series/"+key[7:], 0)
}
p.runOnce(context.Background())
st := p.LaneStatus()
var asura, kagane LaneState
for _, lane := range st.Lanes {
switch lane.Site {
case "asura":
asura = lane
case "kagane":
kagane = lane
}
}
if st.Lanes[0].Site != "asura" {
t.Fatalf("first lane = %q, want asura (snapshot sorted by Site)", st.Lanes[0].Site)
}
if asura.Site == "" {
t.Fatalf("asura missing from snapshot: %+v", st.Lanes)
}
if !asura.LastRun.Equal(now) {
t.Fatalf("asura LastRun = %s, want the frozen clock %s", asura.LastRun, now)
}
if asura.Due != 1 {
t.Fatalf("asura Due = %d, want 1", asura.Due)
}
if asura.Gap == 0 {
t.Fatal("asura Gap = 0, want the Lane's pace")
}
if asura.Browser || asura.Refusing {
t.Fatalf("asura = %+v, want a TLS Lane that is not refusing", asura)
}
if !kagane.Browser || !kagane.Refusing {
t.Fatalf("kagane = %+v, want a browser Lane in refusal backoff", kagane)
}
if !st.BrowserConfigured || !st.BrowserReachable {
t.Fatalf("browser after round = configured=%v reachable=%v, want true/true (sidecar never lost)", st.BrowserConfigured, st.BrowserReachable)
}
if asura.Checked != 1 {
t.Fatalf("asura Checked = %d, want the one Series it read", asura.Checked)
}
// A pass that declines to look (kagane is now in backoff) must not restate
// the figures it never gathered as zeroes: the last real pass's due count
// and pace stand until a pass replaces them.
before := kagane
if before.Due == 0 || before.Gap == 0 {
t.Fatalf("kagane after its refusing pass = %+v, want the figures that pass gathered", before)
}
p.runOnce(context.Background())
for _, lane := range p.LaneStatus().Lanes {
if lane.Site != "kagane" {
continue
}
if lane.Due != before.Due || lane.Gap != before.Gap {
t.Fatalf("kagane after a skipped pass = due %d gap %s, want the previous pass's %d / %s",
lane.Due, lane.Gap, before.Due, before.Gap)
}
}
// A lost sidecar reads as unreachable for the same window the Lanes skip.
p.setBrowserDown(now)
if st := p.LaneStatus(); !st.BrowserConfigured || st.BrowserReachable {
t.Fatalf("browser after loss = configured=%v reachable=%v, want true/false", st.BrowserConfigured, st.BrowserReachable)
}
}
+494
View File
@@ -0,0 +1,494 @@
package latest
import (
"context"
"crypto/sha256"
"fmt"
"log"
"strings"
"testing"
"time"
"bookmarkmanager/backend/internal/store"
)
// Sightings (issue #103) are specified at the Poller seam, with the store as
// the way in: a Sighting is seeded the way handlers.Put performs one, a round
// is run against the injected fetcher and a frozen clock, and the assertions
// are the two observable facts — whether the Series was fetched, and what the
// stored Latest Chapter is afterwards. Nothing here asserts counter arithmetic
// through an internal call or reads how a deferral is represented in a row.
const (
sightingSlug = "chronicles-of-the-demon-faction-f886a8af"
sightingKey = "asura:" + sightingSlug
sightingURL = "https://asurascans.com/comics/" + sightingSlug
)
// sightingFixtureLatest is the newest chapter asuraSeriesFixture publishes.
const sightingFixtureLatest = 181.0
// sight performs one Sighting exactly as the JSON API does (handlers.Put):
// RecordSighting against the row as stored, then the Upsert that stores the
// reported value. The order is load-bearing — the raise comparison has nothing
// to compare against once the Upsert has landed — and the bookmark's own fields
// are carried over untouched, which is what a userscript PUT does when it
// echoes back the row it cached.
func sight(t *testing.T, s *store.Store, readerID int64, key string, num float64, at time.Time) {
t.Helper()
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
b, found, err := s.Get(readerID, key)
if err != nil || !found {
t.Fatalf("sight %q: get: %v found=%v", key, err, found)
}
if err := s.RecordSighting(readerID, site, seriesID, &num, at.UnixMilli()); err != nil {
t.Fatalf("sight %q: %v", key, err)
}
b.LatestChapter = fmt.Sprintf("Chapter %v", num)
b.LatestChapterNum = &num
b.UpdatedAt = at.UnixMilli()
if _, err := s.Upsert(readerID, b); err != nil {
t.Fatalf("sight %q: upsert: %v", key, err)
}
}
// secondReader is another Reader on the same database. The owner seed is the
// only reader-creation path in this package, so a second Open as a different
// owner is how a test gets one (as TestRunOnceFetchesSharedSeriesOnce does).
func secondReader(t *testing.T, dbURL string) *store.Store {
t.Helper()
other, err := store.Open(dbURL,
store.Owner{DiscordID: "second-reader", TokenHash: sha256.Sum256([]byte("second-token-hash"))},
t.TempDir(), testCoverBaseURL)
if err != nil {
t.Fatalf("Open second reader: %v", err)
}
t.Cleanup(func() { other.Close() })
return other
}
func readLatestNum(t *testing.T, s *store.Store, readerID int64, key string) float64 {
t.Helper()
b, ok, err := s.Get(readerID, key)
if err != nil || !ok {
t.Fatalf("Get %q: %v ok=%v", key, err, ok)
}
if b.LatestChapterNum == nil {
t.Fatalf("%q has no latest chapter", key)
}
return *b.LatestChapterNum
}
// A Series only one Reader bookmarks is the case where being wrong can hurt
// nobody but the Reader who reported it, so their Sighting stands in for the
// Poll and the round leaves the Series alone.
func TestSightingOnSolitarySeriesDefersPoll(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).runOnce(context.Background())
if got := f.callCount(); got != 0 {
t.Fatalf("fetched %d times after a Sighting on a solitary Series, want 0", got)
}
}
// On a shared Series the Sighting still writes the Latest Chapter for everyone,
// but the Poll happens on schedule anyway — which is what corrects a wrong
// value within the hour instead of letting it persist.
func TestSightingOnSharedSeriesDoesNotDeferPoll(t *testing.T) {
s, dbURL := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
other := secondReader(t, dbURL)
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
}); err != nil {
t.Fatalf("seed second reader: %v", err)
}
sight(t, s, s.OwnerID(), sightingKey, 200, now.Add(-10*time.Minute))
// The Sighting updated the shared row immediately, before any Poll.
if got := readLatestNum(t, s, s.OwnerID(), sightingKey); got != 200 {
t.Fatalf("latest after the Sighting = %v, want 200", got)
}
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times after a Sighting on a shared Series, want 1", got)
}
if got := readLatestNum(t, s, s.OwnerID(), sightingKey); got != sightingFixtureLatest {
t.Fatalf("latest after the Poll = %v, want the Site's own %v", got, sightingFixtureLatest)
}
}
// Reporting a chapter is not reading one: a Sighting may move the Latest
// Chapter and nothing else. Both the solitary and the shared case, because the
// deferral branch must not be where this guarantee lives.
func TestSightingLeavesProgressAndOrderingUntouched(t *testing.T) {
for _, shared := range []bool{false, true} {
name := "solitary"
if shared {
name = "shared"
}
t.Run(name, func(t *testing.T) {
s, dbURL := newTestStore(t)
read := 5.0
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, SeriesURL: sightingURL,
LastChapter: "Chapter 5", LastChapterNum: read,
LastChapterURL: sightingURL + "/chapter/5", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if shared {
other := secondReader(t, dbURL)
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
}); err != nil {
t.Fatalf("seed second reader: %v", err)
}
}
sight(t, s, s.OwnerID(), sightingKey, 200, time.UnixMilli(9_000_000))
b, ok, err := s.Get(s.OwnerID(), sightingKey)
if err != nil || !ok {
t.Fatalf("Get: %v ok=%v", err, ok)
}
if b.LatestChapterNum == nil || *b.LatestChapterNum != 200 {
t.Fatalf("LatestChapterNum = %v, want 200", b.LatestChapterNum)
}
if b.LastChapterNum != read {
t.Fatalf("LastChapterNum = %v, want %v: a Sighting is not Progress", b.LastChapterNum, read)
}
if b.UpdatedAt != 1000 {
t.Fatalf("updated_at moved to %d: a Sighting must not reorder the list", b.UpdatedAt)
}
})
}
}
// The ceiling is what makes trusting a client report safe: however recently a
// Series was sighted, one that has not been Polled in six hours is Polled.
func TestSightingCeilingForcesPoll(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-7*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-time.Minute))
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times past the %s ceiling, want 1", got, sightingCeilingRests*defaultRest)
}
}
// Deferral is decided from live facts every round, so a Series that gains a
// second Bookmark stops deferring at once — and one that loses it defers again.
func TestDeferralFollowsTheBookmarkCount(t *testing.T) {
s, dbURL := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
p := newTestPoller(t, s, f, now)
p.runOnce(context.Background())
if got := f.callCount(); got != 0 {
t.Fatalf("solitary Series fetched %d times, want 0", got)
}
other := secondReader(t, dbURL)
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
}); err != nil {
t.Fatalf("seed second reader: %v", err)
}
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("shared Series fetched %d times, want 1", got)
}
// The Poll above consumed the rest, so move past it before asking again.
if err := other.Delete(other.OwnerID(), sightingKey); err != nil {
t.Fatalf("delete second bookmark: %v", err)
}
later := now.Add(2 * time.Hour)
p.Now = func() time.Time { return later }
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, later.Add(-time.Minute))
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("Series fetched %d times after returning to one Bookmark, want 1", got)
}
}
// A Series nobody reports any more returns to the normal schedule on its own:
// the Sighting's standing lasts one rest, not forever.
func TestDeferralExpiresWithoutFurtherSightings(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
p := newTestPoller(t, s, f, now)
p.runOnce(context.Background())
if got := f.callCount(); got != 0 {
t.Fatalf("fetched %d times while the Sighting stood, want 0", got)
}
p.Now = func() time.Time { return now.Add(90 * time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times once the Sighting aged out, want 1", got)
}
}
// demonicFixture publishes one chapter in demonicscans' live page shape, so a
// test can make a Site publish an arbitrary number rather than the one the
// captured fixture froze.
func demonicFixture(num float64) string {
return fmt.Sprintf(
`<a href="/chaptered.php?manga=11799&chapter=%v" class="chplinks" title="Catastrophic Necromancer %v">Chapter %v</a>`,
num, num, num)
}
const (
demonicKey = "demonic:Catastrophic-Necromancer"
demonicURL = "https://demonicscans.org/manga/Catastrophic-Necromancer"
)
// contradictOnce reports a chapter that does not exist and then runs the round
// that catches it, returning when that round ran so a caller can chain the
// next one. The wait is one rest and a minute: a Sighting stands in for exactly
// one rest, so that is the first moment this solitary Series is Polled again.
func contradictOnce(t *testing.T, s *store.Store, p *Poller, sightAt time.Time, real float64) time.Time {
t.Helper()
sight(t, s, s.OwnerID(), demonicKey, real+500, sightAt)
at := sightAt.Add(defaultRest + time.Minute)
p.Now = func() time.Time { return at }
p.runOnce(context.Background())
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != real {
t.Fatalf("latest after the Poll = %v, want the Site's own %v", got, real)
}
return at
}
func seedDemonic(t *testing.T, s *store.Store, checkedAt int64) {
t.Helper()
seedForCheck(t, s, demonicKey, demonicURL, checkedAt)
}
// A Poll finding a lower number than stored means the Sighting that raised it
// was false. The Reader is named — not the Series flagged — and both numbers are
// logged, because that is what tells a broken adapter from a deliberate lie.
func TestPollContradictingASightingNamesTheReaderAndBothNumbers(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
var logs strings.Builder
prev := log.Writer()
log.SetOutput(&logs)
t.Cleanup(func() { log.SetOutput(prev) })
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, now)
contradictOnce(t, s, p, now, 296)
got := logs.String()
for _, want := range []string{
fmt.Sprintf("reader %d", s.OwnerID()), "796", "296", demonicKey,
} {
if !strings.Contains(got, want) {
t.Fatalf("contradiction log = %q, want it to name %q", got, want)
}
}
}
// Three contradictions cost the Reader the right to defer. Nothing here writes
// a counter: the marks are earned through Polls, which is the only way
// production produces them.
func TestThreeContradictionsStopDeferral(t *testing.T) {
s, _ := newTestStore(t)
start := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, start)
at := start
for range store.SightingDisagreementLimit {
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
}
fetchesSoFar := f.callCount()
// The marked Reader sights the same solitary Series again. It still writes
// the Latest Chapter — the penalty removes a privilege, it does not silence
// anyone — but the Poll is no longer postponed: the round below runs while a
// trusted Reader's Sighting would still be standing, and fetches anyway.
sight(t, s, s.OwnerID(), demonicKey, 900, at.Add(31*time.Minute))
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != 900 {
t.Fatalf("latest after a marked Reader's Sighting = %v, want 900", got)
}
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != fetchesSoFar+1 {
t.Fatalf("marked Reader's Sighting still deferred the Poll (fetches %d, want %d)",
got, fetchesSoFar+1)
}
}
// The owner's remedy for a mark a broken Site adapter produced restores the
// privilege without a wait and without SQL.
func TestClearingMarksRestoresDeferral(t *testing.T) {
s, _ := newTestStore(t)
start := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, start)
at := start
for range store.SightingDisagreementLimit {
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
}
if err := s.ClearReaderMarks(s.OwnerID()); err != nil {
t.Fatalf("ClearReaderMarks: %v", err)
}
fetchesSoFar := f.callCount()
sight(t, s, s.OwnerID(), demonicKey, 900, at.Add(31*time.Minute))
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != fetchesSoFar {
t.Fatalf("fetched %d times after the marks were cleared, want %d: deferral must resume",
got, fetchesSoFar)
}
}
// Recovery is automatic but expensive: twenty Polls that each confirm a
// Sighting of this Reader's clear the marks. Each round needs a new chapter,
// because only a report that raises the stored number is attributed and so only
// that one can be confirmed.
func TestTwentyAgreementsClearTheMarks(t *testing.T) {
s, _ := newTestStore(t)
start := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, start)
at := start
for range store.SightingDisagreementLimit {
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
}
chapter := 296.0
for range store.SightingAgreementsToClear {
chapter++
sight(t, s, s.OwnerID(), demonicKey, chapter, at.Add(time.Minute))
f.body = demonicFixture(chapter) // the Site publishes what was reported
at = at.Add(defaultRest + time.Minute)
p.Now = func() time.Time { return at }
p.runOnce(context.Background())
}
fetchesSoFar := f.callCount()
chapter++
sight(t, s, s.OwnerID(), demonicKey, chapter, at.Add(31*time.Minute))
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != fetchesSoFar {
t.Fatalf("fetched %d times after %d confirmations, want %d: the marks must be forgiven",
got, store.SightingAgreementsToClear, fetchesSoFar)
}
}
// A Poll finding a higher number is the Site publishing since the Sighting and
// means nothing about the Reader — no mark, and no credit either.
func TestPollFindingHigherNumberIsNotAContradiction(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
// Reported truthfully, then the Site published one more.
sight(t, s, s.OwnerID(), demonicKey, 295, now.Add(-10*time.Minute))
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, now)
// One rest on, the Sighting has lapsed and the Poll happens.
p.Now = func() time.Time { return now.Add(7 * time.Hour) }
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times past the ceiling, want 1", got)
}
// Unmarked, so a fresh Sighting still defers.
at := now.Add(9 * time.Hour)
sight(t, s, s.OwnerID(), demonicKey, 296, at.Add(-time.Minute))
p.Now = func() time.Time { return at }
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("a Reader whose report the Site overtook lost the right to defer (fetches %d, want 1)", got)
}
}
// A Poll that overtakes a Sighting takes ownership of the row: the value stored
// afterwards is the Poll's own, so a later retraction is not the Reader's fault
// and must not be charged to them.
func TestAttributionDoesNotSurviveAPollThatOvertookIt(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), demonicKey, 295, now.Add(-10*time.Minute))
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, now)
at := now.Add(defaultRest + time.Minute)
p.Now = func() time.Time { return at }
p.runOnce(context.Background())
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != 296 {
t.Fatalf("latest after the Poll = %v, want the Site's own 296", got)
}
var logs strings.Builder
prev := log.Writer()
log.SetOutput(&logs)
t.Cleanup(func() { log.SetOutput(prev) })
f.body = demonicFixture(290) // the Site retracts what only the Poll wrote
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
p.runOnce(context.Background())
if strings.Contains(logs.String(), "sighting contradicted") {
t.Fatalf("a retraction of the Poll's own value was charged to a Reader: %s", logs.String())
}
}
// A PUT with no Latest Chapter in it — a favourite toggle, progress written
// from a chapter page — is nobody looking at the Series page, so it buys no
// deferral. Otherwise a client could suppress a Series' Polls while reporting
// nothing, and with nothing reported there would be nothing to judge.
func TestPutWithoutALatestChapterDoesNotDefer(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
// The handler's own call, with the field the client omitted.
if err := s.RecordSighting(s.OwnerID(), "demonic", "Catastrophic-Necromancer",
nil, now.Add(-time.Minute).UnixMilli()); err != nil {
t.Fatalf("RecordSighting: %v", err)
}
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, now)
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times after a PUT carrying no chapter, want 1", got)
}
}
+8
View File
@@ -409,6 +409,14 @@ const (
// asleep (ADR-0005 on-demand browser).
browserWakeCount = 5
browserWakeAge = 15 * time.Minute
// sightingCeilingRests caps Sighting deferral (issue #103): however many
// Sightings arrive, a Series unpolled for this many of its Site's rests is
// Polled. It is what makes a client report safe to trust — a wrong Latest
// Chapter dies within the ceiling deterministically, rather than in
// expectation the way a randomised audit would have it. Six, so a Series a
// Reader visits constantly still gets one authoritative check per working
// day-part.
sightingCeilingRests = 6
)
// effectiveGap is a Site's pace: the registry gap, or one rest divided by the
+79
View File
@@ -0,0 +1,79 @@
package latest
import "time"
// LaneState is the administrative page's view of one Poll Lane (issue #102):
// what the Lane's last pass saw. Due, Gap and Checked are filled in as the
// pass computes them; a pass that returned before reaching a figure (refusal
// backoff, sidecar down) carries the previous pass's figures forward rather
// than overwriting them with zeroes the page would state as fact.
type LaneState struct {
Site string
Due int
LastRun time.Time
Gap time.Duration
// Checked is how many Series this pass actually read. A Lane with Series
// due and nothing checked has stopped working; one with nothing due is
// merely quiet, and the page must not draw the two the same (story 13).
Checked int
Clamped bool
Refusing bool
Browser bool
// Asleep marks a browser Lane whose last pass declined to wake Chrome
// because it was under both wake thresholds (ADR-0005). Due without
// Checked then means "waiting for the group to gather", not "stopped", and
// the page must not draw it as a stall.
Asleep bool
}
// Status is the owner's page snapshot of the whole poller (issue #102).
type Status struct {
Lanes []LaneState
BrowserConfigured bool
BrowserReachable bool
}
// LaneStatus returns a copy of the poller's Lane state for the owner's page.
// Only Sites that have completed a pass appear — a restart therefore renders
// "no data yet" instead of confident zeroes — in the same order Run iterates.
// Refusing is derived at snapshot time from the refusal backoff, not stored,
// so a Lane that cooled down between passes reports false without a new pass.
// BrowserReachable mirrors the Lanes' own gate: the sidecar is down only
// within the refuseBackoff window since its last loss.
func (p *Poller) LaneStatus() Status {
p.mu.Lock()
defer p.mu.Unlock()
lanes := make([]LaneState, 0, len(p.laneStates))
now := p.Now()
for _, name := range laneNames() {
st, ok := p.laneStates[name]
if !ok {
continue
}
st.Refusing = now.Before(p.refuseUntil[name])
lanes = append(lanes, st)
}
configured := p.BrowserFetch != nil
reachable := configured
if reachable && !p.browserDownAt.IsZero() && now.Sub(p.browserDownAt) < refuseBackoff {
reachable = false
}
return Status{Lanes: lanes, BrowserConfigured: configured, BrowserReachable: reachable}
}
// recordLaneState stores one Lane's last pass for LaneStatus. Called deferred
// from runLanePass so every return path records, even a pass that refused.
// A pass that never reached the pace (Gap zero) keeps the last pass's figures:
// the Lane's due count and gap did not become zero because this pass declined
// to look, and the row's own marks say why it declined.
func (p *Poller) recordLaneState(st LaneState) {
p.mu.Lock()
defer p.mu.Unlock()
if p.laneStates == nil {
p.laneStates = make(map[string]LaneState)
}
if prev, ok := p.laneStates[st.Site]; ok && st.Gap == 0 {
st.Due, st.Gap, st.Clamped, st.Checked = prev.Due, prev.Gap, prev.Clamped, prev.Checked
}
p.laneStates[st.Site] = st
}
@@ -0,0 +1,6 @@
-- The owner's administrative page (issue #102) renders these counters and
-- offers a control to clear them, deliberately shipped before the Sighting
-- feature (issue #103) that fills them, so a false mark never needs SQL
-- against production. Zero counters mean a trusted Reader.
ALTER TABLE readers ADD COLUMN sighting_agreements integer NOT NULL DEFAULT 0;
ALTER TABLE readers ADD COLUMN sighting_disagreements integer NOT NULL DEFAULT 0;
@@ -0,0 +1,10 @@
-- Sighting deferral (issue #103). latest_sighted_at is when a Reader's report
-- last stood in for a Poll; it is separate from latest_checked_at because the
-- six-hour ceiling has to know when the Series was last really fetched, and a
-- Sighting writing the Poll's own column would erase that.
-- latest_raised_by is attribution: whoever last raised this Series' Latest
-- Chapter by Sighting, so a Poll that contradicts the value downwards names a
-- Reader rather than flagging a row. Cleared by the Poll that judges it, NULL
-- whenever the stored value is the Poll's own.
ALTER TABLE series ADD COLUMN latest_sighted_at bigint NOT NULL DEFAULT 0;
ALTER TABLE series ADD COLUMN latest_raised_by bigint REFERENCES readers(id) ON DELETE SET NULL;
+174 -12
View File
@@ -79,6 +79,11 @@ type Series struct {
LatestChapter string
LatestChapterNum *float64 // nil until first captured
LatestCheckedAt int64 // unix ms; see MarkLatestChecked
// LatestRaisedBy is the Reader whose Sighting last raised LatestChapter,
// and nil when the stored value is a Poll's own finding. It is what lets a
// Poll that contradicts the value downwards name a Reader instead of
// merely flagging the row (issue #103); the Poll that judges it clears it.
LatestRaisedBy *int64
// readerCount is the number of bookmarks referencing this series, filled
// only by the due-queue query that orders on it.
@@ -195,7 +200,7 @@ const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover_add
// due query. latest_checked_at lives only on series — see MarkLatestChecked
// for why it stays off every client-visible write.
const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover, s.cover_address,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at`
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at, s.latest_raised_by`
// Owner is the person running the service: the first Reader, seeded at startup
// so a fresh deployment has a library before anyone logs in. The seed makes
@@ -315,25 +320,42 @@ func (s *Store) EnsureReader(discordID string, epochZeroHash [32]byte) (int64, e
return id, nil
}
// SightingDisagreementLimit is the number of contradictions that stop that
// Reader's Sightings from deferring a Poll (issue #103). The counters exist
// before the mechanism that moves them, so the admin page (issue #102) can
// clear a false mark without waiting for the Sighting feature.
const SightingDisagreementLimit = 3
// Blocked reports whether this Reader's Sighting marks have reached the
// disagreement limit, which stops their Sightings from deferring a Poll.
func (r ReaderSummary) Blocked() bool {
return r.Disagreements >= SightingDisagreementLimit
}
// ReaderSummary is one Reader as the owner's administration panel sees them:
// who they are and how many live sessions they hold. No credential material,
// hashed or otherwise, is exposed.
// who they are, how many live sessions they hold, and their Sighting marks.
// No credential material, hashed or otherwise, is exposed.
type ReaderSummary struct {
ID int64
DiscordID string
// Sessions counts unexpired session rows — what the owner revokes.
Sessions int
// Agreements and Disagreements are the Sighting counters (issue #102);
// zero means a trusted Reader.
Agreements int
Disagreements int
}
// Readers lists every Reader with their live session count, oldest first, so
// the owner row (always the oldest) heads the list.
// Readers lists every Reader with their live session count and Sighting
// marks, oldest first, so the owner row (always the oldest) heads the list.
func (s *Store) Readers() ([]ReaderSummary, error) {
rows, err := s.db.Query(`
SELECT r.id, r.discord_id,
r.sighting_agreements, r.sighting_disagreements,
count(sess.id) FILTER (WHERE sess.expires_at > now()) AS sessions
FROM readers r
LEFT JOIN sessions sess ON sess.reader_id = r.id
GROUP BY r.id, r.discord_id
GROUP BY r.id, r.discord_id, r.sighting_agreements, r.sighting_disagreements
ORDER BY r.id`)
if err != nil {
return nil, fmt.Errorf("query readers: %w", err)
@@ -343,7 +365,7 @@ func (s *Store) Readers() ([]ReaderSummary, error) {
out := []ReaderSummary{}
for rows.Next() {
var r ReaderSummary
if err := rows.Scan(&r.ID, &r.DiscordID, &r.Sessions); err != nil {
if err := rows.Scan(&r.ID, &r.DiscordID, &r.Agreements, &r.Disagreements, &r.Sessions); err != nil {
return nil, fmt.Errorf("scan reader: %w", err)
}
out = append(out, r)
@@ -351,6 +373,18 @@ func (s *Store) Readers() ([]ReaderSummary, error) {
return out, rows.Err()
}
// ClearReaderMarks zeroes a Reader's Sighting counters. It is the owner's
// remedy for a mark produced by a broken Site adapter rather than a dishonest
// Reader: it restores a privilege, it is not destruction.
func (s *Store) ClearReaderMarks(readerID int64) error {
if _, err := s.db.Exec(`
UPDATE readers SET sighting_agreements = 0, sighting_disagreements = 0
WHERE id = $1`, readerID); err != nil {
return fmt.Errorf("clear reader marks for reader %d: %w", readerID, err)
}
return nil
}
// readersMigration is the version that creates the readers table. The owner
// seed runs between two migrate passes, so that the run-once migration which
// attaches existing bookmarks (0004) finds the owner row.
@@ -555,16 +589,17 @@ func (s *Store) scanBookmark(scan func(...any) error) (Bookmark, error) {
}
// scanSeries reads one row in seriesColumns order, plus the due query's
// reader_count column. latest_chapter_num is NULL until the first capture,
// same as on the bookmark read path.
// reader_count column. latest_chapter_num and latest_raised_by are both
// nullable, same as latest_chapter_num on the bookmark read path.
func scanSeries(scan func(...any) error) (Series, error) {
var (
sr Series
latestChapterNum sql.NullFloat64
latestRaisedBy sql.NullInt64
)
if err := scan(
&sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover, &sr.CoverAddress,
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt,
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt, &latestRaisedBy,
&sr.readerCount,
); err != nil {
return Series{}, err
@@ -572,6 +607,9 @@ func scanSeries(scan func(...any) error) (Series, error) {
if latestChapterNum.Valid {
sr.LatestChapterNum = &latestChapterNum.Float64
}
if latestRaisedBy.Valid {
sr.LatestRaisedBy = &latestRaisedBy.Int64
}
return sr, nil
}
@@ -918,7 +956,20 @@ func (s *Store) Delete(readerID int64, key string) error {
// burns requests. Archived bookmarks still count — knowing what a shelved
// series is up to is the whole reason for archiving instead of deleting.
// A series with no bookmarks at all never appears: the join excludes it.
func (s *Store) DueForLatestCheck(site string, cutoffMs int64) ([]Series, error) {
//
// ceilingMs is the Sighting deferral ceiling (issue #103): a Series whose last
// real Poll is older than it appears however recently it was sighted. That is
// what bounds the whole mechanism — a wrong Latest Chapter dies within the
// ceiling deterministically rather than in expectation. Deferral itself is
// decided here, from two facts the query already computes, so a Lane gains no
// query per round: a Sighting younger than cutoffMs holds the Series back, but
// only while COUNT(*) is 1. A Series a second Reader bookmarks is Polled on
// schedule, so a wrong value the whole guild can see is corrected by a check
// that was never postponed; on a solitary Series the only person a wrong value
// reaches is the Reader who reported it. Whether the reporting Reader is
// allowed to defer at all was settled when the Sighting was recorded — see
// RecordSighting.
func (s *Store) DueForLatestCheck(site string, cutoffMs, ceilingMs int64) ([]Series, error) {
rows, err := s.db.Query(`SELECT `+seriesColumns+`, COUNT(*) AS reader_count
FROM series s
JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id
@@ -928,7 +979,10 @@ func (s *Store) DueForLatestCheck(site string, cutoffMs int64) ([]Series, error)
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at
HAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
ORDER BY reader_count DESC, s.latest_checked_at ASC`, site, cutoffMs)
AND (COUNT(*) > 1
OR s.latest_sighted_at <= $2::bigint
OR s.latest_checked_at <= $3::bigint)
ORDER BY reader_count DESC, s.latest_checked_at ASC`, site, cutoffMs, ceilingMs)
if err != nil {
return nil, fmt.Errorf("query due series: %w", err)
}
@@ -1015,3 +1069,111 @@ func (s *Store) SetLatestChapter(site, seriesID, label string, num float64) erro
}
return nil
}
// RecordSighting notes that a Reader's browser reported this Series' Latest
// Chapter, which is the half of a Sighting the client body cannot express
// (issue #103). It must be called *before* the Upsert that stores the reported
// value: the raise test compares against what is still on the row, and after
// the Upsert there is nothing left to compare with. A Series that does not
// exist yet — the first Bookmark of it — is not a Sighting at all: nothing has
// ever been Polled, so there is nothing to defer and nobody to attribute.
//
// Two independent effects, hence the two CASE arms. The deferral stamp is only
// written for a Reader below the disagreement limit, so a marked Reader's
// reports keep updating the Latest Chapter but stop postponing anything, and
// clearing their marks restores the privilege on their next Sighting. The
// attribution is written whenever the report raises the stored number,
// including for a marked Reader — their Sightings are still judged, which is
// how they earn the privilege back.
//
// num is the reported chapter number. A PUT that carries none — a favourite
// toggle, or progress written from a chapter page — is no Sighting at all:
// nobody read the Series page, so there is nothing to stand in for a Poll and
// nothing that could later be judged.
func (s *Store) RecordSighting(readerID int64, site, seriesID string, num *float64, ts int64) error {
if num == nil {
return nil
}
if _, err := s.db.Exec(`
UPDATE series SET
latest_sighted_at = CASE
WHEN (SELECT sighting_disagreements FROM readers WHERE id = $3) < $6
THEN $4::bigint ELSE latest_sighted_at END,
latest_raised_by = CASE
WHEN latest_chapter_num IS NULL OR $5::double precision > latest_chapter_num
THEN $3::bigint ELSE latest_raised_by END
WHERE site = $1 AND series_id = $2`,
site, seriesID, readerID, ts, *num, SightingDisagreementLimit); err != nil {
return fmt.Errorf("record sighting %s:%s: %w", site, seriesID, err)
}
return nil
}
// SightingAgreementsToClear is how many Polls must confirm a Reader's
// Sightings in a row before their disagreements are forgiven. An agreement is
// only recorded when a Poll later confirms a Sighting, so this is twenty Polls
// of Series that Reader bookmarks — hours to days, not twenty page views. That
// is the intended price: recovery is automatic but cannot be outwaited, and a
// disagreement resets the run to zero, so credit cannot be banked in advance.
const SightingAgreementsToClear = 20
// RecordSightingOutcome settles what a Poll decided about the Reader whose
// Sighting last raised this Series' Latest Chapter, and clears the attribution
// in the same transaction so one Sighting is judged exactly once. agreed is
// the Poll confirming the stored value; its opposite is the Poll finding a
// lower number, which means the raise was false.
//
// A Poll finding a *higher* number is neither — the Site published — and takes
// ClearSightingAttribution instead.
func (s *Store) RecordSightingOutcome(site, seriesID string, readerID int64, agreed bool) error {
tx, err := s.db.Begin()
if err != nil {
return fmt.Errorf("begin sighting outcome %s:%s: %w", site, seriesID, err)
}
defer tx.Rollback()
// The run length is what "consecutive" means: a disagreement zeroes the
// agreements, and completing a run zeroes both, so the next run starts
// from nothing rather than forgiving every later disagreement instantly.
q := `UPDATE readers SET sighting_disagreements = sighting_disagreements + 1,
sighting_agreements = 0
WHERE id = $1`
args := []any{readerID}
if agreed {
q = `UPDATE readers SET
sighting_agreements = CASE WHEN sighting_agreements + 1 >= $2 THEN 0
ELSE sighting_agreements + 1 END,
sighting_disagreements = CASE WHEN sighting_agreements + 1 >= $2 THEN 0
ELSE sighting_disagreements END
WHERE id = $1`
args = append(args, SightingAgreementsToClear)
}
if _, err := tx.Exec(q, args...); err != nil {
return fmt.Errorf("record sighting outcome for reader %d: %w", readerID, err)
}
if _, err := tx.Exec(clearAttributionSQL, site, seriesID, readerID); err != nil {
return fmt.Errorf("clear sighting attribution %s:%s: %w", site, seriesID, err)
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("commit sighting outcome %s:%s: %w", site, seriesID, err)
}
return nil
}
// ClearSightingAttribution answers a Sighting without judging it: the Poll
// found a higher number, so the value about to be stored is its own and this
// Reader is no longer answerable for the row. Without it the next Poll's
// agreement would be credited to a Reader who did not earn it.
func (s *Store) ClearSightingAttribution(site, seriesID string, readerID int64) error {
if _, err := s.db.Exec(clearAttributionSQL, site, seriesID, readerID); err != nil {
return fmt.Errorf("clear sighting attribution %s:%s: %w", site, seriesID, err)
}
return nil
}
// clearAttributionSQL drops the attribution only while it still names the
// Reader being judged: a Sighting landing between the due query's snapshot and
// this write is a fresh, unjudged one and must not be erased by the previous
// one's verdict.
const clearAttributionSQL = `UPDATE series SET latest_raised_by = NULL
WHERE site = $1 AND series_id = $2 AND latest_raised_by = $3`
+91 -7
View File
@@ -277,6 +277,10 @@ func seedForCheck(t *testing.T, s *Store, key, seriesURL string, checkedAt int64
}
}
// noCeiling is a Sighting deferral ceiling no Series can reach, for the tests
// that predate the ceiling and are about rest, ordering or buckets instead.
const noCeiling = int64(-1)
func TestDueForLatestCheck(t *testing.T) {
const hour = int64(3600_000)
now := 10 * hour
@@ -298,7 +302,7 @@ func TestDueForLatestCheck(t *testing.T) {
s := newTestStore(t)
seedForCheck(t, s, "asura:x", tt.seriesURL, tt.checkedAt)
due, err := s.DueForLatestCheck("asura", now-hour)
due, err := s.DueForLatestCheck("asura", now-hour, noCeiling)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
@@ -319,7 +323,7 @@ func TestDueForLatestCheckOldestFirstAndScopedToSite(t *testing.T) {
// asks for one Site, and no Lane may see another's queue.
seedForCheck(t, s, "demonic:z", "https://demonicscans.org/manga/z", 0)
due, err := s.DueForLatestCheck("asura", 1000)
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
@@ -499,7 +503,7 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
}
}
due, err := store.DueForLatestCheck("asura", time.Now().UnixMilli())
due, err := store.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
@@ -1005,7 +1009,7 @@ func TestDueForLatestCheckOrdersByReaderCountThenAge(t *testing.T) {
seedSecondReader(t, s, "asura:pop:2", "asura", "pop", 1001)
seedForCheck(t, s, "asura:solo", "https://asurascans.com/comics/solo", 100)
due, err := s.DueForLatestCheck("asura", 1000)
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
@@ -1031,7 +1035,7 @@ func TestDueForLatestCheckExcludesOrphanSeries(t *testing.T) {
t.Fatalf("seed orphan series: %v", err)
}
due, err := s.DueForLatestCheck("asura", 1000)
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
@@ -1334,6 +1338,86 @@ func TestReadersAndSessionRevocation(t *testing.T) {
}
}
// The Sighting counters ship before the mechanism that fills them (issue
// #102 before #103), so the admin page depends on their default: a fresh
// Reader reads back trusted. Marking one directly proves Readers() reports
// the counters and Blocked() flips at the limit, and that ClearReaderMarks —
// the owner's remedy for a false mark — zeroes them again.
func TestReaderSightingMarks(t *testing.T) {
s := newTestStore(t)
other := secondReader(t, s)
readers, err := s.Readers()
if err != nil {
t.Fatalf("Readers: %v", err)
}
if len(readers) != 2 {
t.Fatalf("readers = %d, want the owner and the second Reader", len(readers))
}
for _, r := range readers {
if r.Agreements != 0 || r.Disagreements != 0 || r.Blocked() {
t.Fatalf("fresh reader %d has marks: %+v", r.ID, r)
}
}
// The counters' default is the trusted state; writing them directly is
// the only way to exercise the read path until issue #103 moves them.
if _, err := s.db.Exec(`
UPDATE readers SET sighting_agreements = 5, sighting_disagreements = 2
WHERE id = $1`, other); err != nil {
t.Fatalf("mark reader: %v", err)
}
readers, err = s.Readers()
if err != nil {
t.Fatalf("Readers: %v", err)
}
var marked *ReaderSummary
for i := range readers {
if readers[i].ID == other {
marked = &readers[i]
}
}
if marked == nil || marked.Agreements != 5 || marked.Disagreements != 2 {
t.Fatalf("marked reader = %+v, want agreements 5, disagreements 2", marked)
}
if marked.Blocked() {
t.Fatalf("reader with 2 disagreements is blocked; limit is %d", SightingDisagreementLimit)
}
if _, err := s.db.Exec(`
UPDATE readers SET sighting_disagreements = 3 WHERE id = $1`, other); err != nil {
t.Fatalf("block reader: %v", err)
}
readers, err = s.Readers()
if err != nil {
t.Fatalf("Readers: %v", err)
}
for _, r := range readers {
if r.ID == other && !r.Blocked() {
t.Fatalf("reader at the disagreement limit is not blocked: %+v", r)
}
if r.ID == s.OwnerID() && r.Blocked() {
t.Fatalf("untouched owner became blocked: %+v", r)
}
}
if err := s.ClearReaderMarks(other); err != nil {
t.Fatalf("ClearReaderMarks: %v", err)
}
if err := s.ClearReaderMarks(other + 9999); err != nil {
t.Fatalf("ClearReaderMarks(unknown id): %v", err)
}
readers, err = s.Readers()
if err != nil {
t.Fatalf("Readers: %v", err)
}
for _, r := range readers {
if r.Agreements != 0 || r.Disagreements != 0 || r.Blocked() {
t.Fatalf("reader %d not cleared: %+v", r.ID, r)
}
}
}
// Two Readers on one Series: one series row, two independent progresses. The
// second Reader starts at zero however far the first has read, and the shared
// row is still due exactly once.
@@ -1369,7 +1453,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
t.Fatalf("series rows = %d, want 1 shared row for two bookmarks", series)
}
due, err := s.DueForLatestCheck("asura", time.Now().UnixMilli())
due, err := s.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
@@ -1385,7 +1469,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
if b, ok, err := s.Get(s.OwnerID(), "asura:solo"); err != nil || !ok || b.LastChapterNum != 200 {
t.Fatalf("owner's bookmark after the other's delete = %+v ok=%v err=%v, want it intact", b, ok, err)
}
due, err = s.DueForLatestCheck("asura", time.Now().UnixMilli())
due, err = s.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
if err != nil {
t.Fatalf("DueForLatestCheck after delete: %v", err)
}
+256
View File
@@ -0,0 +1,256 @@
package web
import (
"log"
"net/http"
"strconv"
"time"
"bookmarkmanager/backend/internal/latest"
"bookmarkmanager/backend/internal/store"
)
// LaneReporter is the administrative page's whole window onto the running
// poller: one snapshot of Poll Lane state, copied out of memory on request.
// The Poller satisfies it in production and a fake with fixed values satisfies
// it in tests, so the page's tests need neither a poller nor a Site.
type LaneReporter interface {
LaneStatus() latest.Status
}
// adminView is what the administrative page and the roster fragment receive.
type adminView struct {
Readers []store.ReaderSummary
// OwnerID travels with the roster so it can tell the owner's own row from
// the Readers they may act on.
OwnerID int64
Lanes lanesView
}
// lanesView is the Lane status block: one row per Site that has run, plus the
// browser fact, which is shared by the three browser Sites rather than held
// once per Site.
type lanesView struct {
Rows []laneRow
// PollerOff means no poller is running at all (disabled by config, or its
// client could not be built). The browser line must not answer "not
// configured" then: the sidecar is not the reason nothing is polled.
PollerOff bool
BrowserConfigured bool
BrowserReachable bool
}
// laneRow is one Lane formatted for reading rather than for arithmetic: the
// template renders strings and flags, and every judgement about what they mean
// is made here.
type laneRow struct {
Site string
Due int
Ran string
// Checked is how many Series the last pass read. Due without Checked is a
// Lane that has stopped working; the two figures side by side are what
// separate that from a Lane with nothing to do.
Checked int
// Gap is empty when no pass has reached the pace yet, so the row omits the
// figure instead of stating a zero.
Gap string
Clamped bool
Refusing bool
// BrowserLost marks a Lane whose pages can only be read through the
// sidecar while the sidecar is unreachable — including the case where none
// is configured, which stops those Series just as completely.
BrowserLost bool
// Stalled marks a Lane with Series waiting that its last pass did not read
// — the difference between a stopped Lane and a quiet one (story 13). A
// browser Lane holding Chrome asleep under the wake thresholds is neither,
// so it carries Asleep instead and never Stalled.
Stalled bool
Asleep bool
// Attention is the one flag the template colours on, so an unhealthy Lane
// is found at a glance rather than read for.
Attention bool
}
// adminRoute pairs a route pattern with its handler so the route list and the
// gate cannot drift apart.
type adminRoute struct {
pattern string
handler http.HandlerFunc
}
// adminRoutes is every route that reaches past the acting Reader. Register
// wraps each one in requireOwner, so a new administrative route is gated by
// being listed here rather than by remembering to write a check inside it.
func (h *Handler) adminRoutes() []adminRoute {
return []adminRoute{
{"GET /admin", h.admin},
{"GET /ui/admin/lanes", h.uiLanes},
{"POST /readers/{id}/revoke", h.revokeReaderSessions},
{"POST /readers/{id}/clear-marks", h.clearReaderMarks},
}
}
// AdminPatterns names every administrative route, so one test can prove the
// owner gate covers all of them rather than one test per route. The receiver is
// nil because only the patterns are read; the bound handlers are never called.
func AdminPatterns() []string {
routes := (*Handler)(nil).adminRoutes()
out := make([]string, 0, len(routes))
for _, rt := range routes {
out = append(out, rt.pattern)
}
return out
}
// requireOwner is the owner test, in one place, layered on the session gate: no
// session is still 401, and a signed-in Reader who is not the owner gets 404
// rather than 403 — a refusal that confirms the address exists is a refusal
// that helps whoever is probing for it.
func (h *Handler) requireOwner(next http.HandlerFunc) http.HandlerFunc {
return h.requireSession(func(w http.ResponseWriter, r *http.Request) {
if readerOf(r) != h.store.OwnerID() {
http.NotFound(w, r)
return
}
next(w, r)
})
}
// admin renders the owner's page: the Reader roster and Poll Lane status.
func (h *Handler) admin(w http.ResponseWriter, r *http.Request) {
readers, err := h.store.Readers()
if err != nil {
log.Printf("admin: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.render(w, http.StatusOK, "admin", adminView{
Readers: readers,
OwnerID: h.store.OwnerID(),
Lanes: h.lanesView(),
})
}
// uiLanes answers the status block's own refresh. Only the block refreshes on a
// timer; the roster re-renders after an action, as it always has.
func (h *Handler) uiLanes(w http.ResponseWriter, r *http.Request) {
h.render(w, http.StatusOK, "lanes", h.lanesView())
}
// lanesView copies the poller's snapshot into display form. A nil reporter (no
// poller running) and a poller no Lane has reported to yet are the same thing
// to the page: no data, which it must say rather than draw as confident zeroes
// — an empty page a few seconds after a restart must not read as a stopped one.
func (h *Handler) lanesView() lanesView {
if h.lanes == nil {
return lanesView{PollerOff: true}
}
snap := h.lanes.LaneStatus()
v := lanesView{
Rows: make([]laneRow, 0, len(snap.Lanes)),
BrowserConfigured: snap.BrowserConfigured,
BrowserReachable: snap.BrowserReachable,
}
now := time.Now()
for _, l := range snap.Lanes {
lost := l.Browser && !snap.BrowserReachable
// Series waiting and none read is the shape of a Lane that has stopped
// working, as distinct from one that is quiet for want of work — or one
// deliberately leaving Chrome asleep until its group gathers.
stalled := l.Due > 0 && l.Checked == 0 && !l.Asleep
gap := ""
if l.Gap > 0 {
gap = l.Gap.Truncate(time.Second).String()
}
v.Rows = append(v.Rows, laneRow{
Site: l.Site,
Due: l.Due,
Ran: since(now, l.LastRun),
Checked: l.Checked,
Gap: gap,
Clamped: l.Clamped,
Refusing: l.Refusing,
BrowserLost: lost,
Stalled: stalled,
Asleep: l.Asleep,
Attention: l.Clamped || l.Refusing || lost || stalled,
})
}
return v
}
// since formats how long ago a Lane last ran, at second resolution: the block
// refreshes every thirty seconds, so anything finer is noise the owner would
// have to ignore.
func since(now, then time.Time) string {
d := now.Sub(then).Truncate(time.Second)
if d < time.Second {
return "just now"
}
return d.String() + " ago"
}
// revokeReaderSessions logs one Reader out of every browser they are signed in
// on. The owner gate is the route's, not this handler's.
func (h *Handler) revokeReaderSessions(w http.ResponseWriter, r *http.Request) {
target, ok := readerPathID(w, r)
if !ok {
return
}
// The owner is not one of the Readers this endpoint reaches: revoking
// themselves would sign out the browser making the request, which is what
// logout is for. The roster hides the button; this refuses the hand-rolled
// POST behind it.
if target == h.store.OwnerID() {
http.NotFound(w, r)
return
}
if err := h.store.DeleteReaderSessions(target); err != nil {
log.Printf("revoke sessions: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.renderRoster(w, "revoke sessions")
}
// clearReaderMarks zeroes one Reader's Sighting counters. The guard those
// counters feed has one known false positive — a Site changing its page shape
// makes a correct adapter read a wrong high number and marks every honest
// Reader of that Site at once (issue #103) — and this is its remedy. It
// restores a privilege rather than destroying anything, so the control is
// confirmed but never wears the destruction accent.
func (h *Handler) clearReaderMarks(w http.ResponseWriter, r *http.Request) {
target, ok := readerPathID(w, r)
if !ok {
return
}
if err := h.store.ClearReaderMarks(target); err != nil {
log.Printf("clear marks: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.renderRoster(w, "clear marks")
}
// readerPathID reads the Reader a route names, answering the request itself
// when there is nobody to act on.
func readerPathID(w http.ResponseWriter, r *http.Request) (int64, bool) {
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil {
http.Error(w, "bad reader id", http.StatusBadRequest)
return 0, false
}
return id, true
}
// renderRoster answers an action with the whole roster, so the counts and marks
// it shows cannot describe the state before the tap.
func (h *Handler) renderRoster(w http.ResponseWriter, what string) {
readers, err := h.store.Readers()
if err != nil {
log.Printf("%s: %v", what, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.render(w, http.StatusOK, "readers", adminView{Readers: readers, OwnerID: h.store.OwnerID()})
}
+53 -4
View File
@@ -87,6 +87,11 @@
--moss: #7fae86; /* finished */
--clay: #b5906f; /* set chapter */
--trash: #977671; /* remove, resting — icons need 3:1, not 4.5:1 */
/* A Lane needing attention: the admin page's only accent. Verdigris — cool,
the far side of the wheel from ember's crimson, and clear of the archive
blue. Neither ember (new chapter) nor danger (destruction) may say
"system unhealthy". */
--patina: #5fb3a6;
/* Desktop cell borders for the two coloured action states. */
--play-hot-line: #3a1d18;
@@ -146,6 +151,7 @@
--moss: #3d6c46;
--clay: #7c5533;
--trash: #8c6558;
--patina: #1f6f66;
--play-hot-line: #f0cfc6;
--fav-line: #e3d3a4;
@@ -290,9 +296,21 @@ button { cursor: pointer; }
letter-spacing: .04em;
}
/* ---- reader roster (owner only): same hairline panel, one row per Reader ---- */
.readerlist { margin: 0; padding: 0; list-style: none; }
.readerlist li {
/* ---- admin page: two sections on the same measured sheet, no cards ----
The reading page is a list of series; this is a list of facts. Both are
sheets of hairline-separated rows, so the roster keeps the shape it had as
a fold-out and the Lane block copies it. */
.readers, .lanes { margin: 0 20px; padding: 12px 0 16px; border-bottom: 1px solid var(--rule); }
.readers h2, .lanes h2 {
margin: 0;
padding: 8px 0;
font: 500 10px/1 var(--font-mono);
letter-spacing: .2em;
text-transform: uppercase;
color: var(--mute-2);
}
.readerlist, .lanelist { margin: 0; padding: 0; list-style: none; }
.readerlist li, .lanelist li {
display: flex;
align-items: center;
flex-wrap: wrap;
@@ -300,7 +318,8 @@ button { cursor: pointer; }
min-height: 44px;
border-top: 1px solid var(--rule);
}
.readerlist form { margin: 0 0 0 auto; }
.reader-actions { display: flex; gap: 18px; margin-left: auto; }
.readerlist form { margin: 0; }
.reader-id {
font: 500 13px/1.4 var(--font-mono);
letter-spacing: .04em;
@@ -312,6 +331,30 @@ button { cursor: pointer; }
text-transform: uppercase;
color: var(--mute);
}
.reader-sightings, .lane-fact {
font: 500 10px/1 var(--font-mono);
letter-spacing: .14em;
text-transform: uppercase;
color: var(--mute-2);
}
/* Two states the owner is meant to find rather than read for: a Reader whose
reports no longer defer a Poll, and a Lane that is not keeping its promise.
Both wear --patina — never ember, which means one thing, and never danger,
which is destruction. */
.reader-blocked, .lane-mark {
font: 500 10px/1 var(--font-mono);
letter-spacing: .14em;
text-transform: uppercase;
color: var(--patina);
}
.lane-site {
font: 400 19px/1.2 var(--font-display);
color: var(--paper-dim);
}
/* The whole row leans patina when the Lane needs attention, so the scan is one
pass down the left edge rather than a read of every mark. */
.lanelist li.attention .lane-site { color: var(--patina); }
.lane-browser { padding: 12px 0 0; }
/* Revocation cuts someone off, so it wears --danger. Ember stays reserved for
the new-chapter signal. */
.ghost.danger { color: var(--danger); }
@@ -600,6 +643,9 @@ button { cursor: pointer; }
box-shadow: inset 0 -2px 0 var(--ember);
}
.topbar form { margin-left: 18px; }
/* The admin page's topbar has no switch to fill the middle, so its back link
keeps company with Log out at the right edge instead of floating centre. */
.topbar .back { margin-left: auto; }
/* At phone width brand + switch + Log out do not fit on one line, so the
switch takes its own row under the wordmark rather than pushing Log out
off-screen. */
@@ -609,6 +655,9 @@ button { cursor: pointer; }
.libswitch { order: 3; margin-left: 0; }
.libswitch a { flex: 1; text-align: center; padding: 8px 14px; }
.topbar form { margin-left: 12px; }
/* The admin page has no switch to take the second row, so its brand claims
the first outright and the back link keeps Log out company below. */
.topbar:has(.back) .brand { flex: 1 1 100%; }
}
/* ---- action strip: full-width on a phone, hairline-divided cells ---- */
+39
View File
@@ -0,0 +1,39 @@
{{/* The owner's administrative page: everything that reaches past one Reader,
at its own address so it can be bookmarked rather than hunted for inside
the reading page. Owner-only at route registration (requireOwner), which
is why nothing in here re-tests who is asking. */}}
{{define "admin"}}
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
<meta name="color-scheme" content="dark light">
<title>BookmarkManager — Admin</title>
<link rel="icon" href="/static/logo.svg" type="image/svg+xml">
<link rel="stylesheet" href="/static/style.css">
<link rel="preload" href="/static/fonts/instrument-serif-400-latin.woff2" as="font" type="font/woff2" crossorigin>
<script src="/static/htmx.min.js" defer></script>
</head>
<body>
<div class="sheet">
<header class="topbar">
<h1 class="brand">{{template "mark" .}}<span>Bookmark<em>Manager</em></span></h1>
{{/* Back to the library, no switch: this page belongs to neither library,
and the ember-lit switch says which library you are reading. */}}
<a class="ghost back" href="/">Library</a>
<form method="post" action="/logout">
<button type="submit" class="ghost">Log out</button>
</form>
</header>
{{/* The live region wraps the swapped block rather than being it: the
refresh replaces the section wholesale, and a region recreated on every
update is never announced. */}}
<div aria-live="polite">{{template "lanes" .Lanes}}</div>
{{template "readers" .}}
</div>
</body>
</html>
{{end}}
+4 -2
View File
@@ -28,6 +28,10 @@
<a href="/?lib=novel&amp;tab=all" class="{{if eq .Lib "novel"}}active{{end}}"
{{if eq .Lib "novel"}}aria-current="page"{{end}}>Novels</a>
</nav>
{{/* The owner's only difference on this page: a link out to the
administrative one. It sits beside Log out rather than in the library
switch — that switch says which library, not which page. */}}
{{if .Owner}}<a class="ghost" href="/admin">Admin</a>{{end}}
<form method="post" action="/logout">
<button type="submit" class="ghost">Log out</button>
</form>
@@ -76,8 +80,6 @@
{{template "setup" .}}
{{if .Owner}}{{template "readers" .}}{{end}}
{{template "keyrow" .}}
{{template "recent" .}}
+42
View File
@@ -0,0 +1,42 @@
{{/* Poll Lane status: one row per Site, refreshing itself so a run can be
watched rather than sampled by reloading. The refresh is one attribute on
the fragment root and the endpoint answers with this same fragment, so the
swap replaces the element that asked for it.
Every figure here is read out of the running poller, never out of a table:
a Site absent from Rows has not completed a pass since the last restart,
which the empty state must say — zeroes would read as a stopped Lane. */}}
{{define "lanes"}}
<section class="lanes" id="lanes"
hx-get="/ui/admin/lanes" hx-trigger="every 30s" hx-swap="outerHTML">
<h2>Poll Lanes</h2>
{{if .Rows}}
<ul class="lanelist">
{{range .Rows}}
<li{{if .Attention}} class="attention"{{end}}>
<span class="lane-site">{{.Site}}</span>
<span class="lane-fact">{{.Due}} due</span>
<span class="lane-fact">{{.Checked}} checked</span>
<span class="lane-fact">ran {{.Ran}}</span>
{{if .Gap}}<span class="lane-fact">gap {{.Gap}}</span>{{end}}
{{if .Clamped}}<span class="lane-mark">gap at floor</span>{{end}}
{{if .Refusing}}<span class="lane-mark">refusing</span>{{end}}
{{if .BrowserLost}}<span class="lane-mark">no browser</span>{{end}}
{{if .Stalled}}<span class="lane-mark">not checking</span>{{end}}
{{if .Asleep}}<span class="lane-mark">browser asleep</span>{{end}}
</li>
{{end}}
</ul>
{{else}}
<p class="setup-copy">No data yet — no Lane has completed a pass since the
backend started.</p>
{{end}}
<p class="setup-copy lane-browser">
{{if .PollerOff}}Polling is switched off in this deployment: no Lane runs,
and Latest Chapter comes from the userscripts alone.
{{else}}Browser sidecar:
{{if not .BrowserConfigured}}not configured — comix, kagane and novelfull
pages are not fetched through it{{else if .BrowserReachable}}reachable
{{else}}unreachable{{end}}.{{end}}</p>
</section>
{{end}}
+36 -17
View File
@@ -1,29 +1,48 @@
{{/* The owner's Reader roster. Rendered only for the owner (listView.Owner),
and re-rendered whole as the response to a revocation so the session
counts cannot describe the state before the tap. Revocation is
confirm-gated: it signs someone out of every device at once. */}}
{{/* The Reader roster, on the owner's administrative page. Re-rendered whole
as the response to an action so the counts and marks it shows cannot
describe the state before the tap. Both actions are confirm-gated: one
signs a Reader out of every device at once, the other wipes a record.
Owner-only at route registration, so nothing here re-tests who is asking. */}}
{{define "readers"}}
<details class="setup" id="readers">
<summary>Readers</summary>
<section class="readers" id="readers">
<h2>Readers</h2>
<p class="setup-copy">Everyone who has signed in through Discord. Revoking
signs a Reader out of every device; their library and bookmarks are
untouched, and they can sign in again.</p>
untouched, and they can sign in again. The Sighting counters record how
often a later Poll confirmed or contradicted what that Reader's browser
reported; enough contradictions stop their reports deferring a Poll, and
clearing the marks gives that back.</p>
<ul class="readerlist">
{{range .Readers}}
<li>
<span class="reader-id">{{.DiscordID}}</span>
<span class="reader-sessions">{{.Sessions}} session{{if ne .Sessions 1}}s{{end}}</span>
{{/* The owner's own row never offers Revoke: it is the one row where the
button would sign the tapping browser out, and the endpoint refuses
it anyway. Logout is the deliberate way to do that. */}}
{{if and .Sessions (ne .ID $.OwnerID)}}
<form hx-post="/readers/{{.ID}}/revoke" hx-target="#readers" hx-swap="outerHTML"
hx-confirm="Revoking signs this Reader out on every device immediately. Revoke?">
<button type="submit" class="ghost danger">Revoke sessions</button>
</form>
{{end}}
<span class="reader-sightings">{{.Agreements}} confirmed / {{.Disagreements}} contradicted</span>
{{/* Blocked is spelled out rather than left to be worked out from two
numbers and a threshold. */}}
{{if .Blocked}}<span class="reader-blocked">deferral blocked</span>{{end}}
<span class="reader-actions">
{{/* Clearing restores a privilege, so it is a plain ghost button —
the destruction accent belongs to revocation alone. It is offered
on every row, including one reading zero: the remedy must be
findable before the counters climb, not after. */}}
<form hx-post="/readers/{{.ID}}/clear-marks" hx-target="#readers" hx-swap="outerHTML"
hx-confirm="Clearing wipes this Reader's whole Sighting record, confirmations included. Clear?">
<button type="submit" class="ghost">Clear marks</button>
</form>
{{/* The owner's own row never offers Revoke: it is the one row where the
button would sign the tapping browser out, and the endpoint refuses
it anyway. Logout is the deliberate way to do that. */}}
{{if and .Sessions (ne .ID $.OwnerID)}}
<form hx-post="/readers/{{.ID}}/revoke" hx-target="#readers" hx-swap="outerHTML"
hx-confirm="Revoking signs this Reader out on every device immediately. Revoke?">
<button type="submit" class="ghost danger">Revoke sessions</button>
</form>
{{end}}
</span>
</li>
{{end}}
</ul>
</details>
</section>
{{end}}
+18 -56
View File
@@ -50,6 +50,9 @@ type Handler struct {
// httpClient is the plain stdlib client that talks to Discord. It is not
// an injected interface: tests point APIBase at a stub server instead.
httpClient *http.Client
// lanes is the Poll Lane snapshot source the administrative page reads.
// Nil is a running deployment with no poller, not a bug.
lanes LaneReporter
}
// listView is what every list-rendering template receives.
@@ -77,14 +80,9 @@ type listView struct {
// It is not "newly registered": a Reader who deletes their last bookmark is
// in the same position and needs the same links.
EmptyLibrary bool
// Owner marks the acting Reader as the deployment's owner, which unlocks
// the Readers panel. Nothing else in the UI differs.
// Owner marks the acting Reader as the deployment's owner, which offers
// the link to the administrative page. Nothing else in the UI differs.
Owner bool
// Readers is the owner's roster, populated only for the owner's own page
// render and the revocation fragment. OwnerID travels with it so the roster
// can tell the owner's own row apart from the Readers they may revoke.
Readers []store.ReaderSummary
OwnerID int64
}
// PageURL and ListURL are the two link shapes every tab needs. Building them
@@ -111,7 +109,10 @@ type loginView struct {
// New parses every template up front so a broken one kills the process at
// startup rather than the first request that touches it.
func New(s *store.Store, discord DiscordConfig, tokenKey []byte, mangaPath, novelPath string) (*Handler, error) {
//
// lanes is the administrative page's window onto the running Poller; nil means
// nothing is polling, which the page reports rather than hides.
func New(s *store.Store, discord DiscordConfig, tokenKey []byte, mangaPath, novelPath string, lanes LaneReporter) (*Handler, error) {
tmpl, err := template.ParseFS(templateFS, "templates/*.html")
if err != nil {
return nil, err
@@ -126,6 +127,7 @@ func New(s *store.Store, discord DiscordConfig, tokenKey []byte, mangaPath, nove
states: newOAuthStates(),
limiter: session.NewLoginLimiter(),
httpClient: &http.Client{Timeout: discordTimeout},
lanes: lanes,
}, nil
}
@@ -149,9 +151,11 @@ func (h *Handler) Register(mux *http.ServeMux) {
mux.HandleFunc("GET /install/manga-bookmark.user.js", h.requireSession(h.installUserscript("manga-bookmark.user.js")))
mux.HandleFunc("GET /install/novel-bookmark.user.js", h.requireSession(h.installUserscript("novel-bookmark.user.js")))
mux.HandleFunc("POST /rotate-token", h.requireSession(h.rotateToken))
// Owner-only: the one place the UI crosses the Reader boundary.
mux.HandleFunc("POST /readers/{id}/revoke", h.requireSession(h.revokeReaderSessions))
// Owner-only: every route that reaches past the acting Reader is gated in
// one place, so a missing gate is visible in the route list.
for _, rt := range h.adminRoutes() {
mux.HandleFunc(rt.pattern, h.requireOwner(rt.handler))
}
}
// staticHandler serves the embedded assets. An hour, not longer: assets are
@@ -240,14 +244,9 @@ func (h *Handler) index(w http.ResponseWriter, r *http.Request) {
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if readerID == h.store.OwnerID() {
view.Owner, view.OwnerID = true, readerID
if view.Readers, err = h.store.Readers(); err != nil {
log.Printf("index readers: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
}
// The owner's page differs only by the link to the administrative page:
// the roster lives there now, so the page read every day is only reading.
view.Owner = readerID == h.store.OwnerID()
h.render(w, http.StatusOK, "app", view)
}
@@ -618,40 +617,3 @@ func (h *Handler) rotateToken(w http.ResponseWriter, r *http.Request) {
view := listView{Lib: store.KindManga, Rotated: true}
h.render(w, http.StatusOK, "setup", view)
}
// revokeReaderSessions logs one Reader out of every browser they are signed
// in on. Owner-only: it reaches across the Reader boundary every other handler
// respects, so the guard is a comparison against the seeded owner rather than
// a role a Reader could acquire. A non-owner gets 404 — the panel does not
// exist for them, so neither should the endpoint.
func (h *Handler) revokeReaderSessions(w http.ResponseWriter, r *http.Request) {
if readerOf(r) != h.store.OwnerID() {
http.NotFound(w, r)
return
}
target, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil {
http.Error(w, "bad reader id", http.StatusBadRequest)
return
}
// The owner is not one of the Readers this endpoint reaches: revoking
// themselves would sign out the browser making the request, which is what
// logout is for. The roster hides the button; this refuses the hand-rolled
// POST behind it.
if target == h.store.OwnerID() {
http.NotFound(w, r)
return
}
if err := h.store.DeleteReaderSessions(target); err != nil {
log.Printf("revoke sessions: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
readers, err := h.store.Readers()
if err != nil {
log.Printf("revoke sessions: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.render(w, http.StatusOK, "readers", listView{Owner: true, Readers: readers, OwnerID: h.store.OwnerID()})
}
+19 -8
View File
@@ -129,7 +129,10 @@ func loadConfig() Config {
// newRouter wires routes and middleware. CORS is the outermost layer so
// preflight OPTIONS short-circuits before auth; /bookmarks* is auth-protected,
// /healthz is public.
func newRouter(s *store.Store, cfg Config) http.Handler {
//
// lanes may be nil — polling disabled, or its client could not be built. The
// admin page reports that rather than pretending Lanes exist.
func newRouter(s *store.Store, cfg Config, lanes web.LaneReporter) http.Handler {
mux := http.NewServeMux()
h := &api.Handler{Store: s}
mux.HandleFunc("GET /healthz", api.Healthz)
@@ -160,7 +163,7 @@ func newRouter(s *store.Store, cfg Config) http.Handler {
// The browser UI is always registered; signing in is Discord OAuth, so
// there is no password to forget and no gate to leave unset.
wh, err := web.New(s, cfg.Discord, []byte(cfg.TokenKey),
cfg.UserscriptPath, cfg.NovelUserscriptPath)
cfg.UserscriptPath, cfg.NovelUserscriptPath, lanes)
if err != nil {
log.Fatalf("web handler: %v", err)
}
@@ -275,11 +278,17 @@ func main() {
}
s.OnSeriesCreated = acq.Acquire
}
startLatestPoller(pollCtx, s, cfg.LatestPoll, browser)
// A nil *Poller must not become a non-nil interface holding a nil pointer:
// the admin page tests the reporter for nil to decide whether anything is
// polling at all.
var lanes web.LaneReporter
if poller := startLatestPoller(pollCtx, s, cfg.LatestPoll, browser); poller != nil {
lanes = poller
}
srv := &http.Server{
Addr: ":" + cfg.Port,
Handler: newRouter(s, cfg),
Handler: newRouter(s, cfg, lanes),
ReadHeaderTimeout: 10 * time.Second,
}
@@ -326,16 +335,17 @@ func newLatestPoller(s *store.Store, cfg LatestPoll, fetch, browser latest.Fetch
// startLatestPoller launches the background poller unless it is disabled or its
// HTTP client cannot be built. Any problem here is logged and skipped: this
// feature going missing degrades the service to userscript-only latest-chapter
// tracking, which is exactly how it behaved before.
func startLatestPoller(ctx context.Context, s *store.Store, cfg LatestPoll, browser latest.Fetcher) {
// tracking, which is exactly how it behaved before. It returns the running
// Poller, or nil when there is none — the admin page's Lane status reads it.
func startLatestPoller(ctx context.Context, s *store.Store, cfg LatestPoll, browser latest.Fetcher) *latest.Poller {
if !cfg.Enabled {
log.Println("latest-chapter poller: disabled by config")
return
return nil
}
f, err := latest.NewTLSFetcher()
if err != nil {
log.Printf("latest-chapter poller: disabled, cannot build client: %v", err)
return
return nil
}
// Nil browser: sites behind a JavaScript challenge are simply not polled,
// and their latest_chapter comes from the userscript alone — which is how
@@ -343,4 +353,5 @@ func startLatestPoller(ctx context.Context, s *store.Store, cfg LatestPoll, brow
p := newLatestPoller(s, cfg, f, browser)
go p.Run(ctx)
return p
}
+3 -3
View File
@@ -49,7 +49,7 @@ func withBody(req *http.Request, body string) *http.Request {
// A refused credential is refused however plausible it looks: only a hash the
// readers table holds authenticates anything.
func TestUnknownCredentialRejected(t *testing.T) {
srv := newRouter(newTestStore(t), testConfig())
srv := newRouter(newTestStore(t), testConfig(), nil)
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", readerCredential("never-registered")))
@@ -69,7 +69,7 @@ func TestUnknownCredentialRejected(t *testing.T) {
func TestPerReaderIsolation(t *testing.T) {
s := newTestStore(t)
registerReader(t, s, "other-reader")
srv := newRouter(s, testConfig())
srv := newRouter(s, testConfig(), nil)
ownerKey := "asura:solo"
putBookmark(t, srv, ownerKey, store.Bookmark{
@@ -267,7 +267,7 @@ func TestRotateCredentialViaWebUI(t *testing.T) {
}
cfg := testConfig()
cfg.UserscriptPath = path
srv := newRouter(s, cfg)
srv := newRouter(s, cfg, nil)
oldCred := ownerCredential()
rr := httptest.NewRecorder()
+277 -16
View File
@@ -1,6 +1,7 @@
package main
import (
"database/sql"
"encoding/json"
"fmt"
"io"
@@ -15,6 +16,7 @@ import (
"testing"
"time"
"bookmarkmanager/backend/internal/latest"
"bookmarkmanager/backend/internal/session"
"bookmarkmanager/backend/internal/store"
"bookmarkmanager/backend/internal/web"
@@ -26,11 +28,17 @@ import (
const testOwnerID = "owner-snowflake"
// newWebTestServer returns the full router plus the store behind it, so tests
// can seed rows and assert on what the handlers wrote back.
func newWebTestServer(t *testing.T, cfg Config) (http.Handler, *store.Store) {
// can seed rows and assert on what the handlers wrote back. An optional lane
// reporter stands in for the running poller; omitted means none is running,
// which is what every test that is not about the admin page wants.
func newWebTestServer(t *testing.T, cfg Config, lanes ...web.LaneReporter) (http.Handler, *store.Store) {
t.Helper()
st := newTestStore(t)
return newRouter(st, cfg), st
var reporter web.LaneReporter
if len(lanes) > 0 {
reporter = lanes[0]
}
return newRouter(st, cfg, reporter), st
}
// sessionCookie mints a live session row for the owner and returns the cookie
@@ -141,12 +149,12 @@ func discordConfig(stubURL string) web.DiscordConfig {
// oauthWebTestServer returns the full router, its store, and a Discord stub
// wired as the configured API — the starting point for sign-in tests.
func oauthWebTestServer(t *testing.T) (http.Handler, *store.Store, *discordStub) {
func oauthWebTestServer(t *testing.T, lanes ...web.LaneReporter) (http.Handler, *store.Store, *discordStub) {
t.Helper()
stub, srv := newDiscordStub(t)
cfg := testConfig()
cfg.Discord = discordConfig(srv.URL)
router, st := newWebTestServer(t, cfg)
router, st := newWebTestServer(t, cfg, lanes...)
return router, st, stub
}
@@ -410,7 +418,7 @@ func TestDiscordLoginRefusesNonMember(t *testing.T) {
cfg.Discord = discordConfig(srv.URL)
cfg.Discord.RequiredRole = tc.require
st := newTestStore(t)
router := newRouter(st, cfg)
router := newRouter(st, cfg, nil)
rr := completeSignIn(t, router, startSignIn(t, router))
if rr.Code != http.StatusForbidden {
@@ -626,24 +634,52 @@ func TestOwnerRevokesAnotherReadersSessions(t *testing.T) {
}
}
// The owner's own page carries the roster; nobody else's does.
func TestOwnerSeesReadersPanel(t *testing.T) {
// fakeLanes is the admin page's poller stand-in: one fixed snapshot, so the
// page's tests need neither a poller nor a Site.
type fakeLanes struct{ status latest.Status }
func (f fakeLanes) LaneStatus() latest.Status { return f.status }
// The roster moved off the reading page onto its own address: the owner gets a
// link, everyone else gets nothing, and the page itself lists every Reader with
// the counters and the two controls.
func TestAdminPageCarriesRosterAndOwnerLink(t *testing.T) {
router, st, _ := oauthWebTestServer(t)
signInCookie(t, router)
theirCookie := signInCookie(t, router)
ownerCookie := sessionCookie(t, st)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(sessionCookie(t, st))
req.AddCookie(ownerCookie)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
body := rr.Body.String()
if !strings.Contains(body, `id="readers"`) {
t.Fatal("the owner's page lacks the Readers panel")
if strings.Contains(body, `id="readers"`) {
t.Error("the reading page still carries the roster; it belongs on /admin")
}
if !strings.Contains(body, testOwnerID) {
t.Fatalf("the roster does not list the registered Reader:\n%s", body)
if !strings.Contains(body, `href="/admin"`) {
t.Error("the owner's reading page offers no link to the admin page")
}
if !strings.Contains(body, "Revoke sessions") {
t.Fatal("the roster offers no revocation control for a signed-in Reader")
req = httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(theirCookie)
rr = httptest.NewRecorder()
router.ServeHTTP(rr, req)
if strings.Contains(rr.Body.String(), `href="/admin"`) {
t.Error("a non-owner was offered the admin link")
}
req = httptest.NewRequest(http.MethodGet, "/admin", nil)
req.AddCookie(ownerCookie)
rr = httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("GET /admin status = %d, want 200", rr.Code)
}
body = rr.Body.String()
for _, want := range []string{`id="readers"`, testOwnerID, "Revoke sessions", "Clear marks", "confirmed"} {
if !strings.Contains(body, want) {
t.Errorf("admin page lacks %q:\n%s", want, body)
}
}
// Exactly one revocable row: the other Reader's. The owner's own row carries
// the same session count and no button.
@@ -652,6 +688,231 @@ func TestOwnerSeesReadersPanel(t *testing.T) {
}
}
// Every administrative route is gated the same way, so the test walks the list
// the router registers rather than naming routes by hand: no session is 401,
// a signed-in non-owner is 404, and the address is not confirmed to either.
func TestAdminRoutesAreOwnerOnly(t *testing.T) {
router, st, _ := oauthWebTestServer(t)
theirCookie := signInCookie(t, router)
ownerCookie := sessionCookie(t, st)
target := strconv.FormatInt(st.OwnerID(), 10)
patterns := web.AdminPatterns()
if len(patterns) == 0 {
t.Fatal("no administrative routes to test")
}
for _, pattern := range patterns {
method, path, ok := strings.Cut(pattern, " ")
if !ok {
t.Fatalf("route pattern %q has no method", pattern)
}
path = strings.Replace(path, "{id}", target, 1)
for _, tc := range []struct {
name string
cookie *http.Cookie
want int
}{
{"no session", nil, http.StatusUnauthorized},
{"non-owner", theirCookie, http.StatusNotFound},
} {
req := httptest.NewRequest(method, path, nil)
if tc.cookie != nil {
req.AddCookie(tc.cookie)
}
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != tc.want {
t.Errorf("%s %s as %s: status = %d, want %d", method, path, tc.name, rr.Code, tc.want)
}
}
req := httptest.NewRequest(method, path, nil)
req.AddCookie(ownerCookie)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code == http.StatusUnauthorized {
t.Errorf("%s %s as the owner: status = 401, the gate rejects the owner", method, path)
}
}
}
// The Lane block reports what the poller says, and marks the Lanes that need
// attention — a clamped gap, a refusal, a Site whose pages can only be read
// through a sidecar that is not there, and a Lane with Series waiting that its
// last pass did not read.
func TestAdminPageShowsLaneStatus(t *testing.T) {
lanes := fakeLanes{latest.Status{
Lanes: []latest.LaneState{
{Site: "asura", Due: 12, Checked: 12, LastRun: time.Now().Add(-90 * time.Second), Gap: 40 * time.Second},
{Site: "kagane", Due: 3, Checked: 3, LastRun: time.Now().Add(-time.Minute), Gap: time.Minute, Browser: true},
{Site: "demonic", Due: 400, Checked: 400, LastRun: time.Now(), Gap: 8 * time.Second, Clamped: true},
{Site: "comix", Due: 7, LastRun: time.Now(), Gap: time.Minute, Browser: true},
},
BrowserConfigured: true,
BrowserReachable: true,
}}
router, st, _ := oauthWebTestServer(t, lanes)
req := httptest.NewRequest(http.MethodGet, "/ui/admin/lanes", nil)
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("GET /ui/admin/lanes status = %d, want 200", rr.Code)
}
body := rr.Body.String()
for _, want := range []string{"asura", "kagane", "12 due", "12 checked", "gap 40s", "ran 1m30s ago", "gap at floor", "not checking", "reachable"} {
if !strings.Contains(body, want) {
t.Errorf("lane status lacks %q:\n%s", want, body)
}
}
// Nothing is refusing and the sidecar is up, so neither mark may appear:
// a mark the owner cannot act on is worse than none.
for _, unwanted := range []string{"refusing", "no browser"} {
if strings.Contains(body, unwanted) {
t.Errorf("lane status marks %q on a healthy run:\n%s", unwanted, body)
}
}
}
// A browser Lane under both wake thresholds holds Chrome asleep (ADR-0005), so
// Series due with none checked is the design working, not a stopped Lane. The
// two must not render the same mark: "not checking" is the owner's cue to go
// looking, and spending it on the commonest healthy browser-Lane state trains
// them to ignore it.
func TestAsleepBrowserLaneIsNotMarkedStalled(t *testing.T) {
lanes := fakeLanes{latest.Status{
Lanes: []latest.LaneState{
{Site: "kagane", Due: 1, LastRun: time.Now(), Gap: 10 * time.Second, Browser: true, Asleep: true},
},
BrowserConfigured: true,
BrowserReachable: true,
}}
router, st, _ := oauthWebTestServer(t, lanes)
req := httptest.NewRequest(http.MethodGet, "/ui/admin/lanes", nil)
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
body := rr.Body.String()
if strings.Contains(body, "not checking") {
t.Errorf("an asleep browser Lane is marked as stalled:\n%s", body)
}
if !strings.Contains(body, "browser asleep") {
t.Errorf("an asleep browser Lane says nothing about why it read nothing:\n%s", body)
}
if strings.Contains(body, `class="attention"`) {
t.Errorf("an asleep browser Lane is coloured as unhealthy:\n%s", body)
}
}
// A Lane whose pass never reached a figure must not have that figure drawn as
// a zero: a refusing Lane still reports the due count and gap its last real
// pass saw, and a Lane that has never reached one omits it entirely.
func TestLaneStatusOmitsUnknownGap(t *testing.T) {
lanes := fakeLanes{latest.Status{
Lanes: []latest.LaneState{{Site: "comix", LastRun: time.Now(), Refusing: true, Browser: true}},
BrowserConfigured: true,
BrowserReachable: true,
}}
router, st, _ := oauthWebTestServer(t, lanes)
req := httptest.NewRequest(http.MethodGet, "/ui/admin/lanes", nil)
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
body := rr.Body.String()
if strings.Contains(body, "gap 0s") {
t.Errorf("a Lane with no pace yet states a zero gap:\n%s", body)
}
if !strings.Contains(body, "refusing") {
t.Errorf("a refusing Lane is not marked as such:\n%s", body)
}
}
// No poller and a poller that has not finished a pass both render "no data
// yet" rather than zeroes that read as a stopped backend — but they are not
// the same fact, so the page must not blame the sidecar when nothing polls.
func TestAdminPageWithoutAPollerSaysSo(t *testing.T) {
for _, tc := range []struct {
name string
lanes []web.LaneReporter
want, unwant string
}{
{"no poller", nil, "Polling is switched off", "not configured"},
{"poller, no pass yet", []web.LaneReporter{fakeLanes{}}, "not configured", "Polling is switched off"},
} {
t.Run(tc.name, func(t *testing.T) {
router, st, _ := oauthWebTestServer(t, tc.lanes...)
req := httptest.NewRequest(http.MethodGet, "/admin", nil)
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
body := rr.Body.String()
if !strings.Contains(body, "No data yet") {
t.Errorf("admin page with no Lane data does not say so:\n%s", body)
}
if !strings.Contains(body, tc.want) {
t.Errorf("admin page lacks %q:\n%s", tc.want, body)
}
if strings.Contains(body, tc.unwant) {
t.Errorf("admin page states %q, which is not what is wrong:\n%s", tc.unwant, body)
}
})
}
}
// A Reader past the disagreement threshold is rendered as blocked, and
// clearing their marks both zeroes the counters and lifts the block in the
// roster the response carries back.
func TestOwnerClearsReaderMarks(t *testing.T) {
st, dsn := newTestStoreURL(t)
router := newRouter(st, testConfig(), nil)
cookie := sessionCookie(t, st)
// The counters are filled by issue #103; until it lands the only way to
// stand a marked Reader up is to write the columns directly.
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
if _, err := db.Exec(`UPDATE readers SET sighting_agreements = 4, sighting_disagreements = 3 WHERE id = $1`, st.OwnerID()); err != nil {
t.Fatalf("mark reader: %v", err)
}
req := httptest.NewRequest(http.MethodGet, "/admin", nil)
req.AddCookie(cookie)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
body := rr.Body.String()
if !strings.Contains(body, "4 confirmed / 3 contradicted") {
t.Errorf("roster does not report the Reader's marks:\n%s", body)
}
if !strings.Contains(body, "deferral blocked") {
t.Errorf("a Reader at the threshold is not rendered as blocked:\n%s", body)
}
req = httptest.NewRequest(http.MethodPost,
"/readers/"+strconv.FormatInt(st.OwnerID(), 10)+"/clear-marks", nil)
req.AddCookie(cookie)
rr = httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("clear marks: status = %d, want 200 (body %s)", rr.Code, rr.Body.String())
}
body = rr.Body.String()
if !strings.Contains(body, `id="readers"`) {
t.Fatalf("clear marks did not re-render the roster:\n%s", body)
}
if !strings.Contains(body, "0 confirmed / 0 contradicted") {
t.Errorf("roster does not report the cleared counters:\n%s", body)
}
if strings.Contains(body, "deferral blocked") {
t.Errorf("a cleared Reader is still marked blocked:\n%s", body)
}
}
func TestDiscordLoginTokenEndpointDown(t *testing.T) {
stub, srv := newDiscordStub(t)
stub.tokenStatus = http.StatusInternalServerError
@@ -0,0 +1,139 @@
# ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them
Date: 2026-08-16
Status: accepted
## Decision
A **Sighting** is the Latest Chapter the Reader's own browser read off the
Series page and PUT to the backend. It is now allowed to stand in for a Poll,
under one restriction and one ceiling:
- **Solitary Series only.** A Sighting defers the Poll of a Series exactly one
Bookmark points at. A Series two Readers share is Polled on schedule no matter
how recently it was sighted.
- **One rest of standing.** A Sighting postpones Polls for one Rest
(`defaultRest`, an hour), not forever: a Series nobody visits again returns to
the normal schedule by itself.
- **Six-rest ceiling.** `sightingCeilingRests = 6`, counted in the Site's own
Rest — six hours everywhere today. However many Sightings arrive, a Series
unpolled that long is Polled.
Both live in the due query's HAVING clause (`store.DueForLatestCheck`), beside
the Rest cutoff — the same place the schedule has always been decided, so no
timer and no second code path can disagree with it.
Attribution and judgement:
- `Store.RecordSighting` runs *before* the Upsert that stores the reported
value, because the raise test needs the row as it stands. A report that raises
the stored Latest Chapter names its Reader in `series.latest_raised_by`.
- The Poll is the oracle. `Poller.checkOne` already compares what the Site
publishes against what is stored, so judgement costs no extra request: a lower
number contradicts the Sighting (`sighting_disagreements + 1`, both numbers and
the Reader logged), the same number confirms it (`sighting_agreements + 1`), a
**higher** number is the Site publishing and means nothing either way — but it
does clear the attribution (`Store.ClearSightingAttribution`), because the
value stored afterwards is the Poll's own and nobody must answer for it.
- At `SightingDisagreementLimit` (3) that Reader's Sightings stop deferring
anything. They still write the Latest Chapter — the penalty removes a
privilege, it does not silence anyone.
- `SightingAgreementsToClear` (20) consecutive confirmations forgive the
disagreements. A disagreement resets the run to zero.
- The owner clears marks from the administration page (issue #102, shipped
first precisely so a false mark has a remedy the day the mechanism lands).
One client change was required, and only one. Both userscripts stopped short of
PUTting a read whose number had not moved (`applyLatestChapterIfChanged`), so
the case this whole mechanism exists for — visiting a Series with nothing new —
never reached the backend. `reportLatestChapter` now sends it, skipping only the
local write and the re-render. A numberless PUT (favourite toggle, progress from
a chapter page) is not a Sighting and defers nothing: nobody read the Series
page, so there would be nothing to judge later.
## Why
Most of the backend's work was redundant. The userscript reads the Latest
Chapter on every Series page visit; minutes later the Poll Lane fetches the same
page for the same number. Deferring on a report converts a visit into a Poll
saved, which is Lane capacity handed back to Series nobody is
reading.
The restriction is the whole safety argument, and it is about **blast radius**,
not about trust arithmetic:
- On a solitary Series, a wrong report can only mislead the Reader who made it.
There is nobody else's ember to falsify.
- On a shared Series it could mislead someone else, so a report never postpones
anything there.
The ceiling bounds the damage in time: a false value dies within six hours
whatever happens, because the Poll that finds it is guaranteed. That is also
what makes lying pointless — the six-hour audit is certain, not sampled, so a
determined attacker buys at most three ceilings' worth of a wrong number on
their own Series and then loses deferral entirely.
The cost of recovery is deliberate. An agreement is only recorded when a later
Poll confirms a Sighting, so twenty agreements are twenty Polls of Series that
Reader bookmarks — hours to days of real time, not twenty page views. Waiting is
therefore not a strategy, and credit cannot be banked in advance.
## Tradeoffs and rejections
- **Trusting a Sighting on a shared Series** rejected: it is the only case where
one Reader's mistake reaches another Reader's list, and no amount of
reputation makes that recoverable within the six-hour window.
- **Cross-Reader agreement, voting, weighting, consensus scoring** rejected on
evidence: every truth-discovery method estimates source reliability by
comparing sources on the same object, and the standard survey states outright
that an object provided by very few sources cannot have its confidence
evaluated — Li, Gao, Meng, Li, Su, Zhao, Fan, Han, *A Survey on Truth
Discovery*, SIGMOD Record 45(1), 2016 (arXiv:1505.02463), §"Challenges" on
sparse sources. With the two Readers this backend actually has, a
disagreement is a coin flip. The Poll is an authoritative oracle, so it is
the only judge.
- **A randomised audit** (Poll a fraction of deferred Series) rejected in favour
of the fixed ceiling. Sampling an oracle against untrusted reports is the
gold-question technique from crowdsourcing quality control — Le, Edmonds,
Hester, Biewald, *Ensuring quality in crowdsourced search relevance
evaluation: the effects of training question distribution*, SIGIR 2010
Workshop on Crowdsourcing for Search Evaluation, which inserts known answers
sporadically and adjusts each worker's trust from them. The ceiling is the
same idea made deterministic: sampling prices an attack in expectation, a
guaranteed six-hour audit prices it as a certainty, which is what makes the
solitary-Series rule defensible in one sentence.
- **A trust *ratio*** (agreements over judgements, as that same gold-question
scheme uses) rejected for two thresholds: a ratio lets an attacker bank
credit first and spend it on lies later, and it needs the owner watching a
score to act. Three-and-twenty is a threshold both ways — a disagreement
resets the run to zero, so credit cannot be pre-bought, and recovery happens
without the owner in the loop.
- **Blocking a marked Reader's writes** rejected: the Latest Chapter they report
is still the best available value, and their Sightings must keep being judged
or they could never earn the privilege back.
- **Per-Series flagging** rejected in favour of per-Reader marks: a Series is
not the thing that can be wrong. Naming the Reader and logging both numbers is
also what distinguishes a broken Site adapter (every Reader of that Site
contradicted at once) from one bad actor.
- **Timers or a background reputation job** rejected: deferral is recomputed
from live facts every round — Bookmark count and sighting timestamp — so a
Series that gains a second Bookmark stops deferring at once, with nothing to
invalidate. The Reader's marks are the one input read earlier, when the
Sighting is recorded rather than when the round runs: a Reader who crosses
the threshold, or has their marks cleared, changes behaviour from their next
Sighting on, and the standing they already bought lasts out its rest. That is
bounded by one rest and costs one subselect instead of joining `readers` into
the due query on every round.
## Constraints preserved
- A Sighting is not Progress: it may move the Latest Chapter and nothing else.
`updated_at` never moves, so a report cannot reorder the list (ADR-0004).
- The Latest Chapter is a Series-level fact (ADR-0003): a Sighting writes the
shared row, so every Reader of a shared Series sees it immediately — deferral
is the only thing the solitary rule withholds.
- Ember means new chapter only (`docs/design-system.md`): a marked Reader
renders no differently in their own list, and nothing about the trust model
reaches the Series list's colour.
- The Poll remains authoritative. Where a Sighting and a Poll disagree, the
Poll's value is what gets stored.
+17 -4
View File
@@ -10,7 +10,7 @@ Implemented in:
| Surface | Files |
| --- | --- |
| Web UI (login, list, card, empty, errors) | `backend/internal/web/static/style.css`, `backend/internal/web/templates/{app,card,list,login,chrome,icons}.html`, `backend/internal/web/static/filter.js` |
| Web UI (login, list, card, empty, errors, admin) | `backend/internal/web/static/style.css`, `backend/internal/web/templates/{app,admin,lanes,readers,card,list,login,chrome,icons}.html`, `backend/internal/web/static/filter.js` |
| Userscript panel (Shadow DOM) | `userscript/manga-bookmark.user.js` — `TEMPLATE` and `CSS` at the bottom of the IIFE |
## 1. The one idea
@@ -73,6 +73,7 @@ Defined once in `backend/internal/web/static/style.css` `:root`, mirrored in the
| `--moss` | `#7fae86` | `#3d6c46` | finished accent |
| `--clay` | `#b5906f` | `#7c5533` | set-chapter accent |
| `--trash` | `#977671` | `#8c6558` | remove, at rest — icons need 3:1, not 4.5:1 |
| `--patina` | `#5fb3a6` | `#1f6f66` | admin page only — a Poll Lane needing attention, a Reader whose reports are blocked |
| `--play-hot-line` | `#3a1d18` | `#f0cfc6` | desktop cell border, play when `.is-new` |
| `--fav-line` | `#332b14` | `#e3d3a4` | desktop cell border, favourite when on |
| `--asura` | `#7d93a5` | `#4f6b80` | site tag |
@@ -81,9 +82,13 @@ Defined once in `backend/internal/web/static/style.css` `:root`, mirrored in the
| `--kagane` | `#9a8aa5` | `#6f5f7d` | site tag |
| `--hatch` / `--hatch-dim` | 135° 5px stripe | paper stripe | missing-cover slot |
`--slate`/`--moss`/`--clay`/`--brass` are held at the same weight deliberately:
one accent per action, so a press says which lane it belongs to, with none of
them competing with ember. Dark is the default (`color-scheme: dark light`);
`--slate`/`--moss`/`--clay`/`--brass`/`--patina` are held at the same weight
deliberately: one accent per meaning, so a press says which lane it belongs to,
with none of them competing with ember. `--patina` is the admin page's only
colour — a cool verdigris, the far side of the wheel from ember's crimson and
clear of the archive blue: system health is neither a new chapter nor
destruction, so it borrows neither `--ember` nor `--danger`.
Dark is the default (`color-scheme: dark light`);
light is a `@media (prefers-color-scheme: light)` override of the same names.
**Any new colour must be added in both branches** — light is not a filter over
dark, the hues are re-tuned.
@@ -140,6 +145,14 @@ Recurring specs (copy these rather than inventing sizes):
main#list article.card … | .empty
```
The owner's admin page (`admin.html`) is the same sheet with two sections in
place of the list — `.lanes` (Poll Lane rows) and `.readers` (the roster) —
and no library switch: it belongs to neither library, so its topbar carries a
plain `.ghost.back` link home. Both sections are eyebrow + hairline-separated
rows, the shape the roster already had as a fold-out. `.lanes` refreshes itself
every 30s via `hx-get="/ui/admin/lanes"` with `hx-swap="outerHTML"`; the roster
re-renders only in answer to an action.
**Brand mark**: an inline `<svg class="mark">` (`viewBox="0 0 200 172"`),
defined once in `chrome.html`'s `mark` template and reused by `app.html` and
`login.html` so it takes the page's `--ink`/`currentColor`/`--ember` rather
+344
View File
@@ -0,0 +1,344 @@
# GIF — maximum byte size of a file
Research note for Gitea issue #71 (backend `maxBodyBytes` = 4 MiB rejects the
8,571,192-byte animated cover GIF at
`https://cdn.asurascans.com/asura-images/covers/a-dragonslayers-peerless-regression.gif`).
All facts fetched live on **2026-08-17**: the GIF89a spec at
`https://www.w3.org/Graphics/GIF/spec-gif89a.txt`, Go stdlib `image/gif`
sources at `/usr/local/go/src/image/gif/reader.go` (Go 1.26.5), Chromium
`blink/renderer/platform/image-decoders/` sources via
`chromium.googlesource.com`, Firefox `image/decoders/nsGIFDecoder2.cpp` via
`hg.mozilla.org`, and cover bytes probed with plain `curl` (desktop Chrome UA;
`HEAD`/ranged `GET`). **No Cloudflare challenge was encountered on any CDN
probe** — every request returned real headers, consistent with the AGENTS.md
note of 2026-07-26 that plain `curl` works against both scan sites from the
dev machine and the VPS.
Every claim carries the URL it came from, or a reproducible command.
Interpretation rather than observation is marked `[INFERENCE]`.
---
## 1. Summary answer table
| Question | Answer | Evidence |
|---|---|---|
| Does the GIF89a spec define a maximum file size? | **No.** There is no file-size field anywhere in the format; the only numeric ceilings are per-field (16-bit screen/image dimensions, 255-byte sub-blocks, 12-bit LZW codes). | §2 |
| Maximum logical screen | 65535 × 65535 pixels (unsigned 16-bit width/height). | §2.1 |
| Number of frames / image descriptors | Unbounded — "An unlimited number of images may be present per Data Stream." | §2.2 |
| Formal max byte size of any single GIF | None. Single-frame worst case ≈ **6.44 GB** (12-bit LZW, max canvas); animated GIFs are **unbounded** because frames are unbounded. | §3 |
| Does the backend's decoder (Go `image/gif`) bound size? | **No.** It reads 16-bit dimensions and allocates `width×height` bytes per frame; a 65535² frame forces a ~4 GiB allocation. No total-size or dimension guard. | §4.1 |
| Do browsers bound on-disk GIF size? | Chromium and Firefox: no on-wire size cap in their GIF readers; Chromium caps *decoded* memory at min(4 B × pixels, platform budget). | §4.3, §4.4 |
| Real cover sizes (asurascans, n=25) | min 190,410 B · median 1,275,082 B · p90 4,524,788 B · max 8,571,192 B · **3/25 > 4 MiB** (two JPEGs and the animated GIF) | §5 |
| Real cover sizes (demonicscans/readermc, n=78) | min 13,298 B · median 63,061 B · max 801,200 B · 0/78 > 4 MiB | §5 |
| Comparable service caps | GitHub: 10 MB for images/GIFs. Discord API: default 10 MiB per file. Wikimedia: 100 MiB upload / 5 GiB host. | §6 |
| Recommended cover cap for #71 | **10 MiB** (separate from the 4 MiB series-page cap). Covers 100% of the 103 observed covers; matches GitHub/Discord calibration; ≤ 20 MiB worst-case transient per concurrent fetch+serve on a 1974 MiB swapless VPS. | §7 |
---
## 2. What the GIF89a specification actually bounds
Source: `https://www.w3.org/Graphics/GIF/spec-gif89a.txt` (fetched 2026-08-17).
### 2.1 Fixed-width fields — the only hard ceilings
The format is a stream of fixed-width blocks; the numeric fields that *do*
have a ceiling are all 16-bit unsigned, little-endian ("multi-byte numeric
fields are ordered Least Significant Byte first", §4 of the spec):
- **Logical Screen Width / Height** — "Unsigned" 2-byte fields (§18, Logical
Screen Descriptor) → maximum **65535 × 65535** pixels.
- **Image Left / Top Position, Image Width / Height** — "Unsigned" 2-byte
fields (§20, Image Descriptor). Each image "must fit within the boundaries
of the Logical Screen" (§20a), so an image cannot exceed the 65535² canvas
even though its own fields would allow it.
- **Data sub-blocks** — "A data sub-block may contain from 0 to 255 data
bytes" (§15); each sub-block is preceded by a 1-byte size field and the
stream is terminated by a 0x00 Block Terminator (§16). This bounds a
*chunk*, not the stream.
- **Global/Local Color Tables** — optional, "3 x 2^(Size of Global Color
Table+1)" bytes with a 3-bit size field → at most 3 × 2⁸ = **768 bytes**
each (§19, §21).
- **LZW codes** — "The output codes are of variable length, starting at
<code size>+1 bits per code, **up to 12 bits per code**. This defines a
maximum code value of 4095 (0xFFF)" (Appendix F, COMPRESSION, rule 4).
- **Trailer** — a single byte, fixed value 0x3B, "indicating the end of the
GIF Data Stream" (§27).
### 2.2 What is unbounded
- **Number of images (frames).** §20a, verbatim: "This block is REQUIRED for
an image. Exactly one Image Descriptor must be present per image in the
Data Stream. **An unlimited number of images may be present per Data
Stream.**"
- **The Data Stream itself.** The grammar in Appendix B is
`<GIF Data Stream> ::= Header <Logical Screen> <Data>* Trailer`, and the
spec states "the entity Data … may be repeated any number of times,
including 0 times." There is **no field anywhere that carries a file size,
byte count, frame count, or total-length value**. §13 (Block Sizes) only
defines sizes *within* blocks.
### 2.3 Verdict
**The GIF89a specification defines no maximum file size.** The only hard
bounds are per-field: 65535×65535 pixels per screen/image, 255 bytes per
sub-block, 12 bits per LZW code, and one trailer byte. A compliant decoder
must process whatever stream the blocks describe. Any byte ceiling a
particular GIF actually hits is therefore *implicit* — 16-bit dimensions,
LZW code width, decoder memory, or an external policy — never something the
format itself enforces. `[INFERENCE]` This is why real-world GIFs cap out at
"a few GB at most" and every service that wants a bound has to impose one
itself (see §6; Wikimedia explicitly documents that a 4 GiB host limit was a
storage-representation artifact of 32-bit integers, `phab:T191805`, not a
format limit).
---
## 3. Theoretical worst case
### 3.1 Single frame, maximal canvas, 8-bit pixels
| Quantity | Value | Derivation |
|---|---|---|
| Max pixels | 4,294,836,225 | 65535 × 65535 |
| Raw 8-bit palette-index raster | 4,294,836,225 B ≈ **4.29 GB / 4.00 GiB** | 1 byte per pixel (Table Based Image Data, §22; Go's `image.Paletted` uses exactly 1 byte/pixel) |
| LZW worst case | ≈ **6.44 GB / 6.00 GiB** | codes ≤ 12 bits each (Appendix F), at most ~1 code per pixel for incompressible data → ≤ 12 bits/px = 1.5 B/px → 4,294,836,225 × 1.5 B |
| Sub-block overhead | ≈ +25.3 MB | every ≤255-byte chunk carries a 1-byte size field (§15): ⌈6,442,254,338 / 255⌉ ≈ 25,263,743 size bytes, + 1 block terminator |
| Fixed overhead | ≈ +1.6 KB | header 6 B (§17) + logical screen descriptor 7 B (§18) + global color table ≤ 768 B (§19) + image descriptor 10 B (§20) + local color table ≤ 768 B (§21) + LZW minimum code size 1 B (§22) |
So a **single maximal-frame GIF cannot exceed ≈ 6.47 GB on the wire**
(12-bit LZW bound), and LZW being lossless means the real byte count depends
entirely on image content — the same canvas can be a few KB (flat color) or
~6 GB (noise).
Two caveats, both marked `[INFERENCE]`:
- The "1.5 B/px" figure assumes ~one emitted code per pixel. An encoder is
permitted to emit a Clear code at any point (Appendix F: "The Clear code
can appear at any point in the image data stream"), so a
pathological-but-compliant encoder emitting clear+pixel per pixel reaches
~24 bits/px ≈ 12.9 GB for the max canvas. Real encoders do not do this;
12-bit/px is the practical bound.
- The spec's deferred-clear note (cover sheet) explicitly allows an encoder
to keep using a full table at 12-bit codes without clearing, so the 12-bit
cap holds for the whole stream, it cannot "grow" past 12 bits.
### 3.2 Animated GIFs: unbounded
Every frame is one Image Descriptor, each bounded by the 65535² canvas, but
the *count* of frames is unbounded (§2.2). Total bytes = sum over frames —
therefore **there is no finite maximum byte size for an animated GIF** in
the format. The only thing that stops a real one is decoder memory, a
service cap, or disk space. `[INFERENCE]` This is the category the issue #71
cover falls into: it is an animated GIF (NETSCAPE2.0 loop extension found at
offset 0x310 of the file, verified 2026-08-17 by a ranged GET), and its
8,571,192 bytes are ~2.04× the current 4 MiB backend cap.
---
## 4. Decoder-side real limits
### 4.1 Go `image/gif` (the backend's decoder path, stdlib)
Source: `/usr/local/go/src/image/gif/reader.go`, Go 1.26.5.
- Dimensions are read as little-endian uint16 — `left/top/width/height :=
int(d.tmp[N]) + int(d.tmp[N+1])<<8` (reader.go:490-493) — so the format
ceiling 65535 applies, and nothing smaller is enforced.
- The only geometric check is that each frame fits inside the logical
screen: `if left+width > d.width || top+height > d.height` →
`errors.New("gif: frame bounds larger than image bounds")` (reader.go:512-513).
- **There is no file-size, byte-count, frame-count, or pixel-count guard.**
Each frame allocates `image.NewPaletted(...)` (reader.go:515) — a
`[]byte` of width×height — so decoding one legal 65535² frame attempts a
**~4.29 GB allocation**. `DecodeAll` (reader.go:603-605) additionally
retains every frame's `Pix` slice for the lifetime of the returned `*GIF`.
- `[INFERENCE]` On the 1974 MiB swapless VPS (root AGENTS.md), decoding such
a file would OOM rather than error cleanly; nothing in stdlib protects
the process. This matters for §7: the backend stores cover bytes without
decoding them (see §5.3), so the fetch path never triggers this — but any
future "validate/re-encode server-side" scheme would.
- Grep for `MaxInt|limit|too large|bounds` in reader.go: the only hits are
the frame-bounds check above and the `tmp [1024]byte` scratch buffer
(reader.go:109); no size caps exist.
### 4.2 giflib / libgif
**Not verified from source.** On 2026-08-17 the giflib sources were not
reachable from this network: `github.com/giflib/giflib` returns 404 (repo
gone/moved), `gitlab.com/giflib/giflib/-/raw/...` answers a Cloudflare
"Just a moment…" challenge, and the SourceForge project download path
404s. No limit claim about giflib is made here. `[INFERENCE]` giflib is
widely known to be allocation-driven with no dimension cap, but that is not
checked against source and is not needed for issue #71 (the backend uses Go
stdlib, not giflib).
### 4.3 Chromium (browser behaviour, first-party source)
- `third_party/blink/renderer/platform/image-decoders/gif/gif_image_reader.cc`
(via `chromium.googlesource.com/chromium/src/+/main/...`, fetched
2026-08-17): **no GIF byte-size or dimension cap found** — grep for
`max|limit|too large|dimension|65535|overflow` matches only license text.
- The base `ImageDecoder` caps *decoded memory*, not transfer size:
`CalculateMaxDecodedBytes` computes `min(4 * num_pixels, platform_max_decoded_bytes)`
(8 bytes/pixel for high-bit-depth), and the header comment says "Ignoring
this limit can cause excessive memory use or even crashes on low-memory
devices"
(`image_decoder.cc:94-117`, `image_decoder.h:545-549`). The GIF reader
itself is untouched by this — it is a decoded-buffer budget.
- Practical consequence `[INFERENCE]`: a browser will happily download and
store a multi-GB GIF from its own cache perspective; Chromium only limits
what it *decodes* into pixels.
### 4.4 Firefox
`image/decoders/nsGIFDecoder2.cpp` (via `hg.mozilla.org/mozilla-central/
raw-file/tip/...`, fetched 2026-08-17): **no dimension or size limit**; the
only guards are on LZW code width (`MAX_BITS` = 12, "maximum codeword size
of 12 bits") and the decode stack. Nothing bounds the on-disk byte size.
### 4.5 Summary
No mainstream decoder enforces a byte-size ceiling; they stop at the 16-bit
dimension ceiling (Go, by construction) or at decoded-memory budgets
(Chromium) or nowhere (Firefox). A GIF's byte size is policed only by
*storage* policies — which is what §6 calibrates and §7 sets.
---
## 5. Practical distribution — what real manga covers weigh
Probed **2026-08-17** with `curl -sI` (HEAD) and ranged GETs, desktop Chrome
UA. No Cloudflare block on any request. Sample = covers *as the backend
would fetch them* (the `og:image`/page-listed cover URL), not thumbnails we
chose by hand.
### 5.1 Exact commands
```sh
# asurascans.com — harvest cover URLs from the homepage, then HEAD each
curl -s -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/126.0" https://asurascans.com/ -o home.html
grep -oE 'https://cdn\.asurascans\.com/asura-images/covers/[^"&\\< ]+\.(webp|gif|jpg|jpeg|png)' home.html \
| sort -u | grep -v '\-400\.' | head -25 > sample.txt # one full-res cover per series, no -400 thumbs
while read -r u; do curl -s -A "…Chrome/126.0" -I "$u" | tr -d '\r' \
| grep -iE '^content-length:'; done < sample.txt
# demonicscans.org — covers live on readermc.org (ADR-0007), URLs contain spaces/UTF-8
curl -s -A "…Chrome/126.0" https://demonicscans.org/ -o demonic.html
grep -oE 'src="https://readermc\.org/images/thumbnails/[^"]+"' demonic.html | tr -d 'src="' > demonic.txt
# …plus og:image from 5 manga pages (Catastrophic-Necromancer, Magic-Emperor, …)
# each URL percent-encoded per path segment (urllib.parse.quote, safe=':/') before HEAD
```
### 5.2 asurascans — 25 full-res covers (mixed formats)
Homepage fetched 200 (664,700 B). All 25 returned `200` with a real
`Content-Length`. Distribution:
| Statistic | Bytes |
|---|---|
| n | 25 |
| min | 190,410 |
| median | 1,275,082 |
| p90 | 4,524,788 |
| max | 8,571,192 |
| mean | 1,943,651 |
| **> 4 MiB (4,194,304)** | **3 (12%)** — `a-dragonslayers-peerless-regression.gif` 8,571,192 (the issue #71 cover, animated: NETSCAPE2.0 at 0x310, 550×733, 256 colors); `bad-born-blood.3008f6.webp` 4,524,788 `image/jpeg`; `ending-maker.cfbf53.webp` 4,619,303 `image/jpeg` |
Notes: the CDN serves `Content-Type` by stored bytes, not by URL extension
(the `.webp` URLs return `image/png`, `image/jpeg`, or `image/webp` — the
sample spans all four of `png/jpeg/webp/gif`). Two of the three over-cap
files are **not GIFs**, so the current 4 MiB cap already silently drops 12%
of asura covers of any format. p90 itself (4.52 MB) exceeds the cap.
### 5.3 demonicscans — 78 covers on readermc.org
78 unique cover URLs (73 from the homepage's `/images/thumbnails/` plus 5
`og:image` values from manga pages — demonicscans publishes the thumbnail
file as the full cover, so that is exactly what the backend would fetch).
**78/78 returned 200 with a real Content-Length** (spaces and UTF-8 in the
filenames were percent-encoded per path segment; the homepage's raw HTML
carries `’`-style mojibake for curly quotes, which was repaired by
latin-1→utf-8 re-encoding before probing).
| Statistic | Bytes |
|---|---|
| n | 78 |
| min | 13,298 |
| median | 63,061 |
| p90 | 206,994 |
| max | 801,200 |
| mean | 110,038 |
| > 4 MiB | 0 |
### 5.4 Reading
`[INFERENCE]` asurascans covers are the heavy tail (median 1.3 MB, top
decile > 4 MiB, occasional ~5–9 MB), demonicscans covers are tiny (all
< 0.8 MB). A cover cap must be chosen against the *asura* distribution —
the 8.57 MB animated GIF is not a freak one-off outlier; the 90th
percentile already crosses 4 MiB and two JPEGs sit between 4.5–4.7 MB.
---
## 6. Comparable documented byte caps (first-party docs only)
| Service | Cap | Source (fetched 2026-08-17) |
|---|---|---|
| GitHub (issues/PR comments) | **10 MB for images and gifs**; 25 MB other files; 10/100 MB video | `https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/attaching-files` — "The maximum file size is: 10MB for images and gifs … 25MB for all other files" |
| Discord (API uploads) | default **10 MiB per file**, higher with Nitro / boost tier | `https://discord.com/developers/docs/reference#uploading-files` — "The file upload size limit applies to each file in a request. The default limit is `10 MiB` for all users" (help-center article `support.discord.com/hc/en-us/articles/115002935588` exists but answered 403 from this network on the probe date, so its figures were not verified here) |
| Wikimedia Commons | **100 MiB** upload limit; hosting up to **5 GiB**; GIF thumbnails limited to **100 megapixels**; prior 4 GiB host cap was a 32-bit storage artifact (phab:T191805) | `https://commons.wikimedia.org/wiki/Commons:Maximum_file_size` |
| MDN | nothing — MDN documents no byte-size limit for images; browsers impose none (see §4.3–4.4) | `[INFERENCE]` from absence in the platform docs read in §4 |
Calibration takeaway: two major platforms independently land on **~10 MB**
as the ceiling for an uploadable image/GIF (GitHub exactly 10 MB, Discord
exactly 10 MiB), with Wikimedia the outlier at 100 MiB/5 GiB because it is a
media *archive*. A 10 MiB cover cap is therefore squarely inside industry
normal.
---
## 7. Recommendation for issue #71
**Raise the cover cap to 10 MiB (10,485,760 B) — as a separate constant, not
by moving the shared one.**
Why:
- **Fits the measured reality.** The largest observed cover is 8,571,192 B
(the issue's animated GIF) = 82% of 10 MiB; 10 MiB covers **100% of the
103 sampled covers** and the *entire* asura distribution, including its
heavy tail. 4 MiB rejects 12% of asura covers (two of them plain JPEGs).
- **Matches industry calibration** (§6): GitHub 10 MB images/GIFs, Discord
10 MiB default. A 10 MiB cap is a number every engineer recognizes, and
it leaves ~18% headroom over the current worst observed file.
- **Costs little on the target hardware.** The backend buffers cover bytes
whole during fetch (`backend/internal/latest/cover.go`: `ContentLength >
maxBodyBytes` rejection at :155, then `io.ReadAll(io.LimitReader(…,
maxBodyBytes+1))` at :158) and loads the full body per `GET /covers/…`
(`backend/internal/api/handlers.go`, `Cover` → `w.Write(body)`). Worst
case per concurrent fetch **+** serve is therefore 2 × cap = 20 MiB; even
ten of each concurrently is ~200 MiB of a 1974 MiB swapless VPS (~10%),
and the browser unit (471 MiB, root AGENTS.md) is no longer on that box.
The 4 MiB series-page cap is *not* the issue — measured pages run
100 KB–1.2 MB (`backend/internal/latest/fetch.go` comment) — so keep it.
- **The cap is a separate knob.** Today one `const maxBodyBytes = 4 << 20`
(`backend/internal/latest/fetch.go:17`) gates *both* series pages and
covers (`cover.go` references it). Raising it wholesale would loosen the
page-side memory guard for no benefit; a cover-specific constant (e.g.
`maxCoverBytes = 10 << 20`) keeps the two policies independent. The fetch
already double-checks `ContentLength` and the post-`LimitReader` length,
so a larger constant changes nothing else.
Alternatives and their costs:
| Option | Cost |
|---|---|
| Keep 4 MiB | 12% of asura covers (incl. non-GIF JPEGs) never stored — current bug, silent missing covers. |
| 16 MiB cap | 2× headroom over the observed max for future GIFs; +60% worst-case transient memory vs 10 MiB; diverges from the GitHub/Discord 10 MB calibration. |
| Server-side re-encode / downscale covers | Requires decoding → Go `image/gif` allocates width×height per frame with **no guard** (§4.1); a legal 65535² GIF forces a ~4.29 GB allocation on a 1974 MiB swapless box — OOM, not an error. Also mutates bytes, which the store treats as immutable/content-addressed (ADR-0007). Highest risk, no upside at this scale. |
| No cap | Unbounded transient memory and disk; rejected outright. |
Decision is the user's; on the evidence, **10 MiB for covers, 4 MiB for
pages** is the defensible middle.
-224
View File
@@ -1,224 +0,0 @@
{
"0": "HTMX Library Internals",
"1": "Cover Fetch Test Helpers",
"2": "Manga Userscript Adapters",
"3": "Novel Userscript Adapters",
"4": "Series Acquisition Tests",
"5": "Bookmarks API Tests",
"6": "Storage Choice ADR",
"7": "Cover & Acquire Internals",
"8": "System Architecture Concepts",
"9": "Session Middleware",
"10": "Go Test Helpers",
"11": "Store Tests",
"12": "Bookmarks API Handler",
"13": "Web UI Handlers",
"14": "Go Error Handling",
"15": "CDP Browser Client",
"16": "Cloudflare bot scoring and poll cadence — what is actually documented",
"17": "Go Code Style Guide",
"18": "Agent Skills",
"19": "Store",
"20": "I/O Performance Patterns",
"21": "CPU Optimization",
"22": "Caching Patterns",
"23": "Browser Entrypoint",
"24": "Memory Allocation & GC",
"25": "Cover Fetcher Tests",
"26": "Open",
"27": "Find Skills Guide",
"28": "Allocation Patterns",
"29": "Observability & Alerting",
"30": "AGENTS.md",
"31": "Store",
"32": "Open",
"33": "Go Testing Guide",
"34": "Session Store",
"35": "Web UI Filter Logic",
"36": "Userscript Test Harness",
"37": "Product & Security Context",
"38": "novel-logic.test.js",
"39": "UI Critique 2026-07-26A",
"40": "UI Critique 2026-07-26B",
"42": "pgtest.go",
"43": "Issue Tracker & Triage",
"44": "Ticket Workflow",
"45": "Go Perf Alert Rules",
"46": "Userscript Display Logic",
"47": "Go Perf Skill Docs",
"48": "Login Page Art",
"49": "BookmarkManager Logo",
"50": "Skills CLI",
"51": "Skills Leaderboard",
"52": "Complex Condition Extraction",
"53": "Sentinel Errors",
"54": "errors.As Patterns",
"55": "errors.Is Patterns",
"56": "errors.Join Patterns",
"57": "Error Wrapping",
"58": "Single Error Handling",
"59": "SIMD Optimizations",
"60": "GOGC Tuning",
"61": "GOMEMLIMIT",
"62": "Bottleneck Decision Tree",
"63": "pprof Profiling",
"64": "Test Timeout Helper",
"65": "httptest Patterns",
"66": "testify Suite Pattern",
"67": "go:embed Fixtures",
"68": "clockwork Time Mocking",
"69": "testify Mocking",
"70": "t.ArtifactDir Helper",
"71": "Subtests Pitfall",
"72": "golang-benchmark Skill",
"73": "golang-concurrency Skill",
"74": "golang-ci Skill",
"75": "golang-database Skill",
"76": "golang-lint Skill",
"77": "testify Skill",
"78": "Build Tag Integration Tests",
"79": "Test Naming Convention",
"80": "UI Critique A Finding",
"81": "UI Critique B Finding",
"82": "P0 Overflow Bug",
"83": "P1 hx-indicator Gap",
"84": "golang-benchmark Skill (ext)",
"85": "golang-concurrency Skill (ext)",
"86": "golang-ci Skill (ext)",
"87": "golang-data-structures Skill (ext)",
"88": "golang-database Skill (ext)",
"89": "golang-design-patterns Skill (ext)",
"90": "golang-documentation Skill (ext)",
"91": "golang-gopls Skill (ext)",
"92": "golang-lint Skill (ext)",
"93": "golang-naming Skill (ext)",
"94": "golang-observability Skill (ext)",
"95": "golang-refactoring Skill (ext)",
"96": "golang-safety Skill (ext)",
"97": "golang-samber-oops Skill (ext)",
"98": "golang-samber-slog Skill (ext)",
"99": "golang-structs-interfaces Skill (ext)",
"100": "golang-troubleshooting Skill (ext)",
"101": "promql-cli Skill",
"102": "Backend Module",
"103": "bookmark-api Service",
"104": "AGENTS.md",
"105": "reviewer.md",
"106": "Redeploy runbook",
"107": "1. Backend",
"108": "Deployment",
"109": "Cinder — BookmarkManager design system",
"110": "Implement tickets",
"111": "SQLite → Postgres cutover runbook",
"112": "Testing the userscript",
"113": "ADR-0007: The backend hosts every Site's Cover bytes",
"114": "Issue tracker: Gitea (`tea` CLI)",
"115": "ADR-0006: The browser runs on the home machine, over the tailnet",
"116": "ADR-0008: A Series identity is discovered from the Site's links, never derived from an address",
"117": "Domain Docs",
"118": "ticket-implementer.md",
"119": "implementer.md",
"120": "Series is a shared entity, and only the Poll may update it",
"121": "Postgres replaces SQLite as the primary datastore",
"122": "Identity comes from Discord OAuth; we store no passwords and send no email",
"123": "The wire format stays flat and deliberately does not mirror the schema",
"124": "ADR-0005: On-demand browser sidecar",
"125": "sessions_test.go",
"126": "Bookmark Manager",
"127": "triage-labels.md",
"128": "Cross-Ticket Contract",
"129": "Implement Tickets Skill",
"130": "Orchestrator Role",
"131": "resolving-merge-conflicts Skill",
"132": "tdd Skill",
"133": "Ticket Wave Batching",
"134": "Four-Object Browser Stub",
"135": "Module Export Hook",
"136": "logic.test.js Test Harness",
"137": "manga-bookmark.user.js",
"138": "stripBuildHash",
"139": "Testing the Userscript Skill",
"140": "cr-spec Agent",
"141": "cr-standards Agent",
"142": "Escalate Rather Than Guess",
"143": "Status Contract",
"144": "Ticket Implementer Agent",
"145": "Worktree Isolation",
"146": "Escalate Rather Than Guess (opencode)",
"147": "Implementer Subagent (opencode)",
"148": "Subagent-Driven Development",
"149": "Code Quality Review",
"150": "Reviewer Subagent (opencode)",
"151": "Finding Severity Rubric",
"152": "Spec Compliance Review",
"161": "Why Use samber/oops",
"162": "singleflight Cache Stampede Prevention",
"163": "Struct Field Alignment",
"164": "testing/synctest Deterministic Goroutine Testing",
"177": "Backend CLAUDE.md Guidance",
"178": "Graphify Knowledge Graph (graphify-out/)",
"179": "CLAUDE.md (Symlink to AGENTS.md)",
"192": "ADR-0001 (Drop modernc.org/sqlite)",
"193": "ADR-0003 (Split Shared Series Facts)",
"194": "SQLite-to-Postgres Cutover Runbook",
"195": "Import SQL Generation Rules",
"196": "Throwaway Import Generator",
"206": "Real scaling limit is the poller outbound fetch budget",
"207": "PostgreSQL (jackc/pgx/v5)",
"208": "SQLite (modernc.org/sqlite)",
"209": "Postgres chosen for future supportability, not concurrency",
"210": "Per-Reader bearer token for userscripts",
"211": "Discord OAuth2 (authorization code grant)",
"212": "ADR-0002: Discord OAuth, no passwords, no email",
"213": "Discord snowflake is the sole identity (lock-in)",
"214": "Bookmark (per-Reader state: Progress, Favourite, Lifecycle)",
"215": "Deduplicate polling per Series (reader_count DESC queue)",
"216": "ADR-0003: Series is shared, only the Poll updates it",
"217": "Only the Poll writes Series fields (security boundary)",
"218": "Series (shared entity keyed site+series_id)",
"219": "ADR-0004: Wire format stays flat, does not mirror schema",
"220": "Flat wire shape is a contract, not an implementation detail",
"221": "Installed userscripts must keep working (14-day grace window)",
"222": "CDP (Chrome DevTools Protocol) endpoint",
"223": "headless-shell service (socat-fronted CDP)",
"224": "Start Chrome on first CDP connection, reap after 300s idle",
"225": "BROWSER_WS_URL configuration seam",
"226": "ADR-0006: Browser runs on the home machine over the tailnet",
"227": "Browser moved home: VPS memory pressure, no requests served",
"228": "Tailnet (Tailscale network)",
"229": "Content-addressed filesystem storage (SHA-256 of source URL)",
"230": "Cover (Series image bytes)",
"231": "Deny-class destination control for outbound fetch",
"232": "ADR-0007: Backend hosts every Site's Cover bytes",
"233": "kagane CORP same-origin cover restriction",
"234": "Backend acquires, stores, serves every Cover (uniformity)",
"235": "a[aria-label='All Chapter'] anchor pointer",
"236": "Series identity is discovered from the Site's links",
"237": "ADR-0008: Series identity discovered, never derived",
"238": "Chapter slug vs series slug divergence (~7% measured)",
"239": "Scan truncated at first wpd-threads marker",
"240": "Surface ADR conflicts explicitly rather than silently overriding",
"241": "Domain docs: single-context layout guidance",
"242": "/domain-modeling skill (lazy CONTEXT.md creation)",
"243": "CONTEXT.md glossary (ubiquitous language)",
"244": "Gitea (tea CLI, gitea.violetcrown.my.id)",
"245": "wayfinder map/ticket mechanism",
"246": "Triage labels: canonical roles to tracker labels",
"247": "Canonical triage role labels (needs-triage ... wontfix)",
"248": "Cinder (BookmarkManager Web UI design system)",
"249": "Heat is typographic: ember reserved for unread chapters",
"250": "Design tokens (dark + light branches, no hardcoded hex)",
"251": "Three type roles: display serif / mono small-caps / sans",
"252": "a[aria-label='All Chapter'] priority pointer",
"253": "Research: lightnovelworld chapter slug vs series slug",
"254": "Gitea issue #77 (chapter vs series slug)",
"255": "Slug divergence measurements (3/41 diverge, 1 split)",
"256": "Unscoped chapter regex is SAFE, truncated at wpd-threads",
"257": "BookmarkManager",
"258": "Bromite (Primary Device)",
"259": "Dark-First Design Constraint",
"260": "Discord Guild Membership",
"261": "Reader Isolation Invariant",
"273": "AGENTS.md",
"279": "Userscript CLAUDE guidance"
}
-1
View File
@@ -1 +0,0 @@
.
-567
View File
@@ -1,567 +0,0 @@
# Graph Report - mangaBookmark (2026-08-16)
## Corpus Check
- 113 files · ~279,161 words
- Verdict: corpus is large enough that graph structure adds value.
## Summary
- 1668 nodes · 3379 edges · 222 communities (68 shown, 154 thin omitted)
- Extraction: 91% EXTRACTED · 9% INFERRED · 0% AMBIGUOUS · INFERRED: 320 edges (avg confidence: 0.77)
- Token cost: 0 input · 0 output
## Graph Freshness
- Built from commit: `4f1cbcfd`
- Run `git rev-parse HEAD` and compare to check if the graph is stale.
- Run `graphify update .` after code changes (no API cost).
## Community Hubs (Navigation)
- [[_COMMUNITY_HTMX Library Internals|HTMX Library Internals]]
- [[_COMMUNITY_Cover Fetch Test Helpers|Cover Fetch Test Helpers]]
- [[_COMMUNITY_Manga Userscript Adapters|Manga Userscript Adapters]]
- [[_COMMUNITY_Novel Userscript Adapters|Novel Userscript Adapters]]
- [[_COMMUNITY_Series Acquisition Tests|Series Acquisition Tests]]
- [[_COMMUNITY_Bookmarks API Tests|Bookmarks API Tests]]
- [[_COMMUNITY_Storage Choice ADR|Storage Choice ADR]]
- [[_COMMUNITY_Cover & Acquire Internals|Cover & Acquire Internals]]
- [[_COMMUNITY_System Architecture Concepts|System Architecture Concepts]]
- [[_COMMUNITY_Session Middleware|Session Middleware]]
- [[_COMMUNITY_Go Test Helpers|Go Test Helpers]]
- [[_COMMUNITY_Store Tests|Store Tests]]
- [[_COMMUNITY_Bookmarks API Handler|Bookmarks API Handler]]
- [[_COMMUNITY_Web UI Handlers|Web UI Handlers]]
- [[_COMMUNITY_Go Error Handling|Go Error Handling]]
- [[_COMMUNITY_CDP Browser Client|CDP Browser Client]]
- [[_COMMUNITY_Cloudflare bot scoring and poll cadence — what is actually documented|Cloudflare bot scoring and poll cadence — what is actually documented]]
- [[_COMMUNITY_Go Code Style Guide|Go Code Style Guide]]
- [[_COMMUNITY_Agent Skills|Agent Skills]]
- [[_COMMUNITY_Store|Store]]
- [[_COMMUNITY_IO Performance Patterns|I/O Performance Patterns]]
- [[_COMMUNITY_CPU Optimization|CPU Optimization]]
- [[_COMMUNITY_Caching Patterns|Caching Patterns]]
- [[_COMMUNITY_Browser Entrypoint|Browser Entrypoint]]
- [[_COMMUNITY_Memory Allocation & GC|Memory Allocation & GC]]
- [[_COMMUNITY_Cover Fetcher Tests|Cover Fetcher Tests]]
- [[_COMMUNITY_Open|Open]]
- [[_COMMUNITY_Find Skills Guide|Find Skills Guide]]
- [[_COMMUNITY_Allocation Patterns|Allocation Patterns]]
- [[_COMMUNITY_Observability & Alerting|Observability & Alerting]]
- [[_COMMUNITY_Store|Store]]
- [[_COMMUNITY_Open|Open]]
- [[_COMMUNITY_Go Testing Guide|Go Testing Guide]]
- [[_COMMUNITY_Session Store|Session Store]]
- [[_COMMUNITY_Web UI Filter Logic|Web UI Filter Logic]]
- [[_COMMUNITY_Userscript Test Harness|Userscript Test Harness]]
- [[_COMMUNITY_Product & Security Context|Product & Security Context]]
- [[_COMMUNITY_novel-logic.test.js|novel-logic.test.js]]
- [[_COMMUNITY_UI Critique 2026-07-26A|UI Critique 2026-07-26A]]
- [[_COMMUNITY_UI Critique 2026-07-26B|UI Critique 2026-07-26B]]
- [[_COMMUNITY_pgtest.go|pgtest.go]]
- [[_COMMUNITY_Issue Tracker & Triage|Issue Tracker & Triage]]
- [[_COMMUNITY_Ticket Workflow|Ticket Workflow]]
- [[_COMMUNITY_Go Perf Alert Rules|Go Perf Alert Rules]]
- [[_COMMUNITY_Userscript Display Logic|Userscript Display Logic]]
- [[_COMMUNITY_Go Perf Skill Docs|Go Perf Skill Docs]]
- [[_COMMUNITY_Login Page Art|Login Page Art]]
- [[_COMMUNITY_BookmarkManager Logo|BookmarkManager Logo]]
- [[_COMMUNITY_Skills CLI|Skills CLI]]
- [[_COMMUNITY_Skills Leaderboard|Skills Leaderboard]]
- [[_COMMUNITY_Complex Condition Extraction|Complex Condition Extraction]]
- [[_COMMUNITY_Sentinel Errors|Sentinel Errors]]
- [[_COMMUNITY_errors.As Patterns|errors.As Patterns]]
- [[_COMMUNITY_errors.Is Patterns|errors.Is Patterns]]
- [[_COMMUNITY_errors.Join Patterns|errors.Join Patterns]]
- [[_COMMUNITY_Error Wrapping|Error Wrapping]]
- [[_COMMUNITY_Single Error Handling|Single Error Handling]]
- [[_COMMUNITY_SIMD Optimizations|SIMD Optimizations]]
- [[_COMMUNITY_GOGC Tuning|GOGC Tuning]]
- [[_COMMUNITY_GOMEMLIMIT|GOMEMLIMIT]]
- [[_COMMUNITY_Bottleneck Decision Tree|Bottleneck Decision Tree]]
- [[_COMMUNITY_pprof Profiling|pprof Profiling]]
- [[_COMMUNITY_Test Timeout Helper|Test Timeout Helper]]
- [[_COMMUNITY_httptest Patterns|httptest Patterns]]
- [[_COMMUNITY_testify Suite Pattern|testify Suite Pattern]]
- [[_COMMUNITY_goembed Fixtures|go:embed Fixtures]]
- [[_COMMUNITY_clockwork Time Mocking|clockwork Time Mocking]]
- [[_COMMUNITY_testify Mocking|testify Mocking]]
- [[_COMMUNITY_t.ArtifactDir Helper|t.ArtifactDir Helper]]
- [[_COMMUNITY_Subtests Pitfall|Subtests Pitfall]]
- [[_COMMUNITY_golang-benchmark Skill|golang-benchmark Skill]]
- [[_COMMUNITY_golang-concurrency Skill|golang-concurrency Skill]]
- [[_COMMUNITY_golang-ci Skill|golang-ci Skill]]
- [[_COMMUNITY_golang-database Skill|golang-database Skill]]
- [[_COMMUNITY_golang-lint Skill|golang-lint Skill]]
- [[_COMMUNITY_testify Skill|testify Skill]]
- [[_COMMUNITY_Build Tag Integration Tests|Build Tag Integration Tests]]
- [[_COMMUNITY_Test Naming Convention|Test Naming Convention]]
- [[_COMMUNITY_UI Critique A Finding|UI Critique A Finding]]
- [[_COMMUNITY_UI Critique B Finding|UI Critique B Finding]]
- [[_COMMUNITY_P0 Overflow Bug|P0 Overflow Bug]]
- [[_COMMUNITY_P1 hx-indicator Gap|P1 hx-indicator Gap]]
- [[_COMMUNITY_golang-benchmark Skill (ext)|golang-benchmark Skill (ext)]]
- [[_COMMUNITY_golang-concurrency Skill (ext)|golang-concurrency Skill (ext)]]
- [[_COMMUNITY_golang-ci Skill (ext)|golang-ci Skill (ext)]]
- [[_COMMUNITY_golang-data-structures Skill (ext)|golang-data-structures Skill (ext)]]
- [[_COMMUNITY_golang-database Skill (ext)|golang-database Skill (ext)]]
- [[_COMMUNITY_golang-design-patterns Skill (ext)|golang-design-patterns Skill (ext)]]
- [[_COMMUNITY_golang-documentation Skill (ext)|golang-documentation Skill (ext)]]
- [[_COMMUNITY_golang-gopls Skill (ext)|golang-gopls Skill (ext)]]
- [[_COMMUNITY_golang-lint Skill (ext)|golang-lint Skill (ext)]]
- [[_COMMUNITY_golang-naming Skill (ext)|golang-naming Skill (ext)]]
- [[_COMMUNITY_golang-observability Skill (ext)|golang-observability Skill (ext)]]
- [[_COMMUNITY_golang-refactoring Skill (ext)|golang-refactoring Skill (ext)]]
- [[_COMMUNITY_golang-safety Skill (ext)|golang-safety Skill (ext)]]
- [[_COMMUNITY_golang-samber-oops Skill (ext)|golang-samber-oops Skill (ext)]]
- [[_COMMUNITY_golang-samber-slog Skill (ext)|golang-samber-slog Skill (ext)]]
- [[_COMMUNITY_golang-structs-interfaces Skill (ext)|golang-structs-interfaces Skill (ext)]]
- [[_COMMUNITY_golang-troubleshooting Skill (ext)|golang-troubleshooting Skill (ext)]]
- [[_COMMUNITY_promql-cli Skill|promql-cli Skill]]
- [[_COMMUNITY_Backend Module|Backend Module]]
- [[_COMMUNITY_bookmark-api Service|bookmark-api Service]]
- [[_COMMUNITY_AGENTS|AGENTS.md]]
- [[_COMMUNITY_reviewer|reviewer.md]]
- [[_COMMUNITY_Redeploy runbook|Redeploy runbook]]
- [[_COMMUNITY_1. Backend|1. Backend]]
- [[_COMMUNITY_Deployment|Deployment]]
- [[_COMMUNITY_Cinder — BookmarkManager design system|Cinder — BookmarkManager design system]]
- [[_COMMUNITY_Implement tickets|Implement tickets]]
- [[_COMMUNITY_SQLite → Postgres cutover runbook|SQLite → Postgres cutover runbook]]
- [[_COMMUNITY_Testing the userscript|Testing the userscript]]
- [[_COMMUNITY_ADR-0007 The backend hosts every Site's Cover bytes|ADR-0007: The backend hosts every Site's Cover bytes]]
- [[_COMMUNITY_Issue tracker Gitea (`tea` CLI)|Issue tracker: Gitea (`tea` CLI)]]
- [[_COMMUNITY_ADR-0006 The browser runs on the home machine, over the tailnet|ADR-0006: The browser runs on the home machine, over the tailnet]]
- [[_COMMUNITY_ADR-0008 A Series identity is discovered from the Site's links, never derived from an address|ADR-0008: A Series identity is discovered from the Site's links, never derived from an address]]
- [[_COMMUNITY_Domain Docs|Domain Docs]]
- [[_COMMUNITY_ticket-implementer|ticket-implementer.md]]
- [[_COMMUNITY_implementer|implementer.md]]
- [[_COMMUNITY_Series is a shared entity, and only the Poll may update it|Series is a shared entity, and only the Poll may update it]]
- [[_COMMUNITY_Postgres replaces SQLite as the primary datastore|Postgres replaces SQLite as the primary datastore]]
- [[_COMMUNITY_Identity comes from Discord OAuth; we store no passwords and send no email|Identity comes from Discord OAuth; we store no passwords and send no email]]
- [[_COMMUNITY_The wire format stays flat and deliberately does not mirror the schema|The wire format stays flat and deliberately does not mirror the schema]]
- [[_COMMUNITY_ADR-0005 On-demand browser sidecar|ADR-0005: On-demand browser sidecar]]
- [[_COMMUNITY_sessions_test.go|sessions_test.go]]
- [[_COMMUNITY_Bookmark Manager|Bookmark Manager]]
- [[_COMMUNITY_triage-labels|triage-labels.md]]
- [[_COMMUNITY_Cross-Ticket Contract|Cross-Ticket Contract]]
- [[_COMMUNITY_Implement Tickets Skill|Implement Tickets Skill]]
- [[_COMMUNITY_Orchestrator Role|Orchestrator Role]]
- [[_COMMUNITY_resolving-merge-conflicts Skill|resolving-merge-conflicts Skill]]
- [[_COMMUNITY_tdd Skill|tdd Skill]]
- [[_COMMUNITY_Ticket Wave Batching|Ticket Wave Batching]]
- [[_COMMUNITY_Four-Object Browser Stub|Four-Object Browser Stub]]
- [[_COMMUNITY_Module Export Hook|Module Export Hook]]
- [[_COMMUNITY_logic.test.js Test Harness|logic.test.js Test Harness]]
- [[_COMMUNITY_manga-bookmark.user.js|manga-bookmark.user.js]]
- [[_COMMUNITY_stripBuildHash|stripBuildHash]]
- [[_COMMUNITY_Testing the Userscript Skill|Testing the Userscript Skill]]
- [[_COMMUNITY_cr-spec Agent|cr-spec Agent]]
- [[_COMMUNITY_cr-standards Agent|cr-standards Agent]]
- [[_COMMUNITY_Escalate Rather Than Guess|Escalate Rather Than Guess]]
- [[_COMMUNITY_Status Contract|Status Contract]]
- [[_COMMUNITY_Ticket Implementer Agent|Ticket Implementer Agent]]
- [[_COMMUNITY_Worktree Isolation|Worktree Isolation]]
- [[_COMMUNITY_Escalate Rather Than Guess (opencode)|Escalate Rather Than Guess (opencode)]]
- [[_COMMUNITY_Implementer Subagent (opencode)|Implementer Subagent (opencode)]]
- [[_COMMUNITY_Subagent-Driven Development|Subagent-Driven Development]]
- [[_COMMUNITY_Code Quality Review|Code Quality Review]]
- [[_COMMUNITY_Reviewer Subagent (opencode)|Reviewer Subagent (opencode)]]
- [[_COMMUNITY_Finding Severity Rubric|Finding Severity Rubric]]
- [[_COMMUNITY_Spec Compliance Review|Spec Compliance Review]]
- [[_COMMUNITY_Why Use samberoops|Why Use samber/oops]]
- [[_COMMUNITY_singleflight Cache Stampede Prevention|singleflight Cache Stampede Prevention]]
- [[_COMMUNITY_Struct Field Alignment|Struct Field Alignment]]
- [[_COMMUNITY_testingsynctest Deterministic Goroutine Testing|testing/synctest Deterministic Goroutine Testing]]
- [[_COMMUNITY_Backend CLAUDE.md Guidance|Backend CLAUDE.md Guidance]]
- [[_COMMUNITY_Graphify Knowledge Graph (graphify-out)|Graphify Knowledge Graph (graphify-out/)]]
- [[_COMMUNITY_CLAUDE.md (Symlink to AGENTS.md)|CLAUDE.md (Symlink to AGENTS.md)]]
- [[_COMMUNITY_ADR-0001 (Drop modernc.orgsqlite)|ADR-0001 (Drop modernc.org/sqlite)]]
- [[_COMMUNITY_ADR-0003 (Split Shared Series Facts)|ADR-0003 (Split Shared Series Facts)]]
- [[_COMMUNITY_SQLite-to-Postgres Cutover Runbook|SQLite-to-Postgres Cutover Runbook]]
- [[_COMMUNITY_Import SQL Generation Rules|Import SQL Generation Rules]]
- [[_COMMUNITY_Throwaway Import Generator|Throwaway Import Generator]]
- [[_COMMUNITY_Real scaling limit is the poller outbound fetch budget|Real scaling limit is the poller outbound fetch budget]]
- [[_COMMUNITY_PostgreSQL (jackcpgxv5)|PostgreSQL (jackc/pgx/v5)]]
- [[_COMMUNITY_SQLite (modernc.orgsqlite)|SQLite (modernc.org/sqlite)]]
- [[_COMMUNITY_Postgres chosen for future supportability, not concurrency|Postgres chosen for future supportability, not concurrency]]
- [[_COMMUNITY_Per-Reader bearer token for userscripts|Per-Reader bearer token for userscripts]]
- [[_COMMUNITY_Discord OAuth2 (authorization code grant)|Discord OAuth2 (authorization code grant)]]
- [[_COMMUNITY_ADR-0002 Discord OAuth, no passwords, no email|ADR-0002: Discord OAuth, no passwords, no email]]
- [[_COMMUNITY_Discord snowflake is the sole identity (lock-in)|Discord snowflake is the sole identity (lock-in)]]
- [[_COMMUNITY_Bookmark (per-Reader state Progress, Favourite, Lifecycle)|Bookmark (per-Reader state: Progress, Favourite, Lifecycle)]]
- [[_COMMUNITY_Deduplicate polling per Series (reader_count DESC queue)|Deduplicate polling per Series (reader_count DESC queue)]]
- [[_COMMUNITY_ADR-0003 Series is shared, only the Poll updates it|ADR-0003: Series is shared, only the Poll updates it]]
- [[_COMMUNITY_Only the Poll writes Series fields (security boundary)|Only the Poll writes Series fields (security boundary)]]
- [[_COMMUNITY_Series (shared entity keyed site+series_id)|Series (shared entity keyed site+series_id)]]
- [[_COMMUNITY_ADR-0004 Wire format stays flat, does not mirror schema|ADR-0004: Wire format stays flat, does not mirror schema]]
- [[_COMMUNITY_Flat wire shape is a contract, not an implementation detail|Flat wire shape is a contract, not an implementation detail]]
- [[_COMMUNITY_Installed userscripts must keep working (14-day grace window)|Installed userscripts must keep working (14-day grace window)]]
- [[_COMMUNITY_CDP (Chrome DevTools Protocol) endpoint|CDP (Chrome DevTools Protocol) endpoint]]
- [[_COMMUNITY_headless-shell service (socat-fronted CDP)|headless-shell service (socat-fronted CDP)]]
- [[_COMMUNITY_Start Chrome on first CDP connection, reap after 300s idle|Start Chrome on first CDP connection, reap after 300s idle]]
- [[_COMMUNITY_BROWSER_WS_URL configuration seam|BROWSER_WS_URL configuration seam]]
- [[_COMMUNITY_ADR-0006 Browser runs on the home machine over the tailnet|ADR-0006: Browser runs on the home machine over the tailnet]]
- [[_COMMUNITY_Browser moved home VPS memory pressure, no requests served|Browser moved home: VPS memory pressure, no requests served]]
- [[_COMMUNITY_Tailnet (Tailscale network)|Tailnet (Tailscale network)]]
- [[_COMMUNITY_Content-addressed filesystem storage (SHA-256 of source URL)|Content-addressed filesystem storage (SHA-256 of source URL)]]
- [[_COMMUNITY_Cover (Series image bytes)|Cover (Series image bytes)]]
- [[_COMMUNITY_Deny-class destination control for outbound fetch|Deny-class destination control for outbound fetch]]
- [[_COMMUNITY_ADR-0007 Backend hosts every Site's Cover bytes|ADR-0007: Backend hosts every Site's Cover bytes]]
- [[_COMMUNITY_kagane CORP same-origin cover restriction|kagane CORP same-origin cover restriction]]
- [[_COMMUNITY_Backend acquires, stores, serves every Cover (uniformity)|Backend acquires, stores, serves every Cover (uniformity)]]
- [[_COMMUNITY_aaria-label='All Chapter' anchor pointer|a[aria-label='All Chapter'] anchor pointer]]
- [[_COMMUNITY_Series identity is discovered from the Site's links|Series identity is discovered from the Site's links]]
- [[_COMMUNITY_ADR-0008 Series identity discovered, never derived|ADR-0008: Series identity discovered, never derived]]
- [[_COMMUNITY_Chapter slug vs series slug divergence (~7% measured)|Chapter slug vs series slug divergence (~7% measured)]]
- [[_COMMUNITY_Scan truncated at first wpd-threads marker|Scan truncated at first wpd-threads marker]]
- [[_COMMUNITY_Surface ADR conflicts explicitly rather than silently overriding|Surface ADR conflicts explicitly rather than silently overriding]]
- [[_COMMUNITY_Domain docs single-context layout guidance|Domain docs: single-context layout guidance]]
- [[_COMMUNITY_domain-modeling skill (lazy CONTEXT.md creation)|/domain-modeling skill (lazy CONTEXT.md creation)]]
- [[_COMMUNITY_CONTEXT.md glossary (ubiquitous language)|CONTEXT.md glossary (ubiquitous language)]]
- [[_COMMUNITY_Gitea (tea CLI, gitea.violetcrown.my.id)|Gitea (tea CLI, gitea.violetcrown.my.id)]]
- [[_COMMUNITY_wayfinder mapticket mechanism|wayfinder map/ticket mechanism]]
- [[_COMMUNITY_Triage labels canonical roles to tracker labels|Triage labels: canonical roles to tracker labels]]
- [[_COMMUNITY_Canonical triage role labels (needs-triage ... wontfix)|Canonical triage role labels (needs-triage ... wontfix)]]
- [[_COMMUNITY_Cinder (BookmarkManager Web UI design system)|Cinder (BookmarkManager Web UI design system)]]
- [[_COMMUNITY_Heat is typographic ember reserved for unread chapters|Heat is typographic: ember reserved for unread chapters]]
- [[_COMMUNITY_Design tokens (dark + light branches, no hardcoded hex)|Design tokens (dark + light branches, no hardcoded hex)]]
- [[_COMMUNITY_Three type roles display serif mono small-caps sans|Three type roles: display serif / mono small-caps / sans]]
- [[_COMMUNITY_aaria-label='All Chapter' priority pointer|a[aria-label='All Chapter'] priority pointer]]
- [[_COMMUNITY_Research lightnovelworld chapter slug vs series slug|Research: lightnovelworld chapter slug vs series slug]]
- [[_COMMUNITY_Gitea issue 77 (chapter vs series slug)|Gitea issue #77 (chapter vs series slug)]]
- [[_COMMUNITY_Slug divergence measurements (341 diverge, 1 split)|Slug divergence measurements (3/41 diverge, 1 split)]]
- [[_COMMUNITY_Unscoped chapter regex is SAFE, truncated at wpd-threads|Unscoped chapter regex is SAFE, truncated at wpd-threads]]
- [[_COMMUNITY_BookmarkManager|BookmarkManager]]
- [[_COMMUNITY_Bromite (Primary Device)|Bromite (Primary Device)]]
- [[_COMMUNITY_Dark-First Design Constraint|Dark-First Design Constraint]]
- [[_COMMUNITY_Discord Guild Membership|Discord Guild Membership]]
- [[_COMMUNITY_Reader Isolation Invariant|Reader Isolation Invariant]]
- [[_COMMUNITY_AGENTS|AGENTS.md]]
- [[_COMMUNITY_Userscript CLAUDE guidance|Userscript CLAUDE guidance]]
## God Nodes (most connected - your core abstractions)
1. `testConfig()` - 53 edges
2. `newWebTestServer()` - 49 edges
3. `newTestStore()` - 48 edges
4. `newTestStore()` - 43 edges
5. `e()` - 33 edges
6. `Open()` - 29 edges
7. `Handler` - 29 edges
8. `ne()` - 28 edges
9. `De()` - 28 edges
10. `Store` - 27 edges
## Surprising Connections (you probably didn't know these)
- `el()` --indirect_call--> `c()` [INFERRED]
userscript/manga-bookmark.user.js → backend/internal/web/static/htmx.min.js
- `el()` --indirect_call--> `c()` [INFERRED]
userscript/novel-bookmark.user.js → backend/internal/web/static/htmx.min.js
- `latestChapterFromAnchors()` --indirect_call--> `re()` [INFERRED]
userscript/manga-bookmark.user.js → backend/internal/web/static/htmx.min.js
- `el()` --indirect_call--> `k()` [INFERRED]
userscript/manga-bookmark.user.js → backend/internal/web/static/htmx.min.js
- `el()` --indirect_call--> `k()` [INFERRED]
userscript/novel-bookmark.user.js → backend/internal/web/static/htmx.min.js
## Import Cycles
- None detected.
## Hyperedges (group relationships)
- **Batch Ticket Implementation Pipeline** — _claude_skills_implement_tickets_skill_implement_tickets, _omp_agents_ticket_implementer_ticket_implementer, _omp_agents_ticket_implementer_cr_spec, _omp_agents_ticket_implementer_cr_standards [INFERRED 0.85]
- **Subagent-Driven Development Pipeline** — _opencode_agent_implementer_implementer, _opencode_agent_reviewer_reviewer, _opencode_agent_implementer_subagent_driven_development [INFERRED 0.85]
- **Go HTML Template Family** — backend_internal_web_templates_app_doc, backend_internal_web_templates_card_doc, backend_internal_web_templates_list_doc, backend_internal_web_templates_chrome_doc, backend_internal_web_templates_login_doc, backend_internal_web_templates_setup_doc, backend_internal_web_templates_readers_doc, backend_internal_web_templates_icons_doc [INFERRED 0.95]
- **htmx Fragment Swap Flow** — backend_internal_web_templates_app_doc, backend_internal_web_templates_card_doc, backend_internal_web_templates_chrome_doc, backend_internal_web_templates_setup_doc, backend_internal_web_templates_readers_doc [INFERRED 0.95]
- **Backend owns the truth (single-writer ownership of shared facts)** — docs_adr_0003_series_shared_and_poll_owned_poll_owned_writes, docs_adr_0004_wire_format_does_not_mirror_the_schema_flat_wire_contract, docs_adr_0007_backend_hosts_cover_bytes_server_side_covers [INFERRED 0.85]
- **Headless browser infrastructure (sidecar, on-demand, home deployment)** — docs_adr_0005_on_demand_browser_headless_shell, docs_adr_0005_on_demand_browser_cdp, docs_adr_0005_on_demand_browser_on_demand_start, docs_adr_0006_browser_on_the_home_machine_home_machine_rationale [INFERRED 0.85]
- **lightnovelworld series-identity investigation and fix** — docs_research_lightnovelworld_chapter_vs_series_slug_issue_77, docs_research_lightnovelworld_chapter_vs_series_slug_unscoped_regex, docs_adr_0008_series_identity_is_discovered_not_derived_discovered_identity [INFERRED 0.85]
## Communities (222 total, 154 thin omitted)
### Community 0 - "HTMX Library Internals"
Cohesion: 0.08
Nodes (101): A(), ae(), an(), at(), B(), be(), bn(), bt() (+93 more)
### Community 1 - "Cover Fetch Test Helpers"
Cohesion: 0.10
Nodes (84): floatPtr(), testConfig(), getCover(), Cookie, Handler, ResponseRecorder, T, TestListRendersAcquiredCover() (+76 more)
### Community 2 - "Manga Userscript Adapters"
Cohesion: 0.06
Nodes (76): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+68 more)
### Community 3 - "Novel Userscript Adapters"
Cohesion: 0.06
Nodes (79): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+71 more)
### Community 4 - "Series Acquisition Tests"
Cohesion: 0.08
Nodes (78): bookmarkNewKaganeSeries(), bookmarkNewNovelfullSeries(), bookmarkNewSeries(), Context, Store, T, newAcquirer(), readBookmark() (+70 more)
### Community 5 - "Bookmarks API Tests"
Cohesion: 0.06
Nodes (74): auth(), getBookmarks(), Handler, Request, Store, T, newTestServer(), newTestStore() (+66 more)
### Community 7 - "Cover & Acquire Internals"
Cohesion: 0.08
Nodes (37): Addr, Context, Store, WaitGroup, isInterstitial(), defaultCoverResolver(), fetchCoverBytes(), Client (+29 more)
### Community 8 - "System Architecture Concepts"
Cohesion: 0.13
Nodes (20): app.html — App Shell Template, Manga/Novel Library Switch, Bookmark Bucket Tabs, Confirm Row (Archive/Finish/Remove), card.html — Series Card Template, htmx /ui/* Mutation Endpoints, chrome.html — Out-of-Band Regions, Action Key (+12 more)
### Community 9 - "Session Middleware"
Cohesion: 0.08
Nodes (35): ClearCookie(), ClientIP(), Duration, Mutex, Request, ResponseWriter, Time, isHTTPS() (+27 more)
### Community 10 - "Go Test Helpers"
Cohesion: 0.05
Nodes (39): Test Helpers, Test Timeout, Basic Handler Test, HTTP Handler Testing, Query Parameters and Headers, Docker Compose Fixture, Integration Testing, SQL Schema Fixture (+31 more)
### Community 11 - "Store Tests"
Cohesion: 0.14
Nodes (45): scanSeries(), Store, T, newTestStore(), readLatestCheckedAt(), readSeries(), secondReader(), seedForCheck() (+37 more)
### Community 12 - "Bookmarks API Handler"
Cohesion: 0.08
Nodes (32): Handler, Request, ResponseWriter, Store, Healthz(), writeJSON(), Auth(), compressible() (+24 more)
### Community 13 - "Web UI Handlers"
Cohesion: 0.14
Nodes (18): currentLib(), currentTab(), filterBookmarks(), Client, HandlerFunc, Request, ResponseWriter, Store (+10 more)
### Community 14 - "Go Error Handling"
Cohesion: 0.06
Nodes (33): Creating Errors, Custom Error Types, Custom types that wrap other errors, Decision table: which error strategy to use, Error Creation, Error String Conventions, Errors as Values, `errors.New` — static error messages (+25 more)
### Community 15 - "CDP Browser Client"
Cohesion: 0.06
Nodes (56): awaitPromise(), browserConnectionLost(), classifyBrowserError(), comixRead(), comixSeriesPageURL(), Action, Context, Mutex (+48 more)
### Community 16 - "Cloudflare bot scoring and poll cadence — what is actually documented"
Cohesion: 0.06
Nodes (33): 1.1 The score itself, 1.2 The detection engines (Enterprise Bot Management), 1.3 Rate limiting is a separate product, 1. What a bot score is and what feeds it, 2.1 What each plan gets, 2.2 Bot Fight Mode specifics (the Free-plan product), 2.3 Does the free tier "score" continuously?, 2. The free-plan reality (+25 more)
### Community 17 - "Go Code Style Guide"
Cohesion: 0.08
Nodes (23): Code Style Details, Extract Complex Conditions, Value vs Pointer Arguments, Code Organization Within Files, Complex Conditions & Init Scope, Composite Literals, Control Flow, Cross-References (+15 more)
### Community 19 - "Store"
Cohesion: 0.33
Nodes (5): ADR-0010: Poll Lanes — one independent Poll stream per Site, Constraints preserved, Decision, Tradeoffs and rejections, Why
### Community 20 - "I/O Performance Patterns"
Cohesion: 0.11
Nodes (18): Avoid io.ReadAll for large payloads, Batch Operations, Buffered I/O, Cgo Overhead, Channel: batch processing from a stream, Concurrent Multi-Stage Pipelines, Connection pooling, Database: batch inserts over row-by-row (+10 more)
### Community 21 - "CPU Optimization"
Cohesion: 0.13
Nodes (15): Cache Locality, Contiguous 2D allocation, CPU Optimization, False Sharing, Function Inlining, Handling CPU-specific instruction sets, Instruction-Level Parallelism, Monotonic Time (+7 more)
### Community 22 - "Caching Patterns"
Cohesion: 0.13
Nodes (14): Algorithmic Complexity, Avoid iterator chains, Caching Patterns, Compiled Pattern Caching, Early returns and short-circuit loops, LRU caches, Map lookups over slice scanning, Precomputed lookup tables (+6 more)
### Community 23 - "Browser Entrypoint"
Cohesion: 0.35
Nodes (14): browser_alive(), connection(), connection_signal(), finish_connection(), has_connections(), lock(), reaper(), entrypoint.sh script (+6 more)
### Community 24 - "Memory Allocation & GC"
Cohesion: 0.13
Nodes (15): Allocation Rate Reduction, Ballast pattern (pre-Go 1.19), Garbage Collector Tuning, GC pacing, GC Profiling and Diagnostics, GODEBUG=gctrace=1, GOGC (default: 100), GOMAXPROCS in Containers (+7 more)
### Community 25 - "Cover Fetcher Tests"
Cohesion: 0.33
Nodes (12): coverResponse(), Request, T, TestCoverFetcherCanonicalisesJpgAlias(), TestCoverFetcherFetchesPublicHTTPSImage(), TestCoverFetcherRefusesUnsafeDestinationsBeforeRequest(), TestCoverFetcherRejectsNonImage(), TestCoverFetcherRejectsOversizedBody() (+4 more)
### Community 26 - "Open"
Cohesion: 0.40
Nodes (5): Map of pointers for large, frequently updated structs, Memory Layout, Pointer receivers for large structs, Struct field alignment, Zero-size field at end of struct
### Community 27 - "Find Skills Guide"
Cohesion: 0.14
Nodes (13): Common Skill Categories, Find Skills, How to Help Users Find Skills, Step 1: Understand What They Need, Step 2: Check the Leaderboard First, Step 3: Search for Skills, Step 4: Verify Quality Before Recommending, Step 5: Present Options to the User (+5 more)
### Community 28 - "Allocation Patterns"
Cohesion: 0.14
Nodes (14): Allocation Patterns, Backing Array Leaks, Direct indexing vs append, Eliminate redundant map lookups, Interface boxing, Map never shrinks, Map size hints, Memory Optimization (+6 more)
### Community 29 - "Observability & Alerting"
Cohesion: 0.22
Nodes (9): Alerting rules (examples), CPU saturation, GC pressure, Goroutine leaks, Grafana Dashboards, Memory leaks, Prometheus Metrics for Go, PromQL Queries for Performance Diagnosis (+1 more)
### Community 31 - "Store"
Cohesion: 0.12
Nodes (3): coverRelativePath(), coverSourceAddress(), Store
### Community 32 - "Open"
Cohesion: 0.14
Nodes (20): applyMigration(), displayChapter(), migrate(), Open(), refreshOwnerToken(), seedOwner(), TestCoverIsContentAddressedOnFilesystem(), TestCoverPersistsAcrossReopen() (+12 more)
### Community 33 - "Go Testing Guide"
Cohesion: 0.20
Nodes (10): CI Regression Detection, Common Mistakes, Core Philosophy, Cross-References, Decision Tree: Where Is Time Spent?, Deep Dives, Go Performance Optimization, Iterative Optimization Methodology (+2 more)
### Community 34 - "Session Store"
Cohesion: 0.28
Nodes (4): Duration, Store, Time, Session
### Community 35 - "Web UI Filter Logic"
Cohesion: 0.31
Nodes (5): closeCardPanels(), setActiveTab(), toggleChapterForm(), toggleConfirmRow(), togglePanel()
### Community 37 - "Product & Security Context"
Cohesion: 0.17
Nodes (11): Accessibility & Inclusion, Brand Commitments, Capabilities and Constraints, Evidence on Hand, Operating Context, Platform, Positioning, Product (+3 more)
### Community 38 - "novel-logic.test.js"
Cohesion: 0.33
Nodes (5): ADR-0009: A Site answers fixed questions; an unusual Site owns its own fetch, Consequences, Considered options, Decision, Why
### Community 39 - "UI Critique 2026-07-26A"
Cohesion: 0.29
Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations, Persona Red Flags, Priority Issues, Questions to Consider
### Community 40 - "UI Critique 2026-07-26B"
Cohesion: 0.29
Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations, Persona Red Flags, Priority Issues, Questions to Consider
### Community 42 - "pgtest.go"
Cohesion: 0.18
Nodes (12): M, TestMain(), M, TestMain(), M, Main(), start(), URL() (+4 more)
### Community 45 - "Go Perf Alert Rules"
Cohesion: 0.50
Nodes (4): Prometheus Alerting Rules (Go Performance), GoroutineLeak Alert, HighGCPauseTime Alert, MemoryNearLimit Alert
### Community 46 - "Userscript Display Logic"
Cohesion: 0.07
Nodes (27): 1. Summary answer table, 2. The two reference pages, 3. Chapter page → series URL: every in-page pointer, in priority order, 4.1 Sample method, 4.2 Divergence results, 4.3 Is there a derivable rule? **No.**, 4.4 The split case — a slug can change *mid-series*, 4. The reverse direction, and how common divergence is (+19 more)
### Community 47 - "Go Perf Skill Docs"
Cohesion: 0.22
Nodes (5): Continuous Profiling, Production Observability for Performance, Pyroscope pull mode (via Grafana Alloy), Pyroscope push mode, Real-Time Visualization (Development)
### Community 48 - "Login Page Art"
Cohesion: 0.67
Nodes (3): Fantasy Sword, Fiery Volcanic Scene, Login Art: Sword in Volcanic Rock
### Community 49 - "BookmarkManager Logo"
Cohesion: 1.00
Nodes (3): Mirrored Double Bookmark Mark, Ember Flame Accent, BookmarkManager Logo
### Community 103 - "bookmark-api Service"
Cohesion: 0.32
Nodes (8): Browser Sidecar Service, CDP Endpoint (Tailnet-Bound :9222), Persistent Chrome Profile Volume, chrome/docker-compose.yml — Browser Deployable Unit, bookmark-api Prod Override, CDP Never on Shared Proxy Network, docker-compose.prod.yml — Production Override, Traefik Reverse Proxy Labels
### Community 104 - "AGENTS.md"
Cohesion: 0.12
Nodes (14): Agent skills, Architecture, Commands, Comments, Design system, Domain docs, Forge: Gitea, not GitHub, graphify (+6 more)
### Community 105 - "reviewer.md"
Cohesion: 0.12
Nodes (15): Assessment, Calibration, Critical (Must Fix), Do Not Trust the Report, Important (Should Fix), Inputs, Issues, Method (+7 more)
### Community 106 - "Redeploy runbook"
Cohesion: 0.12
Nodes (15): 0. Preflight, 1. Back up the database, 2. Pull the new code, 3. Rebuild and restart, 4. Verify the deploy, 5. Smoke-test the full loop, 6. Rollback, 7. The whole thing, as one block (+7 more)
### Community 107 - "1. Backend"
Cohesion: 0.13
Nodes (14): 1. Backend, 2. Userscript, Adapter reference (verified live 2026-07-24), Config (env), Deploy behind your reverse proxy, Desktop iteration (optional), Develop / test, Endpoints (+6 more)
### Community 108 - "Deployment"
Cohesion: 0.14
Nodes (13): 0. Prerequisites, 1. Configure `.env`, 1b. Web UI, 2. Build + start, 3. Verify over HTTPS, 4. Configure the userscript, 5. Install on Bromite, 6. Smoke-test the full loop (+5 more)
### Community 109 - "Cinder — BookmarkManager design system"
Cohesion: 0.20
Nodes (9): 1. The one idea, 2. Tokens, 3. Type, 4. Components (web UI), 5. Components (userscript panel), 6. Motion, 7. Accessibility floor (not negotiable), 8. Adding something new — checklist (+1 more)
### Community 110 - "Implement tickets"
Cohesion: 0.22
Nodes (8): 1. Collect the tickets, 2. Plan the batch, 3. Get the plan approved, 4. Run a wave, 5. Land the wave, 6. Close the batch, Implement tickets, Ticket #<n> — <title>
### Community 111 - "SQLite → Postgres cutover runbook"
Cohesion: 0.22
Nodes (8): 0. The generator is throwaway, 1. Stop the old API and take a fresh export, 2. Bring up Postgres with the schema and the owner Reader, 3. Generate the import SQL, 4. Review it by eye, 5. Apply it, 6. Afterwards, SQLite → Postgres cutover runbook
### Community 112 - "Testing the userscript"
Cohesion: 0.29
Nodes (6): Adding a test, Commands, Gotchas, How the harness works, Testing the userscript, What is NOT testable here
### Community 113 - "ADR-0007: The backend hosts every Site's Cover bytes"
Cohesion: 0.29
Nodes (6): ADR-0007: The backend hosts every Site's Cover bytes, Consequences, Considered options, Decision, Two deliberate relaxations, Why a future reader will find this surprising
### Community 114 - "Issue tracker: Gitea (`tea` CLI)"
Cohesion: 0.29
Nodes (6): Conventions, Issue tracker: Gitea (`tea` CLI), Pull requests as a triage surface, Wayfinding operations, When a skill says "fetch the relevant ticket", When a skill says "publish to the issue tracker"
### Community 115 - "ADR-0006: The browser runs on the home machine, over the tailnet"
Cohesion: 0.33
Nodes (5): ADR-0006: The browser runs on the home machine, over the tailnet, Consequences, Constraints, Decision, Why
### Community 116 - "ADR-0008: A Series identity is discovered from the Site's links, never derived from an address"
Cohesion: 0.33
Nodes (5): ADR-0008: A Series identity is discovered from the Site's links, never derived from an address, Consequences, Considered options, Decision, Why
### Community 117 - "Domain Docs"
Cohesion: 0.33
Nodes (5): Before exploring, read these, Domain Docs, File structure, Flag ADR conflicts, Use the glossary's vocabulary
### Community 118 - "ticket-implementer.md"
Cohesion: 0.33
Nodes (5): Escalate rather than guess, Order of work, Report, Review, The worktree is your whole world
### Community 119 - "implementer.md"
Cohesion: 0.33
Nodes (5): Before You Begin, Report Format, Self-Review Before Reporting, When You're in Over Your Head, Your Job
### Community 120 - "Series is a shared entity, and only the Poll may update it"
Cohesion: 0.40
Nodes (4): Consequences, Only the Poll writes Series fields, Series is a shared entity, and only the Poll may update it, Why
### Community 121 - "Postgres replaces SQLite as the primary datastore"
Cohesion: 0.50
Nodes (3): Consequences, Considered options, Postgres replaces SQLite as the primary datastore
### Community 122 - "Identity comes from Discord OAuth; we store no passwords and send no email"
Cohesion: 0.50
Nodes (3): Consequences, Considered options, Identity comes from Discord OAuth; we store no passwords and send no email
### Community 123 - "The wire format stays flat and deliberately does not mirror the schema"
Cohesion: 0.50
Nodes (3): Consequence, The wire format stays flat and deliberately does not mirror the schema, Why a future reader will find this surprising
### Community 124 - "ADR-0005: On-demand browser sidecar"
Cohesion: 0.50
Nodes (3): ADR-0005: On-demand browser sidecar, Constraints, Decision
### Community 125 - "sessions_test.go"
Cohesion: 0.48
Nodes (6): T, TestCreateAndGetSession(), TestDeleteSessionIsPerReader(), TestDeleteSessionRevokes(), TestExpiredSessionIsGone(), TestGetSessionUnknownID()
### Community 273 - "AGENTS.md"
Cohesion: 0.50
Nodes (3): Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust), Second script: `novel-bookmark.user.js`, Userscript structure (single IIFE, `manga-bookmark.user.js`)
## Knowledge Gaps
- **524 isolated node(s):** `bookmarkmanager/backend`, `ctxKey`, `loginView`, `ctxKey`, `test` (+519 more)
These have ≤1 connection - possible missing edges or undocumented components.
- **154 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes.
## Suggested Questions
_Questions this graph is uniquely positioned to answer:_
- **Why does `New()` connect `Series Acquisition Tests` to `Open`, `Bookmarks API Tests`, `Cover & Acquire Internals`, `Session Middleware`, `Web UI Handlers`?**
_High betweenness centrality (0.045) - this node is a cross-community bridge._
- **Why does `Open()` connect `Open` to `Cover Fetch Test Helpers`, `Series Acquisition Tests`, `Bookmarks API Tests`, `pgtest.go`, `Store Tests`, `Store`?**
_High betweenness centrality (0.033) - this node is a cross-community bridge._
- **Why does `newRouter()` connect `Bookmarks API Tests` to `Cover Fetch Test Helpers`, `Bookmarks API Handler`, `Series Acquisition Tests`?**
_High betweenness centrality (0.027) - this node is a cross-community bridge._
- **Are the 47 inferred relationships involving `testConfig()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverNeverEchoesNonImage()`) actually correct?**
_`testConfig()` has 47 INFERRED edges - model-reasoned connections that need verification._
- **Are the 8 inferred relationships involving `newWebTestServer()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverRejectsUnknownAddress()`) actually correct?**
_`newWebTestServer()` has 8 INFERRED edges - model-reasoned connections that need verification._
- **Are the 12 inferred relationships involving `newTestStore()` (e.g. with `TestAcquireDoesNotBlockTheWrite()` and `TestAcquireFailureLeavesTheBookmarkIntact()`) actually correct?**
_`newTestStore()` has 12 INFERRED edges - model-reasoned connections that need verification._
- **What connects `bookmarkmanager/backend`, `ctxKey`, `loginView` to the rest of the system?**
_557 weakly-connected nodes found - possible documentation gaps or missing edges._
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large Load Diff
-637
View File
@@ -1,637 +0,0 @@
{
".agents/skills/golang-code-style/evals/evals.json": {
"mtime": 1784884678.6627614,
"ast_hash": "bec0e12446e7af3cd05de9b6d42badd8",
"semantic_hash": "bec0e12446e7af3cd05de9b6d42badd8"
},
".agents/skills/golang-error-handling/evals/evals.json": {
"mtime": 1784884678.6655047,
"ast_hash": "275d710b774fba1e1d0bc098d3646c6d",
"semantic_hash": "275d710b774fba1e1d0bc098d3646c6d"
},
".agents/skills/golang-performance/evals/evals.json": {
"mtime": 1784884678.6688662,
"ast_hash": "4f06df87f90aa0e4f6deaa318b47689f",
"semantic_hash": "4f06df87f90aa0e4f6deaa318b47689f"
},
".agents/skills/golang-testing/evals/evals.json": {
"mtime": 1784884678.6721346,
"ast_hash": "60a821bbfd20c6fe8bba996b8b540dd4",
"semantic_hash": "60a821bbfd20c6fe8bba996b8b540dd4"
},
"backend/go.mod": {
"mtime": 1786216141.668644,
"ast_hash": "dac242903b0e98c3e4395159d609e08e",
"semantic_hash": "dac242903b0e98c3e4395159d609e08e"
},
"backend/main.go": {
"mtime": 1786859325.126449,
"ast_hash": "4ad77c286522f97d14b6009ff37118e5",
"semantic_hash": ""
},
"skills-lock.json": {
"mtime": 1784884678.6842625,
"ast_hash": "4a94ac85bad6bce330d085bcc0ae3ffd",
"semantic_hash": "4a94ac85bad6bce330d085bcc0ae3ffd"
},
"userscript/manga-bookmark.user.js": {
"mtime": 1786499529.779988,
"ast_hash": "1f8bcddd3632d709f058a8401af8f127",
"semantic_hash": ""
},
".agents/skills/find-skills/SKILL.md": {
"mtime": 1784884338.760326,
"ast_hash": "62b297abdee9aea84c577ab2e04e1974",
"semantic_hash": "62b297abdee9aea84c577ab2e04e1974"
},
".agents/skills/golang-code-style/SKILL.md": {
"mtime": 1784884678.6623824,
"ast_hash": "d6a01e6f64550a5c8d59dac2e948000e",
"semantic_hash": "d6a01e6f64550a5c8d59dac2e948000e"
},
".agents/skills/golang-code-style/references/details.md": {
"mtime": 1784884678.6627865,
"ast_hash": "19891e396a986f1b24bf34b7a2854fcb",
"semantic_hash": "19891e396a986f1b24bf34b7a2854fcb"
},
".agents/skills/golang-error-handling/SKILL.md": {
"mtime": 1784884678.665052,
"ast_hash": "8b7970f472adb240e5bc4bde863d43a6",
"semantic_hash": "8b7970f472adb240e5bc4bde863d43a6"
},
".agents/skills/golang-error-handling/references/error-creation.md": {
"mtime": 1784884678.665524,
"ast_hash": "248dbf75492c68faef2334b8d83bd080",
"semantic_hash": "248dbf75492c68faef2334b8d83bd080"
},
".agents/skills/golang-error-handling/references/error-handling.md": {
"mtime": 1784884678.6655412,
"ast_hash": "c2424ee3999b05963b199e0100727aa3",
"semantic_hash": "c2424ee3999b05963b199e0100727aa3"
},
".agents/skills/golang-error-handling/references/error-wrapping.md": {
"mtime": 1784884678.6655717,
"ast_hash": "4a15d9266a1ea0c8bb1951e6b9c0f586",
"semantic_hash": "4a15d9266a1ea0c8bb1951e6b9c0f586"
},
".agents/skills/golang-performance/SKILL.md": {
"mtime": 1784884678.667833,
"ast_hash": "35a15fd129c5bedaada3fd4df0b6ba8c",
"semantic_hash": "35a15fd129c5bedaada3fd4df0b6ba8c"
},
".agents/skills/golang-performance/assets/prometheus-alerts.yml": {
"mtime": 1784884678.668527,
"ast_hash": "fa9357ffa87c4f894fc21afa9707c4db",
"semantic_hash": "fa9357ffa87c4f894fc21afa9707c4db"
},
".agents/skills/golang-performance/references/caching.md": {
"mtime": 1784884678.6690567,
"ast_hash": "807c42a82994e5548dbf7eb6e30c71e0",
"semantic_hash": "807c42a82994e5548dbf7eb6e30c71e0"
},
".agents/skills/golang-performance/references/cpu.md": {
"mtime": 1784884678.669087,
"ast_hash": "6d52e532ef51a35cc4134694c944517d",
"semantic_hash": "6d52e532ef51a35cc4134694c944517d"
},
".agents/skills/golang-performance/references/io-networking.md": {
"mtime": 1784884678.6691036,
"ast_hash": "95c5dd51f728fd69c945a92ff021d766",
"semantic_hash": "95c5dd51f728fd69c945a92ff021d766"
},
".agents/skills/golang-performance/references/memory.md": {
"mtime": 1784884678.6691158,
"ast_hash": "3b2108df06b4cfb3980fa80bbd9ebcff",
"semantic_hash": "3b2108df06b4cfb3980fa80bbd9ebcff"
},
".agents/skills/golang-performance/references/observability.md": {
"mtime": 1784884678.6691446,
"ast_hash": "0aa8a498e8d55ccdd4990ad187bae828",
"semantic_hash": "0aa8a498e8d55ccdd4990ad187bae828"
},
".agents/skills/golang-performance/references/runtime.md": {
"mtime": 1784884678.669426,
"ast_hash": "26386c33b3a3794aaef0556713bf8f3a",
"semantic_hash": "26386c33b3a3794aaef0556713bf8f3a"
},
".agents/skills/golang-testing/SKILL.md": {
"mtime": 1784884678.6716368,
"ast_hash": "0a9b9793bba2a239db94e980272a393e",
"semantic_hash": "0a9b9793bba2a239db94e980272a393e"
},
".agents/skills/golang-testing/references/helpers.md": {
"mtime": 1784884678.6722167,
"ast_hash": "0aecb7cfbeb9b374bf61c52e324d9d3f",
"semantic_hash": "0aecb7cfbeb9b374bf61c52e324d9d3f"
},
".agents/skills/golang-testing/references/http-testing.md": {
"mtime": 1784884678.67225,
"ast_hash": "9111110c28a7fbbffc3537aad786b390",
"semantic_hash": "9111110c28a7fbbffc3537aad786b390"
},
".agents/skills/golang-testing/references/integration-testing.md": {
"mtime": 1784884678.67225,
"ast_hash": "fcf9861bc36ae56e3fb7a1c18aa0e77f",
"semantic_hash": "fcf9861bc36ae56e3fb7a1c18aa0e77f"
},
".agents/skills/golang-testing/references/mocking.md": {
"mtime": 1784884678.6722653,
"ast_hash": "3a08979e4603aae5c32a58d5b6c39765",
"semantic_hash": "3a08979e4603aae5c32a58d5b6c39765"
},
"CLAUDE.md": {
"mtime": 1786857336.6414917,
"ast_hash": "c79e49f912d7852f9832565a5f9a1c39",
"semantic_hash": ""
},
"DEPLOY.md": {
"mtime": 1786859620.9977603,
"ast_hash": "b7c2f813aded562ee9291c9baed795ad",
"semantic_hash": ""
},
"README.md": {
"mtime": 1786859273.3238454,
"ast_hash": "81af12a0a2d43e791efd030b5f4d6cbc",
"semantic_hash": ""
},
"docker-compose.prod.yml": {
"mtime": 1786292465.8305523,
"ast_hash": "0751998a532297b8ac507a01ec48dc31",
"semantic_hash": "0751998a532297b8ac507a01ec48dc31"
},
"docker-compose.yml": {
"mtime": 1786859258.6255276,
"ast_hash": "d3b53a8f42a8e0acb4fc4306ea42c093",
"semantic_hash": ""
},
".claude/settings.json": {
"mtime": 1784951973.1869545,
"ast_hash": "e51077b6a7f1f67afc748f1a32a1557d",
"semantic_hash": "e51077b6a7f1f67afc748f1a32a1557d"
},
"backend/web_test.go": {
"mtime": 1786216141.700644,
"ast_hash": "8f1b093b59eb1ed81bc7fc0c22495c50",
"semantic_hash": "8f1b093b59eb1ed81bc7fc0c22495c50"
},
"backend/main_test.go": {
"mtime": 1786859172.6449091,
"ast_hash": "0a5d4dbdc770b40c329ccccc899b59f4",
"semantic_hash": ""
},
".claude/settings.local.json": {
"mtime": 1785697645.350201,
"ast_hash": "9a1ac6369f968e8df4be9dcff0948f70",
"semantic_hash": "9a1ac6369f968e8df4be9dcff0948f70"
},
"PRODUCT.md": {
"mtime": 1786216141.660644,
"ast_hash": "c52072d1978286060087fa0686f9c7f9",
"semantic_hash": "c52072d1978286060087fa0686f9c7f9"
},
"backend/.impeccable/critique/2026-07-26T15-50-42Z__backend-templates-app-html.md": {
"mtime": 1785128576.2412457,
"ast_hash": "d08627c27f22d453125db6c1ab4b71ec",
"semantic_hash": "d08627c27f22d453125db6c1ab4b71ec"
},
"backend/.impeccable/critique/2026-07-26T17-08-41Z__backend-templates-app-html.md": {
"mtime": 1785128576.2452524,
"ast_hash": "e69a8340a371579ca3ea689660f7d7bd",
"semantic_hash": "e69a8340a371579ca3ea689660f7d7bd"
},
"AGENTS.md": {
"mtime": 1786857336.6414917,
"ast_hash": "c79e49f912d7852f9832565a5f9a1c39",
"semantic_hash": ""
},
"userscript/test/logic.test.js": {
"mtime": 1786499529.779988,
"ast_hash": "80d512bfe6fa8b847f3ba6169c321a74",
"semantic_hash": ""
},
".claude/skills/testing-the-userscript/SKILL.md": {
"mtime": 1786363889.5489495,
"ast_hash": "8f3c0132eb4787a2c8736eb99f7689af",
"semantic_hash": "8f3c0132eb4787a2c8736eb99f7689af"
},
"REDEPLOY.md": {
"mtime": 1786857336.6414917,
"ast_hash": "e5e910f0244a040e2ccdc669b17eb4db",
"semantic_hash": ""
},
"docs/design-system.md": {
"mtime": 1786022513.9623306,
"ast_hash": "421cd7e57f02d4b467f120ca6ddd7b6a",
"semantic_hash": "421cd7e57f02d4b467f120ca6ddd7b6a"
},
"backend/api_test.go": {
"mtime": 1786499529.7651505,
"ast_hash": "8e4b9293bc2e45ee3f42027315594fd5",
"semantic_hash": ""
},
"backend/cover_test.go": {
"mtime": 1786363889.552731,
"ast_hash": "c7e313d6c92eb28e6d370e5e89035984",
"semantic_hash": "c7e313d6c92eb28e6d370e5e89035984"
},
"backend/internal/api/handlers.go": {
"mtime": 1786363889.552731,
"ast_hash": "59e6b8767ab19839bb8f82891a7e4616",
"semantic_hash": "59e6b8767ab19839bb8f82891a7e4616"
},
"backend/internal/httpmw/middleware.go": {
"mtime": 1786216141.672644,
"ast_hash": "385b36f58488b7e6d93eb6d6034e9ee3",
"semantic_hash": "385b36f58488b7e6d93eb6d6034e9ee3"
},
"backend/internal/latest/browser.go": {
"mtime": 1786859355.8677974,
"ast_hash": "75f34a974568d122681939dd297944a2",
"semantic_hash": ""
},
"backend/internal/latest/browser_test.go": {
"mtime": 1786857336.6414917,
"ast_hash": "800fa6aa471ada054a3c48943ad17ab7",
"semantic_hash": ""
},
"backend/internal/latest/fetch.go": {
"mtime": 1786499529.7688599,
"ast_hash": "3e20ad86aa46783e9aa95c2b746551ee",
"semantic_hash": ""
},
"backend/internal/latest/poller.go": {
"mtime": 1786860083.6713927,
"ast_hash": "be728405cde3df49baf72bd7837f2971",
"semantic_hash": ""
},
"backend/internal/latest/poller_test.go": {
"mtime": 1786860168.103984,
"ast_hash": "5dba515d28c0523500c1ceb581a54e58",
"semantic_hash": ""
},
"backend/internal/latest/sites.go": {
"mtime": 1786860007.8224697,
"ast_hash": "9b90f9b710ccbd0d0a0a6a0dd721ff3c",
"semantic_hash": ""
},
"backend/internal/latest/sites_test.go": {
"mtime": 1786857336.6414917,
"ast_hash": "0db2028a6073f342fee61ad15c2e5f0f",
"semantic_hash": ""
},
"backend/internal/latest/smoke_image_test.go": {
"mtime": 1786363889.5602942,
"ast_hash": "db068cb59575f8c82669acbaf84bcaed",
"semantic_hash": "db068cb59575f8c82669acbaf84bcaed"
},
"backend/internal/pgtest/pgtest.go": {
"mtime": 1786216141.6766438,
"ast_hash": "f60372d41516e66f7aaeb272da227d6e",
"semantic_hash": "f60372d41516e66f7aaeb272da227d6e"
},
"backend/internal/session/session.go": {
"mtime": 1786216141.6766438,
"ast_hash": "9952474ffb22c825d6f075b866ee26f4",
"semantic_hash": "9952474ffb22c825d6f075b866ee26f4"
},
"backend/internal/session/session_test.go": {
"mtime": 1786216141.680644,
"ast_hash": "37ffd00964e7a67350c68ed50c6503c5",
"semantic_hash": "37ffd00964e7a67350c68ed50c6503c5"
},
"backend/internal/store/migrations/0001_bookmarks.sql": {
"mtime": 1786216141.680644,
"ast_hash": "f87ccfb2c25c43f93021177ced0bfae4",
"semantic_hash": "f87ccfb2c25c43f93021177ced0bfae4"
},
"backend/internal/store/migrations/0002_series.sql": {
"mtime": 1786216141.6820722,
"ast_hash": "5dc98771e0c0f6e8416b434c280b0efb",
"semantic_hash": "5dc98771e0c0f6e8416b434c280b0efb"
},
"backend/internal/store/migrations/0003_reader.sql": {
"mtime": 1786216141.6820722,
"ast_hash": "444a97799f38f6222f87d4e9fb2d6258",
"semantic_hash": "444a97799f38f6222f87d4e9fb2d6258"
},
"backend/internal/store/migrations/0004_owner_bookmarks.sql": {
"mtime": 1786216141.684644,
"ast_hash": "e4fa900cc223865d3ecd4c60c5707a65",
"semantic_hash": "e4fa900cc223865d3ecd4c60c5707a65"
},
"backend/internal/store/migrations/0005_sessions.sql": {
"mtime": 1786216141.684644,
"ast_hash": "5158887ebc57cf8c16a7b821b61cd760",
"semantic_hash": "5158887ebc57cf8c16a7b821b61cd760"
},
"backend/internal/store/migrations/0006_reader_token_epoch.sql": {
"mtime": 1786216141.684644,
"ast_hash": "3093cc1c3aae0cd9643d04105d045402",
"semantic_hash": "3093cc1c3aae0cd9643d04105d045402"
},
"backend/internal/store/sessions.go": {
"mtime": 1786216141.684644,
"ast_hash": "eee3510cc6172ef4b1da820474c26b01",
"semantic_hash": "eee3510cc6172ef4b1da820474c26b01"
},
"backend/internal/store/sessions_test.go": {
"mtime": 1786216141.684644,
"ast_hash": "0b6764a0ee20f5cb7748eecd31a1d220",
"semantic_hash": "0b6764a0ee20f5cb7748eecd31a1d220"
},
"backend/internal/store/store.go": {
"mtime": 1786860012.3431706,
"ast_hash": "c62e73142386c58ab833d404d0c24dc4",
"semantic_hash": ""
},
"backend/internal/store/store_test.go": {
"mtime": 1786858248.9426548,
"ast_hash": "039729554517960d1ae98f8409baf4ba",
"semantic_hash": ""
},
"backend/internal/userscript/userscript.go": {
"mtime": 1786216141.692644,
"ast_hash": "aa13a71b1c9eefe4930fd31f27722328",
"semantic_hash": "aa13a71b1c9eefe4930fd31f27722328"
},
"backend/internal/userscript/userscript_test.go": {
"mtime": 1786216141.692644,
"ast_hash": "6c050968d7b8b67956da1a3136d2c3c7",
"semantic_hash": "6c050968d7b8b67956da1a3136d2c3c7"
},
"backend/internal/web/discord.go": {
"mtime": 1786216141.692644,
"ast_hash": "69e4959c65fa67d7491aedf6a71bb575",
"semantic_hash": "69e4959c65fa67d7491aedf6a71bb575"
},
"backend/internal/web/oauth_test.go": {
"mtime": 1786216141.692644,
"ast_hash": "a3bddeb70dd8d7eb14139da808723ea8",
"semantic_hash": "a3bddeb70dd8d7eb14139da808723ea8"
},
"backend/internal/web/static/filter.js": {
"mtime": 1786022513.9473197,
"ast_hash": "b4ee3306201bfd88b148b96801972617",
"semantic_hash": "b4ee3306201bfd88b148b96801972617"
},
"backend/internal/web/static/htmx.min.js": {
"mtime": 1785873769.5512016,
"ast_hash": "19a573773be4ca22570ca2f8543120c5",
"semantic_hash": "19a573773be4ca22570ca2f8543120c5"
},
"backend/internal/web/web.go": {
"mtime": 1786363889.5678573,
"ast_hash": "8308949c658d3a08ce1c3cdbea6a907c",
"semantic_hash": "8308949c658d3a08ce1c3cdbea6a907c"
},
"backend/reader_credential_test.go": {
"mtime": 1786216141.700644,
"ast_hash": "2a751ea6d9c06635aa3179db0aef1b2b",
"semantic_hash": "2a751ea6d9c06635aa3179db0aef1b2b"
},
"chrome/entrypoint.sh": {
"mtime": 1786363889.5678573,
"ast_hash": "8008a187690764436540fab47ba0cfcc",
"semantic_hash": "8008a187690764436540fab47ba0cfcc"
},
"userscript/novel-bookmark.user.js": {
"mtime": 1786499529.779988,
"ast_hash": "834effb0821f8d6c9f57f6554a5db462",
"semantic_hash": ""
},
"userscript/test/novel-logic.test.js": {
"mtime": 1786499529.779988,
"ast_hash": "b25a7377af210dd0aff8284501fd0252",
"semantic_hash": ""
},
".opencode/agent/implementer.md": {
"mtime": 1785873769.5402634,
"ast_hash": "000de469c18d68352027e10c8ce8acfb",
"semantic_hash": "000de469c18d68352027e10c8ce8acfb"
},
".opencode/agent/reviewer.md": {
"mtime": 1785873769.5416775,
"ast_hash": "e44a2f6f624db044e19508bc5ab05592",
"semantic_hash": "e44a2f6f624db044e19508bc5ab05592"
},
"CONTEXT.md": {
"mtime": 1786859395.4497814,
"ast_hash": "4aafbce0b046e6e34734fb414df818ce",
"semantic_hash": ""
},
"CUTOVER.md": {
"mtime": 1786216141.660644,
"ast_hash": "6c6f3e4c4c2f57867894280bce728c50",
"semantic_hash": "6c6f3e4c4c2f57867894280bce728c50"
},
"backend/AGENTS.md": {
"mtime": 1786860230.7890837,
"ast_hash": "7ceec7c4576f6e91eaf02c200a0dd1b3",
"semantic_hash": ""
},
"backend/CLAUDE.md": {
"mtime": 1786860230.7890837,
"ast_hash": "7ceec7c4576f6e91eaf02c200a0dd1b3",
"semantic_hash": ""
},
"backend/internal/web/templates/app.html": {
"mtime": 1786216141.692644,
"ast_hash": "3965e20e204afb71ba2a3aa86cb7c61c",
"semantic_hash": "3965e20e204afb71ba2a3aa86cb7c61c"
},
"backend/internal/web/templates/card.html": {
"mtime": 1786363889.5678573,
"ast_hash": "ab83ae0dbb34fd40c146a7cc1263173e",
"semantic_hash": "ab83ae0dbb34fd40c146a7cc1263173e"
},
"backend/internal/web/templates/chrome.html": {
"mtime": 1786363889.5678573,
"ast_hash": "d80b27cf3bd9d131075c485dd169dfcc",
"semantic_hash": "d80b27cf3bd9d131075c485dd169dfcc"
},
"backend/internal/web/templates/icons.html": {
"mtime": 1785873769.553139,
"ast_hash": "8e10c507c32934a92463b4bca9e34fe6",
"semantic_hash": "8e10c507c32934a92463b4bca9e34fe6"
},
"backend/internal/web/templates/list.html": {
"mtime": 1786216141.696644,
"ast_hash": "365548aace8c06559a1f66db0ae47256",
"semantic_hash": "365548aace8c06559a1f66db0ae47256"
},
"backend/internal/web/templates/login.html": {
"mtime": 1786216141.696644,
"ast_hash": "bcc3101498a66cf8b79f9d97c6c9cd6b",
"semantic_hash": "bcc3101498a66cf8b79f9d97c6c9cd6b"
},
"backend/internal/web/templates/readers.html": {
"mtime": 1786216141.696644,
"ast_hash": "c5034e76bd20a705d2799cb5ecb328f0",
"semantic_hash": "c5034e76bd20a705d2799cb5ecb328f0"
},
"backend/internal/web/templates/setup.html": {
"mtime": 1786216141.696644,
"ast_hash": "72e93c0b827414063596f7338987d879",
"semantic_hash": "72e93c0b827414063596f7338987d879"
},
"docs/adr/0001-postgresql-over-sqlite.md": {
"mtime": 1786216141.704644,
"ast_hash": "abfb08754cee58be67311377904f8ca4",
"semantic_hash": "abfb08754cee58be67311377904f8ca4"
},
"docs/adr/0002-discord-oauth-no-passwords-no-email.md": {
"mtime": 1786216141.7071996,
"ast_hash": "852a04d86659385085da6ffc8b489933",
"semantic_hash": "852a04d86659385085da6ffc8b489933"
},
"docs/adr/0003-series-shared-and-poll-owned.md": {
"mtime": 1786501942.7367291,
"ast_hash": "6138b113340693e0cc667d1ecbb75f72",
"semantic_hash": ""
},
"docs/adr/0004-wire-format-does-not-mirror-the-schema.md": {
"mtime": 1786216141.7071996,
"ast_hash": "a6ea2770dec2156f78a65b35ba06902a",
"semantic_hash": "a6ea2770dec2156f78a65b35ba06902a"
},
"docs/agents/domain.md": {
"mtime": 1786216141.7071996,
"ast_hash": "6f99318ac6cb9825b613bfde55d76091",
"semantic_hash": "6f99318ac6cb9825b613bfde55d76091"
},
"docs/agents/issue-tracker.md": {
"mtime": 1786216141.7087462,
"ast_hash": "1342e66ccb84a84fd579fb6dc0b8243a",
"semantic_hash": "1342e66ccb84a84fd579fb6dc0b8243a"
},
"docs/agents/triage-labels.md": {
"mtime": 1786216141.7087462,
"ast_hash": "69114d07ed792d6bb1d13758ba5435e1",
"semantic_hash": "69114d07ed792d6bb1d13758ba5435e1"
},
"userscript/AGENTS.md": {
"mtime": 1786499529.7762787,
"ast_hash": "e276ffe9a6e7b55fd3235466a1995c22",
"semantic_hash": ""
},
"userscript/CLAUDE.md": {
"mtime": 1786499529.7762787,
"ast_hash": "e276ffe9a6e7b55fd3235466a1995c22",
"semantic_hash": ""
},
"backend/internal/web/static/login-art.png": {
"mtime": 1786022513.9585779,
"ast_hash": "05d7863cba344a946256719a0c9ef959",
"semantic_hash": "05d7863cba344a946256719a0c9ef959"
},
"backend/internal/web/static/logo.svg": {
"mtime": 1786022513.9585779,
"ast_hash": "d0d34d0f08a25b53176cc55989b7babe",
"semantic_hash": "d0d34d0f08a25b53176cc55989b7babe"
},
"backend/internal/store/migrations/0007_covers.sql": {
"mtime": 1786262323.6964688,
"ast_hash": "6033ce0701be1236ed175362363bd96c",
"semantic_hash": "6033ce0701be1236ed175362363bd96c"
},
"docs/adr/0005-on-demand-browser.md": {
"mtime": 1786292465.8305523,
"ast_hash": "8cf2fb8c0a66c2c7d82ae433b99ca42b",
"semantic_hash": "8cf2fb8c0a66c2c7d82ae433b99ca42b"
},
"chrome/docker-compose.yml": {
"mtime": 1786292465.8305523,
"ast_hash": "5605599395a3f085904e78a2bfec1e58",
"semantic_hash": "5605599395a3f085904e78a2bfec1e58"
},
"docs/adr/0006-browser-on-the-home-machine.md": {
"mtime": 1786501942.7367291,
"ast_hash": "bfcaf39b6c7e96610a7caa00eeb36533",
"semantic_hash": ""
},
"docs/adr/0007-backend-hosts-cover-bytes.md": {
"mtime": 1786292465.8305523,
"ast_hash": "b19e38045b3dcda7dd59634ed9227a68",
"semantic_hash": "b19e38045b3dcda7dd59634ed9227a68"
},
"backend/internal/store/migrations/0008_filesystem_covers.sql": {
"mtime": 1786363889.5602942,
"ast_hash": "46cf7822d4f667e3cab36b547abe5e97",
"semantic_hash": "46cf7822d4f667e3cab36b547abe5e97"
},
"backend/internal/latest/cover.go": {
"mtime": 1786857336.6414917,
"ast_hash": "d5f2248c3d11de74bf5a3977651b17c2",
"semantic_hash": ""
},
"backend/internal/latest/cover_fetch_test.go": {
"mtime": 1786363889.5565126,
"ast_hash": "60d9eb7c59a3751baf4f31c7655217e7",
"semantic_hash": "60d9eb7c59a3751baf4f31c7655217e7"
},
"backend/internal/latest/acquire.go": {
"mtime": 1786859346.3297389,
"ast_hash": "d605e3c94a62fc7787efbc139696b9d2",
"semantic_hash": ""
},
"backend/internal/latest/acquire_test.go": {
"mtime": 1786363889.5565126,
"ast_hash": "7bd9f41814f6bf59d8998dfb81cf990a",
"semantic_hash": "7bd9f41814f6bf59d8998dfb81cf990a"
},
"backend/internal/store/migrations/0009_series_cover_address.sql": {
"mtime": 1786363889.5602942,
"ast_hash": "4c1f6328b2e1a95828fad6d88d474c2d",
"semantic_hash": "4c1f6328b2e1a95828fad6d88d474c2d"
},
".claude/skills/implement-tickets/SKILL.md": {
"mtime": 1786417841.7117643,
"ast_hash": "3060e32e19cc571d91871a98f18afe53",
"semantic_hash": "3060e32e19cc571d91871a98f18afe53"
},
".omp/agents/ticket-implementer.md": {
"mtime": 1786417841.7163916,
"ast_hash": "0150a46c0d21c572b71b3d87d21ac925",
"semantic_hash": "0150a46c0d21c572b71b3d87d21ac925"
},
"docs/adr/0008-series-identity-is-discovered-not-derived.md": {
"mtime": 1786417841.7163916,
"ast_hash": "3ce6d64ef6a8a39f27c257b39065389e",
"semantic_hash": "3ce6d64ef6a8a39f27c257b39065389e"
},
"docs/research/lightnovelworld-chapter-vs-series-slug.md": {
"mtime": 1786417841.7163916,
"ast_hash": "74a4e538875f0a7e8ca3d5dc48c0bb53",
"semantic_hash": "74a4e538875f0a7e8ca3d5dc48c0bb53"
},
"backend/internal/latest/smoke_lnw_test.go": {
"mtime": 1786499529.7725692,
"ast_hash": "2d65da8a081759172918fdf159760f45",
"semantic_hash": ""
},
"docs/adr/0009-a-site-answers-questions-its-own-way.md": {
"mtime": 1786499529.7725692,
"ast_hash": "8039012a5b6de2359ff1a47079f51b66",
"semantic_hash": ""
},
"backend/internal/latest/read.go": {
"mtime": 1786859355.8679621,
"ast_hash": "9f039cc3ad74f803d7621f7ef4157bf2",
"semantic_hash": ""
},
"docs/research/cloudflare-bot-scoring-and-poll-cadence.md": {
"mtime": 1786501942.7367291,
"ast_hash": "1aa17575ab20f2f36583602999a6a60f",
"semantic_hash": ""
},
"backend/internal/latest/smoke_comix_test.go": {
"mtime": 1786857336.6414917,
"ast_hash": "111fdbb75fc68ac2ab1013bc916063cf",
"semantic_hash": ""
},
"docs/adr/0010-poll-lanes-per-site-pace.md": {
"mtime": 1786860214.4637265,
"ast_hash": "dc19d75f034ca920d93b9c71e6ca28e6",
"semantic_hash": ""
}
}
+94 -84
View File
@@ -1,93 +1,103 @@
Guidance for OpenCode (and Claude Code) working under `userscript/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system.
Scope: `userscript/`.
### Userscript structure (single IIFE, `manga-bookmark.user.js`)
Each entry names the code that holds the truth. The prose is only what the code
cannot tell you: rationale, invariants a refactor would break, and dated
observations about sites we don't control.
1. **Site adapters** — one per host, `detect(location, document)` return page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title` from **`og:title`** (or the page heading where a site ships no og: tags), not CSS classes. **No adapter reads a cover**: the backend acquires, stores and serves every Cover from its own origin (ADR-0007), the wire's `cover` is already an address on our origin, and `apiPut` strips any `cover` off an outgoing body.
2. **API client** — `apiGet/apiPut/apiDelete` with bearer header; `localStorage` key `bmgr:manga:cache` for instant render + offline fallback.
3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value.
4. **Retry queue** — every write go through `pushBookmark`/`pushDelete`, so
failed mutation park in `localStorage` (`bmgr:manga:queue`) and replayed on
next navigation, reconnect, or `refresh()`. Entries are markers
(`{key, op, sendStatus, attempts}`), never payloads — body read from
cache at send time, so one entry per key give ordering and coalescing for
free. `sendStatus` is **sticky**: while archive pending, later writes to
that key keep carrying bucket, which stop successful
in-between write from silently un-archiving series. `refresh()` drains
before it fetches and overlays anything still pending, so list never
flaps. 400 drops entry, 401 abort pass and keep queue, and
transient failures retry to cap of 10. Latest-chapter writes deliberately
stay out of queue. See
`docs/superpowers/specs/2026-07-27-offline-retry-queue-design.md`.
5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS
(critical on mobile). Three tabs (All / Favourites / Archived) and row of
link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG
block next to `API_BASE`. FAB is `7 × 44` edge tab whose *hit* area
widened to `28 × 72` by invisible `#hit` child; `#fab` must keep
`touch-action: none` and must **not** regain `overflow: hidden`. Since
`touch-action` resolved at gesture start, strip can't be both
browser-scrolled and script-dragged, so `makeDraggable` splits by intent: swipe
from `#hit` scrolls via `window.scrollBy`, hold of `ARM_MS` arms
reposition drag, visible sliver drags with no hold. See
`docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`.
6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fire without reload. Demonic uses classic reloads (initial `document-idle` run suffice).
### Structure — single IIFE, `manga-bookmark.user.js`
### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust)
Six parts, in file order: site adapters, API client, progress logic, retry
queue, UI, SPA navigation.
- **asurascans.com**: series `/comics/<slug>` (slug carries trailing
site-wide build-hash suffix, e.g. `-059befe1`, that **rotates on every
redeploy**), chapter `/comics/<slug>/chapter/<n>`. `seriesId` must strip
hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in userscript,
`asuraBuildHash` in backend); URLs keep full slug — stale-hash
URLs 302 to current ones. Astro-rendered; chapter links present in raw
server HTML.
- **demonicscans.org**: series `/manga/<slug>` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title/<slug>/chapter/<n>/<page>` (older `chaptered.php?manga=<id>&chapter=<n>` form still exists as redirect, what series-page chapter-list anchors link through).
Encodings (incl. triple-encoded punctuation like `%25252D`) identical
on /manga/ and /title/ pages, so decode-once seriesIds match — verified
2026-07-28.
- **comix.to**: series `/title/<id>-<slug>`, chapter
`/title/<id>-<slug>/<uploadId>-chapter-<n>`. Only the leading `<id>` is
identity — the slug re-renders when a series is renamed (`comixSeriesId`).
An SPA that **never rewrites `og:title`**: the server-rendered head keeps
whatever document loaded first, so on a cold load `og:title` is the homepage's
"Comix — Read Comics online for free" and after an in-page hop it is the
*previous* series' name. `document.title` is the one thing client routing does
update, so titles come from there, with the chapter page's `" · Ch.<n>"` tail
stripped. It publishes no `og:image` either, which is one of the reasons cover
**Site adapters** — one per host, `detect(location, document)` returning page
`type` + IDs.
- Identify type and IDs from **URL regex**, which is the most stable surface a
site exposes; take `title` from **`og:title`** (or the page heading where a
site ships no og: tags), never CSS classes.
- **No adapter reads a cover.** The backend acquires, stores and serves every
Cover from its own origin, the wire `cover` is already an address there, and
`apiPut` strips any `cover` off an outgoing body.
**Progress logic** — auto-upsert `last_chapter` only when
`chapterNum >= stored last_chapter_num`; unparseable sets the current value.
Re-reading an old chapter must not regress progress. A manual panel override
forces any value.
**Retry queue** — every write goes through `pushBookmark`/`pushDelete`.
- Entries are markers (`{key, op, sendStatus, attempts}`), **never payloads**:
the body is read from cache at send time, so one entry per key gives ordering
and coalescing for free.
- `sendStatus` is **sticky** — while an archive is pending, later writes to that
key keep carrying the bucket. Without it a successful in-between write
silently un-archives the series.
- `refresh()` drains before it fetches and overlays anything still pending, so
the list never flaps.
- 400 drops the entry, 401 aborts the pass and keeps the queue, transient
failures retry to a cap. Latest-chapter writes deliberately stay out of the
queue.
**UI** — rendered inside a **Shadow DOM** root to isolate it from site CSS,
which is critical on mobile.
- The FAB's *hit* area is widened by an invisible `#hit` child. `#fab` must keep
`touch-action: none` and must **not** regain `overflow: hidden`.
- `touch-action` is resolved at gesture start, so the strip cannot be both
browser-scrolled and script-dragged. `makeDraggable` therefore splits by
intent: a swipe from `#hit` scrolls via `window.scrollBy`, a hold of `ARM_MS`
arms a reposition drag, and the visible sliver drags with no hold.
**SPA navigation** — Asura is Astro and client-routes on comic/chapter pages, so
`history.pushState`/`replaceState` are patched and `popstate` listened to, and
`detect()` re-runs on URL change. Demonic uses classic reloads, where the
initial `document-idle` run suffices.
### Live URL shapes
Encoded in the adapters; the notes below are the parts a reader of the regex
would get wrong. **Verified 2026-07-26 unless dated otherwise — sites drift, so
re-check against a live page before trusting any of it.**
- **asurascans.com** — the series slug carries a site-wide build-hash suffix
(e.g. `-059befe1`) that **rotates on every redeploy**, so `seriesId` must
strip it (`stripBuildHash` here, `asuraBuildHash` in the backend) while URLs
keep the full slug — stale-hash URLs 302 to current ones.
- **demonicscans.org** — slugs may URL-encode punctuation, and the older
`chaptered.php?manga=<id>&chapter=<n>` form still exists as a redirect, which
is what series-page chapter-list anchors link through. Encodings (including
triple-encoded punctuation like `%25252D`) are identical on `/manga/` and
`/title/` pages, so decode-once seriesIds match (verified 2026-07-28).
- **comix.to** — only the leading `<id>` is identity; the slug re-renders when a
series is renamed (`comixSeriesId`). It is an SPA that **never rewrites
`og:title`**: the server-rendered head keeps whatever document loaded first,
so on a cold load `og:title` is the homepage's name and after an in-page hop
it is the *previous* series'. `document.title` is the one thing client routing
updates, hence titles come from there with the chapter page's `" · Ch.<n>"`
tail stripped. It publishes no `og:image` either, one of the reasons cover
acquisition moved to the backend.
- **kagane.to**: series `/series/<uuid>`, reader
`/series/<uuid>/reader/<bookUuid>`. Reader URLs carry no chapter number, so
the number comes out of `og:title`. Two shapes exist: `"<Series> - Chapter
<n>[ - Episode <n>]"` and, for volume-numbered series, `"<Series> - Volume <v>
Chapter <n>"` with no episode name — both must yield a bare series title, or
the volume tail lands in the bookmark's title.
Its covers are challenge- and CORP-protected, so nothing outside kagane.to can
load one directly; the panel renders the backend's own cover address like every
other Site. Behind a Cloudflare JS challenge, so the backend polls it
through the headless browser.
- **novelfull.com** (novel script): series `/<slug>.html`, chapter
`/<slug>/chapter-<n>[-<title-slug>].html`. No `og:*` tags at all — title from
`h3.title` (series) or `a.truyen-title` (chapter); the script reads no cover.
Behind a Cloudflare JS challenge no TLS fingerprint
clears, so the backend polls it through the headless browser.
- **lightnovelworld.net** (novel script): series `/novel/<slug>/`, chapter
`/<slug>-chapter-<n>/` — flat, at the site root. The chapter path's slug is a
Chapter Slug, not an identity: the Series address is read off the page's
- **kagane.to** — reader URLs carry no chapter number, so the number comes out
of `og:title`. Two shapes exist, `"<Series> - Chapter <n>[ - Episode <n>]"`
and `"<Series> - Volume <v> Chapter <n>"`; both must yield a bare series
title, or the volume tail lands in the bookmark's title. Its covers are
challenge- and CORP-protected, so nothing outside kagane.to can load one —
the panel renders the backend's cover address like every other Site.
- **novelfull.com** (novel script) — no `og:*` tags at all, so the title comes
from `h3.title` (series) or `a.truyen-title` (chapter).
- **lightnovelworld.net** (novel script) — chapter paths are flat at the site
root and their slug is a **Chapter Slug, not an identity**: a Series may
publish under several. The Series address is read off the page's
`a[aria-label='All Chapter']` (fallback: the BreadcrumbList's second crumb),
and a Series may publish under several Chapter Slugs. A chapter page with no
pointer resolves to `other`, so no Bookmark is offered. `h1.entry-title` is
the clean title on a series page and `<Title> Chapter <n>` on a chapter page.
Its series page lists every chapter with an
absolute href, so the backend polls it with the plain TLS client.
The client performs no latest-chapter scan for this Site: the Poll's
one-hour cooldown dominates the client's four-hour throttle, so a scan
would add no freshness, and the page's wpdiscuz thread is a public write
surface a scan would have to truncate at. `computeLatestChapter` yields
null here and `backgroundRefreshLatest` skips the Site before any fetch.
and a chapter page with no pointer resolves to `other` so no Bookmark is
offered. The client runs **no latest-chapter scan** for this Site —
`computeLatestChapter` yields null and `backgroundRefreshLatest` skips it
before any fetch — because the backend Poll's one-hour cooldown dominates the
client's four-hour throttle, so a scan would add no freshness while having to
truncate at the page's wpdiscuz thread, a public write surface.
### Second script: `novel-bookmark.user.js`
### Second script — `novel-bookmark.user.js`
A copy of the manga script with two adapters, `LIBRARY = "novel"` and
`STORE_PREFIX = "bmgr:novel:"`. No migration loop (this script has no previous
installation to carry keys over from). Installed alongside the manga script;
both write to the same backend with the same `LIBRARY` column discriminating
them.
`STORE_PREFIX = "bmgr:novel:"`. No migration loop, because this script has no
previous installation to carry keys over from. Installed alongside the manga
script; both write to the same backend, discriminated by `LIBRARY`.
+24 -14
View File
@@ -905,27 +905,37 @@
}
}
// Records the newest chapter a site has published. Silent: this fires from
// Reports the newest chapter a site has published. Silent: this fires from
// page visits and background checks the user did not ask for, and it never
// reorders the list — updated_at is a candidate the server discards unless
// reading progress moved.
async function applyLatestChapterIfChanged(existing, latest) {
//
// An unchanged number is still sent. It is the read that lets the backend
// skip its own poll of this series (a Sighting, issue #103), so the common
// case — visiting a series with nothing new — is exactly the one worth
// reporting. Only the local write and the re-render are skipped.
async function reportLatestChapter(existing, latest) {
if (!existing || !latest) return;
if (existing.latest_chapter_num === latest.num) return;
const bm = Object.assign({}, existing, {
latest_chapter: latest.label,
latest_chapter_num: latest.num,
updated_at: Date.now(),
});
upsertLocal(bm);
render();
const changed = existing.latest_chapter_num !== latest.num;
let bm = existing;
if (changed) {
bm = Object.assign({}, existing, {
latest_chapter: latest.label,
latest_chapter_num: latest.num,
updated_at: Date.now(),
});
upsertLocal(bm);
render();
}
// A queued write owns this row; the drain sends latest_chapter
// with it, carrying the correct bucket.
if (queueGet(bm.key)) return;
try {
const saved = await apiPut(bm.key, bm);
upsertLocal(saved);
render();
if (changed) {
upsertLocal(saved);
render();
}
} catch (e) {
/* offline — the local cache still shows it, retried on a later visit */
}
@@ -937,7 +947,7 @@
if (p.type !== "series") return;
const existing = state.byKey[keyOf(p)];
if (!existing) return;
applyLatestChapterIfChanged(
reportLatestChapter(
existing,
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
);
@@ -977,7 +987,7 @@
const html = await res.text();
latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
}
await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest);
await reportLatestChapter(state.byKey[bm.key] || bm, latest);
} catch (e) {
/* offline or blocked — try again after the throttle window */
}
+24 -14
View File
@@ -809,27 +809,37 @@
}
}
// Records the newest chapter a site has published. Silent: this fires from
// Reports the newest chapter a site has published. Silent: this fires from
// page visits and background checks the user did not ask for, and it never
// reorders the list — updated_at is a candidate the server discards unless
// reading progress moved.
async function applyLatestChapterIfChanged(existing, latest) {
//
// An unchanged number is still sent. It is the read that lets the backend
// skip its own poll of this series (a Sighting, issue #103), so the common
// case — visiting a series with nothing new — is exactly the one worth
// reporting. Only the local write and the re-render are skipped.
async function reportLatestChapter(existing, latest) {
if (!existing || !latest) return;
if (existing.latest_chapter_num === latest.num) return;
const bm = Object.assign({}, existing, {
latest_chapter: latest.label,
latest_chapter_num: latest.num,
updated_at: Date.now(),
});
upsertLocal(bm);
render();
const changed = existing.latest_chapter_num !== latest.num;
let bm = existing;
if (changed) {
bm = Object.assign({}, existing, {
latest_chapter: latest.label,
latest_chapter_num: latest.num,
updated_at: Date.now(),
});
upsertLocal(bm);
render();
}
// A queued write owns this row; the drain sends latest_chapter
// with it, carrying the correct bucket.
if (queueGet(bm.key)) return;
try {
const saved = await apiPut(bm.key, bm);
upsertLocal(saved);
render();
if (changed) {
upsertLocal(saved);
render();
}
} catch (e) {
/* offline — the local cache still shows it, retried on a later visit */
}
@@ -841,7 +851,7 @@
if (p.type !== "series") return;
const existing = state.byKey[keyOf(p)];
if (!existing) return;
applyLatestChapterIfChanged(
reportLatestChapter(
existing,
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
);
@@ -883,7 +893,7 @@
if (!res.ok) continue;
const html = await res.text();
const latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest);
await reportLatestChapter(state.byKey[bm.key] || bm, latest);
} catch (e) {
/* offline or blocked — try again after the throttle window */
}