From e250762ea6d820bc0e6ecacc5136ef904c1c1fd4 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Sun, 2 Aug 2026 19:18:56 +0700 Subject: [PATCH 01/18] Move userscript to Violentmonkey, sync docs to shipped Cinder design CLAUDE.md: swap Bromite for Violentmonkey throughout, add installed golang skills to relevant skills, add comment-writing rules, add a design-system pointer rule. docs/design-system.md: rewrite against the current Claude Design project and the tokens/components already shipped in backend/static/style.css (danger/slate/moss/clay/trash, action key, brand mark). Co-Authored-By: Claude Sonnet 5 --- CLAUDE.md | 238 ++++++++++++++++++++++++------------------ docs/design-system.md | 173 +++++++++++++++++++++--------- 2 files changed, 260 insertions(+), 151 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index e719837..b57c00f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,137 +1,137 @@ # CLAUDE.md -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +Guidance for Claude Code (claude.ai/code) working in this repo. ## Status -Greenfield. Only `plans/mangaBookmark.md` exists — no code yet. That plan is the spec; read it before building. Two deliverables: a Go sync backend and a single Bromite-compatible userscript. +Greenfield. Only `plans/mangaBookmark.md` exist — no code yet. Plan = spec; read before build. Two deliverables: Go sync backend, single Violentmonkey-compatible userscript. ## What this is -A manga read-progress tracker for a user reading on **asurascans.com** (the current domain; asuracomic.net 301s here) and **demonicscans.org** from **Bromite** (mobile Chromium). A userscript injects on-page UI (floating button + slide-in panel) and syncs progress to a self-hosted Go backend so bookmarks unify across both sites and across devices. +Manga read-progress tracker, user read on **asurascans.com** (current domain; asuracomic.net 301s here) and **demonicscans.org** via **Violentmonkey**. Userscript inject on-page UI (floating button + slide-in panel), sync progress to self-hosted Go backend so bookmarks unify across both sites and devices. -## Hard constraints (these drive the design — do not violate) +## Hard constraints (drive design — don't violate) -Bromite uses Chromium's **native** userscript engine, not Tampermonkey: -- **No `GM_*` APIs anywhere.** No `GM_setValue`/`GM_getValue` (use page `localStorage`), no `GM_registerMenuCommand` (inject on-page UI), no `GM_xmlhttpRequest` for cross-origin (use plain `fetch()`). Keeping the script GM-free also lets it run in desktop Tampermonkey/Violentmonkey for faster iteration. -- Cross-origin `fetch()` works **only** against a CORS-enabled backend. Manga sites are `https://`, so backend **must be HTTPS** (mixed-content block otherwise). -- Asura and Demonic are **separate origins with separate `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync is required, not optional. -- Userscript runs in an **isolated world**, so the embedded API token is safe from the site's JS. -- Cloudflare's block on fetching the manga sites is **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both the CGNAT dev machine *and* the deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. This contradicts an earlier, untested assumption that the CGNAT dev IP would be blocked; it was not, at least on this date. Treat "does curl work right now" as a live, time-varying fact to re-check, not a fixed property of a given machine — Cloudflare's bot scoring can flip a previously-clean IP without notice. Any backend fetcher still needs a graceful-degrade path for when it does get challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, or a direct probe) before finalizing, not assumed from a single earlier test. +Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free where plain web APIs suffice — keeps portability across engines: +- **Avoid `GM_*` unless needed.** Prefer page `localStorage` over `GM_setValue`/`GM_getValue`, on-page UI over `GM_registerMenuCommand`, plain `fetch()` over `GM_xmlhttpRequest` for cross-origin. +- Cross-origin `fetch()` work **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block). +- Asura and Demonic are **separate origins with separate `localStorage`** — shared remote store only way to unify bookmarks. Cloud sync required, not optional. +- Userscript run in **isolated world**, so embedded API token safe from site's JS. +- Cloudflare's block on manga sites **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — Cloudflare's bot scoring can flip previously-clean IP without notice. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, or direct probe) before finalize, not assumed from single earlier test. ## Architecture ``` -Bromite userscript (isolated world, per-site adapters, localStorage cache) +Violentmonkey userscript (isolated world, per-site adapters, localStorage cache) -- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume) ``` -- **Backend** (`backend/`): stdlib `net/http` (a handful of routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). The reverse proxy terminates TLS; the Go service listens plain `:8080`. -- **Single-user store.** One `bookmarks` table keyed `:` (`asura`|`demonic`). Sync is **last-write-wins**. Schema and endpoint list are in the plan. +- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`. +- **Single-user store.** One `bookmarks` table keyed `:` (`asura`|`demonic`). Sync **last-write-wins**. Schema and endpoint list in plan. - **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). -- **Web UI:** the same binary serves a password-gated browser UI on a second - hostname — `GET /` (list, or login page when there is no session), - `POST /login`, `POST /logout`, `GET /static/*`, and htmx fragment endpoints - under `/ui/*`. Templates and assets are `go:embed`-ed, so `backend/Dockerfile` - must copy `templates/` and `static/` as well as `*.go`. Sessions are stateless - HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them and, when empty, - the web routes are not registered at all. UI mutations read-modify-write - through `Store.Get` + `Store.Upsert` so the `updated_at` rule stays in one +- **Web UI:** same binary serve password-gated browser UI on second + hostname — `GET /` (list, or login page when no session), + `POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints + under `/ui/*`. Templates + assets `go:embed`-ed, so `backend/Dockerfile` + must copy `templates/` and `static/` plus `*.go`. Sessions stateless + HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, and when empty, + web routes not registered at all. UI mutations read-modify-write + through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`. - **Design-tool caveat:** the templates link `/static/style.css` root-absolutely - (correct — they are served from `/`), but the impeccable detector resolves a - stylesheet href with `path.resolve(fileDir, href)`, which drops the directory - on a leading `/` and silently skips the file. A relative href does not help - either: the template's directory is not its served path. So - `detect.mjs backend/templates` reports a **false clean** — always pass - `backend/static` too. Its one finding there, `overused-font` on "Instrument - Serif", is a deliberate identity choice, not debt. -- **Every action that moves a series out of the list is confirm-gated.** - Archive, finish, and remove each open their own `.confirm-row` disclosure + **Design-tool caveat:** templates link `/static/style.css` root-absolutely + (correct — served from `/`), but impeccable detector resolves + stylesheet href with `path.resolve(fileDir, href)`, drops directory + on leading `/` and silently skip file. Relative href don't help + either: template's directory isn't its served path. So + `detect.mjs backend/templates` reports **false clean** — always pass + `backend/static` too. One finding there, `overused-font` on "Instrument + Serif", deliberate identity choice, not debt. +- **Every action that moves series out of list is confirm-gated.** + Archive, finish, remove each open own `.confirm-row` disclosure (`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈ - `archive|finish|remove`); restore fires instantly because it is the reversal. - Remove's row wears the ember wash, the two reversible ones wear `.calm` grey. - `--ember` stays reserved for the new-chapter signal: busy bar and inline + `archive|finish|remove`); restore fire instantly since it's the reversal. + Remove's row wear ember wash, two reversible ones wear `.calm` grey. + `--ember` stay reserved for new-chapter signal: busy bar and inline error use `--mute`. -- **Latest-chapter poller:** a ticker goroutine in the same binary re-checks - each bookmarked series' newest published chapter from the backend's own - network access, so `latest_chapter` stays fresh when the user is not - browsing. It is a *second, parallel* signal — the userscript keeps its own +- **Latest-chapter poller:** ticker goroutine in same binary re-check + each bookmarked series' newest published chapter from backend's own + network access, so `latest_chapter` stay fresh when user not + browsing. Second, parallel signal — userscript keep own `maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged. - Two independent clocks: a per-bookmark cooldown (`latest_checked_at` column, - enforced by `Store.DueForLatestCheck`'s WHERE clause) and a wake interval. - The row is stamped *before* the fetch so a broken series waits out a full + Two independent clocks: per-bookmark cooldown (`latest_checked_at` column, + enforced by `Store.DueForLatestCheck`'s WHERE clause) and wake interval. + Row stamped *before* fetch so broken series wait out full cooldown instead of retrying every tick, and writes go through - `Store.Get` + `Store.Upsert` so a new chapter never reorders the list. - Fetches use `bogdanfinn/tls-client` with a Chrome profile as defence in depth - against fingerprint-based blocking; any failure logs and skips. See + `Store.Get` + `Store.Upsert` so new chapter never reorders list. + Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth + against fingerprint-based blocking; any failure log and skip. See `docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`. - The poller's `Store.Get` + `Store.Upsert` is not wrapped in a transaction, so - a userscript `PUT` that commits between the two can be overwritten by the - poller's stale re-read — reverting that read progress and, since the stored - value now differs, moving `updated_at` and reordering the list. This is a - known, accepted limitation for a single-user deployment, not a bug to fix. -- **`updated_at` drives list order, so it moves only on real reading progress:** the server applies its timestamp when the row is new or `last_chapter_num` changes, and otherwise keeps the stored value — favouriting a series or recording a newly published chapter must not reorder the list. `PUT` therefore returns the row **as stored**, and clients must adopt that response rather than their own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4. + Poller's `Store.Get` + `Store.Upsert` not wrapped in transaction, so + userscript `PUT` that commits between the two can get overwritten by + poller's stale re-read — reverting that read progress and, since stored + value now differs, moving `updated_at` and reordering list. Known, + accepted limitation for single-user deployment, not bug to fix. +- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4. - **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` | `finished`, orthogonal to `favorite`. Archived and finished appear only in - their own tab — not in All, Updated, Favourites, or the recent strip. The - poller keeps checking archived series and skips finished ones. `finished` is - settable only from the web UI; `PUT /bookmarks/{key}` rejects it with 400. - **An empty incoming status means "keep the stored one"** — resolved on the - `VALUES` side of `Store.Upsert`, not in the conflict clause, because - `excluded.*` is the post-evaluation row and a default applied there would - wipe the bucket on every PUT from a client that predates the column. See + own tab — not in All, Updated, Favourites, or recent strip. Poller keeps + checking archived series and skip finished ones. `finished` settable + only from web UI; `PUT /bookmarks/{key}` reject it with 400. + **Empty incoming status means "keep stored one"** — resolved on the + `VALUES` side of `Store.Upsert`, not conflict clause, since + `excluded.*` is post-evaluation row and default applied there would + wipe bucket on every PUT from client that predates column. See `docs/superpowers/specs/2026-07-27-status-buckets-design.md`. - **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list), `DB_PATH` (default `/data/bookmarks.db`), `PORT` (default `8080`), `WEB_PASSWORD` - (gates the browser UI; unset disables it), + (gates browser UI; unset disable it), `LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` (background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`). `USERSCRIPT_PATH` (file served at `/u/{token}/manga-bookmark.user.js`, - default `/userscript/manga-bookmark.user.js`, supplied by a bindmount). + default `/userscript/manga-bookmark.user.js`, supplied by bindmount). ### Userscript structure (single IIFE, `manga-bookmark.user.js`) -1. **Site adapters** — one per host, `detect(location, document)` returns page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title`/`cover` from **`og:title`/`og:image` meta tags**, not CSS classes. +1. **Site adapters** — one per host, `detect(location, document)` return page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title`/`cover` from **`og:title`/`og:image` meta tags**, not CSS classes. 2. **API client** — `apiGet/apiPut/apiDelete` with bearer header; `localStorage` key `mangabm:cache` for instant render + offline fallback. 3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value. -4. **Retry queue** — every write goes through `pushBookmark`/`pushDelete`, so a - failed mutation is parked in `localStorage` (`mangabm:queue`) and replayed on - the next navigation, reconnect, or `refresh()`. Entries are markers - (`{key, op, sendStatus, attempts}`), never payloads — the body is read from - the cache at send time, so one entry per key gives ordering and coalescing for - free. `sendStatus` is **sticky**: while an archive is pending, later writes to - that key keep carrying the bucket, which is what stops a successful - in-between write from silently un-archiving the series. `refresh()` drains - before it fetches and overlays anything still pending, so the list never - flaps. A 400 drops the entry, a 401 aborts the pass and keeps the queue, and - transient failures retry to a cap of 10. Latest-chapter writes deliberately - stay out of the queue. See +4. **Retry queue** — every write go through `pushBookmark`/`pushDelete`, so + failed mutation park in `localStorage` (`mangabm:queue`) and replayed on + next navigation, reconnect, or `refresh()`. Entries are markers + (`{key, op, sendStatus, attempts}`), never payloads — body read from + cache at send time, so one entry per key give ordering and coalescing for + free. `sendStatus` is **sticky**: while archive pending, later writes to + that key keep carrying bucket, which stop successful + in-between write from silently un-archiving series. `refresh()` drains + before it fetches and overlays anything still pending, so list never + flaps. 400 drops entry, 401 abort pass and keep queue, and + transient failures retry to cap of 10. Latest-chapter writes deliberately + stay out of queue. See `docs/superpowers/specs/2026-07-27-offline-retry-queue-design.md`. -5. **UI** — rendered inside a **Shadow DOM** root to isolate from site CSS - (critical on mobile). Three tabs (All / Favourites / Archived) and a row of - link chips to the web UI and both manga sites; `WEB_BASE` sits in the CONFIG - block next to `API_BASE`. The FAB is a `7 × 44` edge tab whose *hit* area is - widened to `28 × 72` by an invisible `#hit` child; `#fab` must keep - `touch-action: none` and must **not** regain `overflow: hidden`. Because - `touch-action` is resolved at gesture start, the strip cannot be both - browser-scrolled and script-dragged, so `makeDraggable` splits by intent: a - swipe from `#hit` scrolls via `window.scrollBy`, a hold of `ARM_MS` arms a - reposition drag, and the visible sliver drags with no hold. See +5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS + (critical on mobile). Three tabs (All / Favourites / Archived) and row of + link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG + block next to `API_BASE`. FAB is `7 × 44` edge tab whose *hit* area + widened to `28 × 72` by invisible `#hit` child; `#fab` must keep + `touch-action: none` and must **not** regain `overflow: hidden`. Since + `touch-action` resolved at gesture start, strip can't be both + browser-scrolled and script-dragged, so `makeDraggable` splits by intent: swipe + from `#hit` scrolls via `window.scrollBy`, hold of `ARM_MS` arms + reposition drag, visible sliver drags with no hold. See `docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`. -6. **SPA navigation** — Asura is Astro, client-routed on the comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fires without reload. Demonic uses classic reloads (initial `document-idle` run suffices). +6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fire without reload. Demonic uses classic reloads (initial `document-idle` run suffice). -### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trusting) +### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust) -- **asurascans.com**: series `/comics/` (slug carries a trailing +- **asurascans.com**: series `/comics/` (slug carries trailing site-wide build-hash suffix, e.g. `-059befe1`, that **rotates on every redeploy**), chapter `/comics//chapter/`. `seriesId` must strip - the hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in the userscript, - `asuraBuildHash` in the backend); URLs keep the full slug — stale-hash + hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in userscript, + `asuraBuildHash` in backend); URLs keep full slug — stale-hash URLs 302 to current ones. Astro-rendered; chapter links present in raw server HTML. - **demonicscans.org**: series `/manga/` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title//chapter//` (older `chaptered.php?manga=&chapter=` form still exists as redirect, what series-page chapter-list anchors link through). - Encodings (incl. triple-encoded punctuation like `%25252D`) are identical + Encodings (incl. triple-encoded punctuation like `%25252D`) identical on /manga/ and /title/ pages, so decode-once seriesIds match — verified 2026-07-28. @@ -144,34 +144,74 @@ Backend (`cd backend`): Local stack: `docker compose up` (named volume mounted at `/data`, `restart: unless-stopped`). -Smoke test: `curl` the endpoints with `Authorization: Bearer `; confirm `OPTIONS` preflight returns CORS headers and `/healthz` returns 200. +Smoke test: `curl` endpoints with `Authorization: Bearer `; confirm `OPTIONS` preflight return CORS headers and `/healthz` return 200. ## Forge: Gitea, not GitHub -`origin` is a self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` does not work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones: +`origin` is self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` don't work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones: -- Open a PR: `tea pr create --head --base main --title "..." --description "..."` +- Open PR: `tea pr create --head --base main --title "..." --description "..."` - List / view / check out: `tea pr list`, `tea pr `, `tea pr checkout ` - Issues: `tea issue create`, `tea issue list` -- Auth lives in `tea login`, not a `GH_TOKEN` env var. +- Auth lives in `tea login`, not `GH_TOKEN` env var. -`tea` prints its output as rendered boxes rather than plain text; the PR URL lands on the last line. +`tea` print output as rendered boxes rather than plain text; PR URL lands on last line. + +## Design system + +Web UI + userscript panel follow **Cinder**, rules in `docs/design-system.md` +— source of truth Claude Design project `mangaBookmark Web UI` +(`969ac210-fe02-4c01-ae1b-9a271dcc779a`). Read it before touching +`backend/static/style.css`, `backend/templates/*`, or userscript +`TEMPLATE`/`CSS`. Core law: **ember means new chapter only** — no other +state (busy, error, destruction) may use `--ember`; destruction gets +`--danger`. No cards/corners/shadows, one `--measure: 760px` column, tokens +only (never hardcode hex outside `:root`), both colour branches touched +together. Any move that pulls series out of list (archive/finish/remove) +must be confirm-gated via its own `.confirm-row`; only restore fires +instantly. ## Security invariants - Auth on `/bookmarks*`: require `Authorization: Bearer `, **constant-time compare**, 401 otherwise. - CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` with `204`. +## Comments + +Comment only if code alone can't carry info. Cost per read — must earn spot. + +Write for: +- Why not what. Tradeoffs, non-obvious decisions. +- Load-bearing detail looking incidental — say so if "simplify" breaks it. +- Non-local consequence, invisible from function alone. +- Wire format / encoding / interface contract — save callers re-deriving. +- Gotcha/workaround, with ref if exists. +- Domain/business rule not derivable from code. + +Skip: +- Restating code (no `// increment i` above `i++`). +- Trivial getter/setter/pass-through. +- Banners, dividers, `// helpers`. +- Change narration (`// fix bug`, `// as requested`, `// new impl`) — git's job. +- Commented-out code — delete. +- TODO without concrete action. + +Style: one dense comment over function beats one per line inside. Tight, no worked example unless bug subtle. Wrong comment worse than none — update/delete on change. Default fewer — sparse+high-signal beats comprehensive. + +Test: "competent reader get this from code in few sec?" Yes → skip. Needs detour through another file/spec/git-blame → write it. + ## Relevant skills -`multi-stage-dockerfile` and `docker-compose-orchestration` for the container work (referenced in the plan). +`multi-stage-dockerfile` and `docker-compose-orchestration` for container work (referenced in plan). + +`golang-code-style`, `golang-error-handling`, `golang-performance`, `golang-testing` for backend Go work. ## graphify -This project has a knowledge graph at graphify-out/ with god nodes, community structure, and cross-file relationships. +Project has knowledge graph at graphify-out/ with god nodes, community structure, cross-file relationships. Rules: -- For codebase questions, first run `graphify query ""` when graphify-out/graph.json exists. Use `graphify path "" ""` for relationships and `graphify explain ""` for focused concepts. These return a scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output. -- If graphify-out/wiki/index.md exists, use it for broad navigation instead of raw source browsing. -- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain do not surface enough context. -- After modifying code, run `graphify update .` to keep the graph current (AST-only, no API cost). +- For codebase questions, first run `graphify query ""` when graphify-out/graph.json exists. Use `graphify path "" ""` for relationships and `graphify explain ""` for focused concepts. Return scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output. +- If graphify-out/wiki/index.md exists, use for broad navigation instead of raw source browsing. +- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain don't surface enough context. +- After modifying code, run `graphify update .` to keep graph current (AST-only, no API cost). \ No newline at end of file diff --git a/docs/design-system.md b/docs/design-system.md index a19e051..9aed761 100644 --- a/docs/design-system.md +++ b/docs/design-system.md @@ -1,16 +1,16 @@ # Cinder — mangaBookmark design system -Source of truth: the Claude Design doc **Cinder Sheet** -(`cfa39183-8874-4f76-987c-afef14dceebb`, files `Cinder Sheet.dc.html` for the -static spec and `Cinder Sheet App.dc.html` for the interactive one). This file -records the rules that got implemented so a future agent can extend the UI -without re-reading the design. +Source of truth: the Claude Design project **mangaBookmark Web UI** +(`969ac210-fe02-4c01-ae1b-9a271dcc779a`, `index.html` + siblings +`archived.html`/`fav.html`/`finished.html`/`new.html`/`login.html`/`mobile.html`, +`style.css`, `filter.js`). This file records the rules that got implemented so +a future agent can extend the UI without re-reading the design. Implemented in: | Surface | Files | | --- | --- | -| Web UI (login, list, card, empty, errors) | `backend/static/style.css`, `backend/templates/{app,card,list,login,icons}.html`, `backend/static/filter.js` | +| Web UI (login, list, card, empty, errors) | `backend/static/style.css`, `backend/templates/{app,card,list,login,chrome,icons}.html`, `backend/static/filter.js` | | Userscript panel (Shadow DOM) | `userscript/manga-bookmark.user.js` — `TEMPLATE` and `CSS` at the bottom of the IIFE | ## 1. The one idea @@ -19,9 +19,12 @@ Implemented in: allowed to be crimson: its title turns `--paper-hot` and sits on a 1px ember underline sized to the text, its cover gains a 3px ember rule at the foot, and its `Ch N out` meta and play icon go ember. Everything else — favourites, -status, chrome — stays cool. If a new feature wants to be noticed, it does *not* -get to borrow the ember; find a typographic answer (weight, italic, a rule) or -use brass, which is already spoken for by favourites. +status, chrome, destruction — stays off that one colour. Destruction gets its +own token (`--danger`, a duller oxblood) precisely so a remove confirm is +never mistaken across the room for an unread chapter. If a new feature wants +to be noticed, it does *not* get to borrow the ember; find a typographic +answer (weight, italic, a rule) or reach for one of the named action accents +(§2). Corollaries: @@ -34,7 +37,7 @@ Corollaries: - **Three type roles, never mixed.** Display serif for anything a human reads as a name (brand, titles, tabs, primary buttons, empty-state headings). Mono small-caps for machine facts (site, chapter numbers, labels, status, badges, - ghost buttons). Sans for prose only (empty-state body, hints). + ghost buttons, the action key). Sans for prose only (empty-state body, hints). ## 2. Tokens @@ -44,7 +47,7 @@ Defined once in `backend/static/style.css` `:root`, mirrored in the userscript's | Token | Dark | Light | Use | | --- | --- | --- | --- | | `--ink` | `#100f0e` | `#f7f4ef` | page | -| `--ash` | `#161413` | `#efeae3` | recessed panel (chapter form, toast) | +| `--ash` | `#161413` | `#efeae3` | recessed panel (chapter form) | | `--dim` | `#0d0c0b` | `#f1ede7` | archived / finished row background | | `--rule` | `#221f1d` | `#e0dad2` | hairline between sheets, button borders | | `--rule-soft` | `#1a1817` | `#e8e3dc` | the measure's own side edges | @@ -54,23 +57,34 @@ Defined once in `backend/static/style.css` `:root`, mirrored in the userscript's | `--paper-hot` | `#f0d3cb` | `#a33018` | title of a series with a new chapter | | `--paper-dim` | `#ddd5cb` | `#191715` | resting title | | `--mute` | `#8d857c` | `#6b645d` | secondary text, idle icons | -| `--mute-2` | `#5a5450` | `#857d75` | eyebrow labels, hints | +| `--mute-2` | `#877f76` | `#6c655e` | eyebrow labels, hints (must clear 4.5:1 on both `--ink` and `--ash`) | | `--faint` | `#3a3733` | `#c9c2ba` | the `/` separators in a meta line | | `--faint-2` | `#57504b` | `#a8a098` | cover monogram | | `--ember` | `#e0452c` | `#c23a22` | heat — see §1 | -| `--ember-wash` | `#1a1211` | `#fbeee9` | ember-tinted surface (confirm, error) | +| `--ember-wash` | `#1a1211` | `#fbeee9` | ember-tinted surface | | `--ember-ink` | `#150907` | `#fff` | text on solid ember | | `--ember-soft` | `#eda798` | `#8d2c17` | text on ember wash | -| `--brass` | `#b8912f` | `#8a681c` | favourites, and only favourites | -| `--trash` | `#6b5450` | `#a98276` | remove, at rest | +| `--danger` | `#cf5c4d` | `#97362a` | destruction — remove confirm, never the same as `--ember` | +| `--danger-wash` | `#211311` | `#fbe9e5` | remove-confirm surface | +| `--danger-ink` | `#150808` | `#fff` | text on solid danger | +| `--danger-soft` | `#e2aaa1` | `#7c2c22` | text on danger wash | +| `--brass` | `#b8912f` | `#8a681c` | favourite — a cooler second metal | +| `--slate` | `#7fa0c0` | `#3f6689` | archive accent | +| `--moss` | `#7fae86` | `#3d6c46` | finished accent | +| `--clay` | `#b5906f` | `#7c5533` | set-chapter accent | +| `--trash` | `#977671` | `#8c6558` | remove, at rest — icons need 3:1, not 4.5:1 | +| `--play-hot-line` | `#3a1d18` | `#f0cfc6` | desktop cell border, play when `.is-new` | +| `--fav-line` | `#332b14` | `#e3d3a4` | desktop cell border, favourite when on | | `--asura` | `#7d93a5` | `#4f6b80` | site tag | | `--demonic` | `#a98a78` | `#8a6a55` | site tag | | `--hatch` / `--hatch-dim` | 135° 5px stripe | paper stripe | missing-cover slot | -Dark is the default (`color-scheme: dark light`); light is a -`@media (prefers-color-scheme: light)` override of the same names. **Any new -colour must be added in both branches** — light is not a filter over dark, the -hues are re-tuned. +`--slate`/`--moss`/`--clay`/`--brass` are held at the same weight deliberately: +one accent per action, so a press says which lane it belongs to, with none of +them competing with ember. Dark is the default (`color-scheme: dark light`); +light is a `@media (prefers-color-scheme: light)` override of the same names. +**Any new colour must be added in both branches** — light is not a filter over +dark, the hues are re-tuned. ## 3. Type @@ -83,11 +97,10 @@ hues are re-tuned. The web UI **self-hosts** all three: five latin-subset woff2 files in `backend/static/fonts/` (~120 KB total), declared by the `@font-face` block at the top of `style.css` and embedded in the binary by the existing -`//go:embed static`. There is no request to Google — this UI is read in Bromite, -where `fonts.googleapis.com` is routinely blocked, and over a LAN with no -internet route. `staticHandler()` in `web.go` registers the `.woff2` MIME type -because Go's built-in table lacks it and the scratch image has no -`/etc/mime.types`. +`//go:embed static`. There is no request to Google — this UI needs to survive +on a LAN with no internet route. `staticHandler()` in `web.go` registers the +`.woff2` MIME type because Go's built-in table lacks it and the scratch image +has no `/etc/mime.types`. Adding a weight means adding a file: grab the *latin* `@font-face` block from `https://fonts.googleapis.com/css2?...` **with a browser User-Agent** (Google @@ -102,37 +115,57 @@ root is at the mercy of the host site's CSP. Recurring specs (copy these rather than inventing sizes): -- Brand: `400 26px/1 display`, with `` in ember italic — `mangaBookmark`. -- Row title: `400 19px/1.2 display` (21px ≥720px). -- Tab: `400 17px display` (18px ≥720px), active gets `border-bottom: 2px` in +- Brand: `400 26px/1 display` (`30px` ≥720px), inline SVG mark (§4) + `` in + ember italic — `mangaBookmark`. +- Row title: `400 21px/1.2 display` (`22px` ≥720px). +- Tab: `400 17px display` (`18px` ≥720px), active gets `border-bottom: 2px` in `--paper` (`--ember` for Updated) plus `margin-bottom: -1px` so it lands on the row's own hairline. -- Meta / label / badge: `500 10px mono`, `letter-spacing: .12em`, - `text-transform: uppercase`. Eyebrows ("CONTINUE READING") use `.2em`. +- Meta / label / badge / action key: `500 10–11px mono`, `letter-spacing: + .04em`–`.2em`, `text-transform: uppercase`. Eyebrows use the widest tracking. - Empty-state heading: `400 20px display`; body `400 14px/1.6 sans`, `max-width: 44ch`. -- Primary button: `--paper` fill, `--ink` text, `400 17px display`, no border radius. +- Primary button: `--paper` fill, `--ink` text, `400 17–19px display`, no border radius. - Ghost button: mono small-caps, transparent, `border-bottom: 1px --field-line`. ## 4. Components (web UI) ``` .sheet - .topbar .brand + .ghost (log out) + .topbar .brand (mark + wordmark) + .ghost (log out) .chrome .searchbar + nav.tabs (column on phone, row ≥720px via order:) + .keyrow one-line action key: Read / Fav / Chapter / Archive / Done / Delete .recent h2 eyebrow + .recent-strip > a.recent-card main#list article.card … | .empty ``` +**Brand mark**: an inline `` (`viewBox="0 0 200 172"`), +defined once in `chrome.html`'s `mark` template and reused by `app.html` and +`login.html` so it takes the page's `--ink`/`currentColor`/`--ember` rather +than shipping as a static asset. The blade at its centre strokes +`var(--logo-blade, var(--ember))` — override that custom property, don't +duplicate the SVG, if a surface ever needs a different blade colour. Drawn at +a 5px stroke on a 200-unit grid; at brand size that thins out, so `.brand .mark +g` nudges `stroke-width` up to `6.5` rather than scaling the artwork down. + +**Action key** (`.keyrow`): one permanent line under the tabs naming what +every icon in `.actions` does — Read / Fav / Chapter / Archive / Done / +Delete — so the icon strip on a card is never a guess. On a phone each pair +stacks icon-over-word (`flex-direction: column`) so the word gets the full +cell width and can stay in long form; ≥720px it lays out icon-beside-word and +switches the `.short`/`.full` label pair. `.pair.brass` and `.pair.trash` +carry their icon's resting accent so the key itself teaches the colour +vocabulary in §1/§2. + `article.card` — the row, and the only per-series component: ``` article.card[.is-new|.is-dim]#card-[data-title] .row - a.cover img | span.monogram, + span.foot-rule[.brass] + a.cover[tabindex="-1" aria-hidden] img | span.monogram, + span.foot-rule[.brass] .body .title-line (h3.title + svg.fav-mark) , p.meta - .actions play, favourite, chapter, archive|restore, finish, remove - form.chapter-form[hidden] .hint + .field(input + Save) - .confirm-row[hidden] span + (Remove, Cancel) + .actions play, favourite, chapter | lifecycle: archive/restore, finish, remove + form.chapter-form[hidden] .hint + .field(input + Save) + .hint (latest known) + .confirm-row[.calm][hidden] × one per lifecycle action, span + (go/danger-solid, Cancel) p.error-inline[hidden] ``` @@ -143,9 +176,29 @@ Rules that are easy to break: dim rule is a descendant selector off those two classes, so a new sub-element inherits the state for free. - `.actions` is `flex: 1 0 100%` inside `.row`, which is what makes it a - full-width strip under the row on a phone and a group of 40px squares beside + full-width strip under the row on a phone and a group of 44px squares beside the row at ≥720px. Cells are 46px tall on phone (thumb target) and divided by `border-right: 1px var(--rule)`, last child none. +- Three clusters by consequence, in this order: navigate (`.play`) | organize + (`.fav`, `.pencil`) | lifecycle (`.box`/`.restore`, `.finish`, `.remove`, + each carrying the `.lifecycle` class). Lifecycle cells sit on a recessed + `--ash` ground so the thumb reads "this one moves the series" before it + reads which icon it landed on; ≥720px they separate by a 10px gap instead of + the phone's inset hairline. +- Every lifecycle button that moves a series out of the list is + **confirm-gated**: it opens its own `.confirm-row` (`archive`, `finish`, + `remove` — `toggleConfirmRow(key, kind)` in `filter.js`). Archive and finish + ask in `.calm` grey since they're reversible; remove alone gets the + `--danger-wash` treatment and names the series in its question. Restore + fires instantly — no confirm — because it's the reversal. +- Per-action hover/press accent: `.fav` → `--brass`, `.pencil` → `--clay`, + `.box` → `--slate`, `.finish` → `--moss`. `.play` stays paper/ember (ember + only when `.is-new`). `.remove` stays `--trash` at rest, `--danger` on + hover. Desktop cell borders follow the same accent on hover + (`border-color: currentColor`); the two coloured *resting* states + (`.is-new .play`, `.fav.on`) get their own dim border tokens + (`--play-hot-line`, `--fav-line`) instead of the full accent, since a + resting border needs less contrast than a hover one. - Icons are `` against the sprite in `templates/icons.html`, included once by `app.html`. htmx-swapped card fragments reference the page's sprite, so a card never inlines a path. New icon → add a `` @@ -155,11 +208,15 @@ Rules that are easy to break: - Cover foot rule: ember when new, brass when favourite-and-not-new. Never both. - `[hidden] { display: none !important; }` is load-bearing — every disclosure panel is a flex container, and `display` beats `hidden`. -- Busy state is `.card.htmx-request::before`, a 1px ember bar sliding across the +- Busy state is `.card.htmx-request::before`, a 1px grey bar sliding across the top hairline (`barSlide`), plus the action strip at `opacity: .5`. Never a - spinner. -- `.open` on the pencil / trash cell marks which panel is showing; `filter.js` - `togglePanel()` owns that class alongside `hidden`. + spinner, and deliberately `--mute` not `--ember` — on a list screen ember + means "new chapter" and nothing else, so a system state can't borrow it. +- `.open` on the pencil / lifecycle cell marks which panel is showing; + `filter.js` `togglePanel()`/`toggleConfirmRow()` own that class alongside + `hidden`. An open lifecycle cell needs the next surface step up from + `--hover` (`--rule`) to stay legible as the panel's owner, since the panel + itself already sits on `--ash`. ## 5. Components (userscript panel) @@ -167,9 +224,9 @@ Same tokens, same heat rule, structure unchanged from before the revamp (`#fab`/`#hit`, `#panel`, `#nav` chips, `#context`, `#tabs`, `#list` of `.item`). Cinder-specific: `.item.hot` (new chapter) and `.item.dim` (archived) mirror `.is-new` / `.is-dim`; chips and `.btn`s are mono small-caps with hairline -borders instead of pills; loading is the same sliding ember hairline (`.spinner` -is now a 1px bar, not a rotating ring); toasts are `--ash` with a 2px left rule, -ember-washed when `.err`. +borders instead of pills; loading is the same sliding hairline (`.spinner` +is a 1px bar, not a rotating ring); toasts are `--ash` with a 2px left rule, +`--danger`-washed when `.err`. **Do not touch** the FAB geometry while restyling: `#fab` keeps `touch-action: none`, must not regain `overflow: hidden`, and `#hit` keeps the @@ -179,36 +236,48 @@ ember-washed when `.err`. ## 6. Motion Three animations, all ≤ 1.15s and all disabled under -`prefers-reduced-motion: reduce`: +`prefers-reduced-motion: reduce` (pseudo-elements need naming explicitly in +that query — `*` does not match `::before`/`::after`, so the busy bar and +error dot are listed by name and fall back to their static drawn form): - `sheetIn` — 180ms fade + 4px rise, on a row and on each disclosure panel. -- `barSlide` — the burning hairline, for any busy state. -- `emberPulse` — the 5px dot on `.error-inline`. +- `barSlide` — the sliding hairline, for any busy state. +- `mutePulse` — the 5px dot on `.error-inline`. No transforms on hover, no scale, no easing curves beyond `ease-out`/`linear`. ## 7. Accessibility floor (not negotiable) -- Touch targets on the phone layout are 44–46px; the 40px desktop cells are +- Touch targets on the phone layout are 44–46px; the 44px desktop cells are pointer-only (≥720px). - Every icon-only control keeps `title` + `aria-label`; the SVG inside is - `aria-hidden`. + `aria-hidden`. Lifecycle buttons also carry `aria-expanded` + + `aria-controls` pointing at their `.confirm-row`. - The cover link is `tabindex="-1" aria-hidden="true"` because the title link and the play cell already reach the same URL — do not make it a third tab stop. - Tabs keep `role="tab"` / `role="tablist"`; the active one is marked by class, and `setActiveTab()` in `filter.js` maintains it after an htmx swap. +- `.confirm-row` and `.error-inline` are `role="group"`/`role="status"` with + `aria-live="polite"` so a disclosure opening is announced. - Light and dark are both first-class. Check any new colour in both. ## 8. Adding something new — checklist 1. Can it be a hairline, a small-caps label, or a serif line instead of a new component? Prefer that. -2. Tokens only, both colour branches. +2. Tokens only, both colour branches. A new action gets its own named accent + (like `--slate`/`--moss`/`--clay`) at the same weight as the existing set — + never reuse `--ember` or `--danger` for anything but their one meaning. 3. If it is per-series, hang it off `.is-new` / `.is-dim` rather than adding a third state class. -4. Icon → `templates/icons.html`; nothing inlines SVG paths. -5. Phone first (44px targets, single column), then the ≥720px block. -6. Verify: `cd backend && go test ./...`, then run the binary and screenshot +4. If it removes a series from the current view (archive/finish/remove-shaped), + it is confirm-gated via its own `.confirm-row` — no exceptions, restore is + the only instant action because it's the one that's reversible by nature. +5. Icon → `templates/icons.html`; nothing inlines SVG paths. Brand mark stays + the one exception (`chrome.html`'s `mark` template), since it takes + page-level custom properties the sprite can't carry per-instance. +6. Phone first (44px targets, single column), then the ≥720px block. +7. Verify: `cd backend && go test ./...`, then run the binary and screenshot both widths and both colour schemes (Playwright: `emulateMedia`, `setViewportSize`; disable the browser cache — `/static/*` is served with `max-age=3600`, and templates are `go:embed`ed so the binary must be rebuilt From eeb601cbe2b014244b689b4742a6ed134f335f73 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Sun, 2 Aug 2026 19:41:38 +0700 Subject: [PATCH 02/18] Split backend into internal packages by responsibility All Go files lived flat in backend/ as one package main. Move store, latest-chapter polling, sessions, HTTP middleware, the JSON API, the userscript handler, and the web UI (with its templates/static assets) into backend/internal/{store,latest,session,httpmw,api,userscript,web}, each with an exported API. main.go becomes the composition root wiring them into newRouter; root-level tests cover the assembled router while package-local tests cover unit behavior. Update Dockerfile/.dockerignore for the new internal/ tree and CLAUDE.md to describe the layout. Co-Authored-By: Claude Sonnet 5 --- CLAUDE.md | 23 +- backend/.dockerignore | 10 +- backend/Dockerfile | 9 +- backend/api_test.go | 472 ++++++++++++++++++ backend/{ => internal/api}/handlers.go | 38 +- backend/{ => internal/httpmw}/middleware.go | 14 +- .../latest/fetch.go} | 14 +- .../{latest.go => internal/latest/poller.go} | 60 +-- .../latest/poller_test.go} | 74 ++- .../latest/sites.go} | 6 +- .../latest/sites_test.go} | 2 +- backend/{ => internal/session}/session.go | 58 +-- .../{ => internal/session}/session_test.go | 94 ++-- backend/{ => internal/store}/store.go | 43 +- backend/{ => internal/store}/store_test.go | 470 ++--------------- .../{ => internal/userscript}/userscript.go | 4 +- .../userscript}/userscript_test.go | 51 +- backend/{ => internal/web}/static/filter.js | 0 .../web}/static/fonts/dm-sans-var-latin.woff2 | Bin .../fonts/ibm-plex-mono-500-latin.woff2 | Bin .../fonts/ibm-plex-mono-600-latin.woff2 | Bin .../instrument-serif-400-italic-latin.woff2 | Bin .../fonts/instrument-serif-400-latin.woff2 | Bin backend/{ => internal/web}/static/htmx.min.js | 0 backend/{ => internal/web}/static/logo.svg | 0 backend/{ => internal/web}/static/style.css | 0 backend/{ => internal/web}/templates/app.html | 0 .../{ => internal/web}/templates/card.html | 0 .../{ => internal/web}/templates/chrome.html | 0 .../{ => internal/web}/templates/icons.html | 0 .../{ => internal/web}/templates/list.html | 0 .../{ => internal/web}/templates/login.html | 0 backend/{ => internal/web}/web.go | 129 +++-- backend/main.go | 63 +-- backend/main_test.go | 6 +- backend/web_test.go | 174 +++---- 36 files changed, 971 insertions(+), 843 deletions(-) create mode 100644 backend/api_test.go rename backend/{ => internal/api}/handlers.go (71%) rename backend/{ => internal/httpmw}/middleware.go (88%) rename backend/{latest_http.go => internal/latest/fetch.go} (89%) rename backend/{latest.go => internal/latest/poller.go} (82%) rename backend/{latest_test.go => internal/latest/poller_test.go} (85%) rename backend/{latest_sites.go => internal/latest/sites.go} (96%) rename backend/{latest_sites_test.go => internal/latest/sites_test.go} (99%) rename backend/{ => internal/session}/session.go (75%) rename backend/{ => internal/session}/session_test.go (68%) rename backend/{ => internal/store}/store.go (93%) rename backend/{ => internal/store}/store_test.go (57%) rename backend/{ => internal/userscript}/userscript.go (96%) rename backend/{ => internal/userscript}/userscript_test.go (71%) rename backend/{ => internal/web}/static/filter.js (100%) rename backend/{ => internal/web}/static/fonts/dm-sans-var-latin.woff2 (100%) rename backend/{ => internal/web}/static/fonts/ibm-plex-mono-500-latin.woff2 (100%) rename backend/{ => internal/web}/static/fonts/ibm-plex-mono-600-latin.woff2 (100%) rename backend/{ => internal/web}/static/fonts/instrument-serif-400-italic-latin.woff2 (100%) rename backend/{ => internal/web}/static/fonts/instrument-serif-400-latin.woff2 (100%) rename backend/{ => internal/web}/static/htmx.min.js (100%) rename backend/{ => internal/web}/static/logo.svg (100%) rename backend/{ => internal/web}/static/style.css (100%) rename backend/{ => internal/web}/templates/app.html (100%) rename backend/{ => internal/web}/templates/card.html (100%) rename backend/{ => internal/web}/templates/chrome.html (100%) rename backend/{ => internal/web}/templates/icons.html (100%) rename backend/{ => internal/web}/templates/list.html (100%) rename backend/{ => internal/web}/templates/login.html (100%) rename backend/{ => internal/web}/web.go (78%) diff --git a/CLAUDE.md b/CLAUDE.md index b57c00f..7f41000 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,13 +27,24 @@ Violentmonkey userscript (isolated world, per-site adapters, localStorage cache) ``` - **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`. + Single binary, split into packages under `backend/internal/`: `store` + (Bookmark type, SQLite persistence, migrations), `latest` (background + poller, site parsers, TLS fetcher), `session` (cookie signing, login + rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON + bookmark handlers), `userscript` (userscript-serving handler), `web` + (browser UI handler + `templates/` + `static/`, `go:embed`-ed). + `backend/main.go` is the composition root — the only place that wires + packages together into `newRouter`. Root-level `*_test.go` hold + integration tests that exercise the full router; unit tests for a + package live beside it under `internal/`. - **Single-user store.** One `bookmarks` table keyed `:` (`asura`|`demonic`). Sync **last-write-wins**. Schema and endpoint list in plan. - **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). - **Web UI:** same binary serve password-gated browser UI on second hostname — `GET /` (list, or login page when no session), `POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints - under `/ui/*`. Templates + assets `go:embed`-ed, so `backend/Dockerfile` - must copy `templates/` and `static/` plus `*.go`. Sessions stateless + under `/ui/*`. Templates + assets `go:embed`-ed under + `backend/internal/web/`, so `backend/Dockerfile` must copy the whole + `internal/` tree, not just `*.go`. Sessions stateless HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, and when empty, web routes not registered at all. UI mutations read-modify-write through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one @@ -43,9 +54,9 @@ Violentmonkey userscript (isolated world, per-site adapters, localStorage cache) stylesheet href with `path.resolve(fileDir, href)`, drops directory on leading `/` and silently skip file. Relative href don't help either: template's directory isn't its served path. So - `detect.mjs backend/templates` reports **false clean** — always pass - `backend/static` too. One finding there, `overused-font` on "Instrument - Serif", deliberate identity choice, not debt. + `detect.mjs backend/internal/web/templates` reports **false clean** — + always pass `backend/internal/web/static` too. One finding there, + `overused-font` on "Instrument Serif", deliberate identity choice, not debt. - **Every action that moves series out of list is confirm-gated.** Archive, finish, remove each open own `.confirm-row` disclosure (`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈ @@ -162,7 +173,7 @@ Smoke test: `curl` endpoints with `Authorization: Bearer `; confirm `OPTI Web UI + userscript panel follow **Cinder**, rules in `docs/design-system.md` — source of truth Claude Design project `mangaBookmark Web UI` (`969ac210-fe02-4c01-ae1b-9a271dcc779a`). Read it before touching -`backend/static/style.css`, `backend/templates/*`, or userscript +`backend/internal/web/static/style.css`, `backend/internal/web/templates/*`, or userscript `TEMPLATE`/`CSS`. Core law: **ember means new chapter only** — no other state (busy, error, destruction) may use `--ember`; destruction gets `--danger`. No cards/corners/shadows, one `--measure: 760px` column, tokens diff --git a/backend/.dockerignore b/backend/.dockerignore index 84b07d1..cfc6328 100644 --- a/backend/.dockerignore +++ b/backend/.dockerignore @@ -1,10 +1,8 @@ -# Only go source + module files, plus the go:embed'd templates/static -# directories, are needed in the build context. +# Only go source + module files, plus internal/ (which carries the +# go:embed'd templates/static directories), are needed in the build context. * !go.mod !go.sum !*.go -!templates/ -!templates/** -!static/ -!static/** +!internal/ +!internal/** diff --git a/backend/Dockerfile b/backend/Dockerfile index fb494da..f91faf4 100644 --- a/backend/Dockerfile +++ b/backend/Dockerfile @@ -8,12 +8,11 @@ WORKDIR /src COPY go.mod go.sum ./ RUN go mod download -# Then source (changes often). -# Source plus the go:embed'd assets. Missing either directory turns the embed -# directive into a build error, so both must be copied before `go build`. +# Then source (changes often). internal/web carries the go:embed'd +# templates/static assets — missing them turns the embed directive into a +# build error, so the whole tree must land before `go build`. COPY *.go ./ -COPY templates/ ./templates/ -COPY static/ ./static/ +COPY internal/ ./internal/ # Static binary: pure-Go sqlite means CGO_ENABLED=0 -> no libc dependency. # -trimpath + -ldflags strip paths and debug info for a smaller image. diff --git a/backend/api_test.go b/backend/api_test.go new file mode 100644 index 0000000..19e9450 --- /dev/null +++ b/backend/api_test.go @@ -0,0 +1,472 @@ +package main + +import ( + "bytes" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "mangabm/backend/internal/store" +) + +const testToken = "s3cret-token" + +func testConfig() Config { + return Config{ + Token: testToken, + AllowedOrigins: []string{"https://asuracomic.net", "https://demonicscans.org"}, + Port: "8080", + } +} + +func newTestServer(t *testing.T) http.Handler { + t.Helper() + dbPath := filepath.Join(t.TempDir(), "test.db") + s, err := store.Open(dbPath) + if err != nil { + t.Fatalf("store.Open: %v", err) + } + t.Cleanup(func() { s.Close() }) + return newRouter(s, testConfig()) +} + +func auth(req *http.Request) *http.Request { + req.Header.Set("Authorization", "Bearer "+testToken) + return req +} + +func floatPtr(f float64) *float64 { return &f } + +// seedForCheck inserts a bookmark and forces its latest_checked_at. +func seedForCheck(t *testing.T, s *store.Store, key, seriesURL string, checkedAt int64) { + t.Helper() + if _, err := s.Upsert(store.Bookmark{ + Key: key, + Site: "asura", + SeriesID: key, + SeriesURL: seriesURL, + UpdatedAt: 1000, + }); err != nil { + t.Fatalf("seed %q: %v", key, err) + } + if err := s.MarkLatestChecked(key, checkedAt); err != nil { + t.Fatalf("seed mark %q: %v", key, err) + } +} + +func readLatestCheckedAt(t *testing.T, s *store.Store, key string) int64 { + t.Helper() + ts, err := s.LatestCheckedAt(key) + if err != nil { + t.Fatalf("LatestCheckedAt %q: %v", key, err) + } + return ts +} + +func TestHealthzNoAuth(t *testing.T) { + srv := newTestServer(t) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil)) + if rr.Code != http.StatusOK { + t.Fatalf("healthz status = %d, want 200", rr.Code) + } + if rr.Body.String() != "ok" { + t.Fatalf("healthz body = %q, want ok", rr.Body.String()) + } +} + +func TestAuthRequired(t *testing.T) { + srv := newTestServer(t) + cases := []struct { + name string + header string + }{ + {"no header", ""}, + {"bad token", "Bearer wrong"}, + {"not bearer", "Basic " + testToken}, + {"empty bearer", "Bearer "}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + req := httptest.NewRequest(http.MethodGet, "/bookmarks", nil) + if tc.header != "" { + req.Header.Set("Authorization", tc.header) + } + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + if rr.Code != http.StatusUnauthorized { + t.Fatalf("status = %d, want 401", rr.Code) + } + }) + } +} + +func TestAuthAccepted(t *testing.T) { + srv := newTestServer(t) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + if rr.Code != http.StatusOK { + t.Fatalf("status = %d, want 200", rr.Code) + } + if got := rr.Body.String(); got != "[]\n" { + t.Fatalf("empty list body = %q, want []", got) + } +} + +func TestCORSPreflight(t *testing.T) { + srv := newTestServer(t) + req := httptest.NewRequest(http.MethodOptions, "/bookmarks/asura:foo-1", nil) + req.Header.Set("Origin", "https://asuracomic.net") + req.Header.Set("Access-Control-Request-Method", "PUT") + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + + if rr.Code != http.StatusNoContent { + t.Fatalf("preflight status = %d, want 204", rr.Code) + } + if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "https://asuracomic.net" { + t.Fatalf("Allow-Origin = %q, want reflected origin", got) + } + if got := rr.Header().Get("Access-Control-Allow-Methods"); got == "" { + t.Fatal("Allow-Methods missing") + } + if got := rr.Header().Get("Access-Control-Allow-Headers"); got == "" { + t.Fatal("Allow-Headers missing") + } +} + +func TestCORSDisallowedOrigin(t *testing.T) { + srv := newTestServer(t) + req := httptest.NewRequest(http.MethodOptions, "/bookmarks", nil) + req.Header.Set("Origin", "https://evil.example") + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "" { + t.Fatalf("Allow-Origin = %q, want empty for disallowed origin", got) + } +} + +func TestBookmarkRoundTrip(t *testing.T) { + srv := newTestServer(t) + key := "asura:solo-leveling-123" + in := store.Bookmark{ + Title: "Solo Leveling", + SeriesURL: "https://asuracomic.net/series/solo-leveling-123", + Cover: "https://asuracomic.net/cover.jpg", + LastChapter: "Chapter 10", + LastChapterNum: 10, + LastChapterURL: "https://asuracomic.net/series/solo-leveling-123/chapter/10", + } + body, _ := json.Marshal(in) + + // PUT + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) + if rr.Code != http.StatusOK { + t.Fatalf("PUT status = %d, want 200", rr.Code) + } + var stored store.Bookmark + if err := json.Unmarshal(rr.Body.Bytes(), &stored); err != nil { + t.Fatalf("decode PUT response: %v", err) + } + if stored.Key != key || stored.Site != "asura" || stored.SeriesID != "solo-leveling-123" { + t.Fatalf("derived fields wrong: %+v", stored) + } + if stored.UpdatedAt == 0 { + t.Fatal("server did not set updated_at") + } + + // GET + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + var list []store.Bookmark + if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil { + t.Fatalf("decode list: %v", err) + } + if len(list) != 1 || list[0].Key != key || list[0].LastChapterNum != 10 { + t.Fatalf("GET list wrong: %+v", list) + } + + // PUT again (upsert, progress advance) + in.LastChapter, in.LastChapterNum = "Chapter 11", 11 + body, _ = json.Marshal(in) + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) + if rr.Code != http.StatusOK { + t.Fatalf("second PUT status = %d", rr.Code) + } + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + json.Unmarshal(rr.Body.Bytes(), &list) + if len(list) != 1 || list[0].LastChapterNum != 11 { + t.Fatalf("upsert did not update in place: %+v", list) + } + + // DELETE + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodDelete, "/bookmarks/"+key, nil))) + if rr.Code != http.StatusNoContent { + t.Fatalf("DELETE status = %d, want 204", rr.Code) + } + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + json.Unmarshal(rr.Body.Bytes(), &list) + if len(list) != 0 { + t.Fatalf("after delete list = %+v, want empty", list) + } +} + +// putBookmark PUTs b at key and returns the bookmark the server echoes back, +// which is the row as actually stored (not the request payload). +func putBookmark(t *testing.T, srv http.Handler, key string, b store.Bookmark) store.Bookmark { + t.Helper() + body, _ := json.Marshal(b) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) + if rr.Code != http.StatusOK { + t.Fatalf("PUT %s status = %d, body = %s", key, rr.Code, rr.Body.String()) + } + var out store.Bookmark + if err := json.Unmarshal(rr.Body.Bytes(), &out); err != nil { + t.Fatalf("decode PUT response: %v", err) + } + return out +} + +func getBookmarks(t *testing.T, srv http.Handler) []store.Bookmark { + t.Helper() + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + if rr.Code != http.StatusOK { + t.Fatalf("GET status = %d", rr.Code) + } + var list []store.Bookmark + if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil { + t.Fatalf("decode list: %v", err) + } + return list +} + +// updated_at drives list ordering, so it must move only on a real progress +// advance — never on a favorite toggle or a latest-chapter capture. +func TestUpsertConditionalUpdatedAt(t *testing.T) { + cases := []struct { + name string + mutate func(store.Bookmark) store.Bookmark + wantBumped bool + }{ + { + name: "unchanged progress", + mutate: func(b store.Bookmark) store.Bookmark { return b }, + wantBumped: false, + }, + { + name: "changed progress", + mutate: func(b store.Bookmark) store.Bookmark { + b.LastChapter, b.LastChapterNum = "Chapter 11", 11 + return b + }, + wantBumped: true, + }, + { + name: "favorite only", + mutate: func(b store.Bookmark) store.Bookmark { + b.Favorite = true + return b + }, + wantBumped: false, + }, + { + name: "latest chapter only", + mutate: func(b store.Bookmark) store.Bookmark { + b.LatestChapter, b.LatestChapterNum = "Chapter 15", floatPtr(15) + return b + }, + wantBumped: false, + }, + { + name: "unrelated metadata only", + mutate: func(b store.Bookmark) store.Bookmark { + b.Title, b.Cover = "Renamed", "https://example.test/new.jpg" + return b + }, + wantBumped: false, + }, + } + + for i, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + srv := newTestServer(t) + key := fmt.Sprintf("asura:cond-%d", i) + + first := putBookmark(t, srv, key, store.Bookmark{ + Title: "Test", + LastChapter: "Chapter 10", + LastChapterNum: 10, + }) + if first.UpdatedAt == 0 { + t.Fatal("new bookmark did not get updated_at set") + } + + // Guarantee a later wall-clock ms so a real bump is observable. + time.Sleep(2 * time.Millisecond) + + second := putBookmark(t, srv, key, tc.mutate(first)) + if tc.wantBumped && second.UpdatedAt <= first.UpdatedAt { + t.Fatalf("updated_at = %d, want > %d", second.UpdatedAt, first.UpdatedAt) + } + if !tc.wantBumped && second.UpdatedAt != first.UpdatedAt { + t.Fatalf("updated_at = %d, want preserved %d", second.UpdatedAt, first.UpdatedAt) + } + + // The PUT response must match what a subsequent GET reports. + list := getBookmarks(t, srv) + if len(list) != 1 { + t.Fatalf("list = %+v, want 1 item", list) + } + if list[0].UpdatedAt != second.UpdatedAt { + t.Fatalf("GET updated_at = %d, PUT echoed %d", list[0].UpdatedAt, second.UpdatedAt) + } + }) + } +} + +func TestFavoriteRoundTrip(t *testing.T) { + srv := newTestServer(t) + key := "demonic:some-series" + + stored := putBookmark(t, srv, key, store.Bookmark{Title: "Fav", Favorite: true}) + if !stored.Favorite { + t.Fatalf("PUT response favorite = false, want true") + } + + list := getBookmarks(t, srv) + if len(list) != 1 || !list[0].Favorite { + t.Fatalf("favorite did not round-trip: %+v", list) + } + + // Unfavoriting must persist too (guards against a write that only ever ORs in true). + stored = putBookmark(t, srv, key, store.Bookmark{Title: "Fav", Favorite: false}) + if stored.Favorite { + t.Fatal("PUT response favorite = true after unfavorite") + } + list = getBookmarks(t, srv) + if len(list) != 1 || list[0].Favorite { + t.Fatalf("unfavorite did not round-trip: %+v", list) + } +} + +func TestLatestChapterNullable(t *testing.T) { + srv := newTestServer(t) + key := "asura:latest-test" + + // Never captured: latest_chapter_num must serialize as JSON null. + body, _ := json.Marshal(store.Bookmark{Title: "No latest yet"}) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) + if rr.Code != http.StatusOK { + t.Fatalf("PUT status = %d", rr.Code) + } + if !strings.Contains(rr.Body.String(), `"latest_chapter_num":null`) { + t.Fatalf("want latest_chapter_num null in response, got %s", rr.Body.String()) + } + + list := getBookmarks(t, srv) + if len(list) != 1 || list[0].LatestChapterNum != nil { + t.Fatalf("latest_chapter_num = %v, want nil", list[0].LatestChapterNum) + } + + // Once captured it round-trips as a value. + stored := putBookmark(t, srv, key, store.Bookmark{ + Title: "No latest yet", + LatestChapter: "Chapter 162", + LatestChapterNum: floatPtr(162), + }) + if stored.LatestChapterNum == nil || *stored.LatestChapterNum != 162 { + t.Fatalf("PUT response latest_chapter_num = %v, want 162", stored.LatestChapterNum) + } + list = getBookmarks(t, srv) + if len(list) != 1 || list[0].LatestChapterNum == nil || *list[0].LatestChapterNum != 162 { + t.Fatalf("latest chapter did not round-trip: %+v", list) + } + if list[0].LatestChapter != "Chapter 162" { + t.Fatalf("latest_chapter = %q, want %q", list[0].LatestChapter, "Chapter 162") + } +} + +func TestLoadConfigWebPassword(t *testing.T) { + t.Setenv("API_TOKEN", "token-abc") + t.Setenv("WEB_PASSWORD", "hunter2") + if got := loadConfig().WebPassword; got != "hunter2" { + t.Fatalf("WebPassword = %q, want hunter2", got) + } + + t.Setenv("WEB_PASSWORD", "") + if got := loadConfig().WebPassword; got != "" { + t.Fatalf("WebPassword = %q with the variable unset, want empty", got) + } +} + +// A userscript PUT body has no latest_checked_at field. If the column is ever +// moved into bookmarkColumns, this test catches it: the PUT would reset the +// cooldown and the poller would re-fetch that series on every single tick. +func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) { + dbPath := filepath.Join(t.TempDir(), "test.db") + s, err := store.Open(dbPath) + if err != nil { + t.Fatalf("store.Open: %v", err) + } + t.Cleanup(func() { s.Close() }) + srv := newRouter(s, testConfig()) + + seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", 777) + + // Exactly what the userscript sends: no latest_checked_at key at all. + body := `{"key":"asura:x","site":"asura","series_id":"x", + "series_url":"https://asurascans.com/comics/x", + "last_chapter":"Chapter 5","last_chapter_num":5}` + req := httptest.NewRequest(http.MethodPut, "/bookmarks/asura:x", strings.NewReader(body)) + req.Header.Set("Authorization", "Bearer "+testToken) + req.Header.Set("Content-Type", "application/json") + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("PUT status = %d, want 200 (body %s)", rec.Code, rec.Body.String()) + } + if got := readLatestCheckedAt(t, s, "asura:x"); got != 777 { + t.Fatalf("latest_checked_at = %d after client PUT, want 777 preserved", got) + } +} + +// The userscript route is registered outside the `if cfg.WebPassword != ""` +// block in newRouter, so it must keep working on a deployment that never set +// WEB_PASSWORD — see internal/userscript for the handler's own behaviour. +func TestUserscriptServedWithWebUIDisabled(t *testing.T) { + path := filepath.Join(t.TempDir(), "manga-bookmark.user.js") + if err := os.WriteFile(path, []byte("console.log(1);\n"), 0o644); err != nil { + t.Fatalf("write script: %v", err) + } + + dbPath := filepath.Join(t.TempDir(), "nopass.db") + s, err := store.Open(dbPath) + if err != nil { + t.Fatalf("store.Open: %v", err) + } + t.Cleanup(func() { s.Close() }) + cfg := testConfig() // WebPassword empty + cfg.UserscriptPath = path + + rr := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "/u/"+testToken+"/manga-bookmark.user.js", nil) + newRouter(s, cfg).ServeHTTP(rr, req) + if rr.Code != http.StatusOK { + t.Fatalf("status = %d, want 200", rr.Code) + } +} diff --git a/backend/handlers.go b/backend/internal/api/handlers.go similarity index 71% rename from backend/handlers.go rename to backend/internal/api/handlers.go index 025d670..aa6447c 100644 --- a/backend/handlers.go +++ b/backend/internal/api/handlers.go @@ -1,4 +1,4 @@ -package main +package api import ( "encoding/json" @@ -6,10 +6,13 @@ import ( "net/http" "strings" "time" + + "mangabm/backend/internal/store" ) -type bookmarkHandler struct { - store *Store +// Handler serves the userscript-facing JSON bookmark API. +type Handler struct { + Store *store.Store } func writeJSON(w http.ResponseWriter, status int, v any) { @@ -22,9 +25,9 @@ func writeJSON(w http.ResponseWriter, status int, v any) { } } -// list returns all bookmarks. GET /bookmarks -func (h *bookmarkHandler) list(w http.ResponseWriter, r *http.Request) { - items, err := h.store.List() +// List returns all bookmarks. GET /bookmarks +func (h *Handler) List(w http.ResponseWriter, r *http.Request) { + items, err := h.Store.List() if err != nil { log.Printf("list: %v", err) http.Error(w, "internal error", http.StatusInternalServerError) @@ -33,15 +36,15 @@ func (h *bookmarkHandler) list(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusOK, items) } -// put upserts one bookmark. PUT /bookmarks/{key} -func (h *bookmarkHandler) put(w http.ResponseWriter, r *http.Request) { +// Put upserts one bookmark. PUT /bookmarks/{key} +func (h *Handler) Put(w http.ResponseWriter, r *http.Request) { key := r.PathValue("key") if key == "" { http.Error(w, "missing key", http.StatusBadRequest) return } - var b Bookmark + var b store.Bookmark if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<16)).Decode(&b); err != nil { http.Error(w, "invalid JSON body", http.StatusBadRequest) return @@ -65,9 +68,9 @@ func (h *bookmarkHandler) put(w http.ResponseWriter, r *http.Request) { // Finishing a series is a web-UI decision, so the JSON API refuses it // rather than trusting every client to leave it alone. switch b.Status { - case "", statusReading, statusArchived: - case statusFinished: - http.Error(w, "status "+statusFinished+" can only be set from the web UI", + case "", store.StatusReading, store.StatusArchived: + case store.StatusFinished: + http.Error(w, "status "+store.StatusFinished+" can only be set from the web UI", http.StatusBadRequest) return default: @@ -79,7 +82,7 @@ func (h *bookmarkHandler) put(w http.ResponseWriter, r *http.Request) { // reading progress actually moved. Any client value is ignored. b.UpdatedAt = time.Now().UnixMilli() - stored, err := h.store.Upsert(b) + stored, err := h.Store.Upsert(b) if err != nil { log.Printf("upsert: %v", err) http.Error(w, "internal error", http.StatusInternalServerError) @@ -90,14 +93,14 @@ func (h *bookmarkHandler) put(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusOK, stored) } -// delete removes one bookmark. DELETE /bookmarks/{key} -func (h *bookmarkHandler) delete(w http.ResponseWriter, r *http.Request) { +// Delete removes one bookmark. DELETE /bookmarks/{key} +func (h *Handler) Delete(w http.ResponseWriter, r *http.Request) { key := r.PathValue("key") if key == "" { http.Error(w, "missing key", http.StatusBadRequest) return } - if err := h.store.Delete(key); err != nil { + if err := h.Store.Delete(key); err != nil { log.Printf("delete: %v", err) http.Error(w, "internal error", http.StatusInternalServerError) return @@ -105,7 +108,8 @@ func (h *bookmarkHandler) delete(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusNoContent) } -func healthz(w http.ResponseWriter, r *http.Request) { +// Healthz answers the unauthenticated liveness check. GET /healthz +func Healthz(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "text/plain") w.WriteHeader(http.StatusOK) _, _ = w.Write([]byte("ok")) diff --git a/backend/middleware.go b/backend/internal/httpmw/middleware.go similarity index 88% rename from backend/middleware.go rename to backend/internal/httpmw/middleware.go index 220e4bf..e609531 100644 --- a/backend/middleware.go +++ b/backend/internal/httpmw/middleware.go @@ -1,4 +1,4 @@ -package main +package httpmw import ( "compress/gzip" @@ -9,8 +9,8 @@ import ( const bearerPrefix = "Bearer " -// withAuth guards a handler with a constant-time bearer-token check. -func withAuth(token string, next http.Handler) http.Handler { +// Auth guards a handler with a constant-time bearer-token check. +func Auth(token string, next http.Handler) http.Handler { want := []byte(token) return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { h := r.Header.Get("Authorization") @@ -71,11 +71,11 @@ func (w *gzipWriter) Write(b []byte) (int, error) { return w.ResponseWriter.Write(b) } -// withGzip compresses text responses for clients that ask. The templates, +// Gzip compresses text responses for clients that ask. The templates, // stylesheet and htmx together are ~120 KB uncompressed and roughly a quarter // of that gzipped, which is the difference between a fast and a slow first load // on mobile data. -func withGzip(next http.Handler) http.Handler { +func Gzip(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") { next.ServeHTTP(w, r) @@ -92,11 +92,11 @@ func withGzip(next http.Handler) http.Handler { }) } -// withCORS reflects the request Origin only when it is in allowed, answers +// CORS reflects the request Origin only when it is in allowed, answers // preflight OPTIONS with 204, and passes everything else through. It wraps the // auth middleware so preflight (which carries no Authorization header) is never // rejected by auth. -func withCORS(allowed []string, next http.Handler) http.Handler { +func CORS(allowed []string, next http.Handler) http.Handler { set := make(map[string]struct{}, len(allowed)) for _, o := range allowed { set[o] = struct{}{} diff --git a/backend/latest_http.go b/backend/internal/latest/fetch.go similarity index 89% rename from backend/latest_http.go rename to backend/internal/latest/fetch.go index d6a34ec..866bed4 100644 --- a/backend/latest_http.go +++ b/backend/internal/latest/fetch.go @@ -1,4 +1,4 @@ -package main +package latest import ( "context" @@ -20,20 +20,20 @@ const maxBodyBytes = 4 << 20 const chromeUA = "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 " + "(KHTML, like Gecko) Chrome/133.0.0.0 Mobile Safari/537.36" -// tlsFetcher fetches series pages with a Chrome TLS fingerprint. +// TLSFetcher fetches series pages with a Chrome TLS fingerprint. // // Plain net/http was verified working against both sites on 2026-07-26, so this // is not fixing an observed block — it is deliberate defence-in-depth against a // future fingerprint-based one, chosen up front rather than reacted to later. // The library is pure Go, so CGO_ENABLED=0, the static binary, and the // distroless image are all unaffected. -type tlsFetcher struct { +type TLSFetcher struct { client tls_client.HttpClient } -var _ fetcher = (*tlsFetcher)(nil) +var _ Fetcher = (*TLSFetcher)(nil) -func newTLSFetcher() (*tlsFetcher, error) { +func NewTLSFetcher() (*TLSFetcher, error) { c, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(), tls_client.WithTimeoutSeconds(30), tls_client.WithClientProfile(profiles.Chrome_133), @@ -41,13 +41,13 @@ func newTLSFetcher() (*tlsFetcher, error) { if err != nil { return nil, fmt.Errorf("new tls client: %w", err) } - return &tlsFetcher{client: c}, nil + return &TLSFetcher{client: c}, nil } // Get fetches url and returns the body and status. Redirects are followed: the // demonic chapter anchors are a redirect form, and asura has moved domains // before. -func (f *tlsFetcher) Get(ctx context.Context, url string) (string, int, error) { +func (f *TLSFetcher) Get(ctx context.Context, url string) (string, int, error) { req, err := fhttp.NewRequest(fhttp.MethodGet, url, nil) if err != nil { return "", 0, fmt.Errorf("build request %q: %w", url, err) diff --git a/backend/latest.go b/backend/internal/latest/poller.go similarity index 82% rename from backend/latest.go rename to backend/internal/latest/poller.go index 40a36aa..b2214ca 100644 --- a/backend/latest.go +++ b/backend/internal/latest/poller.go @@ -1,40 +1,42 @@ -package main +package latest import ( "context" "log" "net/url" "time" + + "mangabm/backend/internal/store" ) -// fetcher retrieves a series page. It exists as an interface so tests can inject +// Fetcher retrieves a series page. It exists as an interface so tests can inject // a fake: nothing in the test suite may touch the network or the TLS client. -type fetcher interface { +type Fetcher interface { Get(ctx context.Context, url string) (body string, status int, err error) } -// latestPoller re-checks each bookmarked series' newest published chapter on a +// Poller re-checks each bookmarked series' newest published chapter on a // schedule, independent of the userscript's own in-browser checks. The two run // in parallel and report the same observable fact, so whichever writes last wins // and neither needs to know about the other. // // Two clocks, deliberately independent: // -// - interval is how often this goroutine wakes up and looks. -// - cooldown is how long one bookmark rests since its own last check. +// - Interval is how often this goroutine wakes up and looks. +// - Cooldown is how long one bookmark rests since its own last check. // // Only the cooldown is per bookmark, and it is enforced by the WHERE clause in -// DueForLatestCheck rather than by any timer. Shortening interval therefore +// DueForLatestCheck rather than by any timer. Shortening Interval therefore // cannot shorten anyone's cooldown; it only makes the poller wake up and find // nothing due more often. -type latestPoller struct { - store *Store - fetch fetcher - now func() time.Time // injected so tests can freeze it - cooldown time.Duration - interval time.Duration - stagger time.Duration - batch int +type Poller struct { + Store *store.Store + Fetch Fetcher + Now func() time.Time // injected so tests can freeze it + Cooldown time.Duration + Interval time.Duration + Stagger time.Duration + Batch int } // Run polls until ctx is cancelled. @@ -43,10 +45,10 @@ type latestPoller struct { // next one instead of stacking a second batch on top of it. That is the intended // failure mode for a misconfigured batch x stagger: a slower cadence, never // concurrent fetch storms. -func (p *latestPoller) Run(ctx context.Context) { +func (p *Poller) Run(ctx context.Context) { log.Printf("latest-chapter poller: interval=%s cooldown=%s batch=%d stagger=%s", - p.interval, p.cooldown, p.batch, p.stagger) - t := time.NewTicker(p.interval) + p.Interval, p.Cooldown, p.Batch, p.Stagger) + t := time.NewTicker(p.Interval) defer t.Stop() for { select { @@ -60,9 +62,9 @@ func (p *latestPoller) Run(ctx context.Context) { } // runOnce processes one batch of due bookmarks. -func (p *latestPoller) runOnce(ctx context.Context) { - cutoff := p.now().Add(-p.cooldown).UnixMilli() - due, err := p.store.DueForLatestCheck(cutoff, p.batch) +func (p *Poller) runOnce(ctx context.Context) { + cutoff := p.Now().Add(-p.Cooldown).UnixMilli() + due, err := p.Store.DueForLatestCheck(cutoff, p.Batch) if err != nil { log.Printf("latest poll: due query: %v", err) return @@ -78,11 +80,11 @@ func (p *latestPoller) runOnce(ctx context.Context) { // bot score. This is the server-side analogue of the userscript's "one // series per navigation ... indistinguishable from browsing" (L455-456). stopped := false - if i > 0 && p.stagger > 0 { + if i > 0 && p.Stagger > 0 { select { case <-ctx.Done(): stopped = true - case <-time.After(p.stagger): + case <-time.After(p.Stagger): } } if stopped { @@ -100,7 +102,7 @@ func (p *latestPoller) runOnce(ctx context.Context) { // checkOne re-checks one series. Every failure path here is "log and move on": // the poller is a best-effort enhancement, and no single bad series may stall a // batch or take down the process. -func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { +func (p *Poller) checkOne(ctx context.Context, b store.Bookmark) { defer func() { if r := recover(); r != nil { log.Printf("latest poll %q: recovered from panic: %v", b.Key, r) @@ -111,7 +113,7 @@ func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { // mid-request still consumes the cooldown. Otherwise a renamed or deleted // series would be retried on every single tick forever. The userscript // stamps in the same order and for the same reason (L471-473). - if err := p.store.MarkLatestChecked(b.Key, p.now().UnixMilli()); err != nil { + if err := p.Store.MarkLatestChecked(b.Key, p.Now().UnixMilli()); err != nil { log.Printf("latest poll %q: mark checked: %v", b.Key, err) return } @@ -128,7 +130,7 @@ func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { return } - body, status, err := p.fetch.Get(ctx, b.SeriesURL) + body, status, err := p.Fetch.Get(ctx, b.SeriesURL) if err != nil { log.Printf("latest poll %q: fetch %s: %v", b.Key, b.SeriesURL, err) return @@ -155,7 +157,7 @@ func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { // progress or a status change, and moving updated_at because the stored // value now differs. Accepted for a single-user deployment: the window is // milliseconds and the loser is one poll cycle. - cur, found, err := p.store.Get(b.Key) + cur, found, err := p.Store.Get(b.Key) if err != nil { log.Printf("latest poll %q: reread: %v", b.Key, err) return @@ -174,8 +176,8 @@ func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { cur.LatestChapterNum = &num // A candidate only. last_chapter_num is untouched, so the CASE in Upsert // keeps the stored updated_at and the bookmark list does not reorder. - cur.UpdatedAt = p.now().UnixMilli() - if _, err := p.store.Upsert(cur); err != nil { + cur.UpdatedAt = p.Now().UnixMilli() + if _, err := p.Store.Upsert(cur); err != nil { log.Printf("latest poll %q: upsert: %v", b.Key, err) return } diff --git a/backend/latest_test.go b/backend/internal/latest/poller_test.go similarity index 85% rename from backend/latest_test.go rename to backend/internal/latest/poller_test.go index 04502e0..d60daa4 100644 --- a/backend/latest_test.go +++ b/backend/internal/latest/poller_test.go @@ -1,13 +1,53 @@ -package main +package latest import ( "context" "errors" + "path/filepath" "sync" "testing" "time" + + "mangabm/backend/internal/store" ) +// newTestStore opens a fresh SQLite store in a temp dir. +func newTestStore(t *testing.T) *store.Store { + t.Helper() + s, err := store.Open(filepath.Join(t.TempDir(), "test.db")) + if err != nil { + t.Fatalf("Open: %v", err) + } + t.Cleanup(func() { s.Close() }) + return s +} + +// seedForCheck inserts a bookmark and forces its latest_checked_at. +func seedForCheck(t *testing.T, s *store.Store, key, seriesURL string, checkedAt int64) { + t.Helper() + if _, err := s.Upsert(store.Bookmark{ + Key: key, + Site: "asura", + SeriesID: key, + SeriesURL: seriesURL, + UpdatedAt: 1000, + }); err != nil { + t.Fatalf("seed %q: %v", key, err) + } + if err := s.MarkLatestChecked(key, checkedAt); err != nil { + t.Fatalf("seed mark %q: %v", key, err) + } +} + +func readLatestCheckedAt(t *testing.T, s *store.Store, key string) int64 { + t.Helper() + ts, err := s.LatestCheckedAt(key) + if err != nil { + t.Fatalf("LatestCheckedAt %q: %v", key, err) + } + return ts +} + // fakeFetcher stands in for the network. Every poller test uses it, so nothing // in this file can reach tls-client or a real site. type fakeFetcher struct { @@ -44,16 +84,16 @@ func (f *fakeFetcher) callCount() int { // newTestPoller wires a poller with a frozen clock and no stagger, so tests run // instantly and deterministically. -func newTestPoller(t *testing.T, s *Store, f fetcher, at time.Time) *latestPoller { +func newTestPoller(t *testing.T, s *store.Store, f Fetcher, at time.Time) *Poller { t.Helper() - return &latestPoller{ - store: s, - fetch: f, - now: func() time.Time { return at }, - cooldown: time.Hour, - interval: 10 * time.Minute, - stagger: 0, - batch: 14, + return &Poller{ + Store: s, + Fetch: f, + Now: func() time.Time { return at }, + Cooldown: time.Hour, + Interval: 10 * time.Minute, + Stagger: 0, + Batch: 14, } } @@ -89,7 +129,7 @@ func TestRunOnceDoesNotReorderList(t *testing.T) { const key = "asura:chronicles-of-the-demon-faction-f886a8af" // "other" is the most recently read, so it must stay at the top of List(). - if _, err := s.Upsert(Bookmark{ + if _, err := s.Upsert(store.Bookmark{ Key: "asura:other", Site: "asura", SeriesID: "other", SeriesURL: "https://asurascans.com/comics/other", UpdatedAt: 9_000_000, }); err != nil { @@ -166,7 +206,7 @@ func TestRunOnceRespectsBatchLimit(t *testing.T) { f := &fakeFetcher{body: "", status: 200} p := newTestPoller(t, s, f, time.UnixMilli(5_000_000)) - p.batch = 5 + p.Batch = 5 p.runOnce(context.Background()) if got := f.callCount(); got != 5 { @@ -218,13 +258,13 @@ func TestRunOnceHonoursCooldownAcrossPasses(t *testing.T) { } // Same instant, and again 59 minutes later: both inside the 1h cooldown. p.runOnce(context.Background()) - p.now = func() time.Time { return now.Add(59 * time.Minute) } + p.Now = func() time.Time { return now.Add(59 * time.Minute) } p.runOnce(context.Background()) if got := f.callCount(); got != 1 { t.Fatalf("fetched %d times inside the cooldown, want 1", got) } // Past the cooldown, it is due again. - p.now = func() time.Time { return now.Add(61 * time.Minute) } + p.Now = func() time.Time { return now.Add(61 * time.Minute) } p.runOnce(context.Background()) if got := f.callCount(); got != 2 { t.Fatalf("fetched %d times after the cooldown, want 2", got) @@ -239,7 +279,7 @@ func TestRunOnceCorrectsDownward(t *testing.T) { const key = "demonic:Catastrophic-Necromancer" high := 400.0 - if _, err := s.Upsert(Bookmark{ + if _, err := s.Upsert(store.Bookmark{ Key: key, Site: "demonic", SeriesID: "Catastrophic-Necromancer", SeriesURL: url, LatestChapter: "Chapter 400", LatestChapterNum: &high, UpdatedAt: 1000, @@ -278,7 +318,7 @@ func TestCheckOneValidatesSeriesURLBeforeFetching(t *testing.T) { t.Run(tt.name, func(t *testing.T) { s := newTestStore(t) key := tt.site + ":x" - if _, err := s.Upsert(Bookmark{ + if _, err := s.Upsert(store.Bookmark{ Key: key, Site: tt.site, SeriesID: "x", SeriesURL: tt.seriesURL, UpdatedAt: 1000, }); err != nil { @@ -287,7 +327,7 @@ func TestCheckOneValidatesSeriesURLBeforeFetching(t *testing.T) { now := time.UnixMilli(4_000_000) f := &fakeFetcher{body: asuraSeriesFixture, status: 200} - newTestPoller(t, s, f, now).checkOne(context.Background(), Bookmark{ + newTestPoller(t, s, f, now).checkOne(context.Background(), store.Bookmark{ Key: key, Site: tt.site, SeriesURL: tt.seriesURL, }) diff --git a/backend/latest_sites.go b/backend/internal/latest/sites.go similarity index 96% rename from backend/latest_sites.go rename to backend/internal/latest/sites.go index d503dbf..9b7c72f 100644 --- a/backend/latest_sites.go +++ b/backend/internal/latest/sites.go @@ -1,9 +1,11 @@ -package main +package latest import ( "regexp" "strconv" "strings" + + "mangabm/backend/internal/store" ) // latestChapter is the newest chapter a series page advertises. @@ -53,7 +55,7 @@ func latestChapterFrom(site, seriesURL, body string) (latestChapter, bool) { // chapter hrefs in the fetched body carry the current one. Strip to // the stable ID (same rule as migrateAsuraKeys) and make the hash // optional in the pattern, so scoping survives rotations. - slug := asuraBuildHash.ReplaceAllString(m[1], "") + slug := store.AsuraBuildHash.ReplaceAllString(m[1], "") // Compiled per call rather than cached: this runs once per fetch, which // is at most a few times a minute, and the slug varies per series. re = regexp.MustCompile(`/comics/` + regexp.QuoteMeta(slug) + `(?:-[0-9a-f]{8})?/chapter/([0-9.]+)`) diff --git a/backend/latest_sites_test.go b/backend/internal/latest/sites_test.go similarity index 99% rename from backend/latest_sites_test.go rename to backend/internal/latest/sites_test.go index 443653d..de94005 100644 --- a/backend/latest_sites_test.go +++ b/backend/internal/latest/sites_test.go @@ -1,4 +1,4 @@ -package main +package latest import "testing" diff --git a/backend/session.go b/backend/internal/session/session.go similarity index 75% rename from backend/session.go rename to backend/internal/session/session.go index 34d63c8..3c70592 100644 --- a/backend/session.go +++ b/backend/internal/session/session.go @@ -1,4 +1,4 @@ -package main +package session import ( "crypto/hmac" @@ -14,7 +14,7 @@ import ( ) const ( - sessionCookieName = "mangabm_session" + CookieName = "mangabm_session" // 60 days: long enough that a phone stays logged in between reading spells. sessionTTL = 60 * 24 * time.Hour // Domain separation, so the session key can never collide with any other @@ -23,18 +23,18 @@ const ( sessionKeyPurpose = "mangabm-web-session-v1" ) -// sessionKey derives the cookie-signing key from both secrets. Sessions are +// Key derives the cookie-signing key from both secrets. Sessions are // stateless — there is no session table — so rotating either API_TOKEN or // WEB_PASSWORD invalidates every outstanding cookie at once. The \x00 // separator prevents the concatenation ambiguity a bare apiToken+webPassword // would have (e.g. "ab"+"c" colliding with "a"+"bc"). -func sessionKey(apiToken, webPassword string) []byte { +func Key(apiToken, webPassword string) []byte { sum := sha256.Sum256([]byte(apiToken + "\x00" + webPassword + sessionKeyPurpose)) return sum[:] } -// signSession encodes ".". -func signSession(key []byte, expiryMs int64) string { +// Sign encodes ".". +func Sign(key []byte, expiryMs int64) string { payload := strconv.FormatInt(expiryMs, 10) return payload + "." + sessionMAC(key, payload) } @@ -45,10 +45,10 @@ func sessionMAC(key []byte, payload string) string { return base64.RawURLEncoding.EncodeToString(mac.Sum(nil)) } -// verifySession checks shape, then expiry, then the signature — in that order. +// Verify checks shape, then expiry, then the signature — in that order. // The signature comparison is constant-time; the checks before it only look at // data the holder already supplied, so their timing leaks nothing. -func verifySession(key []byte, value string, nowMs int64) bool { +func Verify(key []byte, value string, nowMs int64) bool { payload, sig, ok := strings.Cut(value, ".") if !ok { return false @@ -69,10 +69,10 @@ func isHTTPS(r *http.Request) bool { return r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https" } -func setSessionCookie(w http.ResponseWriter, r *http.Request, key []byte) { +func SetCookie(w http.ResponseWriter, r *http.Request, key []byte) { http.SetCookie(w, &http.Cookie{ - Name: sessionCookieName, - Value: signSession(key, time.Now().Add(sessionTTL).UnixMilli()), + Name: CookieName, + Value: Sign(key, time.Now().Add(sessionTTL).UnixMilli()), Path: "/", MaxAge: int(sessionTTL / time.Second), HttpOnly: true, @@ -81,9 +81,9 @@ func setSessionCookie(w http.ResponseWriter, r *http.Request, key []byte) { }) } -func clearSessionCookie(w http.ResponseWriter, r *http.Request) { +func ClearCookie(w http.ResponseWriter, r *http.Request) { http.SetCookie(w, &http.Cookie{ - Name: sessionCookieName, + Name: CookieName, Value: "", Path: "/", MaxAge: -1, @@ -94,11 +94,11 @@ func clearSessionCookie(w http.ResponseWriter, r *http.Request) { } const ( - loginMaxFailures = 10 - loginWindow = 20 * time.Minute + MaxFailures = 10 + Window = 20 * time.Minute ) -// clientIP returns the address the reverse proxy actually observed. +// ClientIP returns the address the reverse proxy actually observed. // // Traefik appends the peer address to whatever X-Forwarded-For the client sent, // so the leftmost entry is attacker-controlled and the rightmost is not. Go's @@ -106,7 +106,7 @@ const ( // by sending its own; Values covers every line so the true last hop is found. // RemoteAddr is useless behind the proxy — it is always the Traefik container — // so it serves only as the direct-connection fallback for local development. -func clientIP(r *http.Request) string { +func ClientIP(r *http.Request) string { if vals := r.Header.Values("X-Forwarded-For"); len(vals) > 0 { hops := strings.Split(vals[len(vals)-1], ",") if ip := strings.TrimSpace(hops[len(hops)-1]); ip != "" { @@ -120,8 +120,8 @@ func clientIP(r *http.Request) string { return host } -// loginLimiter throttles password guessing: loginMaxFailures failures inside a -// rolling loginWindow blocks further attempts from that IP until the oldest one +// LoginLimiter throttles password guessing: MaxFailures failures inside a +// rolling Window blocks further attempts from that IP until the oldest one // ages out. There is no permanent ban and no unlock step. // // Behind carrier-grade NAT this budget is shared with every other subscriber on @@ -132,34 +132,34 @@ func clientIP(r *http.Request) string { // State is in memory and per-process, so a restart clears it. Entries are // pruned lazily on access; for a single-user deployment the map cannot grow // past the handful of addresses that ever attempt a login. -type loginLimiter struct { +type LoginLimiter struct { mu sync.Mutex failures map[string][]time.Time } -func newLoginLimiter() *loginLimiter { - return &loginLimiter{failures: make(map[string][]time.Time)} +func NewLoginLimiter() *LoginLimiter { + return &LoginLimiter{failures: make(map[string][]time.Time)} } // retryAfter returns how long ip must wait, or zero when it may try now. -func (l *loginLimiter) retryAfter(ip string, now time.Time) time.Duration { +func (l *LoginLimiter) RetryAfter(ip string, now time.Time) time.Duration { l.mu.Lock() defer l.mu.Unlock() recent := l.pruneLocked(ip, now) - if len(recent) < loginMaxFailures { + if len(recent) < MaxFailures { return 0 } - return recent[0].Add(loginWindow).Sub(now) + return recent[0].Add(Window).Sub(now) } -func (l *loginLimiter) fail(ip string, now time.Time) { +func (l *LoginLimiter) Fail(ip string, now time.Time) { l.mu.Lock() defer l.mu.Unlock() l.failures[ip] = append(l.pruneLocked(ip, now), now) } -func (l *loginLimiter) reset(ip string) { +func (l *LoginLimiter) Reset(ip string) { l.mu.Lock() defer l.mu.Unlock() delete(l.failures, ip) @@ -167,8 +167,8 @@ func (l *loginLimiter) reset(ip string) { // pruneLocked drops attempts older than the window and returns what is left. // The caller must hold l.mu. -func (l *loginLimiter) pruneLocked(ip string, now time.Time) []time.Time { - cutoff := now.Add(-loginWindow) +func (l *LoginLimiter) pruneLocked(ip string, now time.Time) []time.Time { + cutoff := now.Add(-Window) // In-place filter: kept reuses the backing array of the slice being // ranged over. Safe to alias because append writes at index len(kept), // which is always <= the range index i, and element i is read before diff --git a/backend/session_test.go b/backend/internal/session/session_test.go similarity index 68% rename from backend/session_test.go rename to backend/internal/session/session_test.go index 4c261b4..327d168 100644 --- a/backend/session_test.go +++ b/backend/internal/session/session_test.go @@ -1,4 +1,4 @@ -package main +package session import ( "crypto/tls" @@ -10,18 +10,18 @@ import ( ) func TestSessionRoundTrip(t *testing.T) { - key := sessionKey("token-abc", "pw-abc") + key := Key("token-abc", "pw-abc") now := time.Now().UnixMilli() - value := signSession(key, now+60_000) - if !verifySession(key, value, now) { - t.Fatal("verifySession = false for a freshly signed cookie, want true") + value := Sign(key, now+60_000) + if !Verify(key, value, now) { + t.Fatal("Verify = false for a freshly signed cookie, want true") } } func TestSessionRejects(t *testing.T) { - key := sessionKey("token-abc", "pw-abc") + key := Key("token-abc", "pw-abc") now := time.Now().UnixMilli() - valid := signSession(key, now+60_000) + valid := Sign(key, now+60_000) payload, sig, _ := strings.Cut(valid, ".") cases := []struct { @@ -31,15 +31,15 @@ func TestSessionRejects(t *testing.T) { {"empty", ""}, {"no separator", payload + sig}, {"unparseable expiry", "notanumber." + sig}, - {"expired", signSession(key, now-1)}, + {"expired", Sign(key, now-1)}, {"tampered signature", payload + "." + flipLastChar(sig)}, {"tampered expiry", "99999999999999." + sig}, - {"signed with another key", signSession(sessionKey("other-token", "pw-abc"), now+60_000)}, + {"signed with another key", Sign(Key("other-token", "pw-abc"), now+60_000)}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { - if verifySession(key, tc.value, now) { - t.Fatalf("verifySession(%q) = true, want false", tc.value) + if Verify(key, tc.value, now) { + t.Fatalf("Verify(%q) = true, want false", tc.value) } }) } @@ -57,18 +57,18 @@ func flipLastChar(s string) string { } func TestSessionKeyDependsOnToken(t *testing.T) { - a := sessionKey("token-a", "pw-abc") - b := sessionKey("token-b", "pw-abc") + a := Key("token-a", "pw-abc") + b := Key("token-b", "pw-abc") if string(a) == string(b) { - t.Fatal("sessionKey collided for different API tokens") + t.Fatal("Key collided for different API tokens") } } func TestSessionKeyDependsOnWebPassword(t *testing.T) { - a := sessionKey("token-abc", "pw-a") - b := sessionKey("token-abc", "pw-b") + a := Key("token-abc", "pw-a") + b := Key("token-abc", "pw-b") if string(a) == string(b) { - t.Fatal("sessionKey collided for different web passwords with the same API token") + t.Fatal("Key collided for different web passwords with the same API token") } } @@ -94,15 +94,15 @@ func TestSetSessionCookieAttributes(t *testing.T) { r.Header.Set("X-Forwarded-Proto", tc.forwarded) } rr := httptest.NewRecorder() - setSessionCookie(rr, r, sessionKey("token-abc", "pw-abc")) + SetCookie(rr, r, Key("token-abc", "pw-abc")) cookies := rr.Result().Cookies() if len(cookies) != 1 { t.Fatalf("got %d cookies, want 1", len(cookies)) } c := cookies[0] - if c.Name != sessionCookieName { - t.Fatalf("cookie name = %q, want %q", c.Name, sessionCookieName) + if c.Name != CookieName { + t.Fatalf("cookie name = %q, want %q", c.Name, CookieName) } if !c.HttpOnly { t.Fatal("cookie HttpOnly = false, want true") @@ -126,7 +126,7 @@ func TestSetSessionCookieAttributes(t *testing.T) { func TestClearSessionCookie(t *testing.T) { r := httptest.NewRequest(http.MethodPost, "/logout", nil) rr := httptest.NewRecorder() - clearSessionCookie(rr, r) + ClearCookie(rr, r) cookies := rr.Result().Cookies() if len(cookies) != 1 { @@ -168,65 +168,65 @@ func TestClientIP(t *testing.T) { for _, v := range tc.xff { r.Header.Add("X-Forwarded-For", v) } - if got := clientIP(r); got != tc.want { - t.Fatalf("clientIP() = %q, want %q", got, tc.want) + if got := ClientIP(r); got != tc.want { + t.Fatalf("ClientIP() = %q, want %q", got, tc.want) } }) } } func TestLoginLimiterBlocksAfterMaxFailures(t *testing.T) { - l := newLoginLimiter() + l := NewLoginLimiter() now := time.Now() - for i := 0; i < loginMaxFailures; i++ { - if wait := l.retryAfter("1.2.3.4", now); wait != 0 { - t.Fatalf("blocked after %d failures, want block only after %d", i, loginMaxFailures) + for i := 0; i < MaxFailures; i++ { + if wait := l.RetryAfter("1.2.3.4", now); wait != 0 { + t.Fatalf("blocked after %d failures, want block only after %d", i, MaxFailures) } - l.fail("1.2.3.4", now) + l.Fail("1.2.3.4", now) } - wait := l.retryAfter("1.2.3.4", now) + wait := l.RetryAfter("1.2.3.4", now) if wait <= 0 { - t.Fatalf("retryAfter = %v after %d failures, want > 0", wait, loginMaxFailures) + t.Fatalf("retryAfter = %v after %d failures, want > 0", wait, MaxFailures) } - if wait > loginWindow { - t.Fatalf("retryAfter = %v, want <= %v", wait, loginWindow) + if wait > Window { + t.Fatalf("retryAfter = %v, want <= %v", wait, Window) } } func TestLoginLimiterWindowExpires(t *testing.T) { - l := newLoginLimiter() + l := NewLoginLimiter() start := time.Now() - for i := 0; i < loginMaxFailures; i++ { - l.fail("1.2.3.4", start) + for i := 0; i < MaxFailures; i++ { + l.Fail("1.2.3.4", start) } - if l.retryAfter("1.2.3.4", start) == 0 { + if l.RetryAfter("1.2.3.4", start) == 0 { t.Fatal("expected block immediately after the failures") } - later := start.Add(loginWindow + time.Second) - if wait := l.retryAfter("1.2.3.4", later); wait != 0 { + later := start.Add(Window + time.Second) + if wait := l.RetryAfter("1.2.3.4", later); wait != 0 { t.Fatalf("retryAfter = %v once the window passed, want 0", wait) } } func TestLoginLimiterResetClearsCounter(t *testing.T) { - l := newLoginLimiter() + l := NewLoginLimiter() now := time.Now() - for i := 0; i < loginMaxFailures; i++ { - l.fail("1.2.3.4", now) + for i := 0; i < MaxFailures; i++ { + l.Fail("1.2.3.4", now) } - l.reset("1.2.3.4") - if wait := l.retryAfter("1.2.3.4", now); wait != 0 { + l.Reset("1.2.3.4") + if wait := l.RetryAfter("1.2.3.4", now); wait != 0 { t.Fatalf("retryAfter = %v after reset, want 0", wait) } } func TestLoginLimiterIsPerIP(t *testing.T) { - l := newLoginLimiter() + l := NewLoginLimiter() now := time.Now() - for i := 0; i < loginMaxFailures; i++ { - l.fail("1.2.3.4", now) + for i := 0; i < MaxFailures; i++ { + l.Fail("1.2.3.4", now) } - if wait := l.retryAfter("5.6.7.8", now); wait != 0 { + if wait := l.RetryAfter("5.6.7.8", now); wait != 0 { t.Fatalf("retryAfter for a different IP = %v, want 0", wait) } } diff --git a/backend/store.go b/backend/internal/store/store.go similarity index 93% rename from backend/store.go rename to backend/internal/store/store.go index a911ef5..0658d2b 100644 --- a/backend/store.go +++ b/backend/internal/store/store.go @@ -1,4 +1,4 @@ -package main +package store import ( "database/sql" @@ -86,11 +86,21 @@ func (b Bookmark) ContinueURL() string { return b.SeriesURL } +// Initial is the monogram the web UI shows in place of a cover when the +// source site never gave us an og:image. First rune, uppercased; "?" when even +// the title is missing, so the slot is never empty. +func (b Bookmark) Initial() string { + for _, r := range b.Title { + return strings.ToUpper(string(r)) + } + return "?" +} + // Lifecycle buckets. A bookmark is in exactly one; favorite is orthogonal. const ( - statusReading = "reading" - statusArchived = "archived" - statusFinished = "finished" + StatusReading = "reading" + StatusArchived = "archived" + StatusFinished = "finished" ) const schema = ` @@ -137,7 +147,7 @@ type Store struct { } // OpenStore opens (or creates) the SQLite database at path and applies the schema. -func OpenStore(path string) (*Store, error) { +func Open(path string) (*Store, error) { // busy_timeout guards against SQLITE_BUSY under the reverse proxy's // concurrent requests; a single writer connection keeps writes serialized. dsn := path @@ -182,11 +192,11 @@ func migrateColumns(db *sql.DB) error { return nil } -// asuraBuildHash matches the trailing "-xxxxxxxx" site-wide build ID Asura +// AsuraBuildHash matches the trailing "-xxxxxxxx" site-wide build ID Asura // appends to every series slug. It rotates on each site redeploy, so it // must not be part of series_id. Must stay in sync with stripBuildHash in // userscript/manga-bookmark.user.js. -var asuraBuildHash = regexp.MustCompile(`-[0-9a-f]{8}$`) +var AsuraBuildHash = regexp.MustCompile(`-[0-9a-f]{8}$`) // migrateAsuraKeys rewrites asura bookmarks whose series_id still carries // the build hash to the stable, hashless ID. Rows keyed with a hash are @@ -218,7 +228,7 @@ func migrateAsuraKeys(db *sql.DB) error { groups := map[string][]row{} for _, r := range all { - stripped := asuraBuildHash.ReplaceAllString(r.id, "") + stripped := AsuraBuildHash.ReplaceAllString(r.id, "") groups[stripped] = append(groups[stripped], r) } for stripped, g := range groups { @@ -305,8 +315,8 @@ func scanBookmark(scan func(...any) error) (Bookmark, error) { // leave the row in no list at all, so anything outside the three known // buckets reads as the default rather than being passed through. b.Status = status.String - if b.Status != statusReading && b.Status != statusArchived && b.Status != statusFinished { - b.Status = statusReading + if b.Status != StatusReading && b.Status != StatusArchived && b.Status != StatusFinished { + b.Status = StatusReading } return b, nil } @@ -481,3 +491,16 @@ func (s *Store) MarkLatestChecked(key string, ts int64) error { } return nil } + +// LatestCheckedAt reads the column MarkLatestChecked writes. It exists for +// tests outside this package (the poller's own tests assert on cooldown +// bookkeeping) — see MarkLatestChecked for why the field itself stays off +// Bookmark. +func (s *Store) LatestCheckedAt(key string) (int64, error) { + var ts int64 + if err := s.db.QueryRow( + `SELECT latest_checked_at FROM bookmarks WHERE key = ?`, key).Scan(&ts); err != nil { + return 0, fmt.Errorf("latest checked at %q: %w", key, err) + } + return ts, nil +} diff --git a/backend/store_test.go b/backend/internal/store/store_test.go similarity index 57% rename from backend/store_test.go rename to backend/internal/store/store_test.go index 03d4f1b..0653b01 100644 --- a/backend/store_test.go +++ b/backend/internal/store/store_test.go @@ -1,42 +1,15 @@ -package main +package store import ( - "bytes" "database/sql" - "encoding/json" - "fmt" - "net/http" - "net/http/httptest" "path/filepath" - "strings" "testing" "time" ) -const testToken = "s3cret-token" - -func testConfig() Config { - return Config{ - Token: testToken, - AllowedOrigins: []string{"https://asuracomic.net", "https://demonicscans.org"}, - Port: "8080", - } -} - -func newTestServer(t *testing.T) http.Handler { - t.Helper() - dbPath := filepath.Join(t.TempDir(), "test.db") - store, err := OpenStore(dbPath) - if err != nil { - t.Fatalf("OpenStore: %v", err) - } - t.Cleanup(func() { store.Close() }) - return newRouter(store, testConfig()) -} - func newTestStore(t *testing.T) *Store { t.Helper() - store, err := OpenStore(filepath.Join(t.TempDir(), "test.db")) + store, err := Open(filepath.Join(t.TempDir(), "test.db")) if err != nil { t.Fatalf("OpenStore: %v", err) } @@ -44,346 +17,6 @@ func newTestStore(t *testing.T) *Store { return store } -func auth(req *http.Request) *http.Request { - req.Header.Set("Authorization", "Bearer "+testToken) - return req -} - -func TestHealthzNoAuth(t *testing.T) { - srv := newTestServer(t) - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil)) - if rr.Code != http.StatusOK { - t.Fatalf("healthz status = %d, want 200", rr.Code) - } - if rr.Body.String() != "ok" { - t.Fatalf("healthz body = %q, want ok", rr.Body.String()) - } -} - -func TestAuthRequired(t *testing.T) { - srv := newTestServer(t) - cases := []struct { - name string - header string - }{ - {"no header", ""}, - {"bad token", "Bearer wrong"}, - {"not bearer", "Basic " + testToken}, - {"empty bearer", "Bearer "}, - } - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - req := httptest.NewRequest(http.MethodGet, "/bookmarks", nil) - if tc.header != "" { - req.Header.Set("Authorization", tc.header) - } - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, req) - if rr.Code != http.StatusUnauthorized { - t.Fatalf("status = %d, want 401", rr.Code) - } - }) - } -} - -func TestAuthAccepted(t *testing.T) { - srv := newTestServer(t) - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - if rr.Code != http.StatusOK { - t.Fatalf("status = %d, want 200", rr.Code) - } - if got := rr.Body.String(); got != "[]\n" { - t.Fatalf("empty list body = %q, want []", got) - } -} - -func TestCORSPreflight(t *testing.T) { - srv := newTestServer(t) - req := httptest.NewRequest(http.MethodOptions, "/bookmarks/asura:foo-1", nil) - req.Header.Set("Origin", "https://asuracomic.net") - req.Header.Set("Access-Control-Request-Method", "PUT") - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, req) - - if rr.Code != http.StatusNoContent { - t.Fatalf("preflight status = %d, want 204", rr.Code) - } - if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "https://asuracomic.net" { - t.Fatalf("Allow-Origin = %q, want reflected origin", got) - } - if got := rr.Header().Get("Access-Control-Allow-Methods"); got == "" { - t.Fatal("Allow-Methods missing") - } - if got := rr.Header().Get("Access-Control-Allow-Headers"); got == "" { - t.Fatal("Allow-Headers missing") - } -} - -func TestCORSDisallowedOrigin(t *testing.T) { - srv := newTestServer(t) - req := httptest.NewRequest(http.MethodOptions, "/bookmarks", nil) - req.Header.Set("Origin", "https://evil.example") - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, req) - if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "" { - t.Fatalf("Allow-Origin = %q, want empty for disallowed origin", got) - } -} - -func TestBookmarkRoundTrip(t *testing.T) { - srv := newTestServer(t) - key := "asura:solo-leveling-123" - in := Bookmark{ - Title: "Solo Leveling", - SeriesURL: "https://asuracomic.net/series/solo-leveling-123", - Cover: "https://asuracomic.net/cover.jpg", - LastChapter: "Chapter 10", - LastChapterNum: 10, - LastChapterURL: "https://asuracomic.net/series/solo-leveling-123/chapter/10", - } - body, _ := json.Marshal(in) - - // PUT - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) - if rr.Code != http.StatusOK { - t.Fatalf("PUT status = %d, want 200", rr.Code) - } - var stored Bookmark - if err := json.Unmarshal(rr.Body.Bytes(), &stored); err != nil { - t.Fatalf("decode PUT response: %v", err) - } - if stored.Key != key || stored.Site != "asura" || stored.SeriesID != "solo-leveling-123" { - t.Fatalf("derived fields wrong: %+v", stored) - } - if stored.UpdatedAt == 0 { - t.Fatal("server did not set updated_at") - } - - // GET - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - var list []Bookmark - if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil { - t.Fatalf("decode list: %v", err) - } - if len(list) != 1 || list[0].Key != key || list[0].LastChapterNum != 10 { - t.Fatalf("GET list wrong: %+v", list) - } - - // PUT again (upsert, progress advance) - in.LastChapter, in.LastChapterNum = "Chapter 11", 11 - body, _ = json.Marshal(in) - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) - if rr.Code != http.StatusOK { - t.Fatalf("second PUT status = %d", rr.Code) - } - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - json.Unmarshal(rr.Body.Bytes(), &list) - if len(list) != 1 || list[0].LastChapterNum != 11 { - t.Fatalf("upsert did not update in place: %+v", list) - } - - // DELETE - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodDelete, "/bookmarks/"+key, nil))) - if rr.Code != http.StatusNoContent { - t.Fatalf("DELETE status = %d, want 204", rr.Code) - } - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - json.Unmarshal(rr.Body.Bytes(), &list) - if len(list) != 0 { - t.Fatalf("after delete list = %+v, want empty", list) - } -} - -// putBookmark PUTs b at key and returns the bookmark the server echoes back, -// which is the row as actually stored (not the request payload). -func putBookmark(t *testing.T, srv http.Handler, key string, b Bookmark) Bookmark { - t.Helper() - body, _ := json.Marshal(b) - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) - if rr.Code != http.StatusOK { - t.Fatalf("PUT %s status = %d, body = %s", key, rr.Code, rr.Body.String()) - } - var out Bookmark - if err := json.Unmarshal(rr.Body.Bytes(), &out); err != nil { - t.Fatalf("decode PUT response: %v", err) - } - return out -} - -func getBookmarks(t *testing.T, srv http.Handler) []Bookmark { - t.Helper() - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - if rr.Code != http.StatusOK { - t.Fatalf("GET status = %d", rr.Code) - } - var list []Bookmark - if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil { - t.Fatalf("decode list: %v", err) - } - return list -} - -func floatPtr(f float64) *float64 { return &f } - -// updated_at drives list ordering, so it must move only on a real progress -// advance — never on a favorite toggle or a latest-chapter capture. -func TestUpsertConditionalUpdatedAt(t *testing.T) { - cases := []struct { - name string - mutate func(Bookmark) Bookmark - wantBumped bool - }{ - { - name: "unchanged progress", - mutate: func(b Bookmark) Bookmark { return b }, - wantBumped: false, - }, - { - name: "changed progress", - mutate: func(b Bookmark) Bookmark { - b.LastChapter, b.LastChapterNum = "Chapter 11", 11 - return b - }, - wantBumped: true, - }, - { - name: "favorite only", - mutate: func(b Bookmark) Bookmark { - b.Favorite = true - return b - }, - wantBumped: false, - }, - { - name: "latest chapter only", - mutate: func(b Bookmark) Bookmark { - b.LatestChapter, b.LatestChapterNum = "Chapter 15", floatPtr(15) - return b - }, - wantBumped: false, - }, - { - name: "unrelated metadata only", - mutate: func(b Bookmark) Bookmark { - b.Title, b.Cover = "Renamed", "https://example.test/new.jpg" - return b - }, - wantBumped: false, - }, - } - - for i, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - srv := newTestServer(t) - key := fmt.Sprintf("asura:cond-%d", i) - - first := putBookmark(t, srv, key, Bookmark{ - Title: "Test", - LastChapter: "Chapter 10", - LastChapterNum: 10, - }) - if first.UpdatedAt == 0 { - t.Fatal("new bookmark did not get updated_at set") - } - - // Guarantee a later wall-clock ms so a real bump is observable. - time.Sleep(2 * time.Millisecond) - - second := putBookmark(t, srv, key, tc.mutate(first)) - if tc.wantBumped && second.UpdatedAt <= first.UpdatedAt { - t.Fatalf("updated_at = %d, want > %d", second.UpdatedAt, first.UpdatedAt) - } - if !tc.wantBumped && second.UpdatedAt != first.UpdatedAt { - t.Fatalf("updated_at = %d, want preserved %d", second.UpdatedAt, first.UpdatedAt) - } - - // The PUT response must match what a subsequent GET reports. - list := getBookmarks(t, srv) - if len(list) != 1 { - t.Fatalf("list = %+v, want 1 item", list) - } - if list[0].UpdatedAt != second.UpdatedAt { - t.Fatalf("GET updated_at = %d, PUT echoed %d", list[0].UpdatedAt, second.UpdatedAt) - } - }) - } -} - -func TestFavoriteRoundTrip(t *testing.T) { - srv := newTestServer(t) - key := "demonic:some-series" - - stored := putBookmark(t, srv, key, Bookmark{Title: "Fav", Favorite: true}) - if !stored.Favorite { - t.Fatalf("PUT response favorite = false, want true") - } - - list := getBookmarks(t, srv) - if len(list) != 1 || !list[0].Favorite { - t.Fatalf("favorite did not round-trip: %+v", list) - } - - // Unfavoriting must persist too (guards against a write that only ever ORs in true). - stored = putBookmark(t, srv, key, Bookmark{Title: "Fav", Favorite: false}) - if stored.Favorite { - t.Fatal("PUT response favorite = true after unfavorite") - } - list = getBookmarks(t, srv) - if len(list) != 1 || list[0].Favorite { - t.Fatalf("unfavorite did not round-trip: %+v", list) - } -} - -func TestLatestChapterNullable(t *testing.T) { - srv := newTestServer(t) - key := "asura:latest-test" - - // Never captured: latest_chapter_num must serialize as JSON null. - body, _ := json.Marshal(Bookmark{Title: "No latest yet"}) - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) - if rr.Code != http.StatusOK { - t.Fatalf("PUT status = %d", rr.Code) - } - if !strings.Contains(rr.Body.String(), `"latest_chapter_num":null`) { - t.Fatalf("want latest_chapter_num null in response, got %s", rr.Body.String()) - } - - list := getBookmarks(t, srv) - if len(list) != 1 || list[0].LatestChapterNum != nil { - t.Fatalf("latest_chapter_num = %v, want nil", list[0].LatestChapterNum) - } - - // Once captured it round-trips as a value. - stored := putBookmark(t, srv, key, Bookmark{ - Title: "No latest yet", - LatestChapter: "Chapter 162", - LatestChapterNum: floatPtr(162), - }) - if stored.LatestChapterNum == nil || *stored.LatestChapterNum != 162 { - t.Fatalf("PUT response latest_chapter_num = %v, want 162", stored.LatestChapterNum) - } - list = getBookmarks(t, srv) - if len(list) != 1 || list[0].LatestChapterNum == nil || *list[0].LatestChapterNum != 162 { - t.Fatalf("latest chapter did not round-trip: %+v", list) - } - if list[0].LatestChapter != "Chapter 162" { - t.Fatalf("latest_chapter = %q, want %q", list[0].LatestChapter, "Chapter 162") - } -} - -// The deployed database predates favorite/latest_chapter*, and CREATE TABLE -// IF NOT EXISTS will not add them — OpenStore must migrate in place. func TestOpenStoreMigratesLegacySchema(t *testing.T) { dbPath := filepath.Join(t.TempDir(), "legacy.db") @@ -415,7 +48,7 @@ func TestOpenStoreMigratesLegacySchema(t *testing.T) { t.Fatalf("close legacy db: %v", err) } - store, err := OpenStore(dbPath) + store, err := Open(dbPath) if err != nil { t.Fatalf("OpenStore on legacy db: %v", err) } @@ -437,7 +70,7 @@ func TestOpenStoreMigratesLegacySchema(t *testing.T) { } // Reopening an already-migrated database must be a no-op, not an error. - store2, err := OpenStore(dbPath) + store2, err := Open(dbPath) if err != nil { t.Fatalf("OpenStore is not idempotent: %v", err) } @@ -516,19 +149,6 @@ func TestBookmarkContinueURL(t *testing.T) { } } -func TestLoadConfigWebPassword(t *testing.T) { - t.Setenv("API_TOKEN", "token-abc") - t.Setenv("WEB_PASSWORD", "hunter2") - if got := loadConfig().WebPassword; got != "hunter2" { - t.Fatalf("WebPassword = %q, want hunter2", got) - } - - t.Setenv("WEB_PASSWORD", "") - if got := loadConfig().WebPassword; got != "" { - t.Fatalf("WebPassword = %q with the variable unset, want empty", got) - } -} - // readLatestCheckedAt reads the column directly. It is deliberately absent from // Bookmark (see Store.Upsert), so tests cannot assert on it any other way. func readLatestCheckedAt(t *testing.T, s *Store, key string) int64 { @@ -672,7 +292,7 @@ func TestMigrateAddsLatestCheckedAt(t *testing.T) { t.Fatalf("close: %v", err) } - s, err := OpenStore(path) + s, err := Open(path) if err != nil { t.Fatalf("OpenStore on pre-existing db: %v", err) } @@ -691,38 +311,6 @@ func TestMigrateAddsLatestCheckedAt(t *testing.T) { } } -// A userscript PUT body has no latest_checked_at field. If the column is ever -// moved into bookmarkColumns, this test catches it: the PUT would reset the -// cooldown and the poller would re-fetch that series on every single tick. -func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) { - dbPath := filepath.Join(t.TempDir(), "test.db") - store, err := OpenStore(dbPath) - if err != nil { - t.Fatalf("OpenStore: %v", err) - } - t.Cleanup(func() { store.Close() }) - srv := newRouter(store, testConfig()) - - seedForCheck(t, store, "asura:x", "https://asurascans.com/comics/x", 777) - - // Exactly what the userscript sends: no latest_checked_at key at all. - body := `{"key":"asura:x","site":"asura","series_id":"x", - "series_url":"https://asurascans.com/comics/x", - "last_chapter":"Chapter 5","last_chapter_num":5}` - req := httptest.NewRequest(http.MethodPut, "/bookmarks/asura:x", strings.NewReader(body)) - req.Header.Set("Authorization", "Bearer "+testToken) - req.Header.Set("Content-Type", "application/json") - rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req) - - if rec.Code != http.StatusOK { - t.Fatalf("PUT status = %d, want 200 (body %s)", rec.Code, rec.Body.String()) - } - if got := readLatestCheckedAt(t, store, "asura:x"); got != 777 { - t.Fatalf("latest_checked_at = %d after client PUT, want 777 preserved", got) - } -} - func TestUpsertDefaultsStatusToReading(t *testing.T) { store := newTestStore(t) stored, err := store.Upsert(Bookmark{ @@ -732,8 +320,8 @@ func TestUpsertDefaultsStatusToReading(t *testing.T) { if err != nil { t.Fatalf("Upsert: %v", err) } - if stored.Status != statusReading { - t.Fatalf("Status = %q, want %q", stored.Status, statusReading) + if stored.Status != StatusReading { + t.Fatalf("Status = %q, want %q", stored.Status, StatusReading) } } @@ -743,7 +331,7 @@ func TestUpsertEmptyStatusPreservesStored(t *testing.T) { store := newTestStore(t) base := Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", - Status: statusArchived, UpdatedAt: time.Now().UnixMilli(), + Status: StatusArchived, UpdatedAt: time.Now().UnixMilli(), } if _, err := store.Upsert(base); err != nil { t.Fatalf("seed: %v", err) @@ -755,8 +343,8 @@ func TestUpsertEmptyStatusPreservesStored(t *testing.T) { if err != nil { t.Fatalf("Upsert: %v", err) } - if stored.Status != statusArchived { - t.Fatalf("Status = %q, want it preserved as %q", stored.Status, statusArchived) + if stored.Status != StatusArchived { + t.Fatalf("Status = %q, want it preserved as %q", stored.Status, StatusArchived) } } @@ -768,7 +356,7 @@ func TestLatestPollRoundTripPreservesArchived(t *testing.T) { store := newTestStore(t) base := Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", - Status: statusArchived, UpdatedAt: time.Now().UnixMilli(), + Status: StatusArchived, UpdatedAt: time.Now().UnixMilli(), } if _, err := store.Upsert(base); err != nil { t.Fatalf("seed: %v", err) @@ -788,8 +376,8 @@ func TestLatestPollRoundTripPreservesArchived(t *testing.T) { if err != nil { t.Fatalf("Upsert: %v", err) } - if stored.Status != statusArchived { - t.Fatalf("Status = %q, want it preserved as %q", stored.Status, statusArchived) + if stored.Status != StatusArchived { + t.Fatalf("Status = %q, want it preserved as %q", stored.Status, StatusArchived) } } @@ -797,19 +385,19 @@ func TestUpsertReplacesStatusWhenGiven(t *testing.T) { store := newTestStore(t) base := Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", - Status: statusArchived, UpdatedAt: time.Now().UnixMilli(), + Status: StatusArchived, UpdatedAt: time.Now().UnixMilli(), } if _, err := store.Upsert(base); err != nil { t.Fatalf("seed: %v", err) } - base.Status = statusReading + base.Status = StatusReading stored, err := store.Upsert(base) if err != nil { t.Fatalf("Upsert: %v", err) } - if stored.Status != statusReading { - t.Fatalf("Status = %q, want %q", stored.Status, statusReading) + if stored.Status != StatusReading { + t.Fatalf("Status = %q, want %q", stored.Status, StatusReading) } } @@ -826,7 +414,7 @@ func TestUpsertStatusChangeKeepsUpdatedAt(t *testing.T) { t.Fatalf("seed: %v", err) } - base.Status = statusArchived + base.Status = StatusArchived base.UpdatedAt = first.UpdatedAt + 60_000 stored, err := store.Upsert(base) if err != nil { @@ -857,7 +445,7 @@ func TestMigrationAddsStatusToLegacyDatabase(t *testing.T) { } db.Close() - store, err := OpenStore(path) + store, err := Open(path) if err != nil { t.Fatalf("OpenStore: %v", err) } @@ -867,8 +455,8 @@ func TestMigrationAddsStatusToLegacyDatabase(t *testing.T) { if err != nil || !ok { t.Fatalf("Get: ok=%v err=%v", ok, err) } - if b.Status != statusReading { - t.Fatalf("Status = %q, want %q", b.Status, statusReading) + if b.Status != StatusReading { + t.Fatalf("Status = %q, want %q", b.Status, StatusReading) } } @@ -877,9 +465,9 @@ func TestMigrationAddsStatusToLegacyDatabase(t *testing.T) { func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) { store := newTestStore(t) for _, tc := range []struct{ key, status string }{ - {"asura:reading", statusReading}, - {"asura:archived", statusArchived}, - {"asura:finished", statusFinished}, + {"asura:reading", StatusReading}, + {"asura:archived", StatusArchived}, + {"asura:finished", StatusFinished}, } { if _, err := store.Upsert(Bookmark{ Key: tc.key, Site: "asura", SeriesID: tc.key, @@ -912,7 +500,7 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) { func TestOpenStoreMigratesAsuraBuildHashKeys(t *testing.T) { dbPath := filepath.Join(t.TempDir(), "hash.db") - store, err := OpenStore(dbPath) + store, err := Open(dbPath) if err != nil { t.Fatalf("open: %v", err) } @@ -938,7 +526,7 @@ func TestOpenStoreMigratesAsuraBuildHashKeys(t *testing.T) { t.Fatalf("close: %v", err) } - reopened, err := OpenStore(dbPath) + reopened, err := Open(dbPath) if err != nil { t.Fatalf("reopen: %v", err) } @@ -977,7 +565,7 @@ func TestOpenStoreMigratesAsuraBuildHashKeys(t *testing.T) { } // Idempotent: a third open changes nothing. - third, err := OpenStore(dbPath) + third, err := Open(dbPath) if err != nil { t.Fatalf("third open: %v", err) } @@ -990,7 +578,7 @@ func TestOpenStoreMigratesAsuraBuildHashKeys(t *testing.T) { func TestOpenStoreMigratesAsuraHashlessCollision(t *testing.T) { dbPath := filepath.Join(t.TempDir(), "collision.db") - store, err := OpenStore(dbPath) + store, err := Open(dbPath) if err != nil { t.Fatalf("open: %v", err) } @@ -1010,7 +598,7 @@ func TestOpenStoreMigratesAsuraHashlessCollision(t *testing.T) { t.Fatalf("close: %v", err) } - reopened, err := OpenStore(dbPath) + reopened, err := Open(dbPath) if err != nil { t.Fatalf("reopen: %v", err) } diff --git a/backend/userscript.go b/backend/internal/userscript/userscript.go similarity index 96% rename from backend/userscript.go rename to backend/internal/userscript/userscript.go index bcb46cd..c714d90 100644 --- a/backend/userscript.go +++ b/backend/internal/userscript/userscript.go @@ -1,4 +1,4 @@ -package main +package userscript import ( "crypto/subtle" @@ -35,7 +35,7 @@ func stampVersion(src []byte, mod time.Time) []byte { // // The file is read per request — that is what lets a bindmounted copy be edited // on the host without a restart. It is ~50 KB and polled about once a day. -func userscriptHandler(token, path string) http.HandlerFunc { +func Handler(token, path string) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { if subtle.ConstantTimeCompare([]byte(r.PathValue("token")), []byte(token)) != 1 { http.NotFound(w, r) diff --git a/backend/userscript_test.go b/backend/internal/userscript/userscript_test.go similarity index 71% rename from backend/userscript_test.go rename to backend/internal/userscript/userscript_test.go index fec5b34..3196342 100644 --- a/backend/userscript_test.go +++ b/backend/internal/userscript/userscript_test.go @@ -1,4 +1,4 @@ -package main +package userscript import ( "net/http" @@ -10,6 +10,8 @@ import ( "time" ) +const testToken = "s3cret-token" + // sampleScript is a stand-in for the real userscript: a metadata block with a // @version line, plus a body that must survive the rewrite untouched. const sampleScript = `// ==UserScript== @@ -35,16 +37,12 @@ func writeScript(t *testing.T, body string) (path, wantVersion string) { return path, "2026.07.28.1642" } -func newUserscriptServer(t *testing.T, path string) http.Handler { - t.Helper() - store, err := OpenStore(filepath.Join(t.TempDir(), "test.db")) - if err != nil { - t.Fatalf("OpenStore: %v", err) - } - t.Cleanup(func() { store.Close() }) - cfg := testConfig() - cfg.UserscriptPath = path - return newRouter(store, cfg) +// newTestMux registers Handler the same way main.go's router does, without +// pulling in the store or the rest of the app. +func newTestMux(token, path string) http.Handler { + mux := http.NewServeMux() + mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", Handler(token, path)) + return mux } func getScript(t *testing.T, srv http.Handler, token string) *httptest.ResponseRecorder { @@ -56,7 +54,7 @@ func getScript(t *testing.T, srv http.Handler, token string) *httptest.ResponseR func TestUserscriptServedWithStampedVersion(t *testing.T) { path, wantVersion := writeScript(t, sampleScript) - rr := getScript(t, newUserscriptServer(t, path), testToken) + rr := getScript(t, newTestMux(testToken, path), testToken) if rr.Code != http.StatusOK { t.Fatalf("status = %d, want 200", rr.Code) @@ -83,10 +81,13 @@ func TestUserscriptServedWithStampedVersion(t *testing.T) { } } +// The empty-token case ("/u//manga-bookmark.user.js") is covered at the +// router level (see backend's guardEmptyUserscriptToken): ServeMux 307s it to +// "/u/manga-bookmark.user.js" before this handler's own token check ever runs. func TestUserscriptWrongTokenIs404(t *testing.T) { path, _ := writeScript(t, sampleScript) - srv := newUserscriptServer(t, path) - for _, tok := range []string{"wrong", "", testToken + "x", testToken[:3]} { + srv := newTestMux(testToken, path) + for _, tok := range []string{"wrong", testToken + "x", testToken[:3]} { if got := getScript(t, srv, tok).Code; got != http.StatusNotFound { t.Errorf("token %q: status = %d, want 404", tok, got) } @@ -94,7 +95,7 @@ func TestUserscriptWrongTokenIs404(t *testing.T) { } func TestUserscriptMissingFileIs404(t *testing.T) { - srv := newUserscriptServer(t, filepath.Join(t.TempDir(), "absent.user.js")) + srv := newTestMux(testToken, filepath.Join(t.TempDir(), "absent.user.js")) if got := getScript(t, srv, testToken).Code; got != http.StatusNotFound { t.Fatalf("status = %d, want 404", got) } @@ -103,7 +104,7 @@ func TestUserscriptMissingFileIs404(t *testing.T) { func TestUserscriptWithoutVersionLineServedUnmodified(t *testing.T) { const noVersion = "// ==UserScript==\n// @name x\n// ==/UserScript==\nconsole.log(1);\n" path, _ := writeScript(t, noVersion) - rr := getScript(t, newUserscriptServer(t, path), testToken) + rr := getScript(t, newTestMux(testToken, path), testToken) if rr.Code != http.StatusOK { t.Fatalf("status = %d, want 200", rr.Code) @@ -112,21 +113,3 @@ func TestUserscriptWithoutVersionLineServedUnmodified(t *testing.T) { t.Fatalf("body = %q, want it unmodified", rr.Body.String()) } } - -// The endpoint must work on a deployment that never set WEB_PASSWORD, since -// the web routes are not registered at all in that case. -func TestUserscriptServedWithWebUIDisabled(t *testing.T) { - path, _ := writeScript(t, sampleScript) - store, err := OpenStore(filepath.Join(t.TempDir(), "nopass.db")) - if err != nil { - t.Fatalf("OpenStore: %v", err) - } - t.Cleanup(func() { store.Close() }) - cfg := testConfig() - cfg.WebPassword = "" - cfg.UserscriptPath = path - - if got := getScript(t, newRouter(store, cfg), testToken).Code; got != http.StatusOK { - t.Fatalf("status = %d, want 200", got) - } -} diff --git a/backend/static/filter.js b/backend/internal/web/static/filter.js similarity index 100% rename from backend/static/filter.js rename to backend/internal/web/static/filter.js diff --git a/backend/static/fonts/dm-sans-var-latin.woff2 b/backend/internal/web/static/fonts/dm-sans-var-latin.woff2 similarity index 100% rename from backend/static/fonts/dm-sans-var-latin.woff2 rename to backend/internal/web/static/fonts/dm-sans-var-latin.woff2 diff --git a/backend/static/fonts/ibm-plex-mono-500-latin.woff2 b/backend/internal/web/static/fonts/ibm-plex-mono-500-latin.woff2 similarity index 100% rename from backend/static/fonts/ibm-plex-mono-500-latin.woff2 rename to backend/internal/web/static/fonts/ibm-plex-mono-500-latin.woff2 diff --git a/backend/static/fonts/ibm-plex-mono-600-latin.woff2 b/backend/internal/web/static/fonts/ibm-plex-mono-600-latin.woff2 similarity index 100% rename from backend/static/fonts/ibm-plex-mono-600-latin.woff2 rename to backend/internal/web/static/fonts/ibm-plex-mono-600-latin.woff2 diff --git a/backend/static/fonts/instrument-serif-400-italic-latin.woff2 b/backend/internal/web/static/fonts/instrument-serif-400-italic-latin.woff2 similarity index 100% rename from backend/static/fonts/instrument-serif-400-italic-latin.woff2 rename to backend/internal/web/static/fonts/instrument-serif-400-italic-latin.woff2 diff --git a/backend/static/fonts/instrument-serif-400-latin.woff2 b/backend/internal/web/static/fonts/instrument-serif-400-latin.woff2 similarity index 100% rename from backend/static/fonts/instrument-serif-400-latin.woff2 rename to backend/internal/web/static/fonts/instrument-serif-400-latin.woff2 diff --git a/backend/static/htmx.min.js b/backend/internal/web/static/htmx.min.js similarity index 100% rename from backend/static/htmx.min.js rename to backend/internal/web/static/htmx.min.js diff --git a/backend/static/logo.svg b/backend/internal/web/static/logo.svg similarity index 100% rename from backend/static/logo.svg rename to backend/internal/web/static/logo.svg diff --git a/backend/static/style.css b/backend/internal/web/static/style.css similarity index 100% rename from backend/static/style.css rename to backend/internal/web/static/style.css diff --git a/backend/templates/app.html b/backend/internal/web/templates/app.html similarity index 100% rename from backend/templates/app.html rename to backend/internal/web/templates/app.html diff --git a/backend/templates/card.html b/backend/internal/web/templates/card.html similarity index 100% rename from backend/templates/card.html rename to backend/internal/web/templates/card.html diff --git a/backend/templates/chrome.html b/backend/internal/web/templates/chrome.html similarity index 100% rename from backend/templates/chrome.html rename to backend/internal/web/templates/chrome.html diff --git a/backend/templates/icons.html b/backend/internal/web/templates/icons.html similarity index 100% rename from backend/templates/icons.html rename to backend/internal/web/templates/icons.html diff --git a/backend/templates/list.html b/backend/internal/web/templates/list.html similarity index 100% rename from backend/templates/list.html rename to backend/internal/web/templates/list.html diff --git a/backend/templates/login.html b/backend/internal/web/templates/login.html similarity index 100% rename from backend/templates/login.html rename to backend/internal/web/templates/login.html diff --git a/backend/web.go b/backend/internal/web/web.go similarity index 78% rename from backend/web.go rename to backend/internal/web/web.go index e30a4f1..6e8b36e 100644 --- a/backend/web.go +++ b/backend/internal/web/web.go @@ -1,4 +1,4 @@ -package main +package web import ( "crypto/subtle" @@ -13,6 +13,9 @@ import ( "strconv" "strings" "time" + + "mangabm/backend/internal/session" + "mangabm/backend/internal/store" ) //go:embed templates @@ -21,25 +24,25 @@ var templateFS embed.FS //go:embed static var staticFS embed.FS -// recentCount is how many series the "Continue reading" strip shows. -const recentCount = 5 +// RecentCount is how many series the "Continue reading" strip shows. +const RecentCount = 5 -// webHandler serves the browser UI: full pages at / and htmx fragments at /ui/. -// It is a separate handler from bookmarkHandler because the two speak different +// Handler serves the browser UI: full pages at / and htmx fragments at /ui/. +// It is a separate handler from api.Handler because the two speak different // representations (HTML versus JSON) to different clients under different auth. -type webHandler struct { - store *Store +type Handler struct { + store *store.Store tmpl *template.Template key []byte password string - limiter *loginLimiter + limiter *session.LoginLimiter } // listView is what every list-rendering template receives. type listView struct { Tab string // "all", "fav", or "new" - Recent []Bookmark - Items []Bookmark + Recent []store.Bookmark + Items []store.Bookmark // NewCount is the badge on the Updated tab: how many series being read // have a chapter out that has not been read. It is counted over the whole // reading set, not the active tab, so the badge does not change meaning as @@ -50,38 +53,28 @@ type listView struct { OOB bool } -// Initial is the monogram the templates show in place of a cover when the -// source site never gave us an og:image. First rune, uppercased; "?" when even -// the title is missing, so the slot is never empty. -func (b Bookmark) Initial() string { - for _, r := range b.Title { - return strings.ToUpper(string(r)) - } - return "?" -} - // loginView is what the login template receives. type loginView struct { Error string } -// newWebHandler parses every template up front so a broken one kills the -// process at startup rather than the first request that touches it. -func newWebHandler(store *Store, cfg Config) (*webHandler, error) { +// New parses every template up front so a broken one kills the process at +// startup rather than the first request that touches it. +func New(s *store.Store, apiToken, webPassword string) (*Handler, error) { tmpl, err := template.ParseFS(templateFS, "templates/*.html") if err != nil { return nil, err } - return &webHandler{ - store: store, + return &Handler{ + store: s, tmpl: tmpl, - key: sessionKey(cfg.Token, cfg.WebPassword), - password: cfg.WebPassword, - limiter: newLoginLimiter(), + key: session.Key(apiToken, webPassword), + password: webPassword, + limiter: session.NewLoginLimiter(), }, nil } -func (h *webHandler) register(mux *http.ServeMux) { +func (h *Handler) Register(mux *http.ServeMux) { mux.HandleFunc("GET /{$}", h.index) mux.HandleFunc("POST /login", h.login) mux.HandleFunc("POST /logout", h.logout) @@ -118,15 +111,15 @@ func staticHandler() http.Handler { } // authed reports whether the request carries a valid session cookie. -func (h *webHandler) authed(r *http.Request) bool { - c, err := r.Cookie(sessionCookieName) - return err == nil && verifySession(h.key, c.Value, time.Now().UnixMilli()) +func (h *Handler) authed(r *http.Request) bool { + c, err := r.Cookie(session.CookieName) + return err == nil && session.Verify(h.key, c.Value, time.Now().UnixMilli()) } // requireSession guards the fragment endpoints. It answers 401 rather than // redirecting, because htmx swaps whatever body it receives into the page and a // redirected login page would be spliced into the card list. -func (h *webHandler) requireSession(next http.HandlerFunc) http.HandlerFunc { +func (h *Handler) requireSession(next http.HandlerFunc) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { if !h.authed(r) { http.Error(w, "unauthorized", http.StatusUnauthorized) @@ -136,7 +129,7 @@ func (h *webHandler) requireSession(next http.HandlerFunc) http.HandlerFunc { } } -func (h *webHandler) render(w http.ResponseWriter, status int, name string, data any) { +func (h *Handler) render(w http.ResponseWriter, status int, name string, data any) { w.Header().Set("Content-Type", "text/html; charset=utf-8") w.WriteHeader(status) if err := h.tmpl.ExecuteTemplate(w, name, data); err != nil { @@ -148,7 +141,7 @@ func (h *webHandler) render(w http.ResponseWriter, status int, name string, data // index renders the list, or the login page when there is no session. The login // page is served at / with status 200 rather than as a redirect to a separate // URL: one page, no redirect loop to reason about. -func (h *webHandler) index(w http.ResponseWriter, r *http.Request) { +func (h *Handler) index(w http.ResponseWriter, r *http.Request) { if !h.authed(r) { h.render(w, http.StatusOK, "login", loginView{}) return @@ -164,8 +157,8 @@ func (h *webHandler) index(w http.ResponseWriter, r *http.Request) { // filterBookmarks returns the subset keep reports true for, preserving order. // It always returns a non-nil slice so an empty tab renders its empty state. -func filterBookmarks(all []Bookmark, keep func(Bookmark) bool) []Bookmark { - out := []Bookmark{} +func filterBookmarks(all []store.Bookmark, keep func(store.Bookmark) bool) []store.Bookmark { + out := []store.Bookmark{} for _, b := range all { if keep(b) { out = append(out, b) @@ -181,25 +174,25 @@ func filterBookmarks(all []Bookmark, keep func(Bookmark) bool) []Bookmark { // in All, not in Updated, not in Favourites, and not in the recent strip. An // archived favourite therefore shows only under Archived: Favourites means // "favourites I am currently reading". -func (h *webHandler) buildListView(tab string) (listView, error) { +func (h *Handler) buildListView(tab string) (listView, error) { all, err := h.store.List() // already ordered updated_at DESC if err != nil { return listView{}, err } - reading := filterBookmarks(all, func(b Bookmark) bool { return b.Status == statusReading }) + reading := filterBookmarks(all, func(b store.Bookmark) bool { return b.Status == store.StatusReading }) - withNew := filterBookmarks(reading, func(b Bookmark) bool { return b.HasNewChapter() }) + withNew := filterBookmarks(reading, func(b store.Bookmark) bool { return b.HasNewChapter() }) - var items []Bookmark + var items []store.Bookmark switch tab { case "fav": - items = filterBookmarks(reading, func(b Bookmark) bool { return b.Favorite }) + items = filterBookmarks(reading, func(b store.Bookmark) bool { return b.Favorite }) case "new": items = withNew case "archived": - items = filterBookmarks(all, func(b Bookmark) bool { return b.Status == statusArchived }) + items = filterBookmarks(all, func(b store.Bookmark) bool { return b.Status == store.StatusArchived }) case "finished": - items = filterBookmarks(all, func(b Bookmark) bool { return b.Status == statusFinished }) + items = filterBookmarks(all, func(b store.Bookmark) bool { return b.Status == store.StatusFinished }) default: tab = "all" items = reading @@ -213,17 +206,17 @@ func (h *webHandler) buildListView(tab string) (listView, error) { // It therefore disappears entirely on a library with nothing new. That is // the intended reading: an empty strip has nothing to say, and the ~240px it // costs on a phone belongs to the list. - var recent []Bookmark + var recent []store.Bookmark if tab == "all" { recent = withNew - if len(recent) > recentCount { - recent = recent[:recentCount] + if len(recent) > RecentCount { + recent = recent[:RecentCount] } } return listView{Tab: tab, Recent: recent, Items: items, NewCount: len(withNew)}, nil } -func (h *webHandler) uiList(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiList(w http.ResponseWriter, r *http.Request) { view, err := h.buildListView(r.URL.Query().Get("tab")) if err != nil { log.Printf("ui list: %v", err) @@ -253,7 +246,7 @@ func currentTab(r *http.Request) string { // mutation cannot leave them describing the library as it was before the tap. // The key is in here because it is tab-shaped too: archived and finished swap // Archive for Restore. -func (h *webHandler) writeChromeOOB(w http.ResponseWriter, view listView) { +func (h *Handler) writeChromeOOB(w http.ResponseWriter, view listView) { view.OOB = true for _, name := range []string{"recent", "newcount", "keyrow"} { if err := h.tmpl.ExecuteTemplate(w, name, view); err != nil { @@ -266,7 +259,7 @@ func (h *webHandler) writeChromeOOB(w http.ResponseWriter, view listView) { // refreshChrome rebuilds the chrome for the reader's current tab after a // mutation and appends it to the response. -func (h *webHandler) refreshChrome(w http.ResponseWriter, r *http.Request) { +func (h *Handler) refreshChrome(w http.ResponseWriter, r *http.Request) { view, err := h.buildListView(currentTab(r)) if err != nil { log.Printf("ui chrome: %v", err) @@ -275,9 +268,9 @@ func (h *webHandler) refreshChrome(w http.ResponseWriter, r *http.Request) { h.writeChromeOOB(w, view) } -func (h *webHandler) login(w http.ResponseWriter, r *http.Request) { - ip := clientIP(r) - if wait := h.limiter.retryAfter(ip, time.Now()); wait > 0 { +func (h *Handler) login(w http.ResponseWriter, r *http.Request) { + ip := session.ClientIP(r) + if wait := h.limiter.RetryAfter(ip, time.Now()); wait > 0 { secs := int(wait.Seconds()) + 1 w.Header().Set("Retry-After", strconv.Itoa(secs)) h.render(w, http.StatusTooManyRequests, "login", loginView{ @@ -293,38 +286,38 @@ func (h *webHandler) login(w http.ResponseWriter, r *http.Request) { } got := r.PostFormValue("password") if subtle.ConstantTimeCompare([]byte(got), []byte(h.password)) != 1 { - h.limiter.fail(ip, time.Now()) + h.limiter.Fail(ip, time.Now()) h.render(w, http.StatusUnauthorized, "login", loginView{Error: "Wrong password."}) return } - h.limiter.reset(ip) - setSessionCookie(w, r, h.key) + h.limiter.Reset(ip) + session.SetCookie(w, r, h.key) http.Redirect(w, r, "/", http.StatusSeeOther) } -func (h *webHandler) logout(w http.ResponseWriter, r *http.Request) { - clearSessionCookie(w, r) +func (h *Handler) logout(w http.ResponseWriter, r *http.Request) { + session.ClearCookie(w, r) http.Redirect(w, r, "/", http.StatusSeeOther) } // loadForMutation fetches the row a mutation targets, writing the error // response itself when there is nothing to mutate. -func (h *webHandler) loadForMutation(w http.ResponseWriter, r *http.Request) (Bookmark, bool) { +func (h *Handler) loadForMutation(w http.ResponseWriter, r *http.Request) (store.Bookmark, bool) { key := r.PathValue("key") if key == "" { http.Error(w, "missing key", http.StatusBadRequest) - return Bookmark{}, false + return store.Bookmark{}, false } b, ok, err := h.store.Get(key) if err != nil { log.Printf("ui get %q: %v", key, err) http.Error(w, "internal error", http.StatusInternalServerError) - return Bookmark{}, false + return store.Bookmark{}, false } if !ok { http.Error(w, "not found", http.StatusNotFound) - return Bookmark{}, false + return store.Bookmark{}, false } return b, true } @@ -338,7 +331,7 @@ func (h *webHandler) loadForMutation(w http.ResponseWriter, r *http.Request) (Bo // state is the feedback for the tap. The strip and the badge are not: they // describe the whole library, so they are rebuilt out of band on every // mutation, at the cost of one extra list read per toggle. -func (h *webHandler) saveAndRenderCard(w http.ResponseWriter, r *http.Request, b Bookmark) { +func (h *Handler) saveAndRenderCard(w http.ResponseWriter, r *http.Request, b store.Bookmark) { stored, err := h.store.Upsert(b) if err != nil { log.Printf("ui upsert %q: %v", b.Key, err) @@ -351,7 +344,7 @@ func (h *webHandler) saveAndRenderCard(w http.ResponseWriter, r *http.Request, b // uiFavorite flips the favourite flag. last_chapter_num is untouched, so // Upsert keeps the stored updated_at and the list does not reorder. -func (h *webHandler) uiFavorite(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiFavorite(w http.ResponseWriter, r *http.Request) { b, ok := h.loadForMutation(w, r) if !ok { return @@ -367,7 +360,7 @@ func (h *webHandler) uiFavorite(w http.ResponseWriter, r *http.Request) { // // last_chapter_num is untouched, so Upsert keeps the stored updated_at and the // list does not reorder. -func (h *webHandler) uiStatus(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiStatus(w http.ResponseWriter, r *http.Request) { b, ok := h.loadForMutation(w, r) if !ok { return @@ -377,7 +370,7 @@ func (h *webHandler) uiStatus(w http.ResponseWriter, r *http.Request) { return } switch s := r.PostFormValue("status"); s { - case statusReading, statusArchived, statusFinished: + case store.StatusReading, store.StatusArchived, store.StatusFinished: b.Status = s default: http.Error(w, "invalid status", http.StatusBadRequest) @@ -398,7 +391,7 @@ func (h *webHandler) uiStatus(w http.ResponseWriter, r *http.Request) { // pre-filled, so a bare tap of Save is an easy accidental submit; it must not // destroy last_chapter_url, nor rewrite the last_chapter display string ("45.0" // to "45") behind a frozen updated_at. -func (h *webHandler) uiChapter(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiChapter(w http.ResponseWriter, r *http.Request) { b, ok := h.loadForMutation(w, r) if !ok { return @@ -425,7 +418,7 @@ func (h *webHandler) uiChapter(w http.ResponseWriter, r *http.Request) { // uiDelete removes the row and answers with an empty body, which htmx swaps in // place of the card — removing it from the page. -func (h *webHandler) uiDelete(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiDelete(w http.ResponseWriter, r *http.Request) { key := r.PathValue("key") if key == "" { http.Error(w, "missing key", http.StatusBadRequest) diff --git a/backend/main.go b/backend/main.go index aa8f70d..75db207 100644 --- a/backend/main.go +++ b/backend/main.go @@ -11,6 +11,13 @@ import ( "strings" "syscall" "time" + + "mangabm/backend/internal/api" + "mangabm/backend/internal/httpmw" + "mangabm/backend/internal/latest" + "mangabm/backend/internal/store" + "mangabm/backend/internal/userscript" + "mangabm/backend/internal/web" ) // Config holds all runtime settings, sourced from environment variables. @@ -149,22 +156,22 @@ func loadConfig() Config { // newRouter wires routes and middleware. CORS is the outermost layer so // preflight OPTIONS short-circuits before auth; /bookmarks* is auth-protected, // /healthz is public. -func newRouter(store *Store, cfg Config) http.Handler { +func newRouter(s *store.Store, cfg Config) http.Handler { mux := http.NewServeMux() - mux.HandleFunc("GET /healthz", healthz) + mux.HandleFunc("GET /healthz", api.Healthz) - // Outside withAuth (the updater sends no Authorization header) and outside - // the WEB_PASSWORD gate (the script must be installable either way). The - // path segment carries the token instead. - mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", userscriptHandler(cfg.Token, cfg.UserscriptPath)) + // Outside httpmw.Auth (the updater sends no Authorization header) and + // outside the WEB_PASSWORD gate (the script must be installable either + // way). The path segment carries the token instead. + mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", userscript.Handler(cfg.Token, cfg.UserscriptPath)) - h := &bookmarkHandler{store: store} + h := &api.Handler{Store: s} protected := http.NewServeMux() - protected.HandleFunc("GET /bookmarks", h.list) - protected.HandleFunc("PUT /bookmarks/{key}", h.put) - protected.HandleFunc("DELETE /bookmarks/{key}", h.delete) + protected.HandleFunc("GET /bookmarks", h.List) + protected.HandleFunc("PUT /bookmarks/{key}", h.Put) + protected.HandleFunc("DELETE /bookmarks/{key}", h.Delete) - auth := withAuth(cfg.Token, protected) + auth := httpmw.Auth(cfg.Token, protected) mux.Handle("/bookmarks", auth) mux.Handle("/bookmarks/", auth) @@ -172,14 +179,14 @@ func newRouter(store *Store, cfg Config) http.Handler { // deployment that forgets WEB_PASSWORD exposes nothing rather than // exposing an unprotected list. if cfg.WebPassword != "" { - web, err := newWebHandler(store, cfg) + wh, err := web.New(s, cfg.Token, cfg.WebPassword) if err != nil { log.Fatalf("web handler: %v", err) } - web.register(mux) + wh.Register(mux) } - return withCORS(cfg.AllowedOrigins, withGzip(guardEmptyUserscriptToken(mux))) + return httpmw.CORS(cfg.AllowedOrigins, httpmw.Gzip(guardEmptyUserscriptToken(mux))) } // guardEmptyUserscriptToken heads off ServeMux's own path-cleaning redirect: @@ -203,22 +210,22 @@ func main() { log.Fatal("API_TOKEN is required") } - store, err := OpenStore(cfg.DBPath) + s, err := store.Open(cfg.DBPath) if err != nil { log.Fatalf("open store: %v", err) } - defer store.Close() + defer s.Close() // The poller is off the request path entirely: if it cannot start, the // service still serves bookmarks and the userscript still captures latest // chapters on its own. pollCtx, stopPoll := context.WithCancel(context.Background()) defer stopPoll() - startLatestPoller(pollCtx, store, cfg.LatestPoll) + startLatestPoller(pollCtx, s, cfg.LatestPoll) srv := &http.Server{ Addr: ":" + cfg.Port, - Handler: newRouter(store, cfg), + Handler: newRouter(s, cfg), ReadHeaderTimeout: 10 * time.Second, } @@ -248,24 +255,24 @@ func main() { // HTTP client cannot be built. Any problem here is logged and skipped: this // feature going missing degrades the service to userscript-only latest-chapter // tracking, which is exactly how it behaved before. -func startLatestPoller(ctx context.Context, store *Store, cfg LatestPoll) { +func startLatestPoller(ctx context.Context, s *store.Store, cfg LatestPoll) { if !cfg.Enabled { log.Println("latest-chapter poller: disabled by config") return } - f, err := newTLSFetcher() + f, err := latest.NewTLSFetcher() if err != nil { log.Printf("latest-chapter poller: disabled, cannot build client: %v", err) return } - p := &latestPoller{ - store: store, - fetch: f, - now: time.Now, - cooldown: cfg.Cooldown, - interval: cfg.Interval, - stagger: cfg.Stagger, - batch: cfg.Batch, + p := &latest.Poller{ + Store: s, + Fetch: f, + Now: time.Now, + Cooldown: cfg.Cooldown, + Interval: cfg.Interval, + Stagger: cfg.Stagger, + Batch: cfg.Batch, } go p.Run(ctx) } diff --git a/backend/main_test.go b/backend/main_test.go index 1d89df6..66eaa29 100644 --- a/backend/main_test.go +++ b/backend/main_test.go @@ -10,6 +10,8 @@ import ( "strings" "testing" "time" + + "mangabm/backend/internal/store" ) func TestLoadLatestPollDefaults(t *testing.T) { @@ -148,7 +150,7 @@ func TestPutStatusValidation(t *testing.T) { if tc.want != http.StatusOK { return } - var got Bookmark + var got store.Bookmark if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil { t.Fatalf("decode: %v", err) } @@ -187,7 +189,7 @@ func TestPutOmittedStatusPreservesArchivedAndAppliesProgress(t *testing.T) { t.Fatalf("status = %d, want 200 (body %s)", rr.Code, rr.Body.String()) } - var got Bookmark + var got store.Bookmark if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil { t.Fatalf("decode: %v", err) } diff --git a/backend/web_test.go b/backend/web_test.go index 44f90fa..4a9a048 100644 --- a/backend/web_test.go +++ b/backend/web_test.go @@ -10,6 +10,10 @@ import ( "strings" "testing" "time" + + "mangabm/backend/internal/session" + "mangabm/backend/internal/store" + "mangabm/backend/internal/web" ) const testPassword = "hunter2" @@ -22,22 +26,22 @@ func webConfig() Config { // newWebTestServer returns the full router plus the store behind it, so tests // can seed rows and assert on what the handlers wrote back. -func newWebTestServer(t *testing.T, cfg Config) (http.Handler, *Store) { +func newWebTestServer(t *testing.T, cfg Config) (http.Handler, *store.Store) { t.Helper() - store, err := OpenStore(filepath.Join(t.TempDir(), "test.db")) + st, err := store.Open(filepath.Join(t.TempDir(), "test.db")) if err != nil { - t.Fatalf("OpenStore: %v", err) + t.Fatalf("store.Open: %v", err) } - t.Cleanup(func() { store.Close() }) - return newRouter(store, cfg), store + t.Cleanup(func() { st.Close() }) + return newRouter(st, cfg), st } // sessionCookie returns a cookie a handler will accept for cfg's API token. func sessionCookie(t *testing.T, cfg Config) *http.Cookie { t.Helper() return &http.Cookie{ - Name: sessionCookieName, - Value: signSession(sessionKey(cfg.Token, cfg.WebPassword), time.Now().Add(time.Hour).UnixMilli()), + Name: session.CookieName, + Value: session.Sign(session.Key(cfg.Token, cfg.WebPassword), time.Now().Add(time.Hour).UnixMilli()), } } @@ -56,8 +60,8 @@ func TestIndexWithoutSessionShowsLogin(t *testing.T) { func TestIndexWithSessionShowsList(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - if _, err := store.Upsert(Bookmark{ + srv, st := newWebTestServer(t, cfg) + if _, err := st.Upsert(store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45", LastChapterNum: 45, UpdatedAt: time.Now().UnixMilli(), @@ -90,8 +94,8 @@ func TestLoginSuccessSetsCookie(t *testing.T) { t.Fatalf("POST /login status = %d, want 303", rr.Code) } cookies := rr.Result().Cookies() - if len(cookies) != 1 || cookies[0].Name != sessionCookieName || cookies[0].Value == "" { - t.Fatalf("POST /login cookies = %+v, want one non-empty %s", cookies, sessionCookieName) + if len(cookies) != 1 || cookies[0].Name != session.CookieName || cookies[0].Value == "" { + t.Fatalf("POST /login cookies = %+v, want one non-empty %s", cookies, session.CookieName) } } @@ -122,14 +126,14 @@ func TestLoginRateLimited(t *testing.T) { srv.ServeHTTP(rr, req) return rr } - for i := 0; i < loginMaxFailures; i++ { + for i := 0; i < session.MaxFailures; i++ { if code := post().Code; code != http.StatusUnauthorized { t.Fatalf("attempt %d status = %d, want 401", i+1, code) } } rr := post() if rr.Code != http.StatusTooManyRequests { - t.Fatalf("attempt %d status = %d, want 429", loginMaxFailures+1, rr.Code) + t.Fatalf("attempt %d status = %d, want 429", session.MaxFailures+1, rr.Code) } if after := rr.Header().Get("Retry-After"); after == "" { t.Fatal("429 response has no Retry-After header") @@ -202,9 +206,9 @@ func TestStaticAssetsServed(t *testing.T) { } // seed inserts one bookmark and returns it as stored. -func seed(t *testing.T, store *Store, b Bookmark) Bookmark { +func seed(t *testing.T, st *store.Store, b store.Bookmark) store.Bookmark { t.Helper() - stored, err := store.Upsert(b) + stored, err := st.Upsert(b) if err != nil { t.Fatalf("Upsert: %v", err) } @@ -245,8 +249,8 @@ func TestUIRoutesRequireSession(t *testing.T) { func TestFavoriteTogglesWithoutReordering(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - before := seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + before := seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45", LastChapterNum: 45, UpdatedAt: 1_000_000, @@ -258,7 +262,7 @@ func TestFavoriteTogglesWithoutReordering(t *testing.T) { t.Fatalf("favorite status = %d, want 200", rr.Code) } - after, ok, err := store.Get("asura:solo") + after, ok, err := st.Get("asura:solo") if err != nil || !ok { t.Fatalf("Get after favorite: %v ok=%v", err, ok) } @@ -276,7 +280,7 @@ func TestFavoriteTogglesWithoutReordering(t *testing.T) { // Toggling again turns it back off. rr = httptest.NewRecorder() srv.ServeHTTP(rr, uiRequest(t, cfg, http.MethodPost, "/ui/bookmarks/asura:solo/favorite", nil)) - back, _, _ := store.Get("asura:solo") + back, _, _ := st.Get("asura:solo") if back.Favorite { t.Fatal("Favorite = true after a second toggle, want false") } @@ -294,8 +298,8 @@ func TestFavoriteTogglesWithoutReordering(t *testing.T) { // selector, just a regression guard against reintroducing the bare-id form. func TestCardHxTargetIsValidSelectorForColonKey(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45", LastChapterNum: 45, UpdatedAt: 1_000_000, @@ -320,8 +324,8 @@ func TestCardHxTargetIsValidSelectorForColonKey(t *testing.T) { func TestChapterOverrideMovesUpdatedAt(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - before := seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + before := seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45", LastChapterNum: 45, LastChapterURL: "https://example.test/ch/45", SeriesURL: "https://example.test/solo", @@ -335,7 +339,7 @@ func TestChapterOverrideMovesUpdatedAt(t *testing.T) { t.Fatalf("chapter override status = %d, want 200", rr.Code) } - after, ok, err := store.Get("asura:solo") + after, ok, err := st.Get("asura:solo") if err != nil || !ok { t.Fatalf("Get after override: %v ok=%v", err, ok) } @@ -355,8 +359,8 @@ func TestChapterOverrideMovesUpdatedAt(t *testing.T) { func TestChapterOverrideNoOpPreservesURLAndUpdatedAt(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - before := seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + before := seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45.0", LastChapterNum: 45, LastChapterURL: "https://example.test/ch/45", SeriesURL: "https://example.test/solo", @@ -375,7 +379,7 @@ func TestChapterOverrideNoOpPreservesURLAndUpdatedAt(t *testing.T) { t.Fatalf("chapter no-op status = %d, want 200", rr.Code) } - after, ok, err := store.Get("asura:solo") + after, ok, err := st.Get("asura:solo") if err != nil || !ok { t.Fatalf("Get after no-op override: %v ok=%v", err, ok) } @@ -395,8 +399,8 @@ func TestChapterOverrideNoOpPreservesURLAndUpdatedAt(t *testing.T) { func TestChapterOverrideRejectsBadInput(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapterNum: 45, UpdatedAt: 1_000_000, }) @@ -409,7 +413,7 @@ func TestChapterOverrideRejectsBadInput(t *testing.T) { if rr.Code != http.StatusBadRequest { t.Fatalf("status = %d, want 400", rr.Code) } - after, _, _ := store.Get("asura:solo") + after, _, _ := st.Get("asura:solo") if after.LastChapterNum != 45 { t.Fatalf("chapter changed to %v on invalid input", after.LastChapterNum) } @@ -440,8 +444,8 @@ func TestMutationsOnMissingKey(t *testing.T) { func TestUIDeleteRemovesRow(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", UpdatedAt: 1_000_000, }) @@ -460,19 +464,19 @@ func TestUIDeleteRemovesRow(t *testing.T) { if !strings.Contains(body, `id="new-count" hx-swap-oob="true"`) { t.Fatalf("delete body = %q, want the out-of-band badge", body) } - if _, ok, _ := store.Get("asura:solo"); ok { + if _, ok, _ := st.Get("asura:solo"); ok { t.Fatal("row still present after delete") } } func TestUIListFavouritesTab(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", Favorite: true, UpdatedAt: 2_000_000, }) - seed(t, store, Bookmark{ + seed(t, st, store.Bookmark{ Key: "demonic:tower", Site: "demonic", SeriesID: "tower", Title: "Tower of God", Favorite: false, UpdatedAt: 1_000_000, }) @@ -493,14 +497,14 @@ func TestUIListFavouritesTab(t *testing.T) { func TestUIListNewTab(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapterNum: 10, LatestChapter: "Chapter 12", LatestChapterNum: floatPtr(12), UpdatedAt: 2_000_000, }) - seed(t, store, Bookmark{ + seed(t, st, store.Bookmark{ Key: "demonic:tower", Site: "demonic", SeriesID: "tower", Title: "Tower of God", LastChapterNum: 5, LatestChapter: "Chapter 5", LatestChapterNum: floatPtr(5), @@ -524,22 +528,22 @@ func TestUIListNewTab(t *testing.T) { // seedStatusRows puts one series in each bucket, the archived one also // favourited and with a new chapter out, so a leak into any reading-bucket tab // shows up as a failure rather than passing by accident. -func seedStatusRows(t *testing.T, store *Store) { +func seedStatusRows(t *testing.T, st *store.Store) { t.Helper() // floatPtr already exists in store_test.go — same package, reuse it. - rows := []Bookmark{ + rows := []store.Bookmark{ {Key: "asura:reading", Site: "asura", SeriesID: "reading", Title: "ReadingOne", - Status: statusReading, LastChapterNum: 10, Favorite: true, + Status: store.StatusReading, LastChapterNum: 10, Favorite: true, LatestChapter: "11", LatestChapterNum: floatPtr(11)}, {Key: "asura:archived", Site: "asura", SeriesID: "archived", Title: "ArchivedOne", - Status: statusArchived, LastChapterNum: 5, Favorite: true, + Status: store.StatusArchived, LastChapterNum: 5, Favorite: true, LatestChapter: "99", LatestChapterNum: floatPtr(99)}, {Key: "asura:finished", Site: "asura", SeriesID: "finished", Title: "FinishedOne", - Status: statusFinished, LastChapterNum: 200, Favorite: true}, + Status: store.StatusFinished, LastChapterNum: 200, Favorite: true}, } for _, b := range rows { b.UpdatedAt = time.Now().UnixMilli() - if _, err := store.Upsert(b); err != nil { + if _, err := st.Upsert(b); err != nil { t.Fatalf("seed %s: %v", b.Key, err) } } @@ -547,8 +551,8 @@ func seedStatusRows(t *testing.T, store *Store) { func TestTabsShowOnlyTheirBucket(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) cases := []struct { tab string @@ -604,16 +608,16 @@ func stripOf(t *testing.T, srv http.Handler, cfg Config, tab string) string { // updated_at-ordered list below it does not already say — and only on All. func TestRecentStripCarriesUnreadOnlyAndOnlyOnAll(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) // ReadingOne is at 10 with 11 out; the rest are not reading + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) // ReadingOne is at 10 with 11 out; the rest are not reading // A reading series that is caught up has nothing waiting, so it stays out. - caught := Bookmark{ + caught := store.Bookmark{ Key: "asura:caught", Site: "asura", SeriesID: "caught", Title: "CaughtUpOne", - Status: statusReading, LastChapterNum: 40, LatestChapter: "40", + Status: store.StatusReading, LastChapterNum: 40, LatestChapter: "40", LatestChapterNum: floatPtr(40), UpdatedAt: time.Now().UnixMilli(), } - if _, err := store.Upsert(caught); err != nil { + if _, err := st.Upsert(caught); err != nil { t.Fatalf("seed %s: %v", caught.Key, err) } @@ -633,12 +637,12 @@ func TestRecentStripCarriesUnreadOnlyAndOnlyOnAll(t *testing.T) { } // Nothing new anywhere: the strip has nothing to say and does not render. - reading, _, err := store.Get("asura:reading") + reading, _, err := st.Get("asura:reading") if err != nil { t.Fatalf("Get: %v", err) } reading.LatestChapterNum = floatPtr(reading.LastChapterNum) - if _, err := store.Upsert(reading); err != nil { + if _, err := st.Upsert(reading); err != nil { t.Fatalf("Upsert: %v", err) } // The section still ships (an out-of-band swap needs the id to exist) but @@ -652,23 +656,23 @@ func TestRecentStripCarriesUnreadOnlyAndOnlyOnAll(t *testing.T) { } } -// The strip never grows past recentCount, however many series are waiting. +// The strip never grows past web.RecentCount, however many series are waiting. func TestRecentStripCapped(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - for i := 0; i <= recentCount; i++ { - b := Bookmark{ + srv, st := newWebTestServer(t, cfg) + for i := 0; i <= web.RecentCount; i++ { + b := store.Bookmark{ Key: fmt.Sprintf("asura:new%d", i), Site: "asura", SeriesID: fmt.Sprintf("new%d", i), Title: fmt.Sprintf("Waiting%d", i), - Status: statusReading, LastChapterNum: 1, LatestChapter: "2", + Status: store.StatusReading, LastChapterNum: 1, LatestChapter: "2", LatestChapterNum: floatPtr(2), UpdatedAt: time.Now().UnixMilli() + int64(i), } - if _, err := store.Upsert(b); err != nil { + if _, err := st.Upsert(b); err != nil { t.Fatalf("seed %s: %v", b.Key, err) } } - if got := strings.Count(stripOf(t, srv, cfg, "all"), "recent-card"); got != recentCount { - t.Fatalf("strip rendered %d cards, want %d", got, recentCount) + if got := strings.Count(stripOf(t, srv, cfg, "all"), "recent-card"); got != web.RecentCount { + t.Fatalf("strip rendered %d cards, want %d", got, web.RecentCount) } } @@ -686,14 +690,14 @@ func postStatus(t *testing.T, srv http.Handler, cfg Config, key, status string) func TestUIStatusSetsBucket(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) - for _, want := range []string{statusArchived, statusFinished, statusReading} { + for _, want := range []string{store.StatusArchived, store.StatusFinished, store.StatusReading} { if rr := postStatus(t, srv, cfg, "asura:reading", want); rr.Code != http.StatusOK { t.Fatalf("set %s: status = %d, body %s", want, rr.Code, rr.Body.String()) } - b, ok, err := store.Get("asura:reading") + b, ok, err := st.Get("asura:reading") if err != nil || !ok { t.Fatalf("Get: ok=%v err=%v", ok, err) } @@ -705,21 +709,21 @@ func TestUIStatusSetsBucket(t *testing.T) { func TestUIStatusRejectsUnknownValue(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) if rr := postStatus(t, srv, cfg, "asura:reading", "dropped"); rr.Code != http.StatusBadRequest { t.Fatalf("status = %d, want 400", rr.Code) } - b, _, _ := store.Get("asura:reading") - if b.Status != statusReading { + b, _, _ := st.Get("asura:reading") + if b.Status != store.StatusReading { t.Fatalf("stored status = %q, want it untouched", b.Status) } } func TestUIStatusRequiresSession(t *testing.T) { - srv, store := newWebTestServer(t, webConfig()) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, webConfig()) + seedStatusRows(t, st) req := httptest.NewRequest(http.MethodPost, "/ui/bookmarks/asura:reading/status", strings.NewReader("status=archived")) @@ -734,15 +738,15 @@ func TestUIStatusRequiresSession(t *testing.T) { func TestUIStatusDoesNotReorderList(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) - before, _, _ := store.Get("asura:reading") + before, _, _ := st.Get("asura:reading") time.Sleep(2 * time.Millisecond) - if rr := postStatus(t, srv, cfg, "asura:reading", statusArchived); rr.Code != http.StatusOK { + if rr := postStatus(t, srv, cfg, "asura:reading", store.StatusArchived); rr.Code != http.StatusOK { t.Fatalf("status = %d", rr.Code) } - after, _, _ := store.Get("asura:reading") + after, _, _ := st.Get("asura:reading") if after.UpdatedAt != before.UpdatedAt { t.Fatalf("UpdatedAt moved %d -> %d", before.UpdatedAt, after.UpdatedAt) } @@ -750,8 +754,8 @@ func TestUIStatusDoesNotReorderList(t *testing.T) { func TestCardShowsStatusControls(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) cases := []struct { tab string @@ -788,8 +792,8 @@ func TestCardShowsStatusControls(t *testing.T) { func TestAppRendersNewTabs(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) req := httptest.NewRequest(http.MethodGet, "/", nil) req.AddCookie(sessionCookie(t, cfg)) @@ -807,10 +811,10 @@ func TestAppRendersNewTabs(t *testing.T) { // outside the swapped card, so nothing else would correct them. func TestMutationRefreshesChromeOutOfBand(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", - Status: statusReading, LastChapterNum: 10, LatestChapter: "Chapter 11", + Status: store.StatusReading, LastChapterNum: 10, LatestChapter: "Chapter 11", LatestChapterNum: floatPtr(11), UpdatedAt: time.Now().UnixMilli(), }) @@ -820,7 +824,7 @@ func TestMutationRefreshesChromeOutOfBand(t *testing.T) { } req := uiRequest(t, cfg, http.MethodPost, "/ui/bookmarks/asura:solo/status", - url.Values{"status": {statusArchived}}) + url.Values{"status": {store.StatusArchived}}) req.Header.Set("HX-Current-URL", "http://localhost/?tab=all") rr := httptest.NewRecorder() srv.ServeHTTP(rr, req) From e392ec3de0e9306e73e0b1ca495cf8edbc3f9abf Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Mon, 3 Aug 2026 16:57:14 +0700 Subject: [PATCH 03/18] feat(userscript): add comix.to site adapter --- userscript/manga-bookmark.user.js | 88 ++++++++++++++++++++++++++++++- userscript/test/logic.test.js | 69 ++++++++++++++++++++++++ 2 files changed, 155 insertions(+), 2 deletions(-) diff --git a/userscript/manga-bookmark.user.js b/userscript/manga-bookmark.user.js index 9392de1..9f9b4a1 100644 --- a/userscript/manga-bookmark.user.js +++ b/userscript/manga-bookmark.user.js @@ -80,6 +80,14 @@ return slug.replace(/-[0-9a-f]{8}$/, ""); } + // comix path segments are "-", where the slug is a rendering of the + // current title and changes when a series is renamed. Only the id is the + // identity; seriesUrl keeps the full segment because navigation needs it. + function comixSeriesId(segment) { + const i = segment.indexOf("-"); + return i === -1 ? segment : segment.slice(0, i); + } + const asura = { site: "asura", // asuracomic.net deep links 301 to the asurascans.com *root*, dropping the @@ -208,7 +216,83 @@ }, }; - const ADAPTERS = [asura, demonic]; + const comix = { + site: "comix", + matches: (loc) => /(^|\.)comix\.to$/.test(loc.hostname), + detect(loc) { + const path = loc.pathname; + // /title/-/-chapter-. Several uploads (different + // groups or languages) share one chapter number; the number is the + // progress identity, the upload id is not. + let m = path.match(/^\/title\/([^/]+)\/[^/]*-chapter-([\d.]+)/); + if (m) { + const num = parseFloat(m[2]); + return { + type: "chapter", + site: this.site, + seriesId: comixSeriesId(m[1]), + title: cleanTitle(meta("og:title")), + cover: coverFromPage(), + seriesUrl: loc.origin + "/title/" + m[1], + chapterLabel: "Chapter " + m[2], + chapterNum: isNaN(num) ? null : num, + chapterUrl: loc.href, + }; + } + // /title/- + m = path.match(/^\/title\/([^/?#]+)\/?$/); + if (m) { + return { + type: "series", + site: this.site, + seriesId: comixSeriesId(m[1]), + title: cleanTitle(meta("og:title")), + cover: coverFromPage(), + seriesUrl: loc.origin + "/title/" + m[1], + chapterLabel: null, + chapterNum: null, + chapterUrl: null, + }; + } + return { type: "other" }; + + // comix chapter og:title is " · Ch."; series is clean. + function cleanTitle(t) { + if (!t) return ""; + return t.replace(/\s*·\s*Ch\.[\d.]+\s*$/i, "").trim(); + } + + // comix serves no og:image, so this is the one adapter that has to read + // the DOM for a cover. Matching on alt rather than a class keeps it off + // the site's styling: the cover is the image whose alt is the title. + // Do not "simplify" this into meta("og:image") — that returns null. + function coverFromPage() { + const title = cleanTitle(meta("og:title")); + if (!title || !document.querySelectorAll) return ""; + for (const img of document.querySelectorAll("img[alt]")) { + if (img.getAttribute("alt") === title) return img.getAttribute("src") || ""; + } + return ""; + } + }, + // Scoped to this series' own id prefix so a recommendation strip's links + // cannot win the maximum. seriesId is passed in because the anchors alone + // do not say which series the page belongs to. + latestChapterFromAnchors(anchors, seriesId) { + let best = null; + const re = new RegExp("^/title/" + seriesId + "-[^/]*/[^/]*-chapter-([\\d.]+)"); + for (const a of anchors) { + const m = a.href.match(re); + if (!m) continue; + const num = parseFloat(m[1]); + if (isNaN(num)) continue; + if (!best || num > best.num) best = { num, label: "Chapter " + m[1] }; + } + return best; + }, + }; + + const ADAPTERS = [asura, demonic, comix]; function detect() { const loc = window.location; @@ -1608,7 +1692,7 @@ // Exposes pure logic only — see userscript/test/logic.test.js. // ============================================================ if (typeof window === "undefined" && typeof module === "object" && module.exports) { - module.exports = { stripBuildHash, asura, demonic, anchorsFromHTML, statusOf }; + module.exports = { stripBuildHash, comixSeriesId, asura, demonic, comix, anchorsFromHTML, statusOf }; } // ============================================================ diff --git a/userscript/test/logic.test.js b/userscript/test/logic.test.js index 00990cf..3f79812 100644 --- a/userscript/test/logic.test.js +++ b/userscript/test/logic.test.js @@ -44,8 +44,10 @@ globalThis.document = { const { stripBuildHash, + comixSeriesId, asura, demonic, + comix, anchorsFromHTML, statusOf, } = require("../manga-bookmark.user.js"); @@ -166,6 +168,73 @@ test("demonic.latestChapterFromAnchors parses chaptered.php links, including &am assert.deepEqual(best, { num: 12, label: "Chapter 12" }); }); +// ============================================================ +// comix — the id prefix is the stable identity; the slug tail is a title +// rendering that changes when a series is renamed. +// ============================================================ + +test("comixSeriesId keeps only the prefix before the first dash", () => { + assert.equal(comixSeriesId("n8we-dungeons-and-crayons"), "n8we"); + assert.equal(comixSeriesId("20xzd-the-baddest-villainess-is-back"), "20xzd"); +}); + +test("comixSeriesId leaves a bare id untouched", () => { + assert.equal(comixSeriesId("n8we"), "n8we"); +}); + +test("comix detects a series page", () => { + metaTags = { "og:title": "Dungeons and Crayons" }; + const p = comix.detect(loc("https://comix.to/title/n8we-dungeons-and-crayons")); + assert.equal(p.type, "series"); + assert.equal(p.site, "comix"); + assert.equal(p.seriesId, "n8we"); + assert.equal(p.title, "Dungeons and Crayons"); + assert.equal(p.seriesUrl, "https://comix.to/title/n8we-dungeons-and-crayons"); + assert.equal(p.chapterNum, null); +}); + +test("comix detects a chapter page and strips the Ch. suffix from the title", () => { + metaTags = { "og:title": "Dungeons and Crayons · Ch.80" }; + const p = comix.detect( + loc("https://comix.to/title/n8we-dungeons-and-crayons/11139891-chapter-80") + ); + assert.equal(p.type, "chapter"); + assert.equal(p.seriesId, "n8we"); + assert.equal(p.title, "Dungeons and Crayons"); + assert.equal(p.chapterNum, 80); + assert.equal(p.chapterLabel, "Chapter 80"); + assert.equal(p.seriesUrl, "https://comix.to/title/n8we-dungeons-and-crayons"); +}); + +test("comix parses decimal chapter numbers", () => { + metaTags = { "og:title": "Dungeons and Crayons · Ch.80.5" }; + const p = comix.detect( + loc("https://comix.to/title/n8we-dungeons-and-crayons/11139891-chapter-80.5") + ); + assert.equal(p.chapterNum, 80.5); +}); + +test("comix ignores unrelated paths", () => { + assert.equal(comix.detect(loc("https://comix.to/browse")).type, "other"); +}); + +test("comix takes the max chapter and ignores other series' links", () => { + const anchors = anchorsFromHTML(` + Chapter 79 + Chapter 80 + Chapter 78 + Chapter 999 + `); + assert.deepEqual(comix.latestChapterFromAnchors(anchors, "n8we"), { + num: 80, + label: "Chapter 80", + }); +}); + +test("comix latest returns null when no chapter links are present", () => { + assert.equal(comix.latestChapterFromAnchors(anchorsFromHTML("x"), "n8we"), null); +}); + // ============================================================ // Shared helpers // ============================================================ From 78eaecb1198768522550860693587d715e64ed23 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Mon, 3 Aug 2026 17:00:54 +0700 Subject: [PATCH 04/18] Add test coverage for comix coverFromPage() Extend the test harness's document stub with a querySelectorAll("img[alt]") fake (module-level pageImages fixture, mirroring metaTags), then assert on p.cover for a matching alt and for no match. Previously the guard in coverFromPage() always short-circuited under test since querySelectorAll didn't exist on the stub, so the alt-matching loop had zero coverage. --- userscript/test/logic.test.js | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/userscript/test/logic.test.js b/userscript/test/logic.test.js index 3f79812..2271956 100644 --- a/userscript/test/logic.test.js +++ b/userscript/test/logic.test.js @@ -31,6 +31,9 @@ globalThis.location = { // og: meta tags the adapters read through meta(). Reassigned per test. let metaTags = {}; +// img[alt] elements comix's coverFromPage() scans. Reassigned per test; each +// entry is {alt, src}. +let pageImages = []; globalThis.document = { querySelector(sel) { const m = sel.match(/^meta\[property="([^"]+)"\]$/); @@ -38,6 +41,12 @@ globalThis.document = { const v = metaTags[m[1]]; return v == null ? null : { getAttribute: () => v }; }, + querySelectorAll(sel) { + if (sel !== "img[alt]") return []; + return pageImages.map((img) => ({ + getAttribute: (attr) => img[attr] ?? null, + })); + }, addEventListener() {}, body: undefined, }; @@ -214,6 +223,26 @@ test("comix parses decimal chapter numbers", () => { assert.equal(p.chapterNum, 80.5); }); +test("comix.detect reads the cover from an img whose alt matches the cleaned title", () => { + metaTags = { "og:title": "Dungeons and Crayons · Ch.80" }; + pageImages = [ + { alt: "Some Other Series", src: "https://cdn.example/other.jpg" }, + { alt: "Dungeons and Crayons", src: "https://cdn.example/cover.jpg" }, + ]; + const p = comix.detect( + loc("https://comix.to/title/n8we-dungeons-and-crayons/11139891-chapter-80") + ); + assert.equal(p.cover, "https://cdn.example/cover.jpg"); +}); + +test("comix.detect leaves cover empty when no img alt matches the title", () => { + metaTags = { "og:title": "Dungeons and Crayons" }; + pageImages = [{ alt: "Some Other Series", src: "https://cdn.example/other.jpg" }]; + const p = comix.detect(loc("https://comix.to/title/n8we-dungeons-and-crayons")); + assert.equal(p.cover, ""); + pageImages = []; +}); + test("comix ignores unrelated paths", () => { assert.equal(comix.detect(loc("https://comix.to/browse")).type, "other"); }); From b96caf13e63c859588a0a63bb7fa043f016eb918 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Mon, 3 Aug 2026 17:04:19 +0700 Subject: [PATCH 05/18] refactor(userscript): thread seriesId through latest-chapter scan --- userscript/manga-bookmark.user.js | 10 ++++++---- userscript/test/logic.test.js | 10 ++++++++++ 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/userscript/manga-bookmark.user.js b/userscript/manga-bookmark.user.js index 9f9b4a1..f8ef6ba 100644 --- a/userscript/manga-bookmark.user.js +++ b/userscript/manga-bookmark.user.js @@ -311,9 +311,11 @@ } // Highest chapter the site lists, or null when the markup yields nothing. - function computeLatestChapter(site, anchors) { + // seriesId is only consulted by adapters whose pages carry other series' + // chapter links; the rest ignore it. + function computeLatestChapter(site, anchors, seriesId) { const a = adapterFor(site); - return a ? a.latestChapterFromAnchors(anchors) : null; + return a ? a.latestChapterFromAnchors(anchors, seriesId) : null; } function currentSite() { @@ -834,7 +836,7 @@ if (!existing) return; applyLatestChapterIfChanged( existing, - computeLatestChapter(p.site, anchorsFromDocument(document)) + computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId) ); } @@ -864,7 +866,7 @@ const res = await fetch(bm.series_url, { credentials: "same-origin" }); if (!res.ok) continue; const html = await res.text(); - const latest = computeLatestChapter(bm.site, anchorsFromHTML(html)); + const latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id); await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest); } catch (e) { /* offline or blocked — try again after the throttle window */ diff --git a/userscript/test/logic.test.js b/userscript/test/logic.test.js index 2271956..7ed1677 100644 --- a/userscript/test/logic.test.js +++ b/userscript/test/logic.test.js @@ -264,6 +264,16 @@ test("comix latest returns null when no chapter links are present", () => { assert.equal(comix.latestChapterFromAnchors(anchorsFromHTML("x"), "n8we"), null); }); +test("asura and demonic ignore the seriesId argument", () => { + const asuraAnchors = anchorsFromHTML( + `Chapter 12` + ); + assert.deepEqual(asura.latestChapterFromAnchors(asuraAnchors, "ignored"), { + num: 12, + label: "Chapter 12", + }); +}); + // ============================================================ // Shared helpers // ============================================================ From a5c5e11c212e91c6664c39a0ae94f8cd0e6059ba Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Mon, 3 Aug 2026 17:08:33 +0700 Subject: [PATCH 06/18] feat(userscript): add kagane.to site adapter --- userscript/manga-bookmark.user.js | 83 +++++++++++++++++++++++++++++- userscript/test/logic.test.js | 85 +++++++++++++++++++++++++++++++ 2 files changed, 166 insertions(+), 2 deletions(-) diff --git a/userscript/manga-bookmark.user.js b/userscript/manga-bookmark.user.js index f8ef6ba..027f44e 100644 --- a/userscript/manga-bookmark.user.js +++ b/userscript/manga-bookmark.user.js @@ -292,7 +292,86 @@ }, }; - const ADAPTERS = [asura, demonic, comix]; + const kagane = { + site: "kagane", + matches: (loc) => /(^|\.)kagane\.to$/.test(loc.hostname), + detect(loc) { + const path = loc.pathname; + const UUID = "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"; + // /series//reader/ — no chapter number anywhere in the + // URL, so it comes out of og:title instead. + let m = path.match(new RegExp("^/series/(" + UUID + ")/reader/" + UUID)); + if (m) { + const num = chapterNumFromTitle(meta("og:title")); + return { + type: "chapter", + site: this.site, + seriesId: m[1], + title: cleanTitle(meta("og:title")), + cover: meta("og:image") || "", + seriesUrl: loc.origin + "/series/" + m[1], + chapterLabel: num === null ? null : "Chapter " + num, + chapterNum: num, + chapterUrl: loc.href, + }; + } + // /series/ + m = path.match(new RegExp("^/series/(" + UUID + ")/?$")); + if (m) { + return { + type: "series", + site: this.site, + seriesId: m[1], + title: cleanTitle(meta("og:title")), + cover: meta("og:image") || "", + seriesUrl: loc.origin + "/series/" + m[1], + chapterLabel: null, + chapterNum: null, + chapterUrl: null, + }; + } + return { type: "other" }; + + // Reader og:title is " - Chapter <n> - <episode name>". + function chapterNumFromTitle(t) { + const m = t && t.match(/\s-\sChapter\s([\d.]+)\s/); + if (!m) return null; + const num = parseFloat(m[1]); + return isNaN(num) ? null : num; + } + + function cleanTitle(t) { + if (!t) return ""; + return t.replace(/\s-\sChapter\s[\d.]+\s-\s.*$/i, "").trim(); + } + }, + // Reader hrefs are uuids with no number in them, so no maximum can be taken + // from anchors at all. latestChapterFromApi replaces this path entirely. + latestChapterFromAnchors() { + return null; + }, + // Same-origin only: the request needs the Cloudflare clearance cookie that + // this browser already holds for kagane.to. Called cross-origin it would be + // challenged and return nothing. + async latestChapterFromApi(seriesId) { + try { + const res = await fetch("/api/v2/series/" + seriesId); + if (!res.ok) return null; + const data = await res.json(); + let best = null; + for (const b of (data && data.series_books) || []) { + const num = parseFloat(b.chapter_no); + if (isNaN(num)) continue; + if (!best || num > best.num) best = { num, label: "Chapter " + b.chapter_no }; + } + return best; + } catch (e) { + return null; + } + }, + }; + + const ADAPTERS = [asura, demonic, comix, kagane]; function detect() { const loc = window.location; @@ -1694,7 +1773,7 @@ // Exposes pure logic only — see userscript/test/logic.test.js. // ============================================================ if (typeof window === "undefined" && typeof module === "object" && module.exports) { - module.exports = { stripBuildHash, comixSeriesId, asura, demonic, comix, anchorsFromHTML, statusOf }; + module.exports = { stripBuildHash, comixSeriesId, asura, demonic, comix, kagane, anchorsFromHTML, statusOf }; } // ============================================================ diff --git a/userscript/test/logic.test.js b/userscript/test/logic.test.js index 7ed1677..f6b96c3 100644 --- a/userscript/test/logic.test.js +++ b/userscript/test/logic.test.js @@ -57,6 +57,7 @@ const { asura, demonic, comix, + kagane, anchorsFromHTML, statusOf, } = require("../manga-bookmark.user.js"); @@ -274,6 +275,90 @@ test("asura and demonic ignore the seriesId argument", () => { }); }); +// ============================================================ +// kagane — reader URLs carry uuids and no chapter number, so the number has to +// come out of og:title. When that fails, chapterNum is null and the existing +// progress logic records the current chapter rather than guessing. +// ============================================================ + +const KAGANE_SERIES = "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"; +const KAGANE_BOOK = "019fa2e0-6dbd-73ca-b40b-fe06ab75eb0e"; + +test("kagane detects a series page", () => { + metaTags = { + "og:title": "Infinite Decryption: The Strongest Level 0", + "og:image": "https://kagane.to/api/v2/image/abc/compressed", + }; + const p = kagane.detect(loc("https://kagane.to/series/" + KAGANE_SERIES)); + assert.equal(p.type, "series"); + assert.equal(p.site, "kagane"); + assert.equal(p.seriesId, KAGANE_SERIES); + assert.equal(p.title, "Infinite Decryption: The Strongest Level 0"); + assert.equal(p.cover, "https://kagane.to/api/v2/image/abc/compressed"); + assert.equal(p.seriesUrl, "https://kagane.to/series/" + KAGANE_SERIES); +}); + +test("kagane reads the chapter number out of og:title", () => { + metaTags = { + "og:title": "Infinite Decryption: The Strongest Level 0 - Chapter 41 - Episode 41", + "og:image": "https://kagane.to/api/v2/image/abc/compressed", + }; + const p = kagane.detect( + loc("https://kagane.to/series/" + KAGANE_SERIES + "/reader/" + KAGANE_BOOK) + ); + assert.equal(p.type, "chapter"); + assert.equal(p.seriesId, KAGANE_SERIES); + assert.equal(p.title, "Infinite Decryption: The Strongest Level 0"); + assert.equal(p.chapterNum, 41); + assert.equal(p.chapterLabel, "Chapter 41"); + assert.equal(p.seriesUrl, "https://kagane.to/series/" + KAGANE_SERIES); +}); + +test("kagane yields a null chapterNum when og:title has no chapter", () => { + metaTags = { "og:title": "Infinite Decryption: The Strongest Level 0" }; + const p = kagane.detect( + loc("https://kagane.to/series/" + KAGANE_SERIES + "/reader/" + KAGANE_BOOK) + ); + assert.equal(p.type, "chapter"); + assert.equal(p.chapterNum, null); +}); + +test("kagane ignores unrelated paths", () => { + assert.equal(kagane.detect(loc("https://kagane.to/search")).type, "other"); +}); + +test("kagane anchor scanning is structurally impossible and returns null", () => { + const anchors = anchorsFromHTML( + `<a href="/series/${KAGANE_SERIES}/reader/${KAGANE_BOOK}">Chapter 41</a>` + ); + assert.equal(kagane.latestChapterFromAnchors(anchors, KAGANE_SERIES), null); +}); + +test("kagane latestChapterFromApi takes the max chapter_no", async () => { + globalThis.fetch = async (url) => { + assert.equal(url, "/api/v2/series/" + KAGANE_SERIES); + return { + ok: true, + json: async () => ({ + series_books: [ + { chapter_no: "1", title: "Episode 1" }, + { chapter_no: "41", title: "Episode 41" }, + { chapter_no: "40.5", title: "Episode 40.5" }, + ], + }), + }; + }; + assert.deepEqual(await kagane.latestChapterFromApi(KAGANE_SERIES), { + num: 41, + label: "Chapter 41", + }); +}); + +test("kagane latestChapterFromApi returns null on a challenge or error", async () => { + globalThis.fetch = async () => ({ ok: false, status: 403 }); + assert.equal(await kagane.latestChapterFromApi(KAGANE_SERIES), null); +}); + // ============================================================ // Shared helpers // ============================================================ From 076e8bb5d82bfc660cf8dd86c85c7461e7a46674 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 17:12:22 +0700 Subject: [PATCH 07/18] feat(userscript): refresh kagane latest chapters through its API --- userscript/manga-bookmark.user.js | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/userscript/manga-bookmark.user.js b/userscript/manga-bookmark.user.js index 027f44e..28caa27 100644 --- a/userscript/manga-bookmark.user.js +++ b/userscript/manga-bookmark.user.js @@ -919,10 +919,11 @@ ); } - // Everything else is only learned by fetching a series page. Same-origin - // only: these requests carry the session that gets us past the site's bot - // checks, which a request to the other site (or from a server) would not. - // One series per navigation keeps it indistinguishable from browsing. + // Everything else is only learned by fetching a series page — or, where the + // site offers one, its JSON API. Same-origin only: these requests carry the + // session that gets us past the site's bot checks, which a request to the + // other site (or from a server) would not. A few series per navigation keeps + // it indistinguishable from browsing. async function backgroundRefreshLatest() { const site = currentSite(); if (!site) return; @@ -937,15 +938,21 @@ .slice(0, LATEST_CHECK_BATCH); if (due.length === 0) return; + const adapter = adapterFor(site); for (const bm of due) { // Recorded even when the fetch fails, so a broken series is retried on // the next throttle window rather than on every single page load. checked[bm.key] = Date.now(); try { - const res = await fetch(bm.series_url, { credentials: "same-origin" }); - if (!res.ok) continue; - const html = await res.text(); - const latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id); + let latest; + if (adapter && adapter.latestChapterFromApi) { + latest = await adapter.latestChapterFromApi(bm.series_id); + } else { + const res = await fetch(bm.series_url, { credentials: "same-origin" }); + if (!res.ok) continue; + const html = await res.text(); + latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id); + } await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest); } catch (e) { /* offline or blocked — try again after the throttle window */ From 48c2d57d7eeacfd2ffc8644e7821a808f43cf8ab Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 17:15:41 +0700 Subject: [PATCH 08/18] feat(userscript): match comix and kagane, add panel chips --- userscript/manga-bookmark.user.js | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/userscript/manga-bookmark.user.js b/userscript/manga-bookmark.user.js index 28caa27..4e4e51d 100644 --- a/userscript/manga-bookmark.user.js +++ b/userscript/manga-bookmark.user.js @@ -1,14 +1,16 @@ // ==UserScript== // @name Manga Bookmark Sync // @namespace mangabm -// @version 1.5.0 -// @description Track read progress on Asura & Demonic and sync to a self-hosted backend. Bromite-compatible (no GM_* APIs). +// @version 1.6.0 +// @description Track read progress on Asura, Demonic, Comix & Kagane and sync to a self-hosted backend. Bromite-compatible (no GM_* APIs). // @author you // @downloadURL https://manga-api.violetcrown.my.id/u/40d79969b5442f90df4fe306a092c7c50e7b4a7a98099f98cc398f4fb374b1df/manga-bookmark.user.js // @updateURL https://manga-api.violetcrown.my.id/u/40d79969b5442f90df4fe306a092c7c50e7b4a7a98099f98cc398f4fb374b1df/manga-bookmark.user.js // @match https://asuracomic.net/* // @match https://asurascans.com/* // @match https://demonicscans.org/* +// @match https://comix.to/* +// @match https://kagane.to/* // @run-at document-idle // @noframes // ==/UserScript== @@ -1586,6 +1588,8 @@ <a class="chip" href="${WEB_BASE}" target="_blank" rel="noopener">Web</a> <a class="chip" href="https://asurascans.com" target="_blank" rel="noopener">Asura</a> <a class="chip" href="https://demonicscans.org" target="_blank" rel="noopener">Demonic</a> + <a class="chip" href="https://comix.to" target="_blank" rel="noopener">Comix</a> + <a class="chip" href="https://kagane.to" target="_blank" rel="noopener">Kagane</a> <button id="pending" class="chip pending" hidden>⟳ 0 pending</button> </div> <section id="context"></section> From 2fcf882c495d825028184a9b5cbbdd2c96fe9091 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 17:18:38 +0700 Subject: [PATCH 09/18] feat(latest): parse comix SSR state and kagane API for latest chapter --- backend/internal/latest/sites.go | 25 +++++++++++++++ backend/internal/latest/sites_test.go | 46 +++++++++++++++++++++++++++ 2 files changed, 71 insertions(+) diff --git a/backend/internal/latest/sites.go b/backend/internal/latest/sites.go index 9b7c72f..fcc771b 100644 --- a/backend/internal/latest/sites.go +++ b/backend/internal/latest/sites.go @@ -25,6 +25,15 @@ var asuraSlugRe = regexp.MustCompile(`/comics/([^/?#]+)`) // through. Both the raw "&" and the HTML-escaped "&" forms occur. var demonicChapterRe = regexp.MustCompile(`chaptered\.php\?manga=\d+&(?:amp;)?chapter=([0-9.]+)`) +// comixSlugRe pulls the "<id>-<slug>" segment out of a stored series_url. +// Only the id prefix is stable; the slug tail follows the title. +var comixSlugRe = regexp.MustCompile(`/title/([^/?#]+)`) + +// kaganeChapterRe matches the chapter numbers in a kagane API response. This +// branch is fed by the browser fetcher, so the body is JSON rather than HTML — +// there are no anchors to scan. +var kaganeChapterRe = regexp.MustCompile(`"chapter_no":"([0-9.]+)"`) + // latestChapterFrom returns the highest chapter number body advertises for this // series. ok is false when the body yields nothing usable — an unknown site, an // empty body, a Cloudflare challenge page, and a site redesign all land here, @@ -61,6 +70,22 @@ func latestChapterFrom(site, seriesURL, body string) (latestChapter, bool) { re = regexp.MustCompile(`/comics/` + regexp.QuoteMeta(slug) + `(?:-[0-9a-f]{8})?/chapter/([0-9.]+)`) case "demonic": re = demonicChapterRe + case "comix": + m := comixSlugRe.FindStringSubmatch(seriesURL) + if m == nil { + return latestChapter{}, false + } + // comix ships an SPA: the served HTML carries a JSON state blob instead + // of chapter anchors, and latestChapterUrl is the only place the newest + // chapter appears. Scoping to this series' id prefix keeps a + // "recommended" strip's entries from winning the maximum. + id := m[1] + if i := strings.Index(id, "-"); i != -1 { + id = id[:i] + } + re = regexp.MustCompile(`"latestChapterUrl":"/title/` + regexp.QuoteMeta(id) + `-[^"]*-chapter-([0-9.]+)"`) + case "kagane": + re = kaganeChapterRe default: return latestChapter{}, false } diff --git a/backend/internal/latest/sites_test.go b/backend/internal/latest/sites_test.go index de94005..a2bb82d 100644 --- a/backend/internal/latest/sites_test.go +++ b/backend/internal/latest/sites_test.go @@ -34,6 +34,24 @@ const challengeFixture = `<!DOCTYPE html><html><head><title>Just a moment...</ti <script src="/cdn-cgi/challenge-platform/h/b/orchestrate/chl_page/v1"></script></head> <body><div id="challenge-running">Checking your browser</div></body></html>` +// Trimmed from the server-rendered HTML of +// https://comix.to/title/n8we-dungeons-and-crayons fetched 2026-08-03. comix is +// an SPA: the page ships a JSON state blob rather than a list of chapter +// anchors, and latestChapterUrl is where the newest chapter actually lives. +const comixSeriesFixture = ` +{"firstChapterUrl":"/title/n8we-dungeons-and-crayons/5038739-chapter-1","latestChapterUrl":"/title/n8we-dungeons-and-crayons/11139891-chapter-80"}, +{""manga","recommended","n8we",1]":{"items":[{"latestChapterUrl":"/title/qqwrm-full-time-awakening/99999999-chapter-999"}]} +` + +// The kagane branch is fed by the browser fetcher, so the body is API JSON, not +// HTML. Trimmed from GET /api/v2/series/<uuid> on 2026-08-03. +const kaganeAPIFixture = ` +{"series_id":"019f84bc-9ba0-7ed9-86f5-8b905ec7c28b","title":"Infinite Decryption", +"series_books":[{"book_id":"a","title":"Episode 1","chapter_no":"1","sort_no":1}, +{"book_id":"b","title":"Episode 41","chapter_no":"41","sort_no":41}, +{"book_id":"c","title":"Episode 40.5","chapter_no":"40.5","sort_no":40}]} +` + func TestLatestChapterFrom(t *testing.T) { const asuraURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af" const demonicURL = "https://demonicscans.org/manga/Catastrophic-Necromancer" @@ -100,6 +118,34 @@ func TestLatestChapterFrom(t *testing.T) { site: "mangadex", seriesURL: "https://example.com/x", body: asuraSeriesFixture, wantOK: false, }, + { + name: "comix reads latestChapterUrl, scoped to this series", + site: "comix", + seriesURL: "https://comix.to/title/n8we-dungeons-and-crayons", + body: comixSeriesFixture, + wantOK: true, wantNum: 80, wantLabel: "Chapter 80", + }, + { + name: "comix yields nothing on a challenge page", + site: "comix", + seriesURL: "https://comix.to/title/n8we-dungeons-and-crayons", + body: challengeFixture, + wantOK: false, + }, + { + name: "kagane takes the max chapter_no from API json", + site: "kagane", + seriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + body: kaganeAPIFixture, + wantOK: true, wantNum: 41, wantLabel: "Chapter 41", + }, + { + name: "kagane yields nothing on a challenge page", + site: "kagane", + seriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + body: challengeFixture, + wantOK: false, + }, } for _, tt := range tests { From 89eaef70d4a83a5b05d9fad86418a76d89e0973e Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 17:23:10 +0700 Subject: [PATCH 10/18] feat(latest): allow comix and kagane, route kagane to a browser fetcher --- backend/internal/latest/poller.go | 51 +++++++++++--- backend/internal/latest/poller_test.go | 95 ++++++++++++++++++++++++++ 2 files changed, 137 insertions(+), 9 deletions(-) diff --git a/backend/internal/latest/poller.go b/backend/internal/latest/poller.go index b2214ca..f5770d9 100644 --- a/backend/internal/latest/poller.go +++ b/backend/internal/latest/poller.go @@ -32,11 +32,27 @@ type Fetcher interface { type Poller struct { Store *store.Store Fetch Fetcher - Now func() time.Time // injected so tests can freeze it - Cooldown time.Duration - Interval time.Duration - Stagger time.Duration - Batch int + // BrowserFetch handles sites behind a JavaScript challenge that Fetch + // cannot clear. Nil disables those sites entirely rather than falling back + // to Fetch, which would only ever retrieve a challenge page. + BrowserFetch Fetcher + Now func() time.Time // injected so tests can freeze it + Cooldown time.Duration + Interval time.Duration + Stagger time.Duration + Batch int +} + +// fetcherFor returns the fetcher a site needs, or nil when the site cannot be +// fetched at all right now. kagane sits behind a Cloudflare JavaScript +// challenge that no TLS fingerprint clears — verified 2026-08-03 from the +// deployment host with the same Chrome profile TLSFetcher uses — so it is +// browser-only or nothing. +func (p *Poller) fetcherFor(site string) Fetcher { + if site == "kagane" { + return p.BrowserFetch + } + return p.Fetch } // Run polls until ctx is cancelled. @@ -130,7 +146,13 @@ func (p *Poller) checkOne(ctx context.Context, b store.Bookmark) { return } - body, status, err := p.Fetch.Get(ctx, b.SeriesURL) + f := p.fetcherFor(b.Site) + if f == nil { + log.Printf("latest poll %q: no fetcher for site %q", b.Key, b.Site) + return + } + + body, status, err := f.Get(ctx, b.SeriesURL) if err != nil { log.Printf("latest poll %q: fetch %s: %v", b.Key, b.SeriesURL, err) return @@ -185,13 +207,18 @@ func (p *Poller) checkOne(ctx context.Context, b store.Bookmark) { } // fetchableSeriesURL reports whether site is a site latestChapterFrom knows how -// to parse and seriesURL is safe to hand to the fetcher: an https URL with a +// to parse and seriesURL is safe to hand to a fetcher: an https URL with a // non-empty host. series_url comes from client-supplied PUT bodies, so this is // a defence against the poller being used to probe arbitrary hosts from the // server's own network position, not just a check against wasted requests. +// +// kagane is held to a stricter rule: it is fetched by a headless browser, which +// executes JavaScript and carries cookies, and is therefore a far stronger SSRF +// primitive than an HTTP GET. Its host must match exactly, not merely be +// non-empty. func fetchableSeriesURL(site, seriesURL string) bool { switch site { - case "asura", "demonic": + case "asura", "demonic", "comix", "kagane": default: return false } @@ -199,5 +226,11 @@ func fetchableSeriesURL(site, seriesURL string) bool { if err != nil { return false } - return u.Scheme == "https" && u.Host != "" + if u.Scheme != "https" || u.Host == "" { + return false + } + if site == "kagane" { + return u.Hostname() == "kagane.to" + } + return true } diff --git a/backend/internal/latest/poller_test.go b/backend/internal/latest/poller_test.go index d60daa4..a960307 100644 --- a/backend/internal/latest/poller_test.go +++ b/backend/internal/latest/poller_test.go @@ -358,3 +358,98 @@ func TestRunOnceStopsOnCancelledContext(t *testing.T) { t.Fatalf("fetched %d series with a cancelled context, want 0", got) } } + +func TestFetchableSeriesURL(t *testing.T) { + tests := []struct { + name string + site string + seriesURL string + want bool + }{ + {"asura https", "asura", "https://asurascans.com/comics/x-aabbccdd", true}, + {"demonic https", "demonic", "https://demonicscans.org/manga/X", true}, + {"comix https", "comix", "https://comix.to/title/n8we-dungeons-and-crayons", true}, + {"kagane on its own host", "kagane", "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", true}, + // The browser fetcher runs JavaScript and carries cookies, so a + // client-supplied series_url must not be able to aim it anywhere else. + {"kagane on a foreign host", "kagane", "https://evil.example/series/x", false}, + {"kagane on a lookalike host", "kagane", "https://kagane.to.evil.example/series/x", false}, + {"unknown site", "mangadex", "https://mangadex.org/title/x", false}, + {"non-https", "comix", "http://comix.to/title/x", false}, + {"no host", "comix", "https:///title/x", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := fetchableSeriesURL(tt.site, tt.seriesURL); got != tt.want { + t.Errorf("fetchableSeriesURL(%q, %q) = %v, want %v", + tt.site, tt.seriesURL, got, tt.want) + } + }) + } +} + +// A kagane row must not be handed to the plain TLS fetcher: it would only ever +// receive a challenge page, and the browser fetcher is the whole reason kagane +// is pollable at all. +func TestKaganeSkippedWhenNoBrowserFetcher(t *testing.T) { + s := newTestStore(t) + if _, err := s.Upsert(store.Bookmark{ + Key: "kagane:019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + Site: "kagane", + SeriesID: "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + SeriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + UpdatedAt: 1000, + }); err != nil { + t.Fatalf("seed: %v", err) + } + + f := &fakeFetcher{body: kaganeAPIFixture, status: 200} + p := &Poller{ + Store: s, Fetch: f, + Now: func() time.Time { return time.UnixMilli(5_000_000) }, + Cooldown: time.Hour, Interval: time.Hour, Batch: 10, + } + p.runOnce(context.Background()) + + if len(f.calls) != 0 { + t.Errorf("TLS fetcher was called for kagane: %v", f.calls) + } +} + +// With a browser fetcher wired up, kagane goes to it and not to the TLS one. +func TestKaganeUsesBrowserFetcher(t *testing.T) { + s := newTestStore(t) + key := "kagane:019f84bc-9ba0-7ed9-86f5-8b905ec7c28b" + if _, err := s.Upsert(store.Bookmark{ + Key: key, + Site: "kagane", + SeriesID: "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + SeriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + UpdatedAt: 1000, + }); err != nil { + t.Fatalf("seed: %v", err) + } + + tlsF := &fakeFetcher{body: "", status: 200} + browserF := &fakeFetcher{body: kaganeAPIFixture, status: 200} + p := &Poller{ + Store: s, Fetch: tlsF, BrowserFetch: browserF, + Now: func() time.Time { return time.UnixMilli(5_000_000) }, + Cooldown: time.Hour, Interval: time.Hour, Batch: 10, + } + p.runOnce(context.Background()) + + if len(tlsF.calls) != 0 { + t.Errorf("TLS fetcher was called for kagane: %v", tlsF.calls) + } + if len(browserF.calls) != 1 { + t.Fatalf("browser fetcher calls = %v, want 1", browserF.calls) + } + got, found, err := s.Get(key) + if err != nil || !found { + t.Fatalf("Get: %v found=%v", err, found) + } + if got.LatestChapterNum == nil || *got.LatestChapterNum != 41 { + t.Errorf("LatestChapterNum = %v, want 41", got.LatestChapterNum) + } +} From 1d9b1200bbb8550ef5a31d011300928c8f440bf2 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 17:29:21 +0700 Subject: [PATCH 11/18] feat(latest): add chromedp browser fetcher for challenge-gated sites --- backend/go.mod | 11 +- backend/go.sum | 22 +++- backend/internal/latest/browser.go | 146 ++++++++++++++++++++++++ backend/internal/latest/browser_test.go | 38 ++++++ 4 files changed, 213 insertions(+), 4 deletions(-) create mode 100644 backend/internal/latest/browser.go create mode 100644 backend/internal/latest/browser_test.go diff --git a/backend/go.mod b/backend/go.mod index 56a710f..f4fb0e1 100644 --- a/backend/go.mod +++ b/backend/go.mod @@ -1,10 +1,12 @@ module mangabm/backend -go 1.24.1 +go 1.26 require ( github.com/bogdanfinn/fhttp v0.6.8 github.com/bogdanfinn/tls-client v1.15.1 + github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f + github.com/chromedp/chromedp v0.16.0 modernc.org/sqlite v1.34.4 ) @@ -15,7 +17,12 @@ require ( github.com/bogdanfinn/quic-go-utls v1.0.9-utls // indirect github.com/bogdanfinn/utls v1.7.7-barnius // indirect github.com/bogdanfinn/websocket v1.5.5-barnius // indirect + github.com/chromedp/sysutil v1.1.0 // indirect github.com/dustin/go-humanize v1.0.1 // indirect + github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68 // indirect + github.com/gobwas/httphead v0.1.0 // indirect + github.com/gobwas/pool v0.2.1 // indirect + github.com/gobwas/ws v1.4.0 // indirect github.com/google/uuid v1.6.0 // indirect github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect github.com/klauspost/compress v1.18.2 // indirect @@ -26,7 +33,7 @@ require ( github.com/tam7t/hpkp v0.0.0-20160821193359-2b70b4024ed5 // indirect golang.org/x/crypto v0.46.0 // indirect golang.org/x/net v0.48.0 // indirect - golang.org/x/sys v0.39.0 // indirect + golang.org/x/sys v0.47.0 // indirect golang.org/x/text v0.32.0 // indirect modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect modernc.org/libc v1.55.3 // indirect diff --git a/backend/go.sum b/backend/go.sum index cc6daad..ef779f6 100644 --- a/backend/go.sum +++ b/backend/go.sum @@ -14,10 +14,24 @@ github.com/bogdanfinn/utls v1.7.7-barnius h1:OuJ497cc7F3yKNVHRsYPQdGggmk5x6+V5Zl github.com/bogdanfinn/utls v1.7.7-barnius/go.mod h1:aAK1VZQlpKZClF1WEQeq6kyclbkPq4hz6xTbB5xSlmg= github.com/bogdanfinn/websocket v1.5.5-barnius h1:bY+qnxpai1qe7Jmjx+Sds/cmOSpuuLoR8x61rWltjOI= github.com/bogdanfinn/websocket v1.5.5-barnius/go.mod h1:gvvEw6pTKHb7yOiFvIfAFTStQWyrm25BMVCTj5wRSsI= +github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f h1:0Z1zcSLEmnj2c2CmJYBqewtS6pxhB39bNWUSEUAWjgk= +github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f/go.mod h1:RwFsSODCtFExll+GhHM6R92SARHR3Z3oipaxLHj46C0= +github.com/chromedp/chromedp v0.16.0 h1:rOO4deOm4CbZgBCa8mD9g2rDyIoNs0BkgvNrlbp5ouk= +github.com/chromedp/chromedp v0.16.0/go.mod h1:rbuGKFT1vMcFcFqKfPIO1GpX/N+2s8onm2qMxZLbU5U= +github.com/chromedp/sysutil v1.1.0 h1:PUFNv5EcprjqXZD9nJb9b/c9ibAbxiYo4exNWZyipwM= +github.com/chromedp/sysutil v1.1.0/go.mod h1:WiThHUdltqCNKGc4gaU50XgYjwjYIhKWoHGPTUfWTJ8= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68 h1:KZaTBSyshWX3MP5jukJcNSuXDQTO+rNpt0J564dX/eg= +github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68/go.mod h1:tphK2c80bpPhMOI4v6bIc2xWywPfbqi1Z06+RcrMkDg= +github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU= +github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM= +github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og= +github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw= +github.com/gobwas/ws v1.4.0 h1:CTaoG1tojrh4ucGPcoJFiAQUAsEWekEWvLy7GsVNqGs= +github.com/gobwas/ws v1.4.0/go.mod h1:G3gNqMNtPppf5XUz7O4shetPpcZ1VJ7zt18dlUeakrc= github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo= github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= @@ -26,10 +40,14 @@ github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk= github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= +github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80 h1:6Yzfa6GP0rIo/kULo2bwGEkFvCePZ3qHDDTC3/J9Swo= +github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs= github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw= +github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8= @@ -56,8 +74,8 @@ golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk= -golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU= diff --git a/backend/internal/latest/browser.go b/backend/internal/latest/browser.go new file mode 100644 index 0000000..81c2707 --- /dev/null +++ b/backend/internal/latest/browser.go @@ -0,0 +1,146 @@ +package latest + +import ( + "context" + "encoding/json" + "fmt" + "net/url" + "regexp" + "sync" + "time" + + "github.com/chromedp/cdproto/runtime" + "github.com/chromedp/chromedp" +) + +// challengeTimeout bounds one navigate-and-solve. A Cloudflare managed +// challenge clears in a few seconds when it clears at all; anything longer is a +// challenge that is not going to pass, and the caller's cooldown was already +// stamped before this ran. +const challengeTimeout = 45 * time.Second + +var kaganeSeriesRe = regexp.MustCompile(`^/series/([0-9a-f-]{36})/?$`) + +// BrowserFetcher retrieves pages through a remote headless Chrome over the +// DevTools Protocol. +// +// It exists for one reason: kagane.to sits behind a Cloudflare JavaScript +// challenge. Verified 2026-08-03 from the deployment host, plain HTTP and +// bogdanfinn/tls-client with a Chrome_133 profile both get 403 with +// cf-mitigated: challenge on every path, including the API, robots.txt and +// images. Clearing it requires executing the challenge script, which only a +// real browser does. +// +// The request is made *inside* the page rather than by extracting cf_clearance +// and replaying it through TLSFetcher. That cookie is bound to IP, User-Agent +// and often the TLS fingerprint, so replaying it means keeping three things in +// sync that break silently and separately. The browser's own cookie jar +// persists across polls, so the challenge is solved once every few hours. +type BrowserFetcher struct { + allocCtx context.Context + cancel context.CancelFunc + // One page at a time: caps the sidecar's memory and keeps series from + // sharing page state. + mu sync.Mutex +} + +var _ Fetcher = (*BrowserFetcher)(nil) + +// NewBrowserFetcher connects to a headless-shell over CDP. wsURL is the +// container's websocket endpoint, e.g. ws://headless-shell:9222. +// +// NoModifyURL is load-bearing: /json/version advertises a +// webSocketDebuggerUrl pointing at 127.0.0.1, which is meaningless from another +// container, and without this option chromedp follows it and hangs. +func NewBrowserFetcher(wsURL string) (*BrowserFetcher, error) { + if wsURL == "" { + return nil, fmt.Errorf("empty browser websocket url") + } + ctx, cancel := chromedp.NewRemoteAllocator( + context.Background(), wsURL, chromedp.NoModifyURL) + return &BrowserFetcher{allocCtx: ctx, cancel: cancel}, nil +} + +func (f *BrowserFetcher) Close() { + f.cancel() +} + +// Get navigates to seriesURL, lets any challenge resolve, then reads the site's +// JSON API from inside the page so the request carries the clearance cookie. +// The returned body is API JSON, which is what latestChapterFrom's kagane case +// expects — it is not HTML. +func (f *BrowserFetcher) Get(ctx context.Context, seriesURL string) (string, int, error) { + apiURL, ok := kaganeAPIURL(seriesURL) + if !ok { + return "", 0, fmt.Errorf("not a fetchable kagane series url: %q", seriesURL) + } + + f.mu.Lock() + defer f.mu.Unlock() + + ctx, cancel := context.WithTimeout(ctx, challengeTimeout) + defer cancel() + // A fresh tab per fetch, closed on return, so one wedged page cannot + // poison later polls. + tabCtx, cancelTab := chromedp.NewContext(f.allocCtx) + defer cancelTab() + // Bind the caller's deadline to the tab. + tabCtx, cancelDeadline := context.WithCancel(tabCtx) + defer cancelDeadline() + go func() { + <-ctx.Done() + cancelDeadline() + }() + + var body string + err := chromedp.Run(tabCtx, + chromedp.Navigate(seriesURL), + // The challenge reloads the page itself when it passes; waiting for the + // site's own root element is what tells us we are through it. + chromedp.WaitReady("body", chromedp.ByQuery), + chromedp.Evaluate( + `fetch(`+jsString(apiURL)+`).then(r => r.ok ? r.text() : "")`, + &body, + awaitPromise, + ), + ) + if err != nil { + return "", 0, fmt.Errorf("browser fetch %q: %w", seriesURL, err) + } + if body == "" { + // Challenge still up, or the API refused. Indistinguishable from here + // and handled identically by the caller. + return "", 403, nil + } + return body, 200, nil +} + +// kaganeAPIURL maps a stored series_url to the JSON endpoint carrying its +// chapter list. Returning false for anything else is a second line of defence +// behind fetchableSeriesURL: a headless browser is a strong SSRF primitive and +// series_url is client-supplied, so the host is pinned here too. +func kaganeAPIURL(seriesURL string) (string, bool) { + u, err := url.Parse(seriesURL) + if err != nil || u.Scheme != "https" || u.Hostname() != "kagane.to" { + return "", false + } + m := kaganeSeriesRe.FindStringSubmatch(u.Path) + if m == nil { + return "", false + } + return "https://kagane.to/api/v2/series/" + m[1], true +} + +// awaitPromise makes Evaluate resolve the promise rather than returning a +// serialised Promise object. +func awaitPromise(p *runtime.EvaluateParams) *runtime.EvaluateParams { + return p.WithAwaitPromise(true) +} + +// jsString renders s as a JavaScript string literal for embedding in an +// Evaluate expression. The URL is host-pinned by kaganeAPIURL before it gets +// here, but quoting it properly is what keeps that guarantee intact. +func jsString(s string) string { + b, _ := json.Marshal(s) + return string(b) +} diff --git a/backend/internal/latest/browser_test.go b/backend/internal/latest/browser_test.go new file mode 100644 index 0000000..fa91a3a --- /dev/null +++ b/backend/internal/latest/browser_test.go @@ -0,0 +1,38 @@ +package latest + +import "testing" + +func TestKaganeAPIURL(t *testing.T) { + const uuid = "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b" + tests := []struct { + name string + seriesURL string + want string + wantOK bool + }{ + { + name: "series page maps to its API endpoint", + seriesURL: "https://kagane.to/series/" + uuid, + want: "https://kagane.to/api/v2/series/" + uuid, + wantOK: true, + }, + { + name: "trailing slash is tolerated", + seriesURL: "https://kagane.to/series/" + uuid + "/", + want: "https://kagane.to/api/v2/series/" + uuid, + wantOK: true, + }, + {"not a series path", "https://kagane.to/search", "", false}, + {"foreign host", "https://evil.example/series/" + uuid, "", false}, + {"garbage", "://", "", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, ok := kaganeAPIURL(tt.seriesURL) + if ok != tt.wantOK || got != tt.want { + t.Errorf("kaganeAPIURL(%q) = %q, %v; want %q, %v", + tt.seriesURL, got, ok, tt.want, tt.wantOK) + } + }) + } +} From 2c7b4952f31c5c1cd6fc48ada0c8c5a330a92116 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 17:46:29 +0700 Subject: [PATCH 12/18] feat: wire headless-shell sidecar for challenge-gated polling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Also bumps backend/Dockerfile's build stage to golang:1.26-alpine — chromedp v0.16.0 and cdproto both require go 1.26, and the pinned 1.24-alpine base no longer builds the module. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --- .env.example | 4 ++++ CLAUDE.md | 9 +++++++-- DEPLOY.md | 6 ++++++ README.md | 7 ++++--- backend/Dockerfile | 2 +- backend/main.go | 15 +++++++++++++++ docker-compose.prod.yml | 10 ++++++++++ docker-compose.yml | 21 +++++++++++++++++++++ 8 files changed, 68 insertions(+), 6 deletions(-) diff --git a/.env.example b/.env.example index 3ea053b..68000f1 100644 --- a/.env.example +++ b/.env.example @@ -52,3 +52,7 @@ WEB_PASSWORD= # BATCH x (COOLDOWN / INTERVAL) series hold the cooldown cadence — 84 with these # defaults. Beyond that the cadence stretches uniformly rather than breaking; # raise BATCH or lower INTERVAL. Keep BATCH x STAGGER under INTERVAL. + +# Headless-shell CDP endpoint for sites behind a JavaScript challenge (kagane). +# Unset disables browser polling; those sites then rely on the userscript alone. +BROWSER_WS_URL=ws://headless-shell:9222 diff --git a/CLAUDE.md b/CLAUDE.md index 7f41000..2dc42d4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -37,7 +37,7 @@ Violentmonkey userscript (isolated world, per-site adapters, localStorage cache) packages together into `newRouter`. Root-level `*_test.go` hold integration tests that exercise the full router; unit tests for a package live beside it under `internal/`. -- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`). Sync **last-write-wins**. Schema and endpoint list in plan. +- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`|`comix`|`kagane`). Sync **last-write-wins**. Schema and endpoint list in plan. - **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). - **Web UI:** same binary serve password-gated browser UI on second hostname — `GET /` (list, or login page when no session), @@ -75,7 +75,10 @@ Violentmonkey userscript (isolated world, per-site adapters, localStorage cache) cooldown instead of retrying every tick, and writes go through `Store.Get` + `Store.Upsert` so new chapter never reorders list. Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth - against fingerprint-based blocking; any failure log and skip. See + against fingerprint-based blocking; any failure log and skip. kagane sits + behind a Cloudflare JavaScript challenge the TLS client can't clear, so it is + browser-only: fetched over CDP via `BROWSER_WS_URL`, and simply not polled + when that's unset. See `docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`. Poller's `Store.Get` + `Store.Upsert` not wrapped in transaction, so userscript `PUT` that commits between the two can get overwritten by @@ -100,6 +103,8 @@ Violentmonkey userscript (isolated world, per-site adapters, localStorage cache) (background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`). `USERSCRIPT_PATH` (file served at `/u/{token}/manga-bookmark.user.js`, default `/userscript/manga-bookmark.user.js`, supplied by bindmount). + `BROWSER_WS_URL` (headless-shell CDP endpoint for kagane; unset disables + browser polling and leaves that site to the userscript alone). ### Userscript structure (single IIFE, `manga-bookmark.user.js`) diff --git a/DEPLOY.md b/DEPLOY.md index 98bb97f..8f82aea 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -116,6 +116,12 @@ This merges the base file (build/image/env/volume) with the prod override (no host port, Traefik network + router labels). Always pass **both** `-f` flags — the prod file is not standalone. +Two services come up: `manga-api` (the backend) and `headless-shell`, a CDP +sidecar the poller uses to fetch kagane (behind a Cloudflare JS challenge). +It has no published port — only `manga-api` can reach it, over +`BROWSER_WS_URL`. Missing or unreachable, the poller just skips kagane and +logs it; nothing else is affected. + Check it's up and healthy: ```bash diff --git a/README.md b/README.md index 1e6a83a..6ca2428 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,9 @@ # Manga Bookmark -Track manga read-progress on **asurascans.com** (a.k.a. asuracomic.net) and -**demonicscans.org** from a phone (Bromite / mobile Chromium), synced to a -self-hosted Go backend so bookmarks unify across both sites and all devices. +Track manga read-progress on **asurascans.com** (a.k.a. asuracomic.net), +**demonicscans.org**, **comix.to**, and **kagane.to** from a phone (Bromite / +mobile Chromium), synced to a self-hosted Go backend so bookmarks unify across +all four sites and all devices. Two parts: diff --git a/backend/Dockerfile b/backend/Dockerfile index f91faf4..15afbd4 100644 --- a/backend/Dockerfile +++ b/backend/Dockerfile @@ -1,7 +1,7 @@ # syntax=docker/dockerfile:1 # --- build stage: compile a static, CGO-free binary --- -FROM golang:1.24-alpine AS build +FROM golang:1.26-alpine AS build WORKDIR /src # Dependencies first for layer caching (changes rarely). diff --git a/backend/main.go b/backend/main.go index 75db207..a4241c3 100644 --- a/backend/main.go +++ b/backend/main.go @@ -274,5 +274,20 @@ func startLatestPoller(ctx context.Context, s *store.Store, cfg LatestPoll) { Stagger: cfg.Stagger, Batch: cfg.Batch, } + + // Optional: without it, sites behind a JavaScript challenge are simply not + // polled, and their latest_chapter comes from the userscript alone — which + // is how the service behaved before the sidecar existed. + if ws := strings.TrimSpace(os.Getenv("BROWSER_WS_URL")); ws != "" { + bf, err := latest.NewBrowserFetcher(ws) + if err != nil { + log.Printf("latest-chapter poller: browser fetcher disabled: %v", err) + } else { + p.BrowserFetch = bf + context.AfterFunc(ctx, bf.Close) + log.Printf("latest-chapter poller: browser fetcher at %s", ws) + } + } + go p.Run(ctx) } diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml index 78bc45b..9444cf8 100644 --- a/docker-compose.prod.yml +++ b/docker-compose.prod.yml @@ -17,6 +17,10 @@ services: manga-api: # Traffic arrives over the Traefik network, not a published port. ports: !reset [] + environment: + BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://headless-shell:9222} + depends_on: + - headless-shell networks: - proxy labels: @@ -36,6 +40,12 @@ services: - "traefik.http.routers.mangaweb.tls.certresolver=${TRAEFIK_CERTRESOLVER:-le}" - "traefik.http.routers.mangaweb.service=mangabm" + # manga-api only joins `proxy` in this override (see above), so headless-shell + # has to follow it there too or DNS resolution of headless-shell:9222 breaks. + headless-shell: + networks: + - proxy + networks: proxy: external: true diff --git a/docker-compose.yml b/docker-compose.yml index c8adc6b..304ea69 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -29,6 +29,11 @@ services: LATEST_CHAPTER_POLL_INTERVAL: ${LATEST_CHAPTER_POLL_INTERVAL:-10m} LATEST_CHAPTER_POLL_BATCH: ${LATEST_CHAPTER_POLL_BATCH:-14} LATEST_CHAPTER_POLL_STAGGER: ${LATEST_CHAPTER_POLL_STAGGER:-20s} + # CDP endpoint for sites behind a JavaScript challenge (kagane). Unset + # disables browser polling for those sites; the userscript still covers them. + BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://headless-shell:9222} + depends_on: + - headless-shell volumes: - bookmarks-data:/data # The userscript is served from here, read fresh on every request. Editing @@ -41,5 +46,21 @@ services: ports: - "127.0.0.1:8080:8080" + headless-shell: + image: chromedp/headless-shell:stable + restart: unless-stopped + # Chrome allocates shared memory per tab and dies on Docker's 64MB default. + shm_size: '1gb' + # Reaps zombie renderer processes, which otherwise accumulate for the + # container's lifetime. + init: true + # Deliberately no `ports:` — an exposed CDP endpoint is remote code + # execution. Only the backend on the internal network may reach it. + command: + - --remote-debugging-address=0.0.0.0 + - --remote-debugging-port=9222 + - --disable-gpu + - --no-sandbox + volumes: bookmarks-data: From 3adbfb7ad97c9f01e6fc2b3a8a1e7e26482888cc Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 17:52:34 +0700 Subject: [PATCH 13/18] fix: scope headless-shell to a dedicated network, off the Traefik proxy network Prod override put headless-shell on the externally-managed `proxy` network so manga-api (confined there for Traefik routing) could still resolve it. That reopened CDP (port 9222, raw remote code execution) to every other container on that shared network, not just manga-api. Give both services a project-private `browser` network (defined in the base compose file, not `internal: true` since headless Chrome needs outbound access to kagane.to). manga-api joins both `proxy` and `browser` in the prod override; headless-shell never touches `proxy`. --- docker-compose.prod.yml | 13 ++++++++----- docker-compose.yml | 12 +++++++++++- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml index 9444cf8..f27f172 100644 --- a/docker-compose.prod.yml +++ b/docker-compose.prod.yml @@ -21,8 +21,12 @@ services: BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://headless-shell:9222} depends_on: - headless-shell + # `networks:` here replaces the base file's list entirely, so both must be + # named: `proxy` for Traefik routing, `browser` (defined in the base file) + # to keep reaching headless-shell without putting it on `proxy` too. networks: - proxy + - browser labels: - "traefik.enable=true" - "traefik.docker.network=${PROXY_NETWORK:-proxy}" @@ -40,11 +44,10 @@ services: - "traefik.http.routers.mangaweb.tls.certresolver=${TRAEFIK_CERTRESOLVER:-le}" - "traefik.http.routers.mangaweb.service=mangabm" - # manga-api only joins `proxy` in this override (see above), so headless-shell - # has to follow it there too or DNS resolution of headless-shell:9222 breaks. - headless-shell: - networks: - - proxy + # headless-shell is untouched here: it keeps its `browser` network membership + # from the base file and must never join `proxy` — that network is shared + # with whatever else sits behind Traefik on this host, and an exposed + # CDP endpoint on it would be remote code execution for any of them. networks: proxy: diff --git a/docker-compose.yml b/docker-compose.yml index 304ea69..3d53280 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -45,6 +45,8 @@ services: # the public internet does not. ports: - "127.0.0.1:8080:8080" + networks: + - browser headless-shell: image: chromedp/headless-shell:stable @@ -55,12 +57,20 @@ services: # container's lifetime. init: true # Deliberately no `ports:` — an exposed CDP endpoint is remote code - # execution. Only the backend on the internal network may reach it. + # execution. Only manga-api, via the `browser` network below, may reach it. command: - --remote-debugging-address=0.0.0.0 - --remote-debugging-port=9222 - --disable-gpu - --no-sandbox + networks: + - browser volumes: bookmarks-data: + +networks: + # Not `internal: true`: headless Chrome still needs outbound access to reach + # kagane.to. Isolation here comes from membership (only manga-api and + # headless-shell join it), not from cutting egress. + browser: From 877d3df010f180dbf46dc97f8b6936c66864cdb6 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 17:55:50 +0700 Subject: [PATCH 14/18] feat(web): add comix and kagane site colours --- backend/internal/web/static/style.css | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/backend/internal/web/static/style.css b/backend/internal/web/static/style.css index c1e8ce7..6b6a9fe 100644 --- a/backend/internal/web/static/style.css +++ b/backend/internal/web/static/style.css @@ -94,6 +94,8 @@ --asura: #7d93a5; --demonic: #a98a78; + --comix: #8a9a7d; + --kagane: #9a8aa5; /* Covers are often missing; the hatch keeps the slot honest instead of faking artwork. */ @@ -146,6 +148,8 @@ --asura: #4f6b80; --demonic: #8a6a55; + --comix: #5f7250; + --kagane: #6f5f7d; --hatch: repeating-linear-gradient(135deg, #e6e0d8 0 5px, #efeae3 5px 10px); --hatch-dim: repeating-linear-gradient(135deg, #ebe6de 0 5px, #f2eee8 5px 10px); @@ -479,11 +483,14 @@ button { cursor: pointer; } .meta .sep { color: var(--faint); } .site-asura { color: var(--asura); } .site-demonic { color: var(--demonic); } +.site-comix { color: var(--comix); } +.site-kagane { color: var(--kagane); } .new-chapter { color: var(--ember); } .state { display: flex; align-items: center; gap: 4px; color: var(--mute); } .state svg { width: 10px; height: 10px; } .is-dim .meta { color: var(--mute-2); } -.is-dim .site-asura, .is-dim .site-demonic { color: var(--mute); filter: grayscale(.6); } +.is-dim .site-asura, .is-dim .site-demonic, +.is-dim .site-comix, .is-dim .site-kagane { color: var(--mute); filter: grayscale(.6); } /* ---- action strip: full-width on a phone, hairline-divided cells ---- */ .actions { From ba23411a7444120e57a3f7fbfc442542ffe07645 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 18:21:25 +0700 Subject: [PATCH 15/18] fix: address issues found in end-to-end verification Chained defects made the kagane browser-fetch path completely non-functional in Docker Compose: headless-shell's compose command re-declared --remote-debugging-port, colliding with the image's own entrypoint/socat proxy (EOF on every dial); the sidecar was then only reachable by Docker DNS name, which Chrome's DevTools HTTP handler rejects with a 500 (Host-header/DNS-rebinding check); and NewBrowserFetcher's NoModifyURL option skipped /json/version discovery entirely, dialing a bare host:port that Chrome 404s since /devtools/browser/<uuid> is minted fresh per Chrome start. Fixed by trimming the redundant command flags, pinning headless-shell to a static IP so BROWSER_WS_URL can name it directly, and removing NoModifyURL so chromedp's discovery (which echoes the request's Host back into webSocketDebuggerUrl) does the right thing on its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --- backend/internal/latest/browser.go | 22 +++++++++++++++------- docker-compose.prod.yml | 5 ++++- docker-compose.yml | 26 ++++++++++++++++++++++---- 3 files changed, 41 insertions(+), 12 deletions(-) diff --git a/backend/internal/latest/browser.go b/backend/internal/latest/browser.go index 81c2707..10bcf21 100644 --- a/backend/internal/latest/browser.go +++ b/backend/internal/latest/browser.go @@ -46,18 +46,26 @@ type BrowserFetcher struct { var _ Fetcher = (*BrowserFetcher)(nil) -// NewBrowserFetcher connects to a headless-shell over CDP. wsURL is the -// container's websocket endpoint, e.g. ws://headless-shell:9222. +// NewBrowserFetcher connects to a headless-shell over CDP. wsURL must name the +// sidecar by IP, e.g. ws://172.28.0.10:9222 — not by Docker DNS name. Chrome's +// DevTools HTTP handler 500s any /json/version request whose Host header +// isn't an IP or "localhost" (confirmed 2026-08-03 against +// chromedp/headless-shell:stable), so the compose network pins the sidecar's +// address for this to resolve at all. // -// NoModifyURL is load-bearing: /json/version advertises a -// webSocketDebuggerUrl pointing at 127.0.0.1, which is meaningless from another -// container, and without this option chromedp follows it and hangs. +// Do not add chromedp.NoModifyURL here: that option skips the /json/version +// discovery request entirely and dials wsURL as if it were already the full +// debugger endpoint, but Chrome only accepts connections at +// /devtools/browser/<uuid>, a path chosen fresh at every Chrome start — dialing +// the bare host:port 404s. The default (discovery) path works precisely +// because Chrome's /json/version response echoes back the Host header of the +// discovery request in webSocketDebuggerUrl, so as long as wsURL is a +// container-reachable IP, the URL chromedp gets back already points at it. func NewBrowserFetcher(wsURL string) (*BrowserFetcher, error) { if wsURL == "" { return nil, fmt.Errorf("empty browser websocket url") } - ctx, cancel := chromedp.NewRemoteAllocator( - context.Background(), wsURL, chromedp.NoModifyURL) + ctx, cancel := chromedp.NewRemoteAllocator(context.Background(), wsURL) return &BrowserFetcher{allocCtx: ctx, cancel: cancel}, nil } diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml index f27f172..e58e286 100644 --- a/docker-compose.prod.yml +++ b/docker-compose.prod.yml @@ -18,7 +18,10 @@ services: # Traffic arrives over the Traefik network, not a published port. ports: !reset [] environment: - BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://headless-shell:9222} + # Must be an IP, not the DNS name — see the base file's comment on this + # same key: Chrome's DevTools HTTP handler 500s any Host header that + # isn't an IP or "localhost". + BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://172.28.0.10:9222} depends_on: - headless-shell # `networks:` here replaces the base file's list entirely, so both must be diff --git a/docker-compose.yml b/docker-compose.yml index 3d53280..75e49df 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -31,7 +31,13 @@ services: LATEST_CHAPTER_POLL_STAGGER: ${LATEST_CHAPTER_POLL_STAGGER:-20s} # CDP endpoint for sites behind a JavaScript challenge (kagane). Unset # disables browser polling for those sites; the userscript still covers them. - BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://headless-shell:9222} + # Must be an IP, not the "headless-shell" DNS name: Chrome's DevTools HTTP + # handler rejects the discovery request (GET /json/version) with a 500 + # unless the Host header is an IP address or "localhost" — confirmed + # 2026-08-03 against chromedp/headless-shell:stable, independent of + # chromedp's own dial logic. The sidecar's static address below exists so + # this URL survives container recreation. + BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://172.28.0.10:9222} depends_on: - headless-shell volumes: @@ -58,13 +64,22 @@ services: init: true # Deliberately no `ports:` — an exposed CDP endpoint is remote code # execution. Only manga-api, via the `browser` network below, may reach it. + # Don't pass --remote-debugging-address/--remote-debugging-port here: the + # image's own entrypoint (/headless-shell/run.sh) already starts Chrome on + # 127.0.0.1:9223 and fronts it with a socat proxy listening on 0.0.0.0:9222. + # Redeclaring the port flag here overrides Chrome's, so it binds 9222 + # directly (IPv6 loopback only) instead of 9223 — collides with socat's own + # bind on 9222 and leaves nothing listening on 9223, so every external + # connection to headless-shell:9222 fails with EOF. Only pass flags the + # entrypoint doesn't already set. command: - - --remote-debugging-address=0.0.0.0 - - --remote-debugging-port=9222 - --disable-gpu - --no-sandbox networks: - - browser + browser: + # Pinned so BROWSER_WS_URL can name an IP (required, see above) that + # survives `docker compose up` recreating this container. + ipv4_address: 172.28.0.10 volumes: bookmarks-data: @@ -74,3 +89,6 @@ networks: # kagane.to. Isolation here comes from membership (only manga-api and # headless-shell join it), not from cutting egress. browser: + ipam: + config: + - subnet: 172.28.0.0/24 From d8c607455931733eb633b542d6a536b48552a2e9 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 18:32:49 +0700 Subject: [PATCH 16/18] fix: extend CORS origins to comix and kagane, preserve progress on unparseable chapters - .env.example, DEPLOY.md, docker-compose.yml: add comix.to/kagane.to to ALLOWED_ORIGINS so the userscript isn't CORS-blocked on either new site - .env.example: comment out BROWSER_WS_URL's DNS-name default, which overrides the working compose default and 500s Chrome's DevTools handler - userscript: updateToCurrentChapter() now falls back to the stored chapter/label when chapterNum is unparseable, instead of wiping progress (kagane's og:title lacks a number when a chapter has no episode suffix) - README.md: document comix/kagane in the config table, adapter reference, and key examples --- .env.example | 10 +++++++--- DEPLOY.md | 2 +- README.md | 10 +++++++--- docker-compose.yml | 2 +- userscript/manga-bookmark.user.js | 4 ++-- 5 files changed, 18 insertions(+), 10 deletions(-) diff --git a/.env.example b/.env.example index 68000f1..1fb87cc 100644 --- a/.env.example +++ b/.env.example @@ -5,8 +5,8 @@ API_TOKEN=changeme-generate-a-long-random-token # Comma-separated origins allowed to call the API (CORS). Both Asura domains -# plus Demonic. Add/remove as the sites' hostnames change. -ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org +# plus Demonic, Comix, and Kagane. Add/remove as the sites' hostnames change. +ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to # --- Prod override (Traefik) only --- # Subdomain Traefik routes to this service (required by the prod override). @@ -55,4 +55,8 @@ WEB_PASSWORD= # Headless-shell CDP endpoint for sites behind a JavaScript challenge (kagane). # Unset disables browser polling; those sites then rely on the userscript alone. -BROWSER_WS_URL=ws://headless-shell:9222 +# Leave commented — the compose files' own default (ws://172.28.0.10:9222) is +# correct. Do NOT set this to the "headless-shell" DNS name: Chrome's DevTools +# HTTP handler 500s any /json/version request whose Host header isn't an IP or +# "localhost", which silently breaks every kagane poll. +# BROWSER_WS_URL=ws://172.28.0.10:9222 diff --git a/DEPLOY.md b/DEPLOY.md index 8f82aea..eb3f5f5 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -36,7 +36,7 @@ Edit `.env`: API_TOKEN=<paste output of: openssl rand -hex 32> # CORS allowlist — leave as-is unless a site changes hostname. -ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org +ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to # Required for the Traefik override. Both have no fallback — compose refuses # to start without them. MANGA_WEB_HOST is required even if you never set diff --git a/README.md b/README.md index 6ca2428..95be8e6 100644 --- a/README.md +++ b/README.md @@ -26,9 +26,10 @@ Bromite userscript (isolated world, Shadow DOM UI, localStorage cache) | Var | Default | Notes | |-----|---------|-------| | `API_TOKEN` | *(required)* | Bearer token shared with the userscript. | -| `ALLOWED_ORIGINS` | Asura + Demonic origins | Comma-separated CORS allowlist. | +| `ALLOWED_ORIGINS` | Asura + Demonic + Comix + Kagane origins | Comma-separated CORS allowlist. | | `DB_PATH` | `/data/bookmarks.db` | SQLite file location. | | `PORT` | `8080` | Plain HTTP; TLS terminated by the proxy. | +| `BROWSER_WS_URL` | `ws://172.28.0.10:9222` | Headless-shell CDP endpoint used to poll Kagane past its JS challenge. Must be an IP or `localhost` — Chrome's DevTools handler 500s any other Host header. | ### Endpoints @@ -40,8 +41,9 @@ Bromite userscript (isolated world, Shadow DOM UI, localStorage cache) | `GET` | `/healthz` | none | `200 ok`. | | `GET` | `/u/{token}/manga-bookmark.user.js` | token in path | Serves the userscript with an mtime-derived `@version`. | -`key` is `<site>:<series_id>` — e.g. `asura:trash-of-the-counts-family-f886a8af` -or `demonic:Infinite-Level-Up-in-Murim`. Sync is last-write-wins. +`key` is `<site>:<series_id>` — e.g. `asura:trash-of-the-counts-family-f886a8af`, +`demonic:Infinite-Level-Up-in-Murim`, `comix:12345`, or +`kagane:3fa85f64-5717-4562-b3fc-2c963f66afa6`. Sync is last-write-wins. `updated_at` orders the bookmark list, so it moves only on real reading progress: the server applies its timestamp when the row is new or @@ -200,6 +202,8 @@ The site adapters key everything off URL regex, with `title`/`cover` from |------|-----------|-------------|-------------| | **Asura** (`asurascans.com`) | `/comics/<slug-hash>` | `/comics/<slug-hash>/chapter/<n>` | `<slug-hash>` | | **Demonic** (`demonicscans.org`) | `/manga/<slug>` | `/title/<slug>/chapter/<n>/<page>` (`chaptered.php?manga=<id>&chapter=<n>` 301s here) | `<slug>` | +| **Comix** (`comix.to`) | `/title/<id>-<slug>` | `/title/<id>-<slug>/<uploadId>-chapter-<n>` | `<id>` | +| **Kagane** (`kagane.to`) | `/series/<uuid>` | `/series/<uuid>/reader/<bookUuid>` | `<uuid>` | Notes: - **`asuracomic.net` deep links are dead (re-checked 2026-07-25).** They 301 to diff --git a/docker-compose.yml b/docker-compose.yml index 75e49df..a819f16 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -15,7 +15,7 @@ services: environment: # API_TOKEN is required — compose refuses to start without it. API_TOKEN: ${API_TOKEN:?set API_TOKEN in .env} - ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-https://asuracomic.net,https://asurascans.com,https://demonicscans.org} + ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to} DB_PATH: /data/bookmarks.db PORT: "8080" # Gates the browser UI. Unset means the web routes are not served at all. diff --git a/userscript/manga-bookmark.user.js b/userscript/manga-bookmark.user.js index 4e4e51d..02b31d5 100644 --- a/userscript/manga-bookmark.user.js +++ b/userscript/manga-bookmark.user.js @@ -838,8 +838,8 @@ title: existing.title || p.title || p.seriesId, series_url: existing.series_url || p.seriesUrl || "", cover: existing.cover || p.cover || "", - last_chapter: p.chapterLabel || "", - last_chapter_num: p.chapterNum, + last_chapter: p.chapterLabel || existing.last_chapter || "", + last_chapter_num: p.chapterNum != null ? p.chapterNum : existing.last_chapter_num, last_chapter_url: p.chapterUrl || "", updated_at: Date.now(), }); From 0725b11275c0cc42950935e5ee9a5e3aa80f3de9 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Mon, 3 Aug 2026 19:52:40 +0700 Subject: [PATCH 17/18] docs: sync AGENTS.md to current architecture (internal/, Cinder, confirm-gated, edge-tab) Captures what shipped on the branch: - backend split into internal/ packages; composition root = main.go - web UI go:embed now lives under internal/web/; Dockerfile must copy tree - impeccable detector caveat (root-absolute /static/ paths) and false-clean - confirm-row pattern for archive/finish/remove; --ember reserved - edge-tab hitbox design (7x44 visible, 28x72 hit, touch-action + arm hold) - Cinder design system section + ember-law reference --- AGENTS.md | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 76 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index fedf19f..f03f720 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,17 +27,43 @@ Bromite userscript (isolated world, per-site adapters, localStorage cache) ``` - **Backend** (`backend/`): stdlib `net/http` (handful of routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`. + Single binary, split into packages under `backend/internal/`: `store` + (Bookmark type, SQLite persistence, migrations), `latest` (background + poller, site parsers, TLS fetcher), `session` (cookie signing, login + rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON + bookmark handlers), `userscript` (userscript-serving handler), `web` + (browser UI handler + `templates/` + `static/`, `go:embed`-ed). + `backend/main.go` is the composition root — the only place that wires + packages together into `newRouter`. Root-level `*_test.go` hold + integration tests that exercise the full router; unit tests for a + package live beside it under `internal/`. - **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`). Sync **last-write-wins**. Schema + endpoint list in plan. - **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). - **Web UI:** same binary serves password-gated browser UI on second hostname — `GET /` (list, or login page when no session), `POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints - under `/ui/*`. Templates/assets `go:embed`-ed, so `backend/Dockerfile` - must copy `templates/` and `static/` plus `*.go`. Sessions = stateless + under `/ui/*`. Templates + assets `go:embed`-ed under + `backend/internal/web/`, so `backend/Dockerfile` must copy the whole + `internal/` tree, not just `*.go`. Sessions = stateless HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, when empty web routes not registered at all. UI mutations read-modify-write through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`. + **Design-tool caveat:** templates link `/static/style.css` root-absolutely + (correct — served from `/`), but impeccable detector resolves + stylesheet href with `path.resolve(fileDir, href)`, drops directory + on leading `/` and silently skips file. Relative hrefs don't help + either: template's directory isn't its served path. So + `detect.mjs backend/internal/web/templates` reports **false clean** — + always pass `backend/internal/web/static` too. One finding there, + `overused-font` on "Instrument Serif", deliberate identity choice, not debt. +- **Every action that moves series out of list is confirm-gated.** + Archive, finish, remove each open own `.confirm-row` disclosure + (`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈ + `archive|finish|remove`); restore fires instantly since it's the reversal. + Remove's row wears ember wash, two reversible ones wear `.calm` grey. + `--ember` stays reserved for new-chapter signal: busy bar and inline + error use `--mute`. - **Latest-chapter poller:** ticker goroutine in same binary re-checks each bookmarked series' newest published chapter from backend's own network access, so `latest_chapter` stays fresh when user not @@ -96,7 +122,14 @@ Bromite userscript (isolated world, per-site adapters, localStorage cache) 5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS (critical on mobile). Three tabs (All / Favourites / Archived) + row of link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG - block next to `API_BASE`. + block next to `API_BASE`. FAB is `7 × 44` edge tab whose *hit* area + widened to `28 × 72` by invisible `#hit` child; `#fab` must keep + `touch-action: none` and must **not** regain `overflow: hidden`. Since + `touch-action` resolved at gesture start, strip can't be both + browser-scrolled and script-dragged, so `makeDraggable` splits by intent: swipe + from `#hit` scrolls via `window.scrollBy`, hold of `ARM_MS` arms + reposition drag, visible sliver drags with no hold. See + `docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`. 6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fires w/o reload. Demonic uses classic reloads (initial `document-idle` run suffices). ### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trusting) @@ -135,15 +168,55 @@ Smoke test: `curl` endpoints w/ `Authorization: Bearer <token>`; confirm `OPTION `tea` prints output as rendered boxes not plain text; PR URL lands on last line. +## Design system + +Web UI + userscript panel follow **Cinder**, rules in `docs/design-system.md` +— source of truth Claude Design project `mangaBookmark Web UI` +(`969ac210-fe02-4c01-ae1b-9a271dcc779a`). Read it before touching +`backend/internal/web/static/style.css`, `backend/internal/web/templates/*`, or userscript +`TEMPLATE`/`CSS`. Core law: **ember means new chapter only** — no other +state (busy, error, destruction) may use `--ember`; destruction gets +`--danger`. No cards/corners/shadows, one `--measure: 760px` column, tokens +only (never hardcode hex outside `:root`), both colour branches touched +together. Any move that pulls series out of list (archive/finish/remove) +must be confirm-gated via its own `.confirm-row`; only restore fires +instantly. + ## Security invariants - Auth on `/bookmarks*`: require `Authorization: Bearer <API_TOKEN>`, **constant-time compare**, 401 otherwise. - CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` w/ `204`. +## Comments + +Comment only if code alone can't carry info. Cost per read — must earn spot. + +Write for: +- Why not what. Tradeoffs, non-obvious decisions. +- Load-bearing detail looking incidental — say so if "simplify" breaks it. +- Non-local consequence, invisible from function alone. +- Wire format / encoding / interface contract — save callers re-deriving. +- Gotcha/workaround, with ref if exists. +- Domain/business rule not derivable from code. + +Skip: +- Restating code (no `// increment i` above `i++`). +- Trivial getter/setter/pass-through. +- Banners, dividers, `// helpers`. +- Change narration (`// fix bug`, `// as requested`, `// new impl`) — git's job. +- Commented-out code — delete. +- TODO without concrete action. + +Style: one dense comment over function beats one per line inside. Tight, no worked example unless bug subtle. Wrong comment worse than none — update/delete on change. Default fewer — sparse+high-signal beats comprehensive. + +Test: "competent reader get this from code in few sec?" Yes → skip. Needs detour through another file/spec/git-blame → write it. + ## Relevant skills `multi-stage-dockerfile` and `docker-compose-orchestration` for container work (referenced in plan). +`golang-code-style`, `golang-error-handling`, `golang-performance`, `golang-testing` for backend Go work. + ## graphify Project has knowledge graph at graphify-out/ w/ god nodes, community structure, cross-file relationships. From 4ee0b0f092d573316ff1f6fe8ad7beaf9f93c1d8 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki <sultankiki05@gmail.com> Date: Tue, 4 Aug 2026 19:51:33 +0700 Subject: [PATCH 18/18] docs: split CLAUDE.md into per-directory guidance, add opencode agents Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --- .opencode/agent/implementer.md | 47 ++++++++++++ .opencode/agent/reviewer.md | 69 +++++++++++++++++ CLAUDE.md | 131 +-------------------------------- backend/CLAUDE.md | 81 ++++++++++++++++++++ userscript/CLAUDE.md | 46 ++++++++++++ 5 files changed, 245 insertions(+), 129 deletions(-) create mode 100644 .opencode/agent/implementer.md create mode 100644 .opencode/agent/reviewer.md create mode 100644 backend/CLAUDE.md create mode 100644 userscript/CLAUDE.md diff --git a/.opencode/agent/implementer.md b/.opencode/agent/implementer.md new file mode 100644 index 0000000..e3f79f1 --- /dev/null +++ b/.opencode/agent/implementer.md @@ -0,0 +1,47 @@ +--- +description: Code-writer subagent for subagent-driven development. Fast model (ocg/deepseek-v4-flash) for mechanical, well-specified implementation tasks. Escalates complicated tasks so the controller can re-dispatch on minimax-m3. +mode: subagent +model: 9router/ocg/deepseek-v4-flash +--- + +You are the implementer subagent for Subagent-Driven Development. You implement one task, exactly as specified, and report back with evidence. + +## Before You Begin + +If you have questions about requirements, acceptance criteria, approach, dependencies, or anything unclear in the task description — ask now. Raise concerns before starting work. Don't guess or make assumptions. + +## Your Job + +1. Implement exactly what the task specifies +2. Write tests (follow TDD when the task says to) +3. Verify the implementation works (run the focused test while iterating; run the full suite once before committing) +4. Commit your work +5. Self-review (below) +6. Report back + +Follow existing patterns in the codebase. Don't restructure code outside your task. Don't overbuild — only what was requested (YAGNI). + +## When You're in Over Your Head + +It is always OK to stop and say "this is too hard for me." Bad work is worse than no work. STOP and escalate when the task requires architectural judgment, multi-file integration you can't see clearly through, or you're reading file after file without progress. + +**Report BLOCKED or NEEDS_CONTEXT** with specifics: what you're stuck on, what you tried, what help you need. If the task turns out more complicated than mechanical (design judgment, broad codebase understanding), escalate so the controller can re-dispatch you on the more capable minimax-m3 agent. + +## Self-Review Before Reporting + +- **Completeness:** everything in the spec implemented? edge cases handled? +- **Quality:** names accurate? code clean and maintainable? +- **Discipline:** avoided overbuilding? only what was requested? +- **Testing:** do tests verify real behavior? output pristine (no stray warnings)? +Fix what you find before reporting. + +## Report Format + +Report back with ONLY (under 15 lines): +- **Status:** DONE | DONE_WITH_CONCERNS | BLOCKED | NEEDS_CONTEXT +- Commits created (short SHA + subject) +- One-line test summary (e.g. "14/14 passing, output pristine") +- Concerns, if any +- Report file path (if the controller gave you one) + +If BLOCKED or NEEDS_CONTEXT, put the specifics in the final message itself — the controller acts on it directly. Use DONE_WITH_CONCERNS if you completed the work but have doubts. Never silently produce work you're unsure about. diff --git a/.opencode/agent/reviewer.md b/.opencode/agent/reviewer.md new file mode 100644 index 0000000..608f09a --- /dev/null +++ b/.opencode/agent/reviewer.md @@ -0,0 +1,69 @@ +--- +description: Reviewer subagent for subagent-driven development. Capable model (ocg/minimax-m3) for task-scoped and whole-branch code review; also the re-dispatch target when implementation tasks are complicated. +mode: subagent +model: 9router/ocg/minimax-m3 +--- + +You are the reviewer subagent for Subagent-Driven Development. You verify one task's implementation matches its requirements (spec compliance) and is well-built (code quality). You may also be dispatched for whole-branch review. + +## Inputs + +- Task brief file (requirements — use exact values verbatim) +- Implementer's report file +- Diff file (commit list, stat summary, full diff with context) + +## Method + +Read the diff file once — it is your view of the change. The context lines ARE the changed files: do not read a changed file separately unless a hunk you must judge is cut off mid-function (say so in your report). Do not re-run git commands. Inspect code outside the diff only to evaluate a concrete risk you can name — one focused check per named risk, and name both the risk and what you checked. + +Your review is read-only. Do not mutate the working tree, index, HEAD, or branch state. + +## Do Not Trust the Report + +Treat the implementer's report as unverified claims. It may be incomplete, inaccurate, or optimistic. Verify against the diff. Design rationales in the report ("kept it per YAGNI") are the implementer grading their own work — a stated rationale never downgrades a finding's severity. + +## Tests + +The implementer already ran the tests and reported results. Do not re-run the suite to confirm. Run a test only when reading the code raises a specific doubt no existing run answers — a focused test, never a package-wide suite. If heavy validation seems warranted, recommend it in your report instead. Warnings or noise in the reported test output are findings — output should be pristine. + +## Part 1: Spec Compliance + +Compare the diff against the brief: +- **Missing:** requirements skipped, missed, or claimed without implementing +- **Extra:** features not requested, over-engineering, nice-to-haves +- **Misunderstood:** right feature built the wrong way, wrong problem solved + +If a requirement can't be verified from this diff alone (lives in unchanged code or spans tasks), report it as a ⚠️ item instead of broadening your search. + +## Part 2: Code Quality + +- Clean separation of concerns? proper error handling? DRY without premature abstraction? edge cases? +- Do new/changed tests verify real behavior, not mocks? edge cases covered? +- Does each file have one clear responsibility? units independently testable? did this change create/significantly grow large files? + +Point at evidence: file:line references for every finding. A tight report that cites lines gives the controller everything it needs. + +## Calibration + +Not everything is Critical. Important = this task can't be trusted until fixed: incorrect or fragile behavior, a missed requirement, maintainability damage you'd block a merge over (verbatim duplication of a logic block, swallowed errors, tests that assert nothing). "Coverage could be broader" and polish suggestions are Minor. If the plan explicitly mandates something this rubric calls a defect, that IS a finding — report Important, labeled plan-mandated. Acknowledge what was done well before listing issues. + +## Output Format + +### Spec Compliance +- ✅ Spec compliant | ❌ Issues found: [what's missing/extra/misunderstood, with file:line] +- ⚠️ Cannot verify from diff: [requirements you couldn't verify, what the controller should check] + +### Strengths +[What's well done? Be specific.] + +### Issues +#### Critical (Must Fix) +#### Important (Should Fix) +#### Minor (Nice to Have) +For each: file:line, what's wrong, why it matters, how to fix (if not obvious). + +### Assessment +**Task quality:** [Approved | Needs fixes] +**Reasoning:** [1-2 sentence technical assessment] + +Your final message is the report itself: begin directly with the spec-compliance verdict. Every line is a verdict, a finding with file:line, or a check you ran — no preamble, no process narration, no closing summary. diff --git a/CLAUDE.md b/CLAUDE.md index 2dc42d4..2363ed0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,10 +2,6 @@ Guidance for Claude Code (claude.ai/code) working in this repo. -## Status - -Greenfield. Only `plans/mangaBookmark.md` exist — no code yet. Plan = spec; read before build. Two deliverables: Go sync backend, single Violentmonkey-compatible userscript. - ## What this is Manga read-progress tracker, user read on **asurascans.com** (current domain; asuracomic.net 301s here) and **demonicscans.org** via **Violentmonkey**. Userscript inject on-page UI (floating button + slide-in panel), sync progress to self-hosted Go backend so bookmarks unify across both sites and devices. @@ -26,132 +22,9 @@ Violentmonkey userscript (isolated world, per-site adapters, localStorage cache) -- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume) ``` -- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`. - Single binary, split into packages under `backend/internal/`: `store` - (Bookmark type, SQLite persistence, migrations), `latest` (background - poller, site parsers, TLS fetcher), `session` (cookie signing, login - rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON - bookmark handlers), `userscript` (userscript-serving handler), `web` - (browser UI handler + `templates/` + `static/`, `go:embed`-ed). - `backend/main.go` is the composition root — the only place that wires - packages together into `newRouter`. Root-level `*_test.go` hold - integration tests that exercise the full router; unit tests for a - package live beside it under `internal/`. -- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`|`comix`|`kagane`). Sync **last-write-wins**. Schema and endpoint list in plan. -- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). -- **Web UI:** same binary serve password-gated browser UI on second - hostname — `GET /` (list, or login page when no session), - `POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints - under `/ui/*`. Templates + assets `go:embed`-ed under - `backend/internal/web/`, so `backend/Dockerfile` must copy the whole - `internal/` tree, not just `*.go`. Sessions stateless - HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, and when empty, - web routes not registered at all. UI mutations read-modify-write - through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one - place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`. - **Design-tool caveat:** templates link `/static/style.css` root-absolutely - (correct — served from `/`), but impeccable detector resolves - stylesheet href with `path.resolve(fileDir, href)`, drops directory - on leading `/` and silently skip file. Relative href don't help - either: template's directory isn't its served path. So - `detect.mjs backend/internal/web/templates` reports **false clean** — - always pass `backend/internal/web/static` too. One finding there, - `overused-font` on "Instrument Serif", deliberate identity choice, not debt. -- **Every action that moves series out of list is confirm-gated.** - Archive, finish, remove each open own `.confirm-row` disclosure - (`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈ - `archive|finish|remove`); restore fire instantly since it's the reversal. - Remove's row wear ember wash, two reversible ones wear `.calm` grey. - `--ember` stay reserved for new-chapter signal: busy bar and inline - error use `--mute`. -- **Latest-chapter poller:** ticker goroutine in same binary re-check - each bookmarked series' newest published chapter from backend's own - network access, so `latest_chapter` stay fresh when user not - browsing. Second, parallel signal — userscript keep own - `maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged. - Two independent clocks: per-bookmark cooldown (`latest_checked_at` column, - enforced by `Store.DueForLatestCheck`'s WHERE clause) and wake interval. - Row stamped *before* fetch so broken series wait out full - cooldown instead of retrying every tick, and writes go through - `Store.Get` + `Store.Upsert` so new chapter never reorders list. - Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth - against fingerprint-based blocking; any failure log and skip. kagane sits - behind a Cloudflare JavaScript challenge the TLS client can't clear, so it is - browser-only: fetched over CDP via `BROWSER_WS_URL`, and simply not polled - when that's unset. See - `docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`. - Poller's `Store.Get` + `Store.Upsert` not wrapped in transaction, so - userscript `PUT` that commits between the two can get overwritten by - poller's stale re-read — reverting that read progress and, since stored - value now differs, moving `updated_at` and reordering list. Known, - accepted limitation for single-user deployment, not bug to fix. -- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4. -- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` | - `finished`, orthogonal to `favorite`. Archived and finished appear only in - own tab — not in All, Updated, Favourites, or recent strip. Poller keeps - checking archived series and skip finished ones. `finished` settable - only from web UI; `PUT /bookmarks/{key}` reject it with 400. - **Empty incoming status means "keep stored one"** — resolved on the - `VALUES` side of `Store.Upsert`, not conflict clause, since - `excluded.*` is post-evaluation row and default applied there would - wipe bucket on every PUT from client that predates column. See - `docs/superpowers/specs/2026-07-27-status-buckets-design.md`. -- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list), `DB_PATH` - (default `/data/bookmarks.db`), `PORT` (default `8080`), `WEB_PASSWORD` - (gates browser UI; unset disable it), - `LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` - (background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`). - `USERSCRIPT_PATH` (file served at `/u/{token}/manga-bookmark.user.js`, - default `/userscript/manga-bookmark.user.js`, supplied by bindmount). - `BROWSER_WS_URL` (headless-shell CDP endpoint for kagane; unset disables - browser polling and leaves that site to the userscript alone). +Backend-specific architecture (packages, endpoints, poller, config env vars) lives in `backend/CLAUDE.md`. Userscript-specific structure (adapters, retry queue, UI, live URL shapes) lives in `userscript/CLAUDE.md`. -### Userscript structure (single IIFE, `manga-bookmark.user.js`) - -1. **Site adapters** — one per host, `detect(location, document)` return page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title`/`cover` from **`og:title`/`og:image` meta tags**, not CSS classes. -2. **API client** — `apiGet/apiPut/apiDelete` with bearer header; `localStorage` key `mangabm:cache` for instant render + offline fallback. -3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value. -4. **Retry queue** — every write go through `pushBookmark`/`pushDelete`, so - failed mutation park in `localStorage` (`mangabm:queue`) and replayed on - next navigation, reconnect, or `refresh()`. Entries are markers - (`{key, op, sendStatus, attempts}`), never payloads — body read from - cache at send time, so one entry per key give ordering and coalescing for - free. `sendStatus` is **sticky**: while archive pending, later writes to - that key keep carrying bucket, which stop successful - in-between write from silently un-archiving series. `refresh()` drains - before it fetches and overlays anything still pending, so list never - flaps. 400 drops entry, 401 abort pass and keep queue, and - transient failures retry to cap of 10. Latest-chapter writes deliberately - stay out of queue. See - `docs/superpowers/specs/2026-07-27-offline-retry-queue-design.md`. -5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS - (critical on mobile). Three tabs (All / Favourites / Archived) and row of - link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG - block next to `API_BASE`. FAB is `7 × 44` edge tab whose *hit* area - widened to `28 × 72` by invisible `#hit` child; `#fab` must keep - `touch-action: none` and must **not** regain `overflow: hidden`. Since - `touch-action` resolved at gesture start, strip can't be both - browser-scrolled and script-dragged, so `makeDraggable` splits by intent: swipe - from `#hit` scrolls via `window.scrollBy`, hold of `ARM_MS` arms - reposition drag, visible sliver drags with no hold. See - `docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`. -6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fire without reload. Demonic uses classic reloads (initial `document-idle` run suffice). - -### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust) - -- **asurascans.com**: series `/comics/<slug>` (slug carries trailing - site-wide build-hash suffix, e.g. `-059befe1`, that **rotates on every - redeploy**), chapter `/comics/<slug>/chapter/<n>`. `seriesId` must strip - hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in userscript, - `asuraBuildHash` in backend); URLs keep full slug — stale-hash - URLs 302 to current ones. Astro-rendered; chapter links present in raw - server HTML. -- **demonicscans.org**: series `/manga/<slug>` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title/<slug>/chapter/<n>/<page>` (older `chaptered.php?manga=<id>&chapter=<n>` form still exists as redirect, what series-page chapter-list anchors link through). - Encodings (incl. triple-encoded punctuation like `%25252D`) identical - on /manga/ and /title/ pages, so decode-once seriesIds match — verified - 2026-07-28. - -## Commands (once code exists) +## Commands Backend (`cd backend`): - Test all: `go test ./...` diff --git a/backend/CLAUDE.md b/backend/CLAUDE.md new file mode 100644 index 0000000..83d17e8 --- /dev/null +++ b/backend/CLAUDE.md @@ -0,0 +1,81 @@ +Guidance for Claude Code working under `backend/`. See root `CLAUDE.md` for the project-wide architecture diagram, hard constraints, and design system. + +- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`. + Single binary, split into packages under `backend/internal/`: `store` + (Bookmark type, SQLite persistence, migrations), `latest` (background + poller, site parsers, TLS fetcher), `session` (cookie signing, login + rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON + bookmark handlers), `userscript` (userscript-serving handler), `web` + (browser UI handler + `templates/` + `static/`, `go:embed`-ed). + `backend/main.go` is the composition root — the only place that wires + packages together into `newRouter`. Root-level `*_test.go` hold + integration tests that exercise the full router; unit tests for a + package live beside it under `internal/`. +- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`|`comix`|`kagane`). Sync **last-write-wins**. Schema and endpoint list in plan. +- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). +- **Web UI:** same binary serve password-gated browser UI on second + hostname — `GET /` (list, or login page when no session), + `POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints + under `/ui/*`. Templates + assets `go:embed`-ed under + `backend/internal/web/`, so `backend/Dockerfile` must copy the whole + `internal/` tree, not just `*.go`. Sessions stateless + HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, and when empty, + web routes not registered at all. UI mutations read-modify-write + through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one + place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`. + **Design-tool caveat:** templates link `/static/style.css` root-absolutely + (correct — served from `/`), but impeccable detector resolves + stylesheet href with `path.resolve(fileDir, href)`, drops directory + on leading `/` and silently skip file. Relative href don't help + either: template's directory isn't its served path. So + `detect.mjs backend/internal/web/templates` reports **false clean** — + always pass `backend/internal/web/static` too. One finding there, + `overused-font` on "Instrument Serif", deliberate identity choice, not debt. +- **Every action that moves series out of list is confirm-gated.** + Archive, finish, remove each open own `.confirm-row` disclosure + (`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈ + `archive|finish|remove`); restore fire instantly since it's the reversal. + Remove's row wear ember wash, two reversible ones wear `.calm` grey. + `--ember` stay reserved for new-chapter signal: busy bar and inline + error use `--mute`. +- **Latest-chapter poller:** ticker goroutine in same binary re-check + each bookmarked series' newest published chapter from backend's own + network access, so `latest_chapter` stay fresh when user not + browsing. Second, parallel signal — userscript keep own + `maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged. + Two independent clocks: per-bookmark cooldown (`latest_checked_at` column, + enforced by `Store.DueForLatestCheck`'s WHERE clause) and wake interval. + Row stamped *before* fetch so broken series wait out full + cooldown instead of retrying every tick, and writes go through + `Store.Get` + `Store.Upsert` so new chapter never reorders list. + Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth + against fingerprint-based blocking; any failure log and skip. kagane sits + behind a Cloudflare JavaScript challenge the TLS client can't clear, so it is + browser-only: fetched over CDP via `BROWSER_WS_URL`, and simply not polled + when that's unset. See + `docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`. + Poller's `Store.Get` + `Store.Upsert` not wrapped in transaction, so + userscript `PUT` that commits between the two can get overwritten by + poller's stale re-read — reverting that read progress and, since stored + value now differs, moving `updated_at` and reordering list. Known, + accepted limitation for single-user deployment, not bug to fix. +- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4. +- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` | + `finished`, orthogonal to `favorite`. Archived and finished appear only in + own tab — not in All, Updated, Favourites, or recent strip. Poller keeps + checking archived series and skip finished ones. `finished` settable + only from web UI; `PUT /bookmarks/{key}` reject it with 400. + **Empty incoming status means "keep stored one"** — resolved on the + `VALUES` side of `Store.Upsert`, not conflict clause, since + `excluded.*` is post-evaluation row and default applied there would + wipe bucket on every PUT from client that predates column. See + `docs/superpowers/specs/2026-07-27-status-buckets-design.md`. +- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list), `DB_PATH` + (default `/data/bookmarks.db`), `PORT` (default `8080`), `WEB_PASSWORD` + (gates browser UI; unset disable it), + `LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` + (background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`). + `USERSCRIPT_PATH` (file served at `/u/{token}/manga-bookmark.user.js`, + default `/userscript/manga-bookmark.user.js`, supplied by bindmount). + `BROWSER_WS_URL` (headless-shell CDP endpoint for kagane; unset disables + browser polling and leaves that site to the userscript alone). diff --git a/userscript/CLAUDE.md b/userscript/CLAUDE.md new file mode 100644 index 0000000..9f1c225 --- /dev/null +++ b/userscript/CLAUDE.md @@ -0,0 +1,46 @@ +Guidance for Claude Code working under `userscript/`. See root `CLAUDE.md` for the project-wide architecture diagram, hard constraints, and design system. + +### Userscript structure (single IIFE, `manga-bookmark.user.js`) + +1. **Site adapters** — one per host, `detect(location, document)` return page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title`/`cover` from **`og:title`/`og:image` meta tags**, not CSS classes. +2. **API client** — `apiGet/apiPut/apiDelete` with bearer header; `localStorage` key `mangabm:cache` for instant render + offline fallback. +3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value. +4. **Retry queue** — every write go through `pushBookmark`/`pushDelete`, so + failed mutation park in `localStorage` (`mangabm:queue`) and replayed on + next navigation, reconnect, or `refresh()`. Entries are markers + (`{key, op, sendStatus, attempts}`), never payloads — body read from + cache at send time, so one entry per key give ordering and coalescing for + free. `sendStatus` is **sticky**: while archive pending, later writes to + that key keep carrying bucket, which stop successful + in-between write from silently un-archiving series. `refresh()` drains + before it fetches and overlays anything still pending, so list never + flaps. 400 drops entry, 401 abort pass and keep queue, and + transient failures retry to cap of 10. Latest-chapter writes deliberately + stay out of queue. See + `docs/superpowers/specs/2026-07-27-offline-retry-queue-design.md`. +5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS + (critical on mobile). Three tabs (All / Favourites / Archived) and row of + link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG + block next to `API_BASE`. FAB is `7 × 44` edge tab whose *hit* area + widened to `28 × 72` by invisible `#hit` child; `#fab` must keep + `touch-action: none` and must **not** regain `overflow: hidden`. Since + `touch-action` resolved at gesture start, strip can't be both + browser-scrolled and script-dragged, so `makeDraggable` splits by intent: swipe + from `#hit` scrolls via `window.scrollBy`, hold of `ARM_MS` arms + reposition drag, visible sliver drags with no hold. See + `docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`. +6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fire without reload. Demonic uses classic reloads (initial `document-idle` run suffice). + +### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust) + +- **asurascans.com**: series `/comics/<slug>` (slug carries trailing + site-wide build-hash suffix, e.g. `-059befe1`, that **rotates on every + redeploy**), chapter `/comics/<slug>/chapter/<n>`. `seriesId` must strip + hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in userscript, + `asuraBuildHash` in backend); URLs keep full slug — stale-hash + URLs 302 to current ones. Astro-rendered; chapter links present in raw + server HTML. +- **demonicscans.org**: series `/manga/<slug>` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title/<slug>/chapter/<n>/<page>` (older `chaptered.php?manga=<id>&chapter=<n>` form still exists as redirect, what series-page chapter-list anchors link through). + Encodings (incl. triple-encoded punctuation like `%25252D`) identical + on /manga/ and /title/ pages, so decode-once seriesIds match — verified + 2026-07-28.