diff --git a/.env.example b/.env.example index 3ea053b..1fb87cc 100644 --- a/.env.example +++ b/.env.example @@ -5,8 +5,8 @@ API_TOKEN=changeme-generate-a-long-random-token # Comma-separated origins allowed to call the API (CORS). Both Asura domains -# plus Demonic. Add/remove as the sites' hostnames change. -ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org +# plus Demonic, Comix, and Kagane. Add/remove as the sites' hostnames change. +ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to # --- Prod override (Traefik) only --- # Subdomain Traefik routes to this service (required by the prod override). @@ -52,3 +52,11 @@ WEB_PASSWORD= # BATCH x (COOLDOWN / INTERVAL) series hold the cooldown cadence — 84 with these # defaults. Beyond that the cadence stretches uniformly rather than breaking; # raise BATCH or lower INTERVAL. Keep BATCH x STAGGER under INTERVAL. + +# Headless-shell CDP endpoint for sites behind a JavaScript challenge (kagane). +# Unset disables browser polling; those sites then rely on the userscript alone. +# Leave commented — the compose files' own default (ws://172.28.0.10:9222) is +# correct. Do NOT set this to the "headless-shell" DNS name: Chrome's DevTools +# HTTP handler 500s any /json/version request whose Host header isn't an IP or +# "localhost", which silently breaks every kagane poll. +# BROWSER_WS_URL=ws://172.28.0.10:9222 diff --git a/AGENTS.md b/AGENTS.md index fedf19f..f03f720 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,17 +27,43 @@ Bromite userscript (isolated world, per-site adapters, localStorage cache) ``` - **Backend** (`backend/`): stdlib `net/http` (handful of routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`. + Single binary, split into packages under `backend/internal/`: `store` + (Bookmark type, SQLite persistence, migrations), `latest` (background + poller, site parsers, TLS fetcher), `session` (cookie signing, login + rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON + bookmark handlers), `userscript` (userscript-serving handler), `web` + (browser UI handler + `templates/` + `static/`, `go:embed`-ed). + `backend/main.go` is the composition root — the only place that wires + packages together into `newRouter`. Root-level `*_test.go` hold + integration tests that exercise the full router; unit tests for a + package live beside it under `internal/`. - **Single-user store.** One `bookmarks` table keyed `:` (`asura`|`demonic`). Sync **last-write-wins**. Schema + endpoint list in plan. - **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). - **Web UI:** same binary serves password-gated browser UI on second hostname — `GET /` (list, or login page when no session), `POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints - under `/ui/*`. Templates/assets `go:embed`-ed, so `backend/Dockerfile` - must copy `templates/` and `static/` plus `*.go`. Sessions = stateless + under `/ui/*`. Templates + assets `go:embed`-ed under + `backend/internal/web/`, so `backend/Dockerfile` must copy the whole + `internal/` tree, not just `*.go`. Sessions = stateless HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, when empty web routes not registered at all. UI mutations read-modify-write through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`. + **Design-tool caveat:** templates link `/static/style.css` root-absolutely + (correct — served from `/`), but impeccable detector resolves + stylesheet href with `path.resolve(fileDir, href)`, drops directory + on leading `/` and silently skips file. Relative hrefs don't help + either: template's directory isn't its served path. So + `detect.mjs backend/internal/web/templates` reports **false clean** — + always pass `backend/internal/web/static` too. One finding there, + `overused-font` on "Instrument Serif", deliberate identity choice, not debt. +- **Every action that moves series out of list is confirm-gated.** + Archive, finish, remove each open own `.confirm-row` disclosure + (`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈ + `archive|finish|remove`); restore fires instantly since it's the reversal. + Remove's row wears ember wash, two reversible ones wear `.calm` grey. + `--ember` stays reserved for new-chapter signal: busy bar and inline + error use `--mute`. - **Latest-chapter poller:** ticker goroutine in same binary re-checks each bookmarked series' newest published chapter from backend's own network access, so `latest_chapter` stays fresh when user not @@ -96,7 +122,14 @@ Bromite userscript (isolated world, per-site adapters, localStorage cache) 5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS (critical on mobile). Three tabs (All / Favourites / Archived) + row of link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG - block next to `API_BASE`. + block next to `API_BASE`. FAB is `7 × 44` edge tab whose *hit* area + widened to `28 × 72` by invisible `#hit` child; `#fab` must keep + `touch-action: none` and must **not** regain `overflow: hidden`. Since + `touch-action` resolved at gesture start, strip can't be both + browser-scrolled and script-dragged, so `makeDraggable` splits by intent: swipe + from `#hit` scrolls via `window.scrollBy`, hold of `ARM_MS` arms + reposition drag, visible sliver drags with no hold. See + `docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`. 6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fires w/o reload. Demonic uses classic reloads (initial `document-idle` run suffices). ### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trusting) @@ -135,15 +168,55 @@ Smoke test: `curl` endpoints w/ `Authorization: Bearer `; confirm `OPTION `tea` prints output as rendered boxes not plain text; PR URL lands on last line. +## Design system + +Web UI + userscript panel follow **Cinder**, rules in `docs/design-system.md` +— source of truth Claude Design project `mangaBookmark Web UI` +(`969ac210-fe02-4c01-ae1b-9a271dcc779a`). Read it before touching +`backend/internal/web/static/style.css`, `backend/internal/web/templates/*`, or userscript +`TEMPLATE`/`CSS`. Core law: **ember means new chapter only** — no other +state (busy, error, destruction) may use `--ember`; destruction gets +`--danger`. No cards/corners/shadows, one `--measure: 760px` column, tokens +only (never hardcode hex outside `:root`), both colour branches touched +together. Any move that pulls series out of list (archive/finish/remove) +must be confirm-gated via its own `.confirm-row`; only restore fires +instantly. + ## Security invariants - Auth on `/bookmarks*`: require `Authorization: Bearer `, **constant-time compare**, 401 otherwise. - CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` w/ `204`. +## Comments + +Comment only if code alone can't carry info. Cost per read — must earn spot. + +Write for: +- Why not what. Tradeoffs, non-obvious decisions. +- Load-bearing detail looking incidental — say so if "simplify" breaks it. +- Non-local consequence, invisible from function alone. +- Wire format / encoding / interface contract — save callers re-deriving. +- Gotcha/workaround, with ref if exists. +- Domain/business rule not derivable from code. + +Skip: +- Restating code (no `// increment i` above `i++`). +- Trivial getter/setter/pass-through. +- Banners, dividers, `// helpers`. +- Change narration (`// fix bug`, `// as requested`, `// new impl`) — git's job. +- Commented-out code — delete. +- TODO without concrete action. + +Style: one dense comment over function beats one per line inside. Tight, no worked example unless bug subtle. Wrong comment worse than none — update/delete on change. Default fewer — sparse+high-signal beats comprehensive. + +Test: "competent reader get this from code in few sec?" Yes → skip. Needs detour through another file/spec/git-blame → write it. + ## Relevant skills `multi-stage-dockerfile` and `docker-compose-orchestration` for container work (referenced in plan). +`golang-code-style`, `golang-error-handling`, `golang-performance`, `golang-testing` for backend Go work. + ## graphify Project has knowledge graph at graphify-out/ w/ god nodes, community structure, cross-file relationships. diff --git a/CLAUDE.md b/CLAUDE.md index e719837..2dc42d4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,137 +1,153 @@ # CLAUDE.md -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +Guidance for Claude Code (claude.ai/code) working in this repo. ## Status -Greenfield. Only `plans/mangaBookmark.md` exists — no code yet. That plan is the spec; read it before building. Two deliverables: a Go sync backend and a single Bromite-compatible userscript. +Greenfield. Only `plans/mangaBookmark.md` exist — no code yet. Plan = spec; read before build. Two deliverables: Go sync backend, single Violentmonkey-compatible userscript. ## What this is -A manga read-progress tracker for a user reading on **asurascans.com** (the current domain; asuracomic.net 301s here) and **demonicscans.org** from **Bromite** (mobile Chromium). A userscript injects on-page UI (floating button + slide-in panel) and syncs progress to a self-hosted Go backend so bookmarks unify across both sites and across devices. +Manga read-progress tracker, user read on **asurascans.com** (current domain; asuracomic.net 301s here) and **demonicscans.org** via **Violentmonkey**. Userscript inject on-page UI (floating button + slide-in panel), sync progress to self-hosted Go backend so bookmarks unify across both sites and devices. -## Hard constraints (these drive the design — do not violate) +## Hard constraints (drive design — don't violate) -Bromite uses Chromium's **native** userscript engine, not Tampermonkey: -- **No `GM_*` APIs anywhere.** No `GM_setValue`/`GM_getValue` (use page `localStorage`), no `GM_registerMenuCommand` (inject on-page UI), no `GM_xmlhttpRequest` for cross-origin (use plain `fetch()`). Keeping the script GM-free also lets it run in desktop Tampermonkey/Violentmonkey for faster iteration. -- Cross-origin `fetch()` works **only** against a CORS-enabled backend. Manga sites are `https://`, so backend **must be HTTPS** (mixed-content block otherwise). -- Asura and Demonic are **separate origins with separate `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync is required, not optional. -- Userscript runs in an **isolated world**, so the embedded API token is safe from the site's JS. -- Cloudflare's block on fetching the manga sites is **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both the CGNAT dev machine *and* the deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. This contradicts an earlier, untested assumption that the CGNAT dev IP would be blocked; it was not, at least on this date. Treat "does curl work right now" as a live, time-varying fact to re-check, not a fixed property of a given machine — Cloudflare's bot scoring can flip a previously-clean IP without notice. Any backend fetcher still needs a graceful-degrade path for when it does get challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, or a direct probe) before finalizing, not assumed from a single earlier test. +Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free where plain web APIs suffice — keeps portability across engines: +- **Avoid `GM_*` unless needed.** Prefer page `localStorage` over `GM_setValue`/`GM_getValue`, on-page UI over `GM_registerMenuCommand`, plain `fetch()` over `GM_xmlhttpRequest` for cross-origin. +- Cross-origin `fetch()` work **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block). +- Asura and Demonic are **separate origins with separate `localStorage`** — shared remote store only way to unify bookmarks. Cloud sync required, not optional. +- Userscript run in **isolated world**, so embedded API token safe from site's JS. +- Cloudflare's block on manga sites **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — Cloudflare's bot scoring can flip previously-clean IP without notice. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, or direct probe) before finalize, not assumed from single earlier test. ## Architecture ``` -Bromite userscript (isolated world, per-site adapters, localStorage cache) +Violentmonkey userscript (isolated world, per-site adapters, localStorage cache) -- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume) ``` -- **Backend** (`backend/`): stdlib `net/http` (a handful of routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). The reverse proxy terminates TLS; the Go service listens plain `:8080`. -- **Single-user store.** One `bookmarks` table keyed `:` (`asura`|`demonic`). Sync is **last-write-wins**. Schema and endpoint list are in the plan. +- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`. + Single binary, split into packages under `backend/internal/`: `store` + (Bookmark type, SQLite persistence, migrations), `latest` (background + poller, site parsers, TLS fetcher), `session` (cookie signing, login + rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON + bookmark handlers), `userscript` (userscript-serving handler), `web` + (browser UI handler + `templates/` + `static/`, `go:embed`-ed). + `backend/main.go` is the composition root — the only place that wires + packages together into `newRouter`. Root-level `*_test.go` hold + integration tests that exercise the full router; unit tests for a + package live beside it under `internal/`. +- **Single-user store.** One `bookmarks` table keyed `:` (`asura`|`demonic`|`comix`|`kagane`). Sync **last-write-wins**. Schema and endpoint list in plan. - **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). -- **Web UI:** the same binary serves a password-gated browser UI on a second - hostname — `GET /` (list, or login page when there is no session), - `POST /login`, `POST /logout`, `GET /static/*`, and htmx fragment endpoints - under `/ui/*`. Templates and assets are `go:embed`-ed, so `backend/Dockerfile` - must copy `templates/` and `static/` as well as `*.go`. Sessions are stateless - HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them and, when empty, - the web routes are not registered at all. UI mutations read-modify-write - through `Store.Get` + `Store.Upsert` so the `updated_at` rule stays in one +- **Web UI:** same binary serve password-gated browser UI on second + hostname — `GET /` (list, or login page when no session), + `POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints + under `/ui/*`. Templates + assets `go:embed`-ed under + `backend/internal/web/`, so `backend/Dockerfile` must copy the whole + `internal/` tree, not just `*.go`. Sessions stateless + HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, and when empty, + web routes not registered at all. UI mutations read-modify-write + through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`. - **Design-tool caveat:** the templates link `/static/style.css` root-absolutely - (correct — they are served from `/`), but the impeccable detector resolves a - stylesheet href with `path.resolve(fileDir, href)`, which drops the directory - on a leading `/` and silently skips the file. A relative href does not help - either: the template's directory is not its served path. So - `detect.mjs backend/templates` reports a **false clean** — always pass - `backend/static` too. Its one finding there, `overused-font` on "Instrument - Serif", is a deliberate identity choice, not debt. -- **Every action that moves a series out of the list is confirm-gated.** - Archive, finish, and remove each open their own `.confirm-row` disclosure + **Design-tool caveat:** templates link `/static/style.css` root-absolutely + (correct — served from `/`), but impeccable detector resolves + stylesheet href with `path.resolve(fileDir, href)`, drops directory + on leading `/` and silently skip file. Relative href don't help + either: template's directory isn't its served path. So + `detect.mjs backend/internal/web/templates` reports **false clean** — + always pass `backend/internal/web/static` too. One finding there, + `overused-font` on "Instrument Serif", deliberate identity choice, not debt. +- **Every action that moves series out of list is confirm-gated.** + Archive, finish, remove each open own `.confirm-row` disclosure (`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈ - `archive|finish|remove`); restore fires instantly because it is the reversal. - Remove's row wears the ember wash, the two reversible ones wear `.calm` grey. - `--ember` stays reserved for the new-chapter signal: busy bar and inline + `archive|finish|remove`); restore fire instantly since it's the reversal. + Remove's row wear ember wash, two reversible ones wear `.calm` grey. + `--ember` stay reserved for new-chapter signal: busy bar and inline error use `--mute`. -- **Latest-chapter poller:** a ticker goroutine in the same binary re-checks - each bookmarked series' newest published chapter from the backend's own - network access, so `latest_chapter` stays fresh when the user is not - browsing. It is a *second, parallel* signal — the userscript keeps its own +- **Latest-chapter poller:** ticker goroutine in same binary re-check + each bookmarked series' newest published chapter from backend's own + network access, so `latest_chapter` stay fresh when user not + browsing. Second, parallel signal — userscript keep own `maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged. - Two independent clocks: a per-bookmark cooldown (`latest_checked_at` column, - enforced by `Store.DueForLatestCheck`'s WHERE clause) and a wake interval. - The row is stamped *before* the fetch so a broken series waits out a full + Two independent clocks: per-bookmark cooldown (`latest_checked_at` column, + enforced by `Store.DueForLatestCheck`'s WHERE clause) and wake interval. + Row stamped *before* fetch so broken series wait out full cooldown instead of retrying every tick, and writes go through - `Store.Get` + `Store.Upsert` so a new chapter never reorders the list. - Fetches use `bogdanfinn/tls-client` with a Chrome profile as defence in depth - against fingerprint-based blocking; any failure logs and skips. See + `Store.Get` + `Store.Upsert` so new chapter never reorders list. + Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth + against fingerprint-based blocking; any failure log and skip. kagane sits + behind a Cloudflare JavaScript challenge the TLS client can't clear, so it is + browser-only: fetched over CDP via `BROWSER_WS_URL`, and simply not polled + when that's unset. See `docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`. - The poller's `Store.Get` + `Store.Upsert` is not wrapped in a transaction, so - a userscript `PUT` that commits between the two can be overwritten by the - poller's stale re-read — reverting that read progress and, since the stored - value now differs, moving `updated_at` and reordering the list. This is a - known, accepted limitation for a single-user deployment, not a bug to fix. -- **`updated_at` drives list order, so it moves only on real reading progress:** the server applies its timestamp when the row is new or `last_chapter_num` changes, and otherwise keeps the stored value — favouriting a series or recording a newly published chapter must not reorder the list. `PUT` therefore returns the row **as stored**, and clients must adopt that response rather than their own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4. + Poller's `Store.Get` + `Store.Upsert` not wrapped in transaction, so + userscript `PUT` that commits between the two can get overwritten by + poller's stale re-read — reverting that read progress and, since stored + value now differs, moving `updated_at` and reordering list. Known, + accepted limitation for single-user deployment, not bug to fix. +- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4. - **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` | `finished`, orthogonal to `favorite`. Archived and finished appear only in - their own tab — not in All, Updated, Favourites, or the recent strip. The - poller keeps checking archived series and skips finished ones. `finished` is - settable only from the web UI; `PUT /bookmarks/{key}` rejects it with 400. - **An empty incoming status means "keep the stored one"** — resolved on the - `VALUES` side of `Store.Upsert`, not in the conflict clause, because - `excluded.*` is the post-evaluation row and a default applied there would - wipe the bucket on every PUT from a client that predates the column. See + own tab — not in All, Updated, Favourites, or recent strip. Poller keeps + checking archived series and skip finished ones. `finished` settable + only from web UI; `PUT /bookmarks/{key}` reject it with 400. + **Empty incoming status means "keep stored one"** — resolved on the + `VALUES` side of `Store.Upsert`, not conflict clause, since + `excluded.*` is post-evaluation row and default applied there would + wipe bucket on every PUT from client that predates column. See `docs/superpowers/specs/2026-07-27-status-buckets-design.md`. - **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list), `DB_PATH` (default `/data/bookmarks.db`), `PORT` (default `8080`), `WEB_PASSWORD` - (gates the browser UI; unset disables it), + (gates browser UI; unset disable it), `LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` (background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`). `USERSCRIPT_PATH` (file served at `/u/{token}/manga-bookmark.user.js`, - default `/userscript/manga-bookmark.user.js`, supplied by a bindmount). + default `/userscript/manga-bookmark.user.js`, supplied by bindmount). + `BROWSER_WS_URL` (headless-shell CDP endpoint for kagane; unset disables + browser polling and leaves that site to the userscript alone). ### Userscript structure (single IIFE, `manga-bookmark.user.js`) -1. **Site adapters** — one per host, `detect(location, document)` returns page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title`/`cover` from **`og:title`/`og:image` meta tags**, not CSS classes. +1. **Site adapters** — one per host, `detect(location, document)` return page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title`/`cover` from **`og:title`/`og:image` meta tags**, not CSS classes. 2. **API client** — `apiGet/apiPut/apiDelete` with bearer header; `localStorage` key `mangabm:cache` for instant render + offline fallback. 3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value. -4. **Retry queue** — every write goes through `pushBookmark`/`pushDelete`, so a - failed mutation is parked in `localStorage` (`mangabm:queue`) and replayed on - the next navigation, reconnect, or `refresh()`. Entries are markers - (`{key, op, sendStatus, attempts}`), never payloads — the body is read from - the cache at send time, so one entry per key gives ordering and coalescing for - free. `sendStatus` is **sticky**: while an archive is pending, later writes to - that key keep carrying the bucket, which is what stops a successful - in-between write from silently un-archiving the series. `refresh()` drains - before it fetches and overlays anything still pending, so the list never - flaps. A 400 drops the entry, a 401 aborts the pass and keeps the queue, and - transient failures retry to a cap of 10. Latest-chapter writes deliberately - stay out of the queue. See +4. **Retry queue** — every write go through `pushBookmark`/`pushDelete`, so + failed mutation park in `localStorage` (`mangabm:queue`) and replayed on + next navigation, reconnect, or `refresh()`. Entries are markers + (`{key, op, sendStatus, attempts}`), never payloads — body read from + cache at send time, so one entry per key give ordering and coalescing for + free. `sendStatus` is **sticky**: while archive pending, later writes to + that key keep carrying bucket, which stop successful + in-between write from silently un-archiving series. `refresh()` drains + before it fetches and overlays anything still pending, so list never + flaps. 400 drops entry, 401 abort pass and keep queue, and + transient failures retry to cap of 10. Latest-chapter writes deliberately + stay out of queue. See `docs/superpowers/specs/2026-07-27-offline-retry-queue-design.md`. -5. **UI** — rendered inside a **Shadow DOM** root to isolate from site CSS - (critical on mobile). Three tabs (All / Favourites / Archived) and a row of - link chips to the web UI and both manga sites; `WEB_BASE` sits in the CONFIG - block next to `API_BASE`. The FAB is a `7 × 44` edge tab whose *hit* area is - widened to `28 × 72` by an invisible `#hit` child; `#fab` must keep - `touch-action: none` and must **not** regain `overflow: hidden`. Because - `touch-action` is resolved at gesture start, the strip cannot be both - browser-scrolled and script-dragged, so `makeDraggable` splits by intent: a - swipe from `#hit` scrolls via `window.scrollBy`, a hold of `ARM_MS` arms a - reposition drag, and the visible sliver drags with no hold. See +5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS + (critical on mobile). Three tabs (All / Favourites / Archived) and row of + link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG + block next to `API_BASE`. FAB is `7 × 44` edge tab whose *hit* area + widened to `28 × 72` by invisible `#hit` child; `#fab` must keep + `touch-action: none` and must **not** regain `overflow: hidden`. Since + `touch-action` resolved at gesture start, strip can't be both + browser-scrolled and script-dragged, so `makeDraggable` splits by intent: swipe + from `#hit` scrolls via `window.scrollBy`, hold of `ARM_MS` arms + reposition drag, visible sliver drags with no hold. See `docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`. -6. **SPA navigation** — Asura is Astro, client-routed on the comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fires without reload. Demonic uses classic reloads (initial `document-idle` run suffices). +6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fire without reload. Demonic uses classic reloads (initial `document-idle` run suffice). -### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trusting) +### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust) -- **asurascans.com**: series `/comics/` (slug carries a trailing +- **asurascans.com**: series `/comics/` (slug carries trailing site-wide build-hash suffix, e.g. `-059befe1`, that **rotates on every redeploy**), chapter `/comics//chapter/`. `seriesId` must strip - the hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in the userscript, - `asuraBuildHash` in the backend); URLs keep the full slug — stale-hash + hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in userscript, + `asuraBuildHash` in backend); URLs keep full slug — stale-hash URLs 302 to current ones. Astro-rendered; chapter links present in raw server HTML. - **demonicscans.org**: series `/manga/` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title//chapter//` (older `chaptered.php?manga=&chapter=` form still exists as redirect, what series-page chapter-list anchors link through). - Encodings (incl. triple-encoded punctuation like `%25252D`) are identical + Encodings (incl. triple-encoded punctuation like `%25252D`) identical on /manga/ and /title/ pages, so decode-once seriesIds match — verified 2026-07-28. @@ -144,34 +160,74 @@ Backend (`cd backend`): Local stack: `docker compose up` (named volume mounted at `/data`, `restart: unless-stopped`). -Smoke test: `curl` the endpoints with `Authorization: Bearer `; confirm `OPTIONS` preflight returns CORS headers and `/healthz` returns 200. +Smoke test: `curl` endpoints with `Authorization: Bearer `; confirm `OPTIONS` preflight return CORS headers and `/healthz` return 200. ## Forge: Gitea, not GitHub -`origin` is a self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` does not work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones: +`origin` is self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` don't work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones: -- Open a PR: `tea pr create --head --base main --title "..." --description "..."` +- Open PR: `tea pr create --head --base main --title "..." --description "..."` - List / view / check out: `tea pr list`, `tea pr `, `tea pr checkout ` - Issues: `tea issue create`, `tea issue list` -- Auth lives in `tea login`, not a `GH_TOKEN` env var. +- Auth lives in `tea login`, not `GH_TOKEN` env var. -`tea` prints its output as rendered boxes rather than plain text; the PR URL lands on the last line. +`tea` print output as rendered boxes rather than plain text; PR URL lands on last line. + +## Design system + +Web UI + userscript panel follow **Cinder**, rules in `docs/design-system.md` +— source of truth Claude Design project `mangaBookmark Web UI` +(`969ac210-fe02-4c01-ae1b-9a271dcc779a`). Read it before touching +`backend/internal/web/static/style.css`, `backend/internal/web/templates/*`, or userscript +`TEMPLATE`/`CSS`. Core law: **ember means new chapter only** — no other +state (busy, error, destruction) may use `--ember`; destruction gets +`--danger`. No cards/corners/shadows, one `--measure: 760px` column, tokens +only (never hardcode hex outside `:root`), both colour branches touched +together. Any move that pulls series out of list (archive/finish/remove) +must be confirm-gated via its own `.confirm-row`; only restore fires +instantly. ## Security invariants - Auth on `/bookmarks*`: require `Authorization: Bearer `, **constant-time compare**, 401 otherwise. - CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` with `204`. +## Comments + +Comment only if code alone can't carry info. Cost per read — must earn spot. + +Write for: +- Why not what. Tradeoffs, non-obvious decisions. +- Load-bearing detail looking incidental — say so if "simplify" breaks it. +- Non-local consequence, invisible from function alone. +- Wire format / encoding / interface contract — save callers re-deriving. +- Gotcha/workaround, with ref if exists. +- Domain/business rule not derivable from code. + +Skip: +- Restating code (no `// increment i` above `i++`). +- Trivial getter/setter/pass-through. +- Banners, dividers, `// helpers`. +- Change narration (`// fix bug`, `// as requested`, `// new impl`) — git's job. +- Commented-out code — delete. +- TODO without concrete action. + +Style: one dense comment over function beats one per line inside. Tight, no worked example unless bug subtle. Wrong comment worse than none — update/delete on change. Default fewer — sparse+high-signal beats comprehensive. + +Test: "competent reader get this from code in few sec?" Yes → skip. Needs detour through another file/spec/git-blame → write it. + ## Relevant skills -`multi-stage-dockerfile` and `docker-compose-orchestration` for the container work (referenced in the plan). +`multi-stage-dockerfile` and `docker-compose-orchestration` for container work (referenced in plan). + +`golang-code-style`, `golang-error-handling`, `golang-performance`, `golang-testing` for backend Go work. ## graphify -This project has a knowledge graph at graphify-out/ with god nodes, community structure, and cross-file relationships. +Project has knowledge graph at graphify-out/ with god nodes, community structure, cross-file relationships. Rules: -- For codebase questions, first run `graphify query ""` when graphify-out/graph.json exists. Use `graphify path "" ""` for relationships and `graphify explain ""` for focused concepts. These return a scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output. -- If graphify-out/wiki/index.md exists, use it for broad navigation instead of raw source browsing. -- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain do not surface enough context. -- After modifying code, run `graphify update .` to keep the graph current (AST-only, no API cost). +- For codebase questions, first run `graphify query ""` when graphify-out/graph.json exists. Use `graphify path "" ""` for relationships and `graphify explain ""` for focused concepts. Return scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output. +- If graphify-out/wiki/index.md exists, use for broad navigation instead of raw source browsing. +- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain don't surface enough context. +- After modifying code, run `graphify update .` to keep graph current (AST-only, no API cost). \ No newline at end of file diff --git a/DEPLOY.md b/DEPLOY.md index 98bb97f..eb3f5f5 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -36,7 +36,7 @@ Edit `.env`: API_TOKEN= # CORS allowlist — leave as-is unless a site changes hostname. -ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org +ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to # Required for the Traefik override. Both have no fallback — compose refuses # to start without them. MANGA_WEB_HOST is required even if you never set @@ -116,6 +116,12 @@ This merges the base file (build/image/env/volume) with the prod override (no host port, Traefik network + router labels). Always pass **both** `-f` flags — the prod file is not standalone. +Two services come up: `manga-api` (the backend) and `headless-shell`, a CDP +sidecar the poller uses to fetch kagane (behind a Cloudflare JS challenge). +It has no published port — only `manga-api` can reach it, over +`BROWSER_WS_URL`. Missing or unreachable, the poller just skips kagane and +logs it; nothing else is affected. + Check it's up and healthy: ```bash diff --git a/README.md b/README.md index 1e6a83a..95be8e6 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,9 @@ # Manga Bookmark -Track manga read-progress on **asurascans.com** (a.k.a. asuracomic.net) and -**demonicscans.org** from a phone (Bromite / mobile Chromium), synced to a -self-hosted Go backend so bookmarks unify across both sites and all devices. +Track manga read-progress on **asurascans.com** (a.k.a. asuracomic.net), +**demonicscans.org**, **comix.to**, and **kagane.to** from a phone (Bromite / +mobile Chromium), synced to a self-hosted Go backend so bookmarks unify across +all four sites and all devices. Two parts: @@ -25,9 +26,10 @@ Bromite userscript (isolated world, Shadow DOM UI, localStorage cache) | Var | Default | Notes | |-----|---------|-------| | `API_TOKEN` | *(required)* | Bearer token shared with the userscript. | -| `ALLOWED_ORIGINS` | Asura + Demonic origins | Comma-separated CORS allowlist. | +| `ALLOWED_ORIGINS` | Asura + Demonic + Comix + Kagane origins | Comma-separated CORS allowlist. | | `DB_PATH` | `/data/bookmarks.db` | SQLite file location. | | `PORT` | `8080` | Plain HTTP; TLS terminated by the proxy. | +| `BROWSER_WS_URL` | `ws://172.28.0.10:9222` | Headless-shell CDP endpoint used to poll Kagane past its JS challenge. Must be an IP or `localhost` — Chrome's DevTools handler 500s any other Host header. | ### Endpoints @@ -39,8 +41,9 @@ Bromite userscript (isolated world, Shadow DOM UI, localStorage cache) | `GET` | `/healthz` | none | `200 ok`. | | `GET` | `/u/{token}/manga-bookmark.user.js` | token in path | Serves the userscript with an mtime-derived `@version`. | -`key` is `:` — e.g. `asura:trash-of-the-counts-family-f886a8af` -or `demonic:Infinite-Level-Up-in-Murim`. Sync is last-write-wins. +`key` is `:` — e.g. `asura:trash-of-the-counts-family-f886a8af`, +`demonic:Infinite-Level-Up-in-Murim`, `comix:12345`, or +`kagane:3fa85f64-5717-4562-b3fc-2c963f66afa6`. Sync is last-write-wins. `updated_at` orders the bookmark list, so it moves only on real reading progress: the server applies its timestamp when the row is new or @@ -199,6 +202,8 @@ The site adapters key everything off URL regex, with `title`/`cover` from |------|-----------|-------------|-------------| | **Asura** (`asurascans.com`) | `/comics/` | `/comics//chapter/` | `` | | **Demonic** (`demonicscans.org`) | `/manga/` | `/title//chapter//` (`chaptered.php?manga=&chapter=` 301s here) | `` | +| **Comix** (`comix.to`) | `/title/-` | `/title/-/-chapter-` | `` | +| **Kagane** (`kagane.to`) | `/series/` | `/series//reader/` | `` | Notes: - **`asuracomic.net` deep links are dead (re-checked 2026-07-25).** They 301 to diff --git a/backend/.dockerignore b/backend/.dockerignore index 84b07d1..cfc6328 100644 --- a/backend/.dockerignore +++ b/backend/.dockerignore @@ -1,10 +1,8 @@ -# Only go source + module files, plus the go:embed'd templates/static -# directories, are needed in the build context. +# Only go source + module files, plus internal/ (which carries the +# go:embed'd templates/static directories), are needed in the build context. * !go.mod !go.sum !*.go -!templates/ -!templates/** -!static/ -!static/** +!internal/ +!internal/** diff --git a/backend/Dockerfile b/backend/Dockerfile index fb494da..15afbd4 100644 --- a/backend/Dockerfile +++ b/backend/Dockerfile @@ -1,19 +1,18 @@ # syntax=docker/dockerfile:1 # --- build stage: compile a static, CGO-free binary --- -FROM golang:1.24-alpine AS build +FROM golang:1.26-alpine AS build WORKDIR /src # Dependencies first for layer caching (changes rarely). COPY go.mod go.sum ./ RUN go mod download -# Then source (changes often). -# Source plus the go:embed'd assets. Missing either directory turns the embed -# directive into a build error, so both must be copied before `go build`. +# Then source (changes often). internal/web carries the go:embed'd +# templates/static assets — missing them turns the embed directive into a +# build error, so the whole tree must land before `go build`. COPY *.go ./ -COPY templates/ ./templates/ -COPY static/ ./static/ +COPY internal/ ./internal/ # Static binary: pure-Go sqlite means CGO_ENABLED=0 -> no libc dependency. # -trimpath + -ldflags strip paths and debug info for a smaller image. diff --git a/backend/api_test.go b/backend/api_test.go new file mode 100644 index 0000000..19e9450 --- /dev/null +++ b/backend/api_test.go @@ -0,0 +1,472 @@ +package main + +import ( + "bytes" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "mangabm/backend/internal/store" +) + +const testToken = "s3cret-token" + +func testConfig() Config { + return Config{ + Token: testToken, + AllowedOrigins: []string{"https://asuracomic.net", "https://demonicscans.org"}, + Port: "8080", + } +} + +func newTestServer(t *testing.T) http.Handler { + t.Helper() + dbPath := filepath.Join(t.TempDir(), "test.db") + s, err := store.Open(dbPath) + if err != nil { + t.Fatalf("store.Open: %v", err) + } + t.Cleanup(func() { s.Close() }) + return newRouter(s, testConfig()) +} + +func auth(req *http.Request) *http.Request { + req.Header.Set("Authorization", "Bearer "+testToken) + return req +} + +func floatPtr(f float64) *float64 { return &f } + +// seedForCheck inserts a bookmark and forces its latest_checked_at. +func seedForCheck(t *testing.T, s *store.Store, key, seriesURL string, checkedAt int64) { + t.Helper() + if _, err := s.Upsert(store.Bookmark{ + Key: key, + Site: "asura", + SeriesID: key, + SeriesURL: seriesURL, + UpdatedAt: 1000, + }); err != nil { + t.Fatalf("seed %q: %v", key, err) + } + if err := s.MarkLatestChecked(key, checkedAt); err != nil { + t.Fatalf("seed mark %q: %v", key, err) + } +} + +func readLatestCheckedAt(t *testing.T, s *store.Store, key string) int64 { + t.Helper() + ts, err := s.LatestCheckedAt(key) + if err != nil { + t.Fatalf("LatestCheckedAt %q: %v", key, err) + } + return ts +} + +func TestHealthzNoAuth(t *testing.T) { + srv := newTestServer(t) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil)) + if rr.Code != http.StatusOK { + t.Fatalf("healthz status = %d, want 200", rr.Code) + } + if rr.Body.String() != "ok" { + t.Fatalf("healthz body = %q, want ok", rr.Body.String()) + } +} + +func TestAuthRequired(t *testing.T) { + srv := newTestServer(t) + cases := []struct { + name string + header string + }{ + {"no header", ""}, + {"bad token", "Bearer wrong"}, + {"not bearer", "Basic " + testToken}, + {"empty bearer", "Bearer "}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + req := httptest.NewRequest(http.MethodGet, "/bookmarks", nil) + if tc.header != "" { + req.Header.Set("Authorization", tc.header) + } + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + if rr.Code != http.StatusUnauthorized { + t.Fatalf("status = %d, want 401", rr.Code) + } + }) + } +} + +func TestAuthAccepted(t *testing.T) { + srv := newTestServer(t) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + if rr.Code != http.StatusOK { + t.Fatalf("status = %d, want 200", rr.Code) + } + if got := rr.Body.String(); got != "[]\n" { + t.Fatalf("empty list body = %q, want []", got) + } +} + +func TestCORSPreflight(t *testing.T) { + srv := newTestServer(t) + req := httptest.NewRequest(http.MethodOptions, "/bookmarks/asura:foo-1", nil) + req.Header.Set("Origin", "https://asuracomic.net") + req.Header.Set("Access-Control-Request-Method", "PUT") + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + + if rr.Code != http.StatusNoContent { + t.Fatalf("preflight status = %d, want 204", rr.Code) + } + if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "https://asuracomic.net" { + t.Fatalf("Allow-Origin = %q, want reflected origin", got) + } + if got := rr.Header().Get("Access-Control-Allow-Methods"); got == "" { + t.Fatal("Allow-Methods missing") + } + if got := rr.Header().Get("Access-Control-Allow-Headers"); got == "" { + t.Fatal("Allow-Headers missing") + } +} + +func TestCORSDisallowedOrigin(t *testing.T) { + srv := newTestServer(t) + req := httptest.NewRequest(http.MethodOptions, "/bookmarks", nil) + req.Header.Set("Origin", "https://evil.example") + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, req) + if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "" { + t.Fatalf("Allow-Origin = %q, want empty for disallowed origin", got) + } +} + +func TestBookmarkRoundTrip(t *testing.T) { + srv := newTestServer(t) + key := "asura:solo-leveling-123" + in := store.Bookmark{ + Title: "Solo Leveling", + SeriesURL: "https://asuracomic.net/series/solo-leveling-123", + Cover: "https://asuracomic.net/cover.jpg", + LastChapter: "Chapter 10", + LastChapterNum: 10, + LastChapterURL: "https://asuracomic.net/series/solo-leveling-123/chapter/10", + } + body, _ := json.Marshal(in) + + // PUT + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) + if rr.Code != http.StatusOK { + t.Fatalf("PUT status = %d, want 200", rr.Code) + } + var stored store.Bookmark + if err := json.Unmarshal(rr.Body.Bytes(), &stored); err != nil { + t.Fatalf("decode PUT response: %v", err) + } + if stored.Key != key || stored.Site != "asura" || stored.SeriesID != "solo-leveling-123" { + t.Fatalf("derived fields wrong: %+v", stored) + } + if stored.UpdatedAt == 0 { + t.Fatal("server did not set updated_at") + } + + // GET + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + var list []store.Bookmark + if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil { + t.Fatalf("decode list: %v", err) + } + if len(list) != 1 || list[0].Key != key || list[0].LastChapterNum != 10 { + t.Fatalf("GET list wrong: %+v", list) + } + + // PUT again (upsert, progress advance) + in.LastChapter, in.LastChapterNum = "Chapter 11", 11 + body, _ = json.Marshal(in) + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) + if rr.Code != http.StatusOK { + t.Fatalf("second PUT status = %d", rr.Code) + } + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + json.Unmarshal(rr.Body.Bytes(), &list) + if len(list) != 1 || list[0].LastChapterNum != 11 { + t.Fatalf("upsert did not update in place: %+v", list) + } + + // DELETE + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodDelete, "/bookmarks/"+key, nil))) + if rr.Code != http.StatusNoContent { + t.Fatalf("DELETE status = %d, want 204", rr.Code) + } + rr = httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + json.Unmarshal(rr.Body.Bytes(), &list) + if len(list) != 0 { + t.Fatalf("after delete list = %+v, want empty", list) + } +} + +// putBookmark PUTs b at key and returns the bookmark the server echoes back, +// which is the row as actually stored (not the request payload). +func putBookmark(t *testing.T, srv http.Handler, key string, b store.Bookmark) store.Bookmark { + t.Helper() + body, _ := json.Marshal(b) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) + if rr.Code != http.StatusOK { + t.Fatalf("PUT %s status = %d, body = %s", key, rr.Code, rr.Body.String()) + } + var out store.Bookmark + if err := json.Unmarshal(rr.Body.Bytes(), &out); err != nil { + t.Fatalf("decode PUT response: %v", err) + } + return out +} + +func getBookmarks(t *testing.T, srv http.Handler) []store.Bookmark { + t.Helper() + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) + if rr.Code != http.StatusOK { + t.Fatalf("GET status = %d", rr.Code) + } + var list []store.Bookmark + if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil { + t.Fatalf("decode list: %v", err) + } + return list +} + +// updated_at drives list ordering, so it must move only on a real progress +// advance — never on a favorite toggle or a latest-chapter capture. +func TestUpsertConditionalUpdatedAt(t *testing.T) { + cases := []struct { + name string + mutate func(store.Bookmark) store.Bookmark + wantBumped bool + }{ + { + name: "unchanged progress", + mutate: func(b store.Bookmark) store.Bookmark { return b }, + wantBumped: false, + }, + { + name: "changed progress", + mutate: func(b store.Bookmark) store.Bookmark { + b.LastChapter, b.LastChapterNum = "Chapter 11", 11 + return b + }, + wantBumped: true, + }, + { + name: "favorite only", + mutate: func(b store.Bookmark) store.Bookmark { + b.Favorite = true + return b + }, + wantBumped: false, + }, + { + name: "latest chapter only", + mutate: func(b store.Bookmark) store.Bookmark { + b.LatestChapter, b.LatestChapterNum = "Chapter 15", floatPtr(15) + return b + }, + wantBumped: false, + }, + { + name: "unrelated metadata only", + mutate: func(b store.Bookmark) store.Bookmark { + b.Title, b.Cover = "Renamed", "https://example.test/new.jpg" + return b + }, + wantBumped: false, + }, + } + + for i, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + srv := newTestServer(t) + key := fmt.Sprintf("asura:cond-%d", i) + + first := putBookmark(t, srv, key, store.Bookmark{ + Title: "Test", + LastChapter: "Chapter 10", + LastChapterNum: 10, + }) + if first.UpdatedAt == 0 { + t.Fatal("new bookmark did not get updated_at set") + } + + // Guarantee a later wall-clock ms so a real bump is observable. + time.Sleep(2 * time.Millisecond) + + second := putBookmark(t, srv, key, tc.mutate(first)) + if tc.wantBumped && second.UpdatedAt <= first.UpdatedAt { + t.Fatalf("updated_at = %d, want > %d", second.UpdatedAt, first.UpdatedAt) + } + if !tc.wantBumped && second.UpdatedAt != first.UpdatedAt { + t.Fatalf("updated_at = %d, want preserved %d", second.UpdatedAt, first.UpdatedAt) + } + + // The PUT response must match what a subsequent GET reports. + list := getBookmarks(t, srv) + if len(list) != 1 { + t.Fatalf("list = %+v, want 1 item", list) + } + if list[0].UpdatedAt != second.UpdatedAt { + t.Fatalf("GET updated_at = %d, PUT echoed %d", list[0].UpdatedAt, second.UpdatedAt) + } + }) + } +} + +func TestFavoriteRoundTrip(t *testing.T) { + srv := newTestServer(t) + key := "demonic:some-series" + + stored := putBookmark(t, srv, key, store.Bookmark{Title: "Fav", Favorite: true}) + if !stored.Favorite { + t.Fatalf("PUT response favorite = false, want true") + } + + list := getBookmarks(t, srv) + if len(list) != 1 || !list[0].Favorite { + t.Fatalf("favorite did not round-trip: %+v", list) + } + + // Unfavoriting must persist too (guards against a write that only ever ORs in true). + stored = putBookmark(t, srv, key, store.Bookmark{Title: "Fav", Favorite: false}) + if stored.Favorite { + t.Fatal("PUT response favorite = true after unfavorite") + } + list = getBookmarks(t, srv) + if len(list) != 1 || list[0].Favorite { + t.Fatalf("unfavorite did not round-trip: %+v", list) + } +} + +func TestLatestChapterNullable(t *testing.T) { + srv := newTestServer(t) + key := "asura:latest-test" + + // Never captured: latest_chapter_num must serialize as JSON null. + body, _ := json.Marshal(store.Bookmark{Title: "No latest yet"}) + rr := httptest.NewRecorder() + srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) + if rr.Code != http.StatusOK { + t.Fatalf("PUT status = %d", rr.Code) + } + if !strings.Contains(rr.Body.String(), `"latest_chapter_num":null`) { + t.Fatalf("want latest_chapter_num null in response, got %s", rr.Body.String()) + } + + list := getBookmarks(t, srv) + if len(list) != 1 || list[0].LatestChapterNum != nil { + t.Fatalf("latest_chapter_num = %v, want nil", list[0].LatestChapterNum) + } + + // Once captured it round-trips as a value. + stored := putBookmark(t, srv, key, store.Bookmark{ + Title: "No latest yet", + LatestChapter: "Chapter 162", + LatestChapterNum: floatPtr(162), + }) + if stored.LatestChapterNum == nil || *stored.LatestChapterNum != 162 { + t.Fatalf("PUT response latest_chapter_num = %v, want 162", stored.LatestChapterNum) + } + list = getBookmarks(t, srv) + if len(list) != 1 || list[0].LatestChapterNum == nil || *list[0].LatestChapterNum != 162 { + t.Fatalf("latest chapter did not round-trip: %+v", list) + } + if list[0].LatestChapter != "Chapter 162" { + t.Fatalf("latest_chapter = %q, want %q", list[0].LatestChapter, "Chapter 162") + } +} + +func TestLoadConfigWebPassword(t *testing.T) { + t.Setenv("API_TOKEN", "token-abc") + t.Setenv("WEB_PASSWORD", "hunter2") + if got := loadConfig().WebPassword; got != "hunter2" { + t.Fatalf("WebPassword = %q, want hunter2", got) + } + + t.Setenv("WEB_PASSWORD", "") + if got := loadConfig().WebPassword; got != "" { + t.Fatalf("WebPassword = %q with the variable unset, want empty", got) + } +} + +// A userscript PUT body has no latest_checked_at field. If the column is ever +// moved into bookmarkColumns, this test catches it: the PUT would reset the +// cooldown and the poller would re-fetch that series on every single tick. +func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) { + dbPath := filepath.Join(t.TempDir(), "test.db") + s, err := store.Open(dbPath) + if err != nil { + t.Fatalf("store.Open: %v", err) + } + t.Cleanup(func() { s.Close() }) + srv := newRouter(s, testConfig()) + + seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", 777) + + // Exactly what the userscript sends: no latest_checked_at key at all. + body := `{"key":"asura:x","site":"asura","series_id":"x", + "series_url":"https://asurascans.com/comics/x", + "last_chapter":"Chapter 5","last_chapter_num":5}` + req := httptest.NewRequest(http.MethodPut, "/bookmarks/asura:x", strings.NewReader(body)) + req.Header.Set("Authorization", "Bearer "+testToken) + req.Header.Set("Content-Type", "application/json") + rec := httptest.NewRecorder() + srv.ServeHTTP(rec, req) + + if rec.Code != http.StatusOK { + t.Fatalf("PUT status = %d, want 200 (body %s)", rec.Code, rec.Body.String()) + } + if got := readLatestCheckedAt(t, s, "asura:x"); got != 777 { + t.Fatalf("latest_checked_at = %d after client PUT, want 777 preserved", got) + } +} + +// The userscript route is registered outside the `if cfg.WebPassword != ""` +// block in newRouter, so it must keep working on a deployment that never set +// WEB_PASSWORD — see internal/userscript for the handler's own behaviour. +func TestUserscriptServedWithWebUIDisabled(t *testing.T) { + path := filepath.Join(t.TempDir(), "manga-bookmark.user.js") + if err := os.WriteFile(path, []byte("console.log(1);\n"), 0o644); err != nil { + t.Fatalf("write script: %v", err) + } + + dbPath := filepath.Join(t.TempDir(), "nopass.db") + s, err := store.Open(dbPath) + if err != nil { + t.Fatalf("store.Open: %v", err) + } + t.Cleanup(func() { s.Close() }) + cfg := testConfig() // WebPassword empty + cfg.UserscriptPath = path + + rr := httptest.NewRecorder() + req := httptest.NewRequest(http.MethodGet, "/u/"+testToken+"/manga-bookmark.user.js", nil) + newRouter(s, cfg).ServeHTTP(rr, req) + if rr.Code != http.StatusOK { + t.Fatalf("status = %d, want 200", rr.Code) + } +} diff --git a/backend/go.mod b/backend/go.mod index 56a710f..f4fb0e1 100644 --- a/backend/go.mod +++ b/backend/go.mod @@ -1,10 +1,12 @@ module mangabm/backend -go 1.24.1 +go 1.26 require ( github.com/bogdanfinn/fhttp v0.6.8 github.com/bogdanfinn/tls-client v1.15.1 + github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f + github.com/chromedp/chromedp v0.16.0 modernc.org/sqlite v1.34.4 ) @@ -15,7 +17,12 @@ require ( github.com/bogdanfinn/quic-go-utls v1.0.9-utls // indirect github.com/bogdanfinn/utls v1.7.7-barnius // indirect github.com/bogdanfinn/websocket v1.5.5-barnius // indirect + github.com/chromedp/sysutil v1.1.0 // indirect github.com/dustin/go-humanize v1.0.1 // indirect + github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68 // indirect + github.com/gobwas/httphead v0.1.0 // indirect + github.com/gobwas/pool v0.2.1 // indirect + github.com/gobwas/ws v1.4.0 // indirect github.com/google/uuid v1.6.0 // indirect github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect github.com/klauspost/compress v1.18.2 // indirect @@ -26,7 +33,7 @@ require ( github.com/tam7t/hpkp v0.0.0-20160821193359-2b70b4024ed5 // indirect golang.org/x/crypto v0.46.0 // indirect golang.org/x/net v0.48.0 // indirect - golang.org/x/sys v0.39.0 // indirect + golang.org/x/sys v0.47.0 // indirect golang.org/x/text v0.32.0 // indirect modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect modernc.org/libc v1.55.3 // indirect diff --git a/backend/go.sum b/backend/go.sum index cc6daad..ef779f6 100644 --- a/backend/go.sum +++ b/backend/go.sum @@ -14,10 +14,24 @@ github.com/bogdanfinn/utls v1.7.7-barnius h1:OuJ497cc7F3yKNVHRsYPQdGggmk5x6+V5Zl github.com/bogdanfinn/utls v1.7.7-barnius/go.mod h1:aAK1VZQlpKZClF1WEQeq6kyclbkPq4hz6xTbB5xSlmg= github.com/bogdanfinn/websocket v1.5.5-barnius h1:bY+qnxpai1qe7Jmjx+Sds/cmOSpuuLoR8x61rWltjOI= github.com/bogdanfinn/websocket v1.5.5-barnius/go.mod h1:gvvEw6pTKHb7yOiFvIfAFTStQWyrm25BMVCTj5wRSsI= +github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f h1:0Z1zcSLEmnj2c2CmJYBqewtS6pxhB39bNWUSEUAWjgk= +github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f/go.mod h1:RwFsSODCtFExll+GhHM6R92SARHR3Z3oipaxLHj46C0= +github.com/chromedp/chromedp v0.16.0 h1:rOO4deOm4CbZgBCa8mD9g2rDyIoNs0BkgvNrlbp5ouk= +github.com/chromedp/chromedp v0.16.0/go.mod h1:rbuGKFT1vMcFcFqKfPIO1GpX/N+2s8onm2qMxZLbU5U= +github.com/chromedp/sysutil v1.1.0 h1:PUFNv5EcprjqXZD9nJb9b/c9ibAbxiYo4exNWZyipwM= +github.com/chromedp/sysutil v1.1.0/go.mod h1:WiThHUdltqCNKGc4gaU50XgYjwjYIhKWoHGPTUfWTJ8= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= +github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68 h1:KZaTBSyshWX3MP5jukJcNSuXDQTO+rNpt0J564dX/eg= +github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68/go.mod h1:tphK2c80bpPhMOI4v6bIc2xWywPfbqi1Z06+RcrMkDg= +github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU= +github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM= +github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og= +github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw= +github.com/gobwas/ws v1.4.0 h1:CTaoG1tojrh4ucGPcoJFiAQUAsEWekEWvLy7GsVNqGs= +github.com/gobwas/ws v1.4.0/go.mod h1:G3gNqMNtPppf5XUz7O4shetPpcZ1VJ7zt18dlUeakrc= github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo= github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= @@ -26,10 +40,14 @@ github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk= github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= +github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80 h1:6Yzfa6GP0rIo/kULo2bwGEkFvCePZ3qHDDTC3/J9Swo= +github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs= github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4= github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw= +github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8= @@ -56,8 +74,8 @@ golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk= -golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU= diff --git a/backend/handlers.go b/backend/internal/api/handlers.go similarity index 71% rename from backend/handlers.go rename to backend/internal/api/handlers.go index 025d670..aa6447c 100644 --- a/backend/handlers.go +++ b/backend/internal/api/handlers.go @@ -1,4 +1,4 @@ -package main +package api import ( "encoding/json" @@ -6,10 +6,13 @@ import ( "net/http" "strings" "time" + + "mangabm/backend/internal/store" ) -type bookmarkHandler struct { - store *Store +// Handler serves the userscript-facing JSON bookmark API. +type Handler struct { + Store *store.Store } func writeJSON(w http.ResponseWriter, status int, v any) { @@ -22,9 +25,9 @@ func writeJSON(w http.ResponseWriter, status int, v any) { } } -// list returns all bookmarks. GET /bookmarks -func (h *bookmarkHandler) list(w http.ResponseWriter, r *http.Request) { - items, err := h.store.List() +// List returns all bookmarks. GET /bookmarks +func (h *Handler) List(w http.ResponseWriter, r *http.Request) { + items, err := h.Store.List() if err != nil { log.Printf("list: %v", err) http.Error(w, "internal error", http.StatusInternalServerError) @@ -33,15 +36,15 @@ func (h *bookmarkHandler) list(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusOK, items) } -// put upserts one bookmark. PUT /bookmarks/{key} -func (h *bookmarkHandler) put(w http.ResponseWriter, r *http.Request) { +// Put upserts one bookmark. PUT /bookmarks/{key} +func (h *Handler) Put(w http.ResponseWriter, r *http.Request) { key := r.PathValue("key") if key == "" { http.Error(w, "missing key", http.StatusBadRequest) return } - var b Bookmark + var b store.Bookmark if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<16)).Decode(&b); err != nil { http.Error(w, "invalid JSON body", http.StatusBadRequest) return @@ -65,9 +68,9 @@ func (h *bookmarkHandler) put(w http.ResponseWriter, r *http.Request) { // Finishing a series is a web-UI decision, so the JSON API refuses it // rather than trusting every client to leave it alone. switch b.Status { - case "", statusReading, statusArchived: - case statusFinished: - http.Error(w, "status "+statusFinished+" can only be set from the web UI", + case "", store.StatusReading, store.StatusArchived: + case store.StatusFinished: + http.Error(w, "status "+store.StatusFinished+" can only be set from the web UI", http.StatusBadRequest) return default: @@ -79,7 +82,7 @@ func (h *bookmarkHandler) put(w http.ResponseWriter, r *http.Request) { // reading progress actually moved. Any client value is ignored. b.UpdatedAt = time.Now().UnixMilli() - stored, err := h.store.Upsert(b) + stored, err := h.Store.Upsert(b) if err != nil { log.Printf("upsert: %v", err) http.Error(w, "internal error", http.StatusInternalServerError) @@ -90,14 +93,14 @@ func (h *bookmarkHandler) put(w http.ResponseWriter, r *http.Request) { writeJSON(w, http.StatusOK, stored) } -// delete removes one bookmark. DELETE /bookmarks/{key} -func (h *bookmarkHandler) delete(w http.ResponseWriter, r *http.Request) { +// Delete removes one bookmark. DELETE /bookmarks/{key} +func (h *Handler) Delete(w http.ResponseWriter, r *http.Request) { key := r.PathValue("key") if key == "" { http.Error(w, "missing key", http.StatusBadRequest) return } - if err := h.store.Delete(key); err != nil { + if err := h.Store.Delete(key); err != nil { log.Printf("delete: %v", err) http.Error(w, "internal error", http.StatusInternalServerError) return @@ -105,7 +108,8 @@ func (h *bookmarkHandler) delete(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusNoContent) } -func healthz(w http.ResponseWriter, r *http.Request) { +// Healthz answers the unauthenticated liveness check. GET /healthz +func Healthz(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "text/plain") w.WriteHeader(http.StatusOK) _, _ = w.Write([]byte("ok")) diff --git a/backend/middleware.go b/backend/internal/httpmw/middleware.go similarity index 88% rename from backend/middleware.go rename to backend/internal/httpmw/middleware.go index 220e4bf..e609531 100644 --- a/backend/middleware.go +++ b/backend/internal/httpmw/middleware.go @@ -1,4 +1,4 @@ -package main +package httpmw import ( "compress/gzip" @@ -9,8 +9,8 @@ import ( const bearerPrefix = "Bearer " -// withAuth guards a handler with a constant-time bearer-token check. -func withAuth(token string, next http.Handler) http.Handler { +// Auth guards a handler with a constant-time bearer-token check. +func Auth(token string, next http.Handler) http.Handler { want := []byte(token) return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { h := r.Header.Get("Authorization") @@ -71,11 +71,11 @@ func (w *gzipWriter) Write(b []byte) (int, error) { return w.ResponseWriter.Write(b) } -// withGzip compresses text responses for clients that ask. The templates, +// Gzip compresses text responses for clients that ask. The templates, // stylesheet and htmx together are ~120 KB uncompressed and roughly a quarter // of that gzipped, which is the difference between a fast and a slow first load // on mobile data. -func withGzip(next http.Handler) http.Handler { +func Gzip(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") { next.ServeHTTP(w, r) @@ -92,11 +92,11 @@ func withGzip(next http.Handler) http.Handler { }) } -// withCORS reflects the request Origin only when it is in allowed, answers +// CORS reflects the request Origin only when it is in allowed, answers // preflight OPTIONS with 204, and passes everything else through. It wraps the // auth middleware so preflight (which carries no Authorization header) is never // rejected by auth. -func withCORS(allowed []string, next http.Handler) http.Handler { +func CORS(allowed []string, next http.Handler) http.Handler { set := make(map[string]struct{}, len(allowed)) for _, o := range allowed { set[o] = struct{}{} diff --git a/backend/internal/latest/browser.go b/backend/internal/latest/browser.go new file mode 100644 index 0000000..10bcf21 --- /dev/null +++ b/backend/internal/latest/browser.go @@ -0,0 +1,154 @@ +package latest + +import ( + "context" + "encoding/json" + "fmt" + "net/url" + "regexp" + "sync" + "time" + + "github.com/chromedp/cdproto/runtime" + "github.com/chromedp/chromedp" +) + +// challengeTimeout bounds one navigate-and-solve. A Cloudflare managed +// challenge clears in a few seconds when it clears at all; anything longer is a +// challenge that is not going to pass, and the caller's cooldown was already +// stamped before this ran. +const challengeTimeout = 45 * time.Second + +var kaganeSeriesRe = regexp.MustCompile(`^/series/([0-9a-f-]{36})/?$`) + +// BrowserFetcher retrieves pages through a remote headless Chrome over the +// DevTools Protocol. +// +// It exists for one reason: kagane.to sits behind a Cloudflare JavaScript +// challenge. Verified 2026-08-03 from the deployment host, plain HTTP and +// bogdanfinn/tls-client with a Chrome_133 profile both get 403 with +// cf-mitigated: challenge on every path, including the API, robots.txt and +// images. Clearing it requires executing the challenge script, which only a +// real browser does. +// +// The request is made *inside* the page rather than by extracting cf_clearance +// and replaying it through TLSFetcher. That cookie is bound to IP, User-Agent +// and often the TLS fingerprint, so replaying it means keeping three things in +// sync that break silently and separately. The browser's own cookie jar +// persists across polls, so the challenge is solved once every few hours. +type BrowserFetcher struct { + allocCtx context.Context + cancel context.CancelFunc + // One page at a time: caps the sidecar's memory and keeps series from + // sharing page state. + mu sync.Mutex +} + +var _ Fetcher = (*BrowserFetcher)(nil) + +// NewBrowserFetcher connects to a headless-shell over CDP. wsURL must name the +// sidecar by IP, e.g. ws://172.28.0.10:9222 — not by Docker DNS name. Chrome's +// DevTools HTTP handler 500s any /json/version request whose Host header +// isn't an IP or "localhost" (confirmed 2026-08-03 against +// chromedp/headless-shell:stable), so the compose network pins the sidecar's +// address for this to resolve at all. +// +// Do not add chromedp.NoModifyURL here: that option skips the /json/version +// discovery request entirely and dials wsURL as if it were already the full +// debugger endpoint, but Chrome only accepts connections at +// /devtools/browser/, a path chosen fresh at every Chrome start — dialing +// the bare host:port 404s. The default (discovery) path works precisely +// because Chrome's /json/version response echoes back the Host header of the +// discovery request in webSocketDebuggerUrl, so as long as wsURL is a +// container-reachable IP, the URL chromedp gets back already points at it. +func NewBrowserFetcher(wsURL string) (*BrowserFetcher, error) { + if wsURL == "" { + return nil, fmt.Errorf("empty browser websocket url") + } + ctx, cancel := chromedp.NewRemoteAllocator(context.Background(), wsURL) + return &BrowserFetcher{allocCtx: ctx, cancel: cancel}, nil +} + +func (f *BrowserFetcher) Close() { + f.cancel() +} + +// Get navigates to seriesURL, lets any challenge resolve, then reads the site's +// JSON API from inside the page so the request carries the clearance cookie. +// The returned body is API JSON, which is what latestChapterFrom's kagane case +// expects — it is not HTML. +func (f *BrowserFetcher) Get(ctx context.Context, seriesURL string) (string, int, error) { + apiURL, ok := kaganeAPIURL(seriesURL) + if !ok { + return "", 0, fmt.Errorf("not a fetchable kagane series url: %q", seriesURL) + } + + f.mu.Lock() + defer f.mu.Unlock() + + ctx, cancel := context.WithTimeout(ctx, challengeTimeout) + defer cancel() + // A fresh tab per fetch, closed on return, so one wedged page cannot + // poison later polls. + tabCtx, cancelTab := chromedp.NewContext(f.allocCtx) + defer cancelTab() + // Bind the caller's deadline to the tab. + tabCtx, cancelDeadline := context.WithCancel(tabCtx) + defer cancelDeadline() + go func() { + <-ctx.Done() + cancelDeadline() + }() + + var body string + err := chromedp.Run(tabCtx, + chromedp.Navigate(seriesURL), + // The challenge reloads the page itself when it passes; waiting for the + // site's own root element is what tells us we are through it. + chromedp.WaitReady("body", chromedp.ByQuery), + chromedp.Evaluate( + `fetch(`+jsString(apiURL)+`).then(r => r.ok ? r.text() : "")`, + &body, + awaitPromise, + ), + ) + if err != nil { + return "", 0, fmt.Errorf("browser fetch %q: %w", seriesURL, err) + } + if body == "" { + // Challenge still up, or the API refused. Indistinguishable from here + // and handled identically by the caller. + return "", 403, nil + } + return body, 200, nil +} + +// kaganeAPIURL maps a stored series_url to the JSON endpoint carrying its +// chapter list. Returning false for anything else is a second line of defence +// behind fetchableSeriesURL: a headless browser is a strong SSRF primitive and +// series_url is client-supplied, so the host is pinned here too. +func kaganeAPIURL(seriesURL string) (string, bool) { + u, err := url.Parse(seriesURL) + if err != nil || u.Scheme != "https" || u.Hostname() != "kagane.to" { + return "", false + } + m := kaganeSeriesRe.FindStringSubmatch(u.Path) + if m == nil { + return "", false + } + return "https://kagane.to/api/v2/series/" + m[1], true +} + +// awaitPromise makes Evaluate resolve the promise rather than returning a +// serialised Promise object. +func awaitPromise(p *runtime.EvaluateParams) *runtime.EvaluateParams { + return p.WithAwaitPromise(true) +} + +// jsString renders s as a JavaScript string literal for embedding in an +// Evaluate expression. The URL is host-pinned by kaganeAPIURL before it gets +// here, but quoting it properly is what keeps that guarantee intact. +func jsString(s string) string { + b, _ := json.Marshal(s) + return string(b) +} diff --git a/backend/internal/latest/browser_test.go b/backend/internal/latest/browser_test.go new file mode 100644 index 0000000..fa91a3a --- /dev/null +++ b/backend/internal/latest/browser_test.go @@ -0,0 +1,38 @@ +package latest + +import "testing" + +func TestKaganeAPIURL(t *testing.T) { + const uuid = "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b" + tests := []struct { + name string + seriesURL string + want string + wantOK bool + }{ + { + name: "series page maps to its API endpoint", + seriesURL: "https://kagane.to/series/" + uuid, + want: "https://kagane.to/api/v2/series/" + uuid, + wantOK: true, + }, + { + name: "trailing slash is tolerated", + seriesURL: "https://kagane.to/series/" + uuid + "/", + want: "https://kagane.to/api/v2/series/" + uuid, + wantOK: true, + }, + {"not a series path", "https://kagane.to/search", "", false}, + {"foreign host", "https://evil.example/series/" + uuid, "", false}, + {"garbage", "://", "", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, ok := kaganeAPIURL(tt.seriesURL) + if ok != tt.wantOK || got != tt.want { + t.Errorf("kaganeAPIURL(%q) = %q, %v; want %q, %v", + tt.seriesURL, got, ok, tt.want, tt.wantOK) + } + }) + } +} diff --git a/backend/latest_http.go b/backend/internal/latest/fetch.go similarity index 89% rename from backend/latest_http.go rename to backend/internal/latest/fetch.go index d6a34ec..866bed4 100644 --- a/backend/latest_http.go +++ b/backend/internal/latest/fetch.go @@ -1,4 +1,4 @@ -package main +package latest import ( "context" @@ -20,20 +20,20 @@ const maxBodyBytes = 4 << 20 const chromeUA = "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 " + "(KHTML, like Gecko) Chrome/133.0.0.0 Mobile Safari/537.36" -// tlsFetcher fetches series pages with a Chrome TLS fingerprint. +// TLSFetcher fetches series pages with a Chrome TLS fingerprint. // // Plain net/http was verified working against both sites on 2026-07-26, so this // is not fixing an observed block — it is deliberate defence-in-depth against a // future fingerprint-based one, chosen up front rather than reacted to later. // The library is pure Go, so CGO_ENABLED=0, the static binary, and the // distroless image are all unaffected. -type tlsFetcher struct { +type TLSFetcher struct { client tls_client.HttpClient } -var _ fetcher = (*tlsFetcher)(nil) +var _ Fetcher = (*TLSFetcher)(nil) -func newTLSFetcher() (*tlsFetcher, error) { +func NewTLSFetcher() (*TLSFetcher, error) { c, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(), tls_client.WithTimeoutSeconds(30), tls_client.WithClientProfile(profiles.Chrome_133), @@ -41,13 +41,13 @@ func newTLSFetcher() (*tlsFetcher, error) { if err != nil { return nil, fmt.Errorf("new tls client: %w", err) } - return &tlsFetcher{client: c}, nil + return &TLSFetcher{client: c}, nil } // Get fetches url and returns the body and status. Redirects are followed: the // demonic chapter anchors are a redirect form, and asura has moved domains // before. -func (f *tlsFetcher) Get(ctx context.Context, url string) (string, int, error) { +func (f *TLSFetcher) Get(ctx context.Context, url string) (string, int, error) { req, err := fhttp.NewRequest(fhttp.MethodGet, url, nil) if err != nil { return "", 0, fmt.Errorf("build request %q: %w", url, err) diff --git a/backend/latest.go b/backend/internal/latest/poller.go similarity index 68% rename from backend/latest.go rename to backend/internal/latest/poller.go index 40a36aa..f5770d9 100644 --- a/backend/latest.go +++ b/backend/internal/latest/poller.go @@ -1,40 +1,58 @@ -package main +package latest import ( "context" "log" "net/url" "time" + + "mangabm/backend/internal/store" ) -// fetcher retrieves a series page. It exists as an interface so tests can inject +// Fetcher retrieves a series page. It exists as an interface so tests can inject // a fake: nothing in the test suite may touch the network or the TLS client. -type fetcher interface { +type Fetcher interface { Get(ctx context.Context, url string) (body string, status int, err error) } -// latestPoller re-checks each bookmarked series' newest published chapter on a +// Poller re-checks each bookmarked series' newest published chapter on a // schedule, independent of the userscript's own in-browser checks. The two run // in parallel and report the same observable fact, so whichever writes last wins // and neither needs to know about the other. // // Two clocks, deliberately independent: // -// - interval is how often this goroutine wakes up and looks. -// - cooldown is how long one bookmark rests since its own last check. +// - Interval is how often this goroutine wakes up and looks. +// - Cooldown is how long one bookmark rests since its own last check. // // Only the cooldown is per bookmark, and it is enforced by the WHERE clause in -// DueForLatestCheck rather than by any timer. Shortening interval therefore +// DueForLatestCheck rather than by any timer. Shortening Interval therefore // cannot shorten anyone's cooldown; it only makes the poller wake up and find // nothing due more often. -type latestPoller struct { - store *Store - fetch fetcher - now func() time.Time // injected so tests can freeze it - cooldown time.Duration - interval time.Duration - stagger time.Duration - batch int +type Poller struct { + Store *store.Store + Fetch Fetcher + // BrowserFetch handles sites behind a JavaScript challenge that Fetch + // cannot clear. Nil disables those sites entirely rather than falling back + // to Fetch, which would only ever retrieve a challenge page. + BrowserFetch Fetcher + Now func() time.Time // injected so tests can freeze it + Cooldown time.Duration + Interval time.Duration + Stagger time.Duration + Batch int +} + +// fetcherFor returns the fetcher a site needs, or nil when the site cannot be +// fetched at all right now. kagane sits behind a Cloudflare JavaScript +// challenge that no TLS fingerprint clears — verified 2026-08-03 from the +// deployment host with the same Chrome profile TLSFetcher uses — so it is +// browser-only or nothing. +func (p *Poller) fetcherFor(site string) Fetcher { + if site == "kagane" { + return p.BrowserFetch + } + return p.Fetch } // Run polls until ctx is cancelled. @@ -43,10 +61,10 @@ type latestPoller struct { // next one instead of stacking a second batch on top of it. That is the intended // failure mode for a misconfigured batch x stagger: a slower cadence, never // concurrent fetch storms. -func (p *latestPoller) Run(ctx context.Context) { +func (p *Poller) Run(ctx context.Context) { log.Printf("latest-chapter poller: interval=%s cooldown=%s batch=%d stagger=%s", - p.interval, p.cooldown, p.batch, p.stagger) - t := time.NewTicker(p.interval) + p.Interval, p.Cooldown, p.Batch, p.Stagger) + t := time.NewTicker(p.Interval) defer t.Stop() for { select { @@ -60,9 +78,9 @@ func (p *latestPoller) Run(ctx context.Context) { } // runOnce processes one batch of due bookmarks. -func (p *latestPoller) runOnce(ctx context.Context) { - cutoff := p.now().Add(-p.cooldown).UnixMilli() - due, err := p.store.DueForLatestCheck(cutoff, p.batch) +func (p *Poller) runOnce(ctx context.Context) { + cutoff := p.Now().Add(-p.Cooldown).UnixMilli() + due, err := p.Store.DueForLatestCheck(cutoff, p.Batch) if err != nil { log.Printf("latest poll: due query: %v", err) return @@ -78,11 +96,11 @@ func (p *latestPoller) runOnce(ctx context.Context) { // bot score. This is the server-side analogue of the userscript's "one // series per navigation ... indistinguishable from browsing" (L455-456). stopped := false - if i > 0 && p.stagger > 0 { + if i > 0 && p.Stagger > 0 { select { case <-ctx.Done(): stopped = true - case <-time.After(p.stagger): + case <-time.After(p.Stagger): } } if stopped { @@ -100,7 +118,7 @@ func (p *latestPoller) runOnce(ctx context.Context) { // checkOne re-checks one series. Every failure path here is "log and move on": // the poller is a best-effort enhancement, and no single bad series may stall a // batch or take down the process. -func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { +func (p *Poller) checkOne(ctx context.Context, b store.Bookmark) { defer func() { if r := recover(); r != nil { log.Printf("latest poll %q: recovered from panic: %v", b.Key, r) @@ -111,7 +129,7 @@ func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { // mid-request still consumes the cooldown. Otherwise a renamed or deleted // series would be retried on every single tick forever. The userscript // stamps in the same order and for the same reason (L471-473). - if err := p.store.MarkLatestChecked(b.Key, p.now().UnixMilli()); err != nil { + if err := p.Store.MarkLatestChecked(b.Key, p.Now().UnixMilli()); err != nil { log.Printf("latest poll %q: mark checked: %v", b.Key, err) return } @@ -128,7 +146,13 @@ func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { return } - body, status, err := p.fetch.Get(ctx, b.SeriesURL) + f := p.fetcherFor(b.Site) + if f == nil { + log.Printf("latest poll %q: no fetcher for site %q", b.Key, b.Site) + return + } + + body, status, err := f.Get(ctx, b.SeriesURL) if err != nil { log.Printf("latest poll %q: fetch %s: %v", b.Key, b.SeriesURL, err) return @@ -155,7 +179,7 @@ func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { // progress or a status change, and moving updated_at because the stored // value now differs. Accepted for a single-user deployment: the window is // milliseconds and the loser is one poll cycle. - cur, found, err := p.store.Get(b.Key) + cur, found, err := p.Store.Get(b.Key) if err != nil { log.Printf("latest poll %q: reread: %v", b.Key, err) return @@ -174,8 +198,8 @@ func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { cur.LatestChapterNum = &num // A candidate only. last_chapter_num is untouched, so the CASE in Upsert // keeps the stored updated_at and the bookmark list does not reorder. - cur.UpdatedAt = p.now().UnixMilli() - if _, err := p.store.Upsert(cur); err != nil { + cur.UpdatedAt = p.Now().UnixMilli() + if _, err := p.Store.Upsert(cur); err != nil { log.Printf("latest poll %q: upsert: %v", b.Key, err) return } @@ -183,13 +207,18 @@ func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) { } // fetchableSeriesURL reports whether site is a site latestChapterFrom knows how -// to parse and seriesURL is safe to hand to the fetcher: an https URL with a +// to parse and seriesURL is safe to hand to a fetcher: an https URL with a // non-empty host. series_url comes from client-supplied PUT bodies, so this is // a defence against the poller being used to probe arbitrary hosts from the // server's own network position, not just a check against wasted requests. +// +// kagane is held to a stricter rule: it is fetched by a headless browser, which +// executes JavaScript and carries cookies, and is therefore a far stronger SSRF +// primitive than an HTTP GET. Its host must match exactly, not merely be +// non-empty. func fetchableSeriesURL(site, seriesURL string) bool { switch site { - case "asura", "demonic": + case "asura", "demonic", "comix", "kagane": default: return false } @@ -197,5 +226,11 @@ func fetchableSeriesURL(site, seriesURL string) bool { if err != nil { return false } - return u.Scheme == "https" && u.Host != "" + if u.Scheme != "https" || u.Host == "" { + return false + } + if site == "kagane" { + return u.Hostname() == "kagane.to" + } + return true } diff --git a/backend/latest_test.go b/backend/internal/latest/poller_test.go similarity index 65% rename from backend/latest_test.go rename to backend/internal/latest/poller_test.go index 04502e0..a960307 100644 --- a/backend/latest_test.go +++ b/backend/internal/latest/poller_test.go @@ -1,13 +1,53 @@ -package main +package latest import ( "context" "errors" + "path/filepath" "sync" "testing" "time" + + "mangabm/backend/internal/store" ) +// newTestStore opens a fresh SQLite store in a temp dir. +func newTestStore(t *testing.T) *store.Store { + t.Helper() + s, err := store.Open(filepath.Join(t.TempDir(), "test.db")) + if err != nil { + t.Fatalf("Open: %v", err) + } + t.Cleanup(func() { s.Close() }) + return s +} + +// seedForCheck inserts a bookmark and forces its latest_checked_at. +func seedForCheck(t *testing.T, s *store.Store, key, seriesURL string, checkedAt int64) { + t.Helper() + if _, err := s.Upsert(store.Bookmark{ + Key: key, + Site: "asura", + SeriesID: key, + SeriesURL: seriesURL, + UpdatedAt: 1000, + }); err != nil { + t.Fatalf("seed %q: %v", key, err) + } + if err := s.MarkLatestChecked(key, checkedAt); err != nil { + t.Fatalf("seed mark %q: %v", key, err) + } +} + +func readLatestCheckedAt(t *testing.T, s *store.Store, key string) int64 { + t.Helper() + ts, err := s.LatestCheckedAt(key) + if err != nil { + t.Fatalf("LatestCheckedAt %q: %v", key, err) + } + return ts +} + // fakeFetcher stands in for the network. Every poller test uses it, so nothing // in this file can reach tls-client or a real site. type fakeFetcher struct { @@ -44,16 +84,16 @@ func (f *fakeFetcher) callCount() int { // newTestPoller wires a poller with a frozen clock and no stagger, so tests run // instantly and deterministically. -func newTestPoller(t *testing.T, s *Store, f fetcher, at time.Time) *latestPoller { +func newTestPoller(t *testing.T, s *store.Store, f Fetcher, at time.Time) *Poller { t.Helper() - return &latestPoller{ - store: s, - fetch: f, - now: func() time.Time { return at }, - cooldown: time.Hour, - interval: 10 * time.Minute, - stagger: 0, - batch: 14, + return &Poller{ + Store: s, + Fetch: f, + Now: func() time.Time { return at }, + Cooldown: time.Hour, + Interval: 10 * time.Minute, + Stagger: 0, + Batch: 14, } } @@ -89,7 +129,7 @@ func TestRunOnceDoesNotReorderList(t *testing.T) { const key = "asura:chronicles-of-the-demon-faction-f886a8af" // "other" is the most recently read, so it must stay at the top of List(). - if _, err := s.Upsert(Bookmark{ + if _, err := s.Upsert(store.Bookmark{ Key: "asura:other", Site: "asura", SeriesID: "other", SeriesURL: "https://asurascans.com/comics/other", UpdatedAt: 9_000_000, }); err != nil { @@ -166,7 +206,7 @@ func TestRunOnceRespectsBatchLimit(t *testing.T) { f := &fakeFetcher{body: "", status: 200} p := newTestPoller(t, s, f, time.UnixMilli(5_000_000)) - p.batch = 5 + p.Batch = 5 p.runOnce(context.Background()) if got := f.callCount(); got != 5 { @@ -218,13 +258,13 @@ func TestRunOnceHonoursCooldownAcrossPasses(t *testing.T) { } // Same instant, and again 59 minutes later: both inside the 1h cooldown. p.runOnce(context.Background()) - p.now = func() time.Time { return now.Add(59 * time.Minute) } + p.Now = func() time.Time { return now.Add(59 * time.Minute) } p.runOnce(context.Background()) if got := f.callCount(); got != 1 { t.Fatalf("fetched %d times inside the cooldown, want 1", got) } // Past the cooldown, it is due again. - p.now = func() time.Time { return now.Add(61 * time.Minute) } + p.Now = func() time.Time { return now.Add(61 * time.Minute) } p.runOnce(context.Background()) if got := f.callCount(); got != 2 { t.Fatalf("fetched %d times after the cooldown, want 2", got) @@ -239,7 +279,7 @@ func TestRunOnceCorrectsDownward(t *testing.T) { const key = "demonic:Catastrophic-Necromancer" high := 400.0 - if _, err := s.Upsert(Bookmark{ + if _, err := s.Upsert(store.Bookmark{ Key: key, Site: "demonic", SeriesID: "Catastrophic-Necromancer", SeriesURL: url, LatestChapter: "Chapter 400", LatestChapterNum: &high, UpdatedAt: 1000, @@ -278,7 +318,7 @@ func TestCheckOneValidatesSeriesURLBeforeFetching(t *testing.T) { t.Run(tt.name, func(t *testing.T) { s := newTestStore(t) key := tt.site + ":x" - if _, err := s.Upsert(Bookmark{ + if _, err := s.Upsert(store.Bookmark{ Key: key, Site: tt.site, SeriesID: "x", SeriesURL: tt.seriesURL, UpdatedAt: 1000, }); err != nil { @@ -287,7 +327,7 @@ func TestCheckOneValidatesSeriesURLBeforeFetching(t *testing.T) { now := time.UnixMilli(4_000_000) f := &fakeFetcher{body: asuraSeriesFixture, status: 200} - newTestPoller(t, s, f, now).checkOne(context.Background(), Bookmark{ + newTestPoller(t, s, f, now).checkOne(context.Background(), store.Bookmark{ Key: key, Site: tt.site, SeriesURL: tt.seriesURL, }) @@ -318,3 +358,98 @@ func TestRunOnceStopsOnCancelledContext(t *testing.T) { t.Fatalf("fetched %d series with a cancelled context, want 0", got) } } + +func TestFetchableSeriesURL(t *testing.T) { + tests := []struct { + name string + site string + seriesURL string + want bool + }{ + {"asura https", "asura", "https://asurascans.com/comics/x-aabbccdd", true}, + {"demonic https", "demonic", "https://demonicscans.org/manga/X", true}, + {"comix https", "comix", "https://comix.to/title/n8we-dungeons-and-crayons", true}, + {"kagane on its own host", "kagane", "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", true}, + // The browser fetcher runs JavaScript and carries cookies, so a + // client-supplied series_url must not be able to aim it anywhere else. + {"kagane on a foreign host", "kagane", "https://evil.example/series/x", false}, + {"kagane on a lookalike host", "kagane", "https://kagane.to.evil.example/series/x", false}, + {"unknown site", "mangadex", "https://mangadex.org/title/x", false}, + {"non-https", "comix", "http://comix.to/title/x", false}, + {"no host", "comix", "https:///title/x", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := fetchableSeriesURL(tt.site, tt.seriesURL); got != tt.want { + t.Errorf("fetchableSeriesURL(%q, %q) = %v, want %v", + tt.site, tt.seriesURL, got, tt.want) + } + }) + } +} + +// A kagane row must not be handed to the plain TLS fetcher: it would only ever +// receive a challenge page, and the browser fetcher is the whole reason kagane +// is pollable at all. +func TestKaganeSkippedWhenNoBrowserFetcher(t *testing.T) { + s := newTestStore(t) + if _, err := s.Upsert(store.Bookmark{ + Key: "kagane:019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + Site: "kagane", + SeriesID: "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + SeriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + UpdatedAt: 1000, + }); err != nil { + t.Fatalf("seed: %v", err) + } + + f := &fakeFetcher{body: kaganeAPIFixture, status: 200} + p := &Poller{ + Store: s, Fetch: f, + Now: func() time.Time { return time.UnixMilli(5_000_000) }, + Cooldown: time.Hour, Interval: time.Hour, Batch: 10, + } + p.runOnce(context.Background()) + + if len(f.calls) != 0 { + t.Errorf("TLS fetcher was called for kagane: %v", f.calls) + } +} + +// With a browser fetcher wired up, kagane goes to it and not to the TLS one. +func TestKaganeUsesBrowserFetcher(t *testing.T) { + s := newTestStore(t) + key := "kagane:019f84bc-9ba0-7ed9-86f5-8b905ec7c28b" + if _, err := s.Upsert(store.Bookmark{ + Key: key, + Site: "kagane", + SeriesID: "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + SeriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + UpdatedAt: 1000, + }); err != nil { + t.Fatalf("seed: %v", err) + } + + tlsF := &fakeFetcher{body: "", status: 200} + browserF := &fakeFetcher{body: kaganeAPIFixture, status: 200} + p := &Poller{ + Store: s, Fetch: tlsF, BrowserFetch: browserF, + Now: func() time.Time { return time.UnixMilli(5_000_000) }, + Cooldown: time.Hour, Interval: time.Hour, Batch: 10, + } + p.runOnce(context.Background()) + + if len(tlsF.calls) != 0 { + t.Errorf("TLS fetcher was called for kagane: %v", tlsF.calls) + } + if len(browserF.calls) != 1 { + t.Fatalf("browser fetcher calls = %v, want 1", browserF.calls) + } + got, found, err := s.Get(key) + if err != nil || !found { + t.Fatalf("Get: %v found=%v", err, found) + } + if got.LatestChapterNum == nil || *got.LatestChapterNum != 41 { + t.Errorf("LatestChapterNum = %v, want 41", got.LatestChapterNum) + } +} diff --git a/backend/latest_sites.go b/backend/internal/latest/sites.go similarity index 73% rename from backend/latest_sites.go rename to backend/internal/latest/sites.go index d503dbf..fcc771b 100644 --- a/backend/latest_sites.go +++ b/backend/internal/latest/sites.go @@ -1,9 +1,11 @@ -package main +package latest import ( "regexp" "strconv" "strings" + + "mangabm/backend/internal/store" ) // latestChapter is the newest chapter a series page advertises. @@ -23,6 +25,15 @@ var asuraSlugRe = regexp.MustCompile(`/comics/([^/?#]+)`) // through. Both the raw "&" and the HTML-escaped "&" forms occur. var demonicChapterRe = regexp.MustCompile(`chaptered\.php\?manga=\d+&(?:amp;)?chapter=([0-9.]+)`) +// comixSlugRe pulls the "-" segment out of a stored series_url. +// Only the id prefix is stable; the slug tail follows the title. +var comixSlugRe = regexp.MustCompile(`/title/([^/?#]+)`) + +// kaganeChapterRe matches the chapter numbers in a kagane API response. This +// branch is fed by the browser fetcher, so the body is JSON rather than HTML — +// there are no anchors to scan. +var kaganeChapterRe = regexp.MustCompile(`"chapter_no":"([0-9.]+)"`) + // latestChapterFrom returns the highest chapter number body advertises for this // series. ok is false when the body yields nothing usable — an unknown site, an // empty body, a Cloudflare challenge page, and a site redesign all land here, @@ -53,12 +64,28 @@ func latestChapterFrom(site, seriesURL, body string) (latestChapter, bool) { // chapter hrefs in the fetched body carry the current one. Strip to // the stable ID (same rule as migrateAsuraKeys) and make the hash // optional in the pattern, so scoping survives rotations. - slug := asuraBuildHash.ReplaceAllString(m[1], "") + slug := store.AsuraBuildHash.ReplaceAllString(m[1], "") // Compiled per call rather than cached: this runs once per fetch, which // is at most a few times a minute, and the slug varies per series. re = regexp.MustCompile(`/comics/` + regexp.QuoteMeta(slug) + `(?:-[0-9a-f]{8})?/chapter/([0-9.]+)`) case "demonic": re = demonicChapterRe + case "comix": + m := comixSlugRe.FindStringSubmatch(seriesURL) + if m == nil { + return latestChapter{}, false + } + // comix ships an SPA: the served HTML carries a JSON state blob instead + // of chapter anchors, and latestChapterUrl is the only place the newest + // chapter appears. Scoping to this series' id prefix keeps a + // "recommended" strip's entries from winning the maximum. + id := m[1] + if i := strings.Index(id, "-"); i != -1 { + id = id[:i] + } + re = regexp.MustCompile(`"latestChapterUrl":"/title/` + regexp.QuoteMeta(id) + `-[^"]*-chapter-([0-9.]+)"`) + case "kagane": + re = kaganeChapterRe default: return latestChapter{}, false } diff --git a/backend/latest_sites_test.go b/backend/internal/latest/sites_test.go similarity index 70% rename from backend/latest_sites_test.go rename to backend/internal/latest/sites_test.go index 443653d..a2bb82d 100644 --- a/backend/latest_sites_test.go +++ b/backend/internal/latest/sites_test.go @@ -1,4 +1,4 @@ -package main +package latest import "testing" @@ -34,6 +34,24 @@ const challengeFixture = `Just a moment...</ti <script src="/cdn-cgi/challenge-platform/h/b/orchestrate/chl_page/v1"></script></head> <body><div id="challenge-running">Checking your browser</div></body></html>` +// Trimmed from the server-rendered HTML of +// https://comix.to/title/n8we-dungeons-and-crayons fetched 2026-08-03. comix is +// an SPA: the page ships a JSON state blob rather than a list of chapter +// anchors, and latestChapterUrl is where the newest chapter actually lives. +const comixSeriesFixture = ` +{"firstChapterUrl":"/title/n8we-dungeons-and-crayons/5038739-chapter-1","latestChapterUrl":"/title/n8we-dungeons-and-crayons/11139891-chapter-80"}, +{""manga","recommended","n8we",1]":{"items":[{"latestChapterUrl":"/title/qqwrm-full-time-awakening/99999999-chapter-999"}]} +` + +// The kagane branch is fed by the browser fetcher, so the body is API JSON, not +// HTML. Trimmed from GET /api/v2/series/<uuid> on 2026-08-03. +const kaganeAPIFixture = ` +{"series_id":"019f84bc-9ba0-7ed9-86f5-8b905ec7c28b","title":"Infinite Decryption", +"series_books":[{"book_id":"a","title":"Episode 1","chapter_no":"1","sort_no":1}, +{"book_id":"b","title":"Episode 41","chapter_no":"41","sort_no":41}, +{"book_id":"c","title":"Episode 40.5","chapter_no":"40.5","sort_no":40}]} +` + func TestLatestChapterFrom(t *testing.T) { const asuraURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af" const demonicURL = "https://demonicscans.org/manga/Catastrophic-Necromancer" @@ -100,6 +118,34 @@ func TestLatestChapterFrom(t *testing.T) { site: "mangadex", seriesURL: "https://example.com/x", body: asuraSeriesFixture, wantOK: false, }, + { + name: "comix reads latestChapterUrl, scoped to this series", + site: "comix", + seriesURL: "https://comix.to/title/n8we-dungeons-and-crayons", + body: comixSeriesFixture, + wantOK: true, wantNum: 80, wantLabel: "Chapter 80", + }, + { + name: "comix yields nothing on a challenge page", + site: "comix", + seriesURL: "https://comix.to/title/n8we-dungeons-and-crayons", + body: challengeFixture, + wantOK: false, + }, + { + name: "kagane takes the max chapter_no from API json", + site: "kagane", + seriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + body: kaganeAPIFixture, + wantOK: true, wantNum: 41, wantLabel: "Chapter 41", + }, + { + name: "kagane yields nothing on a challenge page", + site: "kagane", + seriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b", + body: challengeFixture, + wantOK: false, + }, } for _, tt := range tests { diff --git a/backend/session.go b/backend/internal/session/session.go similarity index 75% rename from backend/session.go rename to backend/internal/session/session.go index 34d63c8..3c70592 100644 --- a/backend/session.go +++ b/backend/internal/session/session.go @@ -1,4 +1,4 @@ -package main +package session import ( "crypto/hmac" @@ -14,7 +14,7 @@ import ( ) const ( - sessionCookieName = "mangabm_session" + CookieName = "mangabm_session" // 60 days: long enough that a phone stays logged in between reading spells. sessionTTL = 60 * 24 * time.Hour // Domain separation, so the session key can never collide with any other @@ -23,18 +23,18 @@ const ( sessionKeyPurpose = "mangabm-web-session-v1" ) -// sessionKey derives the cookie-signing key from both secrets. Sessions are +// Key derives the cookie-signing key from both secrets. Sessions are // stateless — there is no session table — so rotating either API_TOKEN or // WEB_PASSWORD invalidates every outstanding cookie at once. The \x00 // separator prevents the concatenation ambiguity a bare apiToken+webPassword // would have (e.g. "ab"+"c" colliding with "a"+"bc"). -func sessionKey(apiToken, webPassword string) []byte { +func Key(apiToken, webPassword string) []byte { sum := sha256.Sum256([]byte(apiToken + "\x00" + webPassword + sessionKeyPurpose)) return sum[:] } -// signSession encodes "<expiryMs>.<base64url HMAC(expiryMs)>". -func signSession(key []byte, expiryMs int64) string { +// Sign encodes "<expiryMs>.<base64url HMAC(expiryMs)>". +func Sign(key []byte, expiryMs int64) string { payload := strconv.FormatInt(expiryMs, 10) return payload + "." + sessionMAC(key, payload) } @@ -45,10 +45,10 @@ func sessionMAC(key []byte, payload string) string { return base64.RawURLEncoding.EncodeToString(mac.Sum(nil)) } -// verifySession checks shape, then expiry, then the signature — in that order. +// Verify checks shape, then expiry, then the signature — in that order. // The signature comparison is constant-time; the checks before it only look at // data the holder already supplied, so their timing leaks nothing. -func verifySession(key []byte, value string, nowMs int64) bool { +func Verify(key []byte, value string, nowMs int64) bool { payload, sig, ok := strings.Cut(value, ".") if !ok { return false @@ -69,10 +69,10 @@ func isHTTPS(r *http.Request) bool { return r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https" } -func setSessionCookie(w http.ResponseWriter, r *http.Request, key []byte) { +func SetCookie(w http.ResponseWriter, r *http.Request, key []byte) { http.SetCookie(w, &http.Cookie{ - Name: sessionCookieName, - Value: signSession(key, time.Now().Add(sessionTTL).UnixMilli()), + Name: CookieName, + Value: Sign(key, time.Now().Add(sessionTTL).UnixMilli()), Path: "/", MaxAge: int(sessionTTL / time.Second), HttpOnly: true, @@ -81,9 +81,9 @@ func setSessionCookie(w http.ResponseWriter, r *http.Request, key []byte) { }) } -func clearSessionCookie(w http.ResponseWriter, r *http.Request) { +func ClearCookie(w http.ResponseWriter, r *http.Request) { http.SetCookie(w, &http.Cookie{ - Name: sessionCookieName, + Name: CookieName, Value: "", Path: "/", MaxAge: -1, @@ -94,11 +94,11 @@ func clearSessionCookie(w http.ResponseWriter, r *http.Request) { } const ( - loginMaxFailures = 10 - loginWindow = 20 * time.Minute + MaxFailures = 10 + Window = 20 * time.Minute ) -// clientIP returns the address the reverse proxy actually observed. +// ClientIP returns the address the reverse proxy actually observed. // // Traefik appends the peer address to whatever X-Forwarded-For the client sent, // so the leftmost entry is attacker-controlled and the rightmost is not. Go's @@ -106,7 +106,7 @@ const ( // by sending its own; Values covers every line so the true last hop is found. // RemoteAddr is useless behind the proxy — it is always the Traefik container — // so it serves only as the direct-connection fallback for local development. -func clientIP(r *http.Request) string { +func ClientIP(r *http.Request) string { if vals := r.Header.Values("X-Forwarded-For"); len(vals) > 0 { hops := strings.Split(vals[len(vals)-1], ",") if ip := strings.TrimSpace(hops[len(hops)-1]); ip != "" { @@ -120,8 +120,8 @@ func clientIP(r *http.Request) string { return host } -// loginLimiter throttles password guessing: loginMaxFailures failures inside a -// rolling loginWindow blocks further attempts from that IP until the oldest one +// LoginLimiter throttles password guessing: MaxFailures failures inside a +// rolling Window blocks further attempts from that IP until the oldest one // ages out. There is no permanent ban and no unlock step. // // Behind carrier-grade NAT this budget is shared with every other subscriber on @@ -132,34 +132,34 @@ func clientIP(r *http.Request) string { // State is in memory and per-process, so a restart clears it. Entries are // pruned lazily on access; for a single-user deployment the map cannot grow // past the handful of addresses that ever attempt a login. -type loginLimiter struct { +type LoginLimiter struct { mu sync.Mutex failures map[string][]time.Time } -func newLoginLimiter() *loginLimiter { - return &loginLimiter{failures: make(map[string][]time.Time)} +func NewLoginLimiter() *LoginLimiter { + return &LoginLimiter{failures: make(map[string][]time.Time)} } // retryAfter returns how long ip must wait, or zero when it may try now. -func (l *loginLimiter) retryAfter(ip string, now time.Time) time.Duration { +func (l *LoginLimiter) RetryAfter(ip string, now time.Time) time.Duration { l.mu.Lock() defer l.mu.Unlock() recent := l.pruneLocked(ip, now) - if len(recent) < loginMaxFailures { + if len(recent) < MaxFailures { return 0 } - return recent[0].Add(loginWindow).Sub(now) + return recent[0].Add(Window).Sub(now) } -func (l *loginLimiter) fail(ip string, now time.Time) { +func (l *LoginLimiter) Fail(ip string, now time.Time) { l.mu.Lock() defer l.mu.Unlock() l.failures[ip] = append(l.pruneLocked(ip, now), now) } -func (l *loginLimiter) reset(ip string) { +func (l *LoginLimiter) Reset(ip string) { l.mu.Lock() defer l.mu.Unlock() delete(l.failures, ip) @@ -167,8 +167,8 @@ func (l *loginLimiter) reset(ip string) { // pruneLocked drops attempts older than the window and returns what is left. // The caller must hold l.mu. -func (l *loginLimiter) pruneLocked(ip string, now time.Time) []time.Time { - cutoff := now.Add(-loginWindow) +func (l *LoginLimiter) pruneLocked(ip string, now time.Time) []time.Time { + cutoff := now.Add(-Window) // In-place filter: kept reuses the backing array of the slice being // ranged over. Safe to alias because append writes at index len(kept), // which is always <= the range index i, and element i is read before diff --git a/backend/session_test.go b/backend/internal/session/session_test.go similarity index 68% rename from backend/session_test.go rename to backend/internal/session/session_test.go index 4c261b4..327d168 100644 --- a/backend/session_test.go +++ b/backend/internal/session/session_test.go @@ -1,4 +1,4 @@ -package main +package session import ( "crypto/tls" @@ -10,18 +10,18 @@ import ( ) func TestSessionRoundTrip(t *testing.T) { - key := sessionKey("token-abc", "pw-abc") + key := Key("token-abc", "pw-abc") now := time.Now().UnixMilli() - value := signSession(key, now+60_000) - if !verifySession(key, value, now) { - t.Fatal("verifySession = false for a freshly signed cookie, want true") + value := Sign(key, now+60_000) + if !Verify(key, value, now) { + t.Fatal("Verify = false for a freshly signed cookie, want true") } } func TestSessionRejects(t *testing.T) { - key := sessionKey("token-abc", "pw-abc") + key := Key("token-abc", "pw-abc") now := time.Now().UnixMilli() - valid := signSession(key, now+60_000) + valid := Sign(key, now+60_000) payload, sig, _ := strings.Cut(valid, ".") cases := []struct { @@ -31,15 +31,15 @@ func TestSessionRejects(t *testing.T) { {"empty", ""}, {"no separator", payload + sig}, {"unparseable expiry", "notanumber." + sig}, - {"expired", signSession(key, now-1)}, + {"expired", Sign(key, now-1)}, {"tampered signature", payload + "." + flipLastChar(sig)}, {"tampered expiry", "99999999999999." + sig}, - {"signed with another key", signSession(sessionKey("other-token", "pw-abc"), now+60_000)}, + {"signed with another key", Sign(Key("other-token", "pw-abc"), now+60_000)}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { - if verifySession(key, tc.value, now) { - t.Fatalf("verifySession(%q) = true, want false", tc.value) + if Verify(key, tc.value, now) { + t.Fatalf("Verify(%q) = true, want false", tc.value) } }) } @@ -57,18 +57,18 @@ func flipLastChar(s string) string { } func TestSessionKeyDependsOnToken(t *testing.T) { - a := sessionKey("token-a", "pw-abc") - b := sessionKey("token-b", "pw-abc") + a := Key("token-a", "pw-abc") + b := Key("token-b", "pw-abc") if string(a) == string(b) { - t.Fatal("sessionKey collided for different API tokens") + t.Fatal("Key collided for different API tokens") } } func TestSessionKeyDependsOnWebPassword(t *testing.T) { - a := sessionKey("token-abc", "pw-a") - b := sessionKey("token-abc", "pw-b") + a := Key("token-abc", "pw-a") + b := Key("token-abc", "pw-b") if string(a) == string(b) { - t.Fatal("sessionKey collided for different web passwords with the same API token") + t.Fatal("Key collided for different web passwords with the same API token") } } @@ -94,15 +94,15 @@ func TestSetSessionCookieAttributes(t *testing.T) { r.Header.Set("X-Forwarded-Proto", tc.forwarded) } rr := httptest.NewRecorder() - setSessionCookie(rr, r, sessionKey("token-abc", "pw-abc")) + SetCookie(rr, r, Key("token-abc", "pw-abc")) cookies := rr.Result().Cookies() if len(cookies) != 1 { t.Fatalf("got %d cookies, want 1", len(cookies)) } c := cookies[0] - if c.Name != sessionCookieName { - t.Fatalf("cookie name = %q, want %q", c.Name, sessionCookieName) + if c.Name != CookieName { + t.Fatalf("cookie name = %q, want %q", c.Name, CookieName) } if !c.HttpOnly { t.Fatal("cookie HttpOnly = false, want true") @@ -126,7 +126,7 @@ func TestSetSessionCookieAttributes(t *testing.T) { func TestClearSessionCookie(t *testing.T) { r := httptest.NewRequest(http.MethodPost, "/logout", nil) rr := httptest.NewRecorder() - clearSessionCookie(rr, r) + ClearCookie(rr, r) cookies := rr.Result().Cookies() if len(cookies) != 1 { @@ -168,65 +168,65 @@ func TestClientIP(t *testing.T) { for _, v := range tc.xff { r.Header.Add("X-Forwarded-For", v) } - if got := clientIP(r); got != tc.want { - t.Fatalf("clientIP() = %q, want %q", got, tc.want) + if got := ClientIP(r); got != tc.want { + t.Fatalf("ClientIP() = %q, want %q", got, tc.want) } }) } } func TestLoginLimiterBlocksAfterMaxFailures(t *testing.T) { - l := newLoginLimiter() + l := NewLoginLimiter() now := time.Now() - for i := 0; i < loginMaxFailures; i++ { - if wait := l.retryAfter("1.2.3.4", now); wait != 0 { - t.Fatalf("blocked after %d failures, want block only after %d", i, loginMaxFailures) + for i := 0; i < MaxFailures; i++ { + if wait := l.RetryAfter("1.2.3.4", now); wait != 0 { + t.Fatalf("blocked after %d failures, want block only after %d", i, MaxFailures) } - l.fail("1.2.3.4", now) + l.Fail("1.2.3.4", now) } - wait := l.retryAfter("1.2.3.4", now) + wait := l.RetryAfter("1.2.3.4", now) if wait <= 0 { - t.Fatalf("retryAfter = %v after %d failures, want > 0", wait, loginMaxFailures) + t.Fatalf("retryAfter = %v after %d failures, want > 0", wait, MaxFailures) } - if wait > loginWindow { - t.Fatalf("retryAfter = %v, want <= %v", wait, loginWindow) + if wait > Window { + t.Fatalf("retryAfter = %v, want <= %v", wait, Window) } } func TestLoginLimiterWindowExpires(t *testing.T) { - l := newLoginLimiter() + l := NewLoginLimiter() start := time.Now() - for i := 0; i < loginMaxFailures; i++ { - l.fail("1.2.3.4", start) + for i := 0; i < MaxFailures; i++ { + l.Fail("1.2.3.4", start) } - if l.retryAfter("1.2.3.4", start) == 0 { + if l.RetryAfter("1.2.3.4", start) == 0 { t.Fatal("expected block immediately after the failures") } - later := start.Add(loginWindow + time.Second) - if wait := l.retryAfter("1.2.3.4", later); wait != 0 { + later := start.Add(Window + time.Second) + if wait := l.RetryAfter("1.2.3.4", later); wait != 0 { t.Fatalf("retryAfter = %v once the window passed, want 0", wait) } } func TestLoginLimiterResetClearsCounter(t *testing.T) { - l := newLoginLimiter() + l := NewLoginLimiter() now := time.Now() - for i := 0; i < loginMaxFailures; i++ { - l.fail("1.2.3.4", now) + for i := 0; i < MaxFailures; i++ { + l.Fail("1.2.3.4", now) } - l.reset("1.2.3.4") - if wait := l.retryAfter("1.2.3.4", now); wait != 0 { + l.Reset("1.2.3.4") + if wait := l.RetryAfter("1.2.3.4", now); wait != 0 { t.Fatalf("retryAfter = %v after reset, want 0", wait) } } func TestLoginLimiterIsPerIP(t *testing.T) { - l := newLoginLimiter() + l := NewLoginLimiter() now := time.Now() - for i := 0; i < loginMaxFailures; i++ { - l.fail("1.2.3.4", now) + for i := 0; i < MaxFailures; i++ { + l.Fail("1.2.3.4", now) } - if wait := l.retryAfter("5.6.7.8", now); wait != 0 { + if wait := l.RetryAfter("5.6.7.8", now); wait != 0 { t.Fatalf("retryAfter for a different IP = %v, want 0", wait) } } diff --git a/backend/store.go b/backend/internal/store/store.go similarity index 93% rename from backend/store.go rename to backend/internal/store/store.go index a911ef5..0658d2b 100644 --- a/backend/store.go +++ b/backend/internal/store/store.go @@ -1,4 +1,4 @@ -package main +package store import ( "database/sql" @@ -86,11 +86,21 @@ func (b Bookmark) ContinueURL() string { return b.SeriesURL } +// Initial is the monogram the web UI shows in place of a cover when the +// source site never gave us an og:image. First rune, uppercased; "?" when even +// the title is missing, so the slot is never empty. +func (b Bookmark) Initial() string { + for _, r := range b.Title { + return strings.ToUpper(string(r)) + } + return "?" +} + // Lifecycle buckets. A bookmark is in exactly one; favorite is orthogonal. const ( - statusReading = "reading" - statusArchived = "archived" - statusFinished = "finished" + StatusReading = "reading" + StatusArchived = "archived" + StatusFinished = "finished" ) const schema = ` @@ -137,7 +147,7 @@ type Store struct { } // OpenStore opens (or creates) the SQLite database at path and applies the schema. -func OpenStore(path string) (*Store, error) { +func Open(path string) (*Store, error) { // busy_timeout guards against SQLITE_BUSY under the reverse proxy's // concurrent requests; a single writer connection keeps writes serialized. dsn := path @@ -182,11 +192,11 @@ func migrateColumns(db *sql.DB) error { return nil } -// asuraBuildHash matches the trailing "-xxxxxxxx" site-wide build ID Asura +// AsuraBuildHash matches the trailing "-xxxxxxxx" site-wide build ID Asura // appends to every series slug. It rotates on each site redeploy, so it // must not be part of series_id. Must stay in sync with stripBuildHash in // userscript/manga-bookmark.user.js. -var asuraBuildHash = regexp.MustCompile(`-[0-9a-f]{8}$`) +var AsuraBuildHash = regexp.MustCompile(`-[0-9a-f]{8}$`) // migrateAsuraKeys rewrites asura bookmarks whose series_id still carries // the build hash to the stable, hashless ID. Rows keyed with a hash are @@ -218,7 +228,7 @@ func migrateAsuraKeys(db *sql.DB) error { groups := map[string][]row{} for _, r := range all { - stripped := asuraBuildHash.ReplaceAllString(r.id, "") + stripped := AsuraBuildHash.ReplaceAllString(r.id, "") groups[stripped] = append(groups[stripped], r) } for stripped, g := range groups { @@ -305,8 +315,8 @@ func scanBookmark(scan func(...any) error) (Bookmark, error) { // leave the row in no list at all, so anything outside the three known // buckets reads as the default rather than being passed through. b.Status = status.String - if b.Status != statusReading && b.Status != statusArchived && b.Status != statusFinished { - b.Status = statusReading + if b.Status != StatusReading && b.Status != StatusArchived && b.Status != StatusFinished { + b.Status = StatusReading } return b, nil } @@ -481,3 +491,16 @@ func (s *Store) MarkLatestChecked(key string, ts int64) error { } return nil } + +// LatestCheckedAt reads the column MarkLatestChecked writes. It exists for +// tests outside this package (the poller's own tests assert on cooldown +// bookkeeping) — see MarkLatestChecked for why the field itself stays off +// Bookmark. +func (s *Store) LatestCheckedAt(key string) (int64, error) { + var ts int64 + if err := s.db.QueryRow( + `SELECT latest_checked_at FROM bookmarks WHERE key = ?`, key).Scan(&ts); err != nil { + return 0, fmt.Errorf("latest checked at %q: %w", key, err) + } + return ts, nil +} diff --git a/backend/store_test.go b/backend/internal/store/store_test.go similarity index 57% rename from backend/store_test.go rename to backend/internal/store/store_test.go index 03d4f1b..0653b01 100644 --- a/backend/store_test.go +++ b/backend/internal/store/store_test.go @@ -1,42 +1,15 @@ -package main +package store import ( - "bytes" "database/sql" - "encoding/json" - "fmt" - "net/http" - "net/http/httptest" "path/filepath" - "strings" "testing" "time" ) -const testToken = "s3cret-token" - -func testConfig() Config { - return Config{ - Token: testToken, - AllowedOrigins: []string{"https://asuracomic.net", "https://demonicscans.org"}, - Port: "8080", - } -} - -func newTestServer(t *testing.T) http.Handler { - t.Helper() - dbPath := filepath.Join(t.TempDir(), "test.db") - store, err := OpenStore(dbPath) - if err != nil { - t.Fatalf("OpenStore: %v", err) - } - t.Cleanup(func() { store.Close() }) - return newRouter(store, testConfig()) -} - func newTestStore(t *testing.T) *Store { t.Helper() - store, err := OpenStore(filepath.Join(t.TempDir(), "test.db")) + store, err := Open(filepath.Join(t.TempDir(), "test.db")) if err != nil { t.Fatalf("OpenStore: %v", err) } @@ -44,346 +17,6 @@ func newTestStore(t *testing.T) *Store { return store } -func auth(req *http.Request) *http.Request { - req.Header.Set("Authorization", "Bearer "+testToken) - return req -} - -func TestHealthzNoAuth(t *testing.T) { - srv := newTestServer(t) - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil)) - if rr.Code != http.StatusOK { - t.Fatalf("healthz status = %d, want 200", rr.Code) - } - if rr.Body.String() != "ok" { - t.Fatalf("healthz body = %q, want ok", rr.Body.String()) - } -} - -func TestAuthRequired(t *testing.T) { - srv := newTestServer(t) - cases := []struct { - name string - header string - }{ - {"no header", ""}, - {"bad token", "Bearer wrong"}, - {"not bearer", "Basic " + testToken}, - {"empty bearer", "Bearer "}, - } - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - req := httptest.NewRequest(http.MethodGet, "/bookmarks", nil) - if tc.header != "" { - req.Header.Set("Authorization", tc.header) - } - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, req) - if rr.Code != http.StatusUnauthorized { - t.Fatalf("status = %d, want 401", rr.Code) - } - }) - } -} - -func TestAuthAccepted(t *testing.T) { - srv := newTestServer(t) - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - if rr.Code != http.StatusOK { - t.Fatalf("status = %d, want 200", rr.Code) - } - if got := rr.Body.String(); got != "[]\n" { - t.Fatalf("empty list body = %q, want []", got) - } -} - -func TestCORSPreflight(t *testing.T) { - srv := newTestServer(t) - req := httptest.NewRequest(http.MethodOptions, "/bookmarks/asura:foo-1", nil) - req.Header.Set("Origin", "https://asuracomic.net") - req.Header.Set("Access-Control-Request-Method", "PUT") - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, req) - - if rr.Code != http.StatusNoContent { - t.Fatalf("preflight status = %d, want 204", rr.Code) - } - if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "https://asuracomic.net" { - t.Fatalf("Allow-Origin = %q, want reflected origin", got) - } - if got := rr.Header().Get("Access-Control-Allow-Methods"); got == "" { - t.Fatal("Allow-Methods missing") - } - if got := rr.Header().Get("Access-Control-Allow-Headers"); got == "" { - t.Fatal("Allow-Headers missing") - } -} - -func TestCORSDisallowedOrigin(t *testing.T) { - srv := newTestServer(t) - req := httptest.NewRequest(http.MethodOptions, "/bookmarks", nil) - req.Header.Set("Origin", "https://evil.example") - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, req) - if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "" { - t.Fatalf("Allow-Origin = %q, want empty for disallowed origin", got) - } -} - -func TestBookmarkRoundTrip(t *testing.T) { - srv := newTestServer(t) - key := "asura:solo-leveling-123" - in := Bookmark{ - Title: "Solo Leveling", - SeriesURL: "https://asuracomic.net/series/solo-leveling-123", - Cover: "https://asuracomic.net/cover.jpg", - LastChapter: "Chapter 10", - LastChapterNum: 10, - LastChapterURL: "https://asuracomic.net/series/solo-leveling-123/chapter/10", - } - body, _ := json.Marshal(in) - - // PUT - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) - if rr.Code != http.StatusOK { - t.Fatalf("PUT status = %d, want 200", rr.Code) - } - var stored Bookmark - if err := json.Unmarshal(rr.Body.Bytes(), &stored); err != nil { - t.Fatalf("decode PUT response: %v", err) - } - if stored.Key != key || stored.Site != "asura" || stored.SeriesID != "solo-leveling-123" { - t.Fatalf("derived fields wrong: %+v", stored) - } - if stored.UpdatedAt == 0 { - t.Fatal("server did not set updated_at") - } - - // GET - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - var list []Bookmark - if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil { - t.Fatalf("decode list: %v", err) - } - if len(list) != 1 || list[0].Key != key || list[0].LastChapterNum != 10 { - t.Fatalf("GET list wrong: %+v", list) - } - - // PUT again (upsert, progress advance) - in.LastChapter, in.LastChapterNum = "Chapter 11", 11 - body, _ = json.Marshal(in) - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) - if rr.Code != http.StatusOK { - t.Fatalf("second PUT status = %d", rr.Code) - } - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - json.Unmarshal(rr.Body.Bytes(), &list) - if len(list) != 1 || list[0].LastChapterNum != 11 { - t.Fatalf("upsert did not update in place: %+v", list) - } - - // DELETE - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodDelete, "/bookmarks/"+key, nil))) - if rr.Code != http.StatusNoContent { - t.Fatalf("DELETE status = %d, want 204", rr.Code) - } - rr = httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - json.Unmarshal(rr.Body.Bytes(), &list) - if len(list) != 0 { - t.Fatalf("after delete list = %+v, want empty", list) - } -} - -// putBookmark PUTs b at key and returns the bookmark the server echoes back, -// which is the row as actually stored (not the request payload). -func putBookmark(t *testing.T, srv http.Handler, key string, b Bookmark) Bookmark { - t.Helper() - body, _ := json.Marshal(b) - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) - if rr.Code != http.StatusOK { - t.Fatalf("PUT %s status = %d, body = %s", key, rr.Code, rr.Body.String()) - } - var out Bookmark - if err := json.Unmarshal(rr.Body.Bytes(), &out); err != nil { - t.Fatalf("decode PUT response: %v", err) - } - return out -} - -func getBookmarks(t *testing.T, srv http.Handler) []Bookmark { - t.Helper() - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil))) - if rr.Code != http.StatusOK { - t.Fatalf("GET status = %d", rr.Code) - } - var list []Bookmark - if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil { - t.Fatalf("decode list: %v", err) - } - return list -} - -func floatPtr(f float64) *float64 { return &f } - -// updated_at drives list ordering, so it must move only on a real progress -// advance — never on a favorite toggle or a latest-chapter capture. -func TestUpsertConditionalUpdatedAt(t *testing.T) { - cases := []struct { - name string - mutate func(Bookmark) Bookmark - wantBumped bool - }{ - { - name: "unchanged progress", - mutate: func(b Bookmark) Bookmark { return b }, - wantBumped: false, - }, - { - name: "changed progress", - mutate: func(b Bookmark) Bookmark { - b.LastChapter, b.LastChapterNum = "Chapter 11", 11 - return b - }, - wantBumped: true, - }, - { - name: "favorite only", - mutate: func(b Bookmark) Bookmark { - b.Favorite = true - return b - }, - wantBumped: false, - }, - { - name: "latest chapter only", - mutate: func(b Bookmark) Bookmark { - b.LatestChapter, b.LatestChapterNum = "Chapter 15", floatPtr(15) - return b - }, - wantBumped: false, - }, - { - name: "unrelated metadata only", - mutate: func(b Bookmark) Bookmark { - b.Title, b.Cover = "Renamed", "https://example.test/new.jpg" - return b - }, - wantBumped: false, - }, - } - - for i, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - srv := newTestServer(t) - key := fmt.Sprintf("asura:cond-%d", i) - - first := putBookmark(t, srv, key, Bookmark{ - Title: "Test", - LastChapter: "Chapter 10", - LastChapterNum: 10, - }) - if first.UpdatedAt == 0 { - t.Fatal("new bookmark did not get updated_at set") - } - - // Guarantee a later wall-clock ms so a real bump is observable. - time.Sleep(2 * time.Millisecond) - - second := putBookmark(t, srv, key, tc.mutate(first)) - if tc.wantBumped && second.UpdatedAt <= first.UpdatedAt { - t.Fatalf("updated_at = %d, want > %d", second.UpdatedAt, first.UpdatedAt) - } - if !tc.wantBumped && second.UpdatedAt != first.UpdatedAt { - t.Fatalf("updated_at = %d, want preserved %d", second.UpdatedAt, first.UpdatedAt) - } - - // The PUT response must match what a subsequent GET reports. - list := getBookmarks(t, srv) - if len(list) != 1 { - t.Fatalf("list = %+v, want 1 item", list) - } - if list[0].UpdatedAt != second.UpdatedAt { - t.Fatalf("GET updated_at = %d, PUT echoed %d", list[0].UpdatedAt, second.UpdatedAt) - } - }) - } -} - -func TestFavoriteRoundTrip(t *testing.T) { - srv := newTestServer(t) - key := "demonic:some-series" - - stored := putBookmark(t, srv, key, Bookmark{Title: "Fav", Favorite: true}) - if !stored.Favorite { - t.Fatalf("PUT response favorite = false, want true") - } - - list := getBookmarks(t, srv) - if len(list) != 1 || !list[0].Favorite { - t.Fatalf("favorite did not round-trip: %+v", list) - } - - // Unfavoriting must persist too (guards against a write that only ever ORs in true). - stored = putBookmark(t, srv, key, Bookmark{Title: "Fav", Favorite: false}) - if stored.Favorite { - t.Fatal("PUT response favorite = true after unfavorite") - } - list = getBookmarks(t, srv) - if len(list) != 1 || list[0].Favorite { - t.Fatalf("unfavorite did not round-trip: %+v", list) - } -} - -func TestLatestChapterNullable(t *testing.T) { - srv := newTestServer(t) - key := "asura:latest-test" - - // Never captured: latest_chapter_num must serialize as JSON null. - body, _ := json.Marshal(Bookmark{Title: "No latest yet"}) - rr := httptest.NewRecorder() - srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body)))) - if rr.Code != http.StatusOK { - t.Fatalf("PUT status = %d", rr.Code) - } - if !strings.Contains(rr.Body.String(), `"latest_chapter_num":null`) { - t.Fatalf("want latest_chapter_num null in response, got %s", rr.Body.String()) - } - - list := getBookmarks(t, srv) - if len(list) != 1 || list[0].LatestChapterNum != nil { - t.Fatalf("latest_chapter_num = %v, want nil", list[0].LatestChapterNum) - } - - // Once captured it round-trips as a value. - stored := putBookmark(t, srv, key, Bookmark{ - Title: "No latest yet", - LatestChapter: "Chapter 162", - LatestChapterNum: floatPtr(162), - }) - if stored.LatestChapterNum == nil || *stored.LatestChapterNum != 162 { - t.Fatalf("PUT response latest_chapter_num = %v, want 162", stored.LatestChapterNum) - } - list = getBookmarks(t, srv) - if len(list) != 1 || list[0].LatestChapterNum == nil || *list[0].LatestChapterNum != 162 { - t.Fatalf("latest chapter did not round-trip: %+v", list) - } - if list[0].LatestChapter != "Chapter 162" { - t.Fatalf("latest_chapter = %q, want %q", list[0].LatestChapter, "Chapter 162") - } -} - -// The deployed database predates favorite/latest_chapter*, and CREATE TABLE -// IF NOT EXISTS will not add them — OpenStore must migrate in place. func TestOpenStoreMigratesLegacySchema(t *testing.T) { dbPath := filepath.Join(t.TempDir(), "legacy.db") @@ -415,7 +48,7 @@ func TestOpenStoreMigratesLegacySchema(t *testing.T) { t.Fatalf("close legacy db: %v", err) } - store, err := OpenStore(dbPath) + store, err := Open(dbPath) if err != nil { t.Fatalf("OpenStore on legacy db: %v", err) } @@ -437,7 +70,7 @@ func TestOpenStoreMigratesLegacySchema(t *testing.T) { } // Reopening an already-migrated database must be a no-op, not an error. - store2, err := OpenStore(dbPath) + store2, err := Open(dbPath) if err != nil { t.Fatalf("OpenStore is not idempotent: %v", err) } @@ -516,19 +149,6 @@ func TestBookmarkContinueURL(t *testing.T) { } } -func TestLoadConfigWebPassword(t *testing.T) { - t.Setenv("API_TOKEN", "token-abc") - t.Setenv("WEB_PASSWORD", "hunter2") - if got := loadConfig().WebPassword; got != "hunter2" { - t.Fatalf("WebPassword = %q, want hunter2", got) - } - - t.Setenv("WEB_PASSWORD", "") - if got := loadConfig().WebPassword; got != "" { - t.Fatalf("WebPassword = %q with the variable unset, want empty", got) - } -} - // readLatestCheckedAt reads the column directly. It is deliberately absent from // Bookmark (see Store.Upsert), so tests cannot assert on it any other way. func readLatestCheckedAt(t *testing.T, s *Store, key string) int64 { @@ -672,7 +292,7 @@ func TestMigrateAddsLatestCheckedAt(t *testing.T) { t.Fatalf("close: %v", err) } - s, err := OpenStore(path) + s, err := Open(path) if err != nil { t.Fatalf("OpenStore on pre-existing db: %v", err) } @@ -691,38 +311,6 @@ func TestMigrateAddsLatestCheckedAt(t *testing.T) { } } -// A userscript PUT body has no latest_checked_at field. If the column is ever -// moved into bookmarkColumns, this test catches it: the PUT would reset the -// cooldown and the poller would re-fetch that series on every single tick. -func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) { - dbPath := filepath.Join(t.TempDir(), "test.db") - store, err := OpenStore(dbPath) - if err != nil { - t.Fatalf("OpenStore: %v", err) - } - t.Cleanup(func() { store.Close() }) - srv := newRouter(store, testConfig()) - - seedForCheck(t, store, "asura:x", "https://asurascans.com/comics/x", 777) - - // Exactly what the userscript sends: no latest_checked_at key at all. - body := `{"key":"asura:x","site":"asura","series_id":"x", - "series_url":"https://asurascans.com/comics/x", - "last_chapter":"Chapter 5","last_chapter_num":5}` - req := httptest.NewRequest(http.MethodPut, "/bookmarks/asura:x", strings.NewReader(body)) - req.Header.Set("Authorization", "Bearer "+testToken) - req.Header.Set("Content-Type", "application/json") - rec := httptest.NewRecorder() - srv.ServeHTTP(rec, req) - - if rec.Code != http.StatusOK { - t.Fatalf("PUT status = %d, want 200 (body %s)", rec.Code, rec.Body.String()) - } - if got := readLatestCheckedAt(t, store, "asura:x"); got != 777 { - t.Fatalf("latest_checked_at = %d after client PUT, want 777 preserved", got) - } -} - func TestUpsertDefaultsStatusToReading(t *testing.T) { store := newTestStore(t) stored, err := store.Upsert(Bookmark{ @@ -732,8 +320,8 @@ func TestUpsertDefaultsStatusToReading(t *testing.T) { if err != nil { t.Fatalf("Upsert: %v", err) } - if stored.Status != statusReading { - t.Fatalf("Status = %q, want %q", stored.Status, statusReading) + if stored.Status != StatusReading { + t.Fatalf("Status = %q, want %q", stored.Status, StatusReading) } } @@ -743,7 +331,7 @@ func TestUpsertEmptyStatusPreservesStored(t *testing.T) { store := newTestStore(t) base := Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", - Status: statusArchived, UpdatedAt: time.Now().UnixMilli(), + Status: StatusArchived, UpdatedAt: time.Now().UnixMilli(), } if _, err := store.Upsert(base); err != nil { t.Fatalf("seed: %v", err) @@ -755,8 +343,8 @@ func TestUpsertEmptyStatusPreservesStored(t *testing.T) { if err != nil { t.Fatalf("Upsert: %v", err) } - if stored.Status != statusArchived { - t.Fatalf("Status = %q, want it preserved as %q", stored.Status, statusArchived) + if stored.Status != StatusArchived { + t.Fatalf("Status = %q, want it preserved as %q", stored.Status, StatusArchived) } } @@ -768,7 +356,7 @@ func TestLatestPollRoundTripPreservesArchived(t *testing.T) { store := newTestStore(t) base := Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", - Status: statusArchived, UpdatedAt: time.Now().UnixMilli(), + Status: StatusArchived, UpdatedAt: time.Now().UnixMilli(), } if _, err := store.Upsert(base); err != nil { t.Fatalf("seed: %v", err) @@ -788,8 +376,8 @@ func TestLatestPollRoundTripPreservesArchived(t *testing.T) { if err != nil { t.Fatalf("Upsert: %v", err) } - if stored.Status != statusArchived { - t.Fatalf("Status = %q, want it preserved as %q", stored.Status, statusArchived) + if stored.Status != StatusArchived { + t.Fatalf("Status = %q, want it preserved as %q", stored.Status, StatusArchived) } } @@ -797,19 +385,19 @@ func TestUpsertReplacesStatusWhenGiven(t *testing.T) { store := newTestStore(t) base := Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", - Status: statusArchived, UpdatedAt: time.Now().UnixMilli(), + Status: StatusArchived, UpdatedAt: time.Now().UnixMilli(), } if _, err := store.Upsert(base); err != nil { t.Fatalf("seed: %v", err) } - base.Status = statusReading + base.Status = StatusReading stored, err := store.Upsert(base) if err != nil { t.Fatalf("Upsert: %v", err) } - if stored.Status != statusReading { - t.Fatalf("Status = %q, want %q", stored.Status, statusReading) + if stored.Status != StatusReading { + t.Fatalf("Status = %q, want %q", stored.Status, StatusReading) } } @@ -826,7 +414,7 @@ func TestUpsertStatusChangeKeepsUpdatedAt(t *testing.T) { t.Fatalf("seed: %v", err) } - base.Status = statusArchived + base.Status = StatusArchived base.UpdatedAt = first.UpdatedAt + 60_000 stored, err := store.Upsert(base) if err != nil { @@ -857,7 +445,7 @@ func TestMigrationAddsStatusToLegacyDatabase(t *testing.T) { } db.Close() - store, err := OpenStore(path) + store, err := Open(path) if err != nil { t.Fatalf("OpenStore: %v", err) } @@ -867,8 +455,8 @@ func TestMigrationAddsStatusToLegacyDatabase(t *testing.T) { if err != nil || !ok { t.Fatalf("Get: ok=%v err=%v", ok, err) } - if b.Status != statusReading { - t.Fatalf("Status = %q, want %q", b.Status, statusReading) + if b.Status != StatusReading { + t.Fatalf("Status = %q, want %q", b.Status, StatusReading) } } @@ -877,9 +465,9 @@ func TestMigrationAddsStatusToLegacyDatabase(t *testing.T) { func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) { store := newTestStore(t) for _, tc := range []struct{ key, status string }{ - {"asura:reading", statusReading}, - {"asura:archived", statusArchived}, - {"asura:finished", statusFinished}, + {"asura:reading", StatusReading}, + {"asura:archived", StatusArchived}, + {"asura:finished", StatusFinished}, } { if _, err := store.Upsert(Bookmark{ Key: tc.key, Site: "asura", SeriesID: tc.key, @@ -912,7 +500,7 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) { func TestOpenStoreMigratesAsuraBuildHashKeys(t *testing.T) { dbPath := filepath.Join(t.TempDir(), "hash.db") - store, err := OpenStore(dbPath) + store, err := Open(dbPath) if err != nil { t.Fatalf("open: %v", err) } @@ -938,7 +526,7 @@ func TestOpenStoreMigratesAsuraBuildHashKeys(t *testing.T) { t.Fatalf("close: %v", err) } - reopened, err := OpenStore(dbPath) + reopened, err := Open(dbPath) if err != nil { t.Fatalf("reopen: %v", err) } @@ -977,7 +565,7 @@ func TestOpenStoreMigratesAsuraBuildHashKeys(t *testing.T) { } // Idempotent: a third open changes nothing. - third, err := OpenStore(dbPath) + third, err := Open(dbPath) if err != nil { t.Fatalf("third open: %v", err) } @@ -990,7 +578,7 @@ func TestOpenStoreMigratesAsuraBuildHashKeys(t *testing.T) { func TestOpenStoreMigratesAsuraHashlessCollision(t *testing.T) { dbPath := filepath.Join(t.TempDir(), "collision.db") - store, err := OpenStore(dbPath) + store, err := Open(dbPath) if err != nil { t.Fatalf("open: %v", err) } @@ -1010,7 +598,7 @@ func TestOpenStoreMigratesAsuraHashlessCollision(t *testing.T) { t.Fatalf("close: %v", err) } - reopened, err := OpenStore(dbPath) + reopened, err := Open(dbPath) if err != nil { t.Fatalf("reopen: %v", err) } diff --git a/backend/userscript.go b/backend/internal/userscript/userscript.go similarity index 96% rename from backend/userscript.go rename to backend/internal/userscript/userscript.go index bcb46cd..c714d90 100644 --- a/backend/userscript.go +++ b/backend/internal/userscript/userscript.go @@ -1,4 +1,4 @@ -package main +package userscript import ( "crypto/subtle" @@ -35,7 +35,7 @@ func stampVersion(src []byte, mod time.Time) []byte { // // The file is read per request — that is what lets a bindmounted copy be edited // on the host without a restart. It is ~50 KB and polled about once a day. -func userscriptHandler(token, path string) http.HandlerFunc { +func Handler(token, path string) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { if subtle.ConstantTimeCompare([]byte(r.PathValue("token")), []byte(token)) != 1 { http.NotFound(w, r) diff --git a/backend/userscript_test.go b/backend/internal/userscript/userscript_test.go similarity index 71% rename from backend/userscript_test.go rename to backend/internal/userscript/userscript_test.go index fec5b34..3196342 100644 --- a/backend/userscript_test.go +++ b/backend/internal/userscript/userscript_test.go @@ -1,4 +1,4 @@ -package main +package userscript import ( "net/http" @@ -10,6 +10,8 @@ import ( "time" ) +const testToken = "s3cret-token" + // sampleScript is a stand-in for the real userscript: a metadata block with a // @version line, plus a body that must survive the rewrite untouched. const sampleScript = `// ==UserScript== @@ -35,16 +37,12 @@ func writeScript(t *testing.T, body string) (path, wantVersion string) { return path, "2026.07.28.1642" } -func newUserscriptServer(t *testing.T, path string) http.Handler { - t.Helper() - store, err := OpenStore(filepath.Join(t.TempDir(), "test.db")) - if err != nil { - t.Fatalf("OpenStore: %v", err) - } - t.Cleanup(func() { store.Close() }) - cfg := testConfig() - cfg.UserscriptPath = path - return newRouter(store, cfg) +// newTestMux registers Handler the same way main.go's router does, without +// pulling in the store or the rest of the app. +func newTestMux(token, path string) http.Handler { + mux := http.NewServeMux() + mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", Handler(token, path)) + return mux } func getScript(t *testing.T, srv http.Handler, token string) *httptest.ResponseRecorder { @@ -56,7 +54,7 @@ func getScript(t *testing.T, srv http.Handler, token string) *httptest.ResponseR func TestUserscriptServedWithStampedVersion(t *testing.T) { path, wantVersion := writeScript(t, sampleScript) - rr := getScript(t, newUserscriptServer(t, path), testToken) + rr := getScript(t, newTestMux(testToken, path), testToken) if rr.Code != http.StatusOK { t.Fatalf("status = %d, want 200", rr.Code) @@ -83,10 +81,13 @@ func TestUserscriptServedWithStampedVersion(t *testing.T) { } } +// The empty-token case ("/u//manga-bookmark.user.js") is covered at the +// router level (see backend's guardEmptyUserscriptToken): ServeMux 307s it to +// "/u/manga-bookmark.user.js" before this handler's own token check ever runs. func TestUserscriptWrongTokenIs404(t *testing.T) { path, _ := writeScript(t, sampleScript) - srv := newUserscriptServer(t, path) - for _, tok := range []string{"wrong", "", testToken + "x", testToken[:3]} { + srv := newTestMux(testToken, path) + for _, tok := range []string{"wrong", testToken + "x", testToken[:3]} { if got := getScript(t, srv, tok).Code; got != http.StatusNotFound { t.Errorf("token %q: status = %d, want 404", tok, got) } @@ -94,7 +95,7 @@ func TestUserscriptWrongTokenIs404(t *testing.T) { } func TestUserscriptMissingFileIs404(t *testing.T) { - srv := newUserscriptServer(t, filepath.Join(t.TempDir(), "absent.user.js")) + srv := newTestMux(testToken, filepath.Join(t.TempDir(), "absent.user.js")) if got := getScript(t, srv, testToken).Code; got != http.StatusNotFound { t.Fatalf("status = %d, want 404", got) } @@ -103,7 +104,7 @@ func TestUserscriptMissingFileIs404(t *testing.T) { func TestUserscriptWithoutVersionLineServedUnmodified(t *testing.T) { const noVersion = "// ==UserScript==\n// @name x\n// ==/UserScript==\nconsole.log(1);\n" path, _ := writeScript(t, noVersion) - rr := getScript(t, newUserscriptServer(t, path), testToken) + rr := getScript(t, newTestMux(testToken, path), testToken) if rr.Code != http.StatusOK { t.Fatalf("status = %d, want 200", rr.Code) @@ -112,21 +113,3 @@ func TestUserscriptWithoutVersionLineServedUnmodified(t *testing.T) { t.Fatalf("body = %q, want it unmodified", rr.Body.String()) } } - -// The endpoint must work on a deployment that never set WEB_PASSWORD, since -// the web routes are not registered at all in that case. -func TestUserscriptServedWithWebUIDisabled(t *testing.T) { - path, _ := writeScript(t, sampleScript) - store, err := OpenStore(filepath.Join(t.TempDir(), "nopass.db")) - if err != nil { - t.Fatalf("OpenStore: %v", err) - } - t.Cleanup(func() { store.Close() }) - cfg := testConfig() - cfg.WebPassword = "" - cfg.UserscriptPath = path - - if got := getScript(t, newRouter(store, cfg), testToken).Code; got != http.StatusOK { - t.Fatalf("status = %d, want 200", got) - } -} diff --git a/backend/static/filter.js b/backend/internal/web/static/filter.js similarity index 100% rename from backend/static/filter.js rename to backend/internal/web/static/filter.js diff --git a/backend/static/fonts/dm-sans-var-latin.woff2 b/backend/internal/web/static/fonts/dm-sans-var-latin.woff2 similarity index 100% rename from backend/static/fonts/dm-sans-var-latin.woff2 rename to backend/internal/web/static/fonts/dm-sans-var-latin.woff2 diff --git a/backend/static/fonts/ibm-plex-mono-500-latin.woff2 b/backend/internal/web/static/fonts/ibm-plex-mono-500-latin.woff2 similarity index 100% rename from backend/static/fonts/ibm-plex-mono-500-latin.woff2 rename to backend/internal/web/static/fonts/ibm-plex-mono-500-latin.woff2 diff --git a/backend/static/fonts/ibm-plex-mono-600-latin.woff2 b/backend/internal/web/static/fonts/ibm-plex-mono-600-latin.woff2 similarity index 100% rename from backend/static/fonts/ibm-plex-mono-600-latin.woff2 rename to backend/internal/web/static/fonts/ibm-plex-mono-600-latin.woff2 diff --git a/backend/static/fonts/instrument-serif-400-italic-latin.woff2 b/backend/internal/web/static/fonts/instrument-serif-400-italic-latin.woff2 similarity index 100% rename from backend/static/fonts/instrument-serif-400-italic-latin.woff2 rename to backend/internal/web/static/fonts/instrument-serif-400-italic-latin.woff2 diff --git a/backend/static/fonts/instrument-serif-400-latin.woff2 b/backend/internal/web/static/fonts/instrument-serif-400-latin.woff2 similarity index 100% rename from backend/static/fonts/instrument-serif-400-latin.woff2 rename to backend/internal/web/static/fonts/instrument-serif-400-latin.woff2 diff --git a/backend/static/htmx.min.js b/backend/internal/web/static/htmx.min.js similarity index 100% rename from backend/static/htmx.min.js rename to backend/internal/web/static/htmx.min.js diff --git a/backend/static/logo.svg b/backend/internal/web/static/logo.svg similarity index 100% rename from backend/static/logo.svg rename to backend/internal/web/static/logo.svg diff --git a/backend/static/style.css b/backend/internal/web/static/style.css similarity index 98% rename from backend/static/style.css rename to backend/internal/web/static/style.css index c1e8ce7..6b6a9fe 100644 --- a/backend/static/style.css +++ b/backend/internal/web/static/style.css @@ -94,6 +94,8 @@ --asura: #7d93a5; --demonic: #a98a78; + --comix: #8a9a7d; + --kagane: #9a8aa5; /* Covers are often missing; the hatch keeps the slot honest instead of faking artwork. */ @@ -146,6 +148,8 @@ --asura: #4f6b80; --demonic: #8a6a55; + --comix: #5f7250; + --kagane: #6f5f7d; --hatch: repeating-linear-gradient(135deg, #e6e0d8 0 5px, #efeae3 5px 10px); --hatch-dim: repeating-linear-gradient(135deg, #ebe6de 0 5px, #f2eee8 5px 10px); @@ -479,11 +483,14 @@ button { cursor: pointer; } .meta .sep { color: var(--faint); } .site-asura { color: var(--asura); } .site-demonic { color: var(--demonic); } +.site-comix { color: var(--comix); } +.site-kagane { color: var(--kagane); } .new-chapter { color: var(--ember); } .state { display: flex; align-items: center; gap: 4px; color: var(--mute); } .state svg { width: 10px; height: 10px; } .is-dim .meta { color: var(--mute-2); } -.is-dim .site-asura, .is-dim .site-demonic { color: var(--mute); filter: grayscale(.6); } +.is-dim .site-asura, .is-dim .site-demonic, +.is-dim .site-comix, .is-dim .site-kagane { color: var(--mute); filter: grayscale(.6); } /* ---- action strip: full-width on a phone, hairline-divided cells ---- */ .actions { diff --git a/backend/templates/app.html b/backend/internal/web/templates/app.html similarity index 100% rename from backend/templates/app.html rename to backend/internal/web/templates/app.html diff --git a/backend/templates/card.html b/backend/internal/web/templates/card.html similarity index 100% rename from backend/templates/card.html rename to backend/internal/web/templates/card.html diff --git a/backend/templates/chrome.html b/backend/internal/web/templates/chrome.html similarity index 100% rename from backend/templates/chrome.html rename to backend/internal/web/templates/chrome.html diff --git a/backend/templates/icons.html b/backend/internal/web/templates/icons.html similarity index 100% rename from backend/templates/icons.html rename to backend/internal/web/templates/icons.html diff --git a/backend/templates/list.html b/backend/internal/web/templates/list.html similarity index 100% rename from backend/templates/list.html rename to backend/internal/web/templates/list.html diff --git a/backend/templates/login.html b/backend/internal/web/templates/login.html similarity index 100% rename from backend/templates/login.html rename to backend/internal/web/templates/login.html diff --git a/backend/web.go b/backend/internal/web/web.go similarity index 78% rename from backend/web.go rename to backend/internal/web/web.go index e30a4f1..6e8b36e 100644 --- a/backend/web.go +++ b/backend/internal/web/web.go @@ -1,4 +1,4 @@ -package main +package web import ( "crypto/subtle" @@ -13,6 +13,9 @@ import ( "strconv" "strings" "time" + + "mangabm/backend/internal/session" + "mangabm/backend/internal/store" ) //go:embed templates @@ -21,25 +24,25 @@ var templateFS embed.FS //go:embed static var staticFS embed.FS -// recentCount is how many series the "Continue reading" strip shows. -const recentCount = 5 +// RecentCount is how many series the "Continue reading" strip shows. +const RecentCount = 5 -// webHandler serves the browser UI: full pages at / and htmx fragments at /ui/. -// It is a separate handler from bookmarkHandler because the two speak different +// Handler serves the browser UI: full pages at / and htmx fragments at /ui/. +// It is a separate handler from api.Handler because the two speak different // representations (HTML versus JSON) to different clients under different auth. -type webHandler struct { - store *Store +type Handler struct { + store *store.Store tmpl *template.Template key []byte password string - limiter *loginLimiter + limiter *session.LoginLimiter } // listView is what every list-rendering template receives. type listView struct { Tab string // "all", "fav", or "new" - Recent []Bookmark - Items []Bookmark + Recent []store.Bookmark + Items []store.Bookmark // NewCount is the badge on the Updated tab: how many series being read // have a chapter out that has not been read. It is counted over the whole // reading set, not the active tab, so the badge does not change meaning as @@ -50,38 +53,28 @@ type listView struct { OOB bool } -// Initial is the monogram the templates show in place of a cover when the -// source site never gave us an og:image. First rune, uppercased; "?" when even -// the title is missing, so the slot is never empty. -func (b Bookmark) Initial() string { - for _, r := range b.Title { - return strings.ToUpper(string(r)) - } - return "?" -} - // loginView is what the login template receives. type loginView struct { Error string } -// newWebHandler parses every template up front so a broken one kills the -// process at startup rather than the first request that touches it. -func newWebHandler(store *Store, cfg Config) (*webHandler, error) { +// New parses every template up front so a broken one kills the process at +// startup rather than the first request that touches it. +func New(s *store.Store, apiToken, webPassword string) (*Handler, error) { tmpl, err := template.ParseFS(templateFS, "templates/*.html") if err != nil { return nil, err } - return &webHandler{ - store: store, + return &Handler{ + store: s, tmpl: tmpl, - key: sessionKey(cfg.Token, cfg.WebPassword), - password: cfg.WebPassword, - limiter: newLoginLimiter(), + key: session.Key(apiToken, webPassword), + password: webPassword, + limiter: session.NewLoginLimiter(), }, nil } -func (h *webHandler) register(mux *http.ServeMux) { +func (h *Handler) Register(mux *http.ServeMux) { mux.HandleFunc("GET /{$}", h.index) mux.HandleFunc("POST /login", h.login) mux.HandleFunc("POST /logout", h.logout) @@ -118,15 +111,15 @@ func staticHandler() http.Handler { } // authed reports whether the request carries a valid session cookie. -func (h *webHandler) authed(r *http.Request) bool { - c, err := r.Cookie(sessionCookieName) - return err == nil && verifySession(h.key, c.Value, time.Now().UnixMilli()) +func (h *Handler) authed(r *http.Request) bool { + c, err := r.Cookie(session.CookieName) + return err == nil && session.Verify(h.key, c.Value, time.Now().UnixMilli()) } // requireSession guards the fragment endpoints. It answers 401 rather than // redirecting, because htmx swaps whatever body it receives into the page and a // redirected login page would be spliced into the card list. -func (h *webHandler) requireSession(next http.HandlerFunc) http.HandlerFunc { +func (h *Handler) requireSession(next http.HandlerFunc) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { if !h.authed(r) { http.Error(w, "unauthorized", http.StatusUnauthorized) @@ -136,7 +129,7 @@ func (h *webHandler) requireSession(next http.HandlerFunc) http.HandlerFunc { } } -func (h *webHandler) render(w http.ResponseWriter, status int, name string, data any) { +func (h *Handler) render(w http.ResponseWriter, status int, name string, data any) { w.Header().Set("Content-Type", "text/html; charset=utf-8") w.WriteHeader(status) if err := h.tmpl.ExecuteTemplate(w, name, data); err != nil { @@ -148,7 +141,7 @@ func (h *webHandler) render(w http.ResponseWriter, status int, name string, data // index renders the list, or the login page when there is no session. The login // page is served at / with status 200 rather than as a redirect to a separate // URL: one page, no redirect loop to reason about. -func (h *webHandler) index(w http.ResponseWriter, r *http.Request) { +func (h *Handler) index(w http.ResponseWriter, r *http.Request) { if !h.authed(r) { h.render(w, http.StatusOK, "login", loginView{}) return @@ -164,8 +157,8 @@ func (h *webHandler) index(w http.ResponseWriter, r *http.Request) { // filterBookmarks returns the subset keep reports true for, preserving order. // It always returns a non-nil slice so an empty tab renders its empty state. -func filterBookmarks(all []Bookmark, keep func(Bookmark) bool) []Bookmark { - out := []Bookmark{} +func filterBookmarks(all []store.Bookmark, keep func(store.Bookmark) bool) []store.Bookmark { + out := []store.Bookmark{} for _, b := range all { if keep(b) { out = append(out, b) @@ -181,25 +174,25 @@ func filterBookmarks(all []Bookmark, keep func(Bookmark) bool) []Bookmark { // in All, not in Updated, not in Favourites, and not in the recent strip. An // archived favourite therefore shows only under Archived: Favourites means // "favourites I am currently reading". -func (h *webHandler) buildListView(tab string) (listView, error) { +func (h *Handler) buildListView(tab string) (listView, error) { all, err := h.store.List() // already ordered updated_at DESC if err != nil { return listView{}, err } - reading := filterBookmarks(all, func(b Bookmark) bool { return b.Status == statusReading }) + reading := filterBookmarks(all, func(b store.Bookmark) bool { return b.Status == store.StatusReading }) - withNew := filterBookmarks(reading, func(b Bookmark) bool { return b.HasNewChapter() }) + withNew := filterBookmarks(reading, func(b store.Bookmark) bool { return b.HasNewChapter() }) - var items []Bookmark + var items []store.Bookmark switch tab { case "fav": - items = filterBookmarks(reading, func(b Bookmark) bool { return b.Favorite }) + items = filterBookmarks(reading, func(b store.Bookmark) bool { return b.Favorite }) case "new": items = withNew case "archived": - items = filterBookmarks(all, func(b Bookmark) bool { return b.Status == statusArchived }) + items = filterBookmarks(all, func(b store.Bookmark) bool { return b.Status == store.StatusArchived }) case "finished": - items = filterBookmarks(all, func(b Bookmark) bool { return b.Status == statusFinished }) + items = filterBookmarks(all, func(b store.Bookmark) bool { return b.Status == store.StatusFinished }) default: tab = "all" items = reading @@ -213,17 +206,17 @@ func (h *webHandler) buildListView(tab string) (listView, error) { // It therefore disappears entirely on a library with nothing new. That is // the intended reading: an empty strip has nothing to say, and the ~240px it // costs on a phone belongs to the list. - var recent []Bookmark + var recent []store.Bookmark if tab == "all" { recent = withNew - if len(recent) > recentCount { - recent = recent[:recentCount] + if len(recent) > RecentCount { + recent = recent[:RecentCount] } } return listView{Tab: tab, Recent: recent, Items: items, NewCount: len(withNew)}, nil } -func (h *webHandler) uiList(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiList(w http.ResponseWriter, r *http.Request) { view, err := h.buildListView(r.URL.Query().Get("tab")) if err != nil { log.Printf("ui list: %v", err) @@ -253,7 +246,7 @@ func currentTab(r *http.Request) string { // mutation cannot leave them describing the library as it was before the tap. // The key is in here because it is tab-shaped too: archived and finished swap // Archive for Restore. -func (h *webHandler) writeChromeOOB(w http.ResponseWriter, view listView) { +func (h *Handler) writeChromeOOB(w http.ResponseWriter, view listView) { view.OOB = true for _, name := range []string{"recent", "newcount", "keyrow"} { if err := h.tmpl.ExecuteTemplate(w, name, view); err != nil { @@ -266,7 +259,7 @@ func (h *webHandler) writeChromeOOB(w http.ResponseWriter, view listView) { // refreshChrome rebuilds the chrome for the reader's current tab after a // mutation and appends it to the response. -func (h *webHandler) refreshChrome(w http.ResponseWriter, r *http.Request) { +func (h *Handler) refreshChrome(w http.ResponseWriter, r *http.Request) { view, err := h.buildListView(currentTab(r)) if err != nil { log.Printf("ui chrome: %v", err) @@ -275,9 +268,9 @@ func (h *webHandler) refreshChrome(w http.ResponseWriter, r *http.Request) { h.writeChromeOOB(w, view) } -func (h *webHandler) login(w http.ResponseWriter, r *http.Request) { - ip := clientIP(r) - if wait := h.limiter.retryAfter(ip, time.Now()); wait > 0 { +func (h *Handler) login(w http.ResponseWriter, r *http.Request) { + ip := session.ClientIP(r) + if wait := h.limiter.RetryAfter(ip, time.Now()); wait > 0 { secs := int(wait.Seconds()) + 1 w.Header().Set("Retry-After", strconv.Itoa(secs)) h.render(w, http.StatusTooManyRequests, "login", loginView{ @@ -293,38 +286,38 @@ func (h *webHandler) login(w http.ResponseWriter, r *http.Request) { } got := r.PostFormValue("password") if subtle.ConstantTimeCompare([]byte(got), []byte(h.password)) != 1 { - h.limiter.fail(ip, time.Now()) + h.limiter.Fail(ip, time.Now()) h.render(w, http.StatusUnauthorized, "login", loginView{Error: "Wrong password."}) return } - h.limiter.reset(ip) - setSessionCookie(w, r, h.key) + h.limiter.Reset(ip) + session.SetCookie(w, r, h.key) http.Redirect(w, r, "/", http.StatusSeeOther) } -func (h *webHandler) logout(w http.ResponseWriter, r *http.Request) { - clearSessionCookie(w, r) +func (h *Handler) logout(w http.ResponseWriter, r *http.Request) { + session.ClearCookie(w, r) http.Redirect(w, r, "/", http.StatusSeeOther) } // loadForMutation fetches the row a mutation targets, writing the error // response itself when there is nothing to mutate. -func (h *webHandler) loadForMutation(w http.ResponseWriter, r *http.Request) (Bookmark, bool) { +func (h *Handler) loadForMutation(w http.ResponseWriter, r *http.Request) (store.Bookmark, bool) { key := r.PathValue("key") if key == "" { http.Error(w, "missing key", http.StatusBadRequest) - return Bookmark{}, false + return store.Bookmark{}, false } b, ok, err := h.store.Get(key) if err != nil { log.Printf("ui get %q: %v", key, err) http.Error(w, "internal error", http.StatusInternalServerError) - return Bookmark{}, false + return store.Bookmark{}, false } if !ok { http.Error(w, "not found", http.StatusNotFound) - return Bookmark{}, false + return store.Bookmark{}, false } return b, true } @@ -338,7 +331,7 @@ func (h *webHandler) loadForMutation(w http.ResponseWriter, r *http.Request) (Bo // state is the feedback for the tap. The strip and the badge are not: they // describe the whole library, so they are rebuilt out of band on every // mutation, at the cost of one extra list read per toggle. -func (h *webHandler) saveAndRenderCard(w http.ResponseWriter, r *http.Request, b Bookmark) { +func (h *Handler) saveAndRenderCard(w http.ResponseWriter, r *http.Request, b store.Bookmark) { stored, err := h.store.Upsert(b) if err != nil { log.Printf("ui upsert %q: %v", b.Key, err) @@ -351,7 +344,7 @@ func (h *webHandler) saveAndRenderCard(w http.ResponseWriter, r *http.Request, b // uiFavorite flips the favourite flag. last_chapter_num is untouched, so // Upsert keeps the stored updated_at and the list does not reorder. -func (h *webHandler) uiFavorite(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiFavorite(w http.ResponseWriter, r *http.Request) { b, ok := h.loadForMutation(w, r) if !ok { return @@ -367,7 +360,7 @@ func (h *webHandler) uiFavorite(w http.ResponseWriter, r *http.Request) { // // last_chapter_num is untouched, so Upsert keeps the stored updated_at and the // list does not reorder. -func (h *webHandler) uiStatus(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiStatus(w http.ResponseWriter, r *http.Request) { b, ok := h.loadForMutation(w, r) if !ok { return @@ -377,7 +370,7 @@ func (h *webHandler) uiStatus(w http.ResponseWriter, r *http.Request) { return } switch s := r.PostFormValue("status"); s { - case statusReading, statusArchived, statusFinished: + case store.StatusReading, store.StatusArchived, store.StatusFinished: b.Status = s default: http.Error(w, "invalid status", http.StatusBadRequest) @@ -398,7 +391,7 @@ func (h *webHandler) uiStatus(w http.ResponseWriter, r *http.Request) { // pre-filled, so a bare tap of Save is an easy accidental submit; it must not // destroy last_chapter_url, nor rewrite the last_chapter display string ("45.0" // to "45") behind a frozen updated_at. -func (h *webHandler) uiChapter(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiChapter(w http.ResponseWriter, r *http.Request) { b, ok := h.loadForMutation(w, r) if !ok { return @@ -425,7 +418,7 @@ func (h *webHandler) uiChapter(w http.ResponseWriter, r *http.Request) { // uiDelete removes the row and answers with an empty body, which htmx swaps in // place of the card — removing it from the page. -func (h *webHandler) uiDelete(w http.ResponseWriter, r *http.Request) { +func (h *Handler) uiDelete(w http.ResponseWriter, r *http.Request) { key := r.PathValue("key") if key == "" { http.Error(w, "missing key", http.StatusBadRequest) diff --git a/backend/main.go b/backend/main.go index aa8f70d..a4241c3 100644 --- a/backend/main.go +++ b/backend/main.go @@ -11,6 +11,13 @@ import ( "strings" "syscall" "time" + + "mangabm/backend/internal/api" + "mangabm/backend/internal/httpmw" + "mangabm/backend/internal/latest" + "mangabm/backend/internal/store" + "mangabm/backend/internal/userscript" + "mangabm/backend/internal/web" ) // Config holds all runtime settings, sourced from environment variables. @@ -149,22 +156,22 @@ func loadConfig() Config { // newRouter wires routes and middleware. CORS is the outermost layer so // preflight OPTIONS short-circuits before auth; /bookmarks* is auth-protected, // /healthz is public. -func newRouter(store *Store, cfg Config) http.Handler { +func newRouter(s *store.Store, cfg Config) http.Handler { mux := http.NewServeMux() - mux.HandleFunc("GET /healthz", healthz) + mux.HandleFunc("GET /healthz", api.Healthz) - // Outside withAuth (the updater sends no Authorization header) and outside - // the WEB_PASSWORD gate (the script must be installable either way). The - // path segment carries the token instead. - mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", userscriptHandler(cfg.Token, cfg.UserscriptPath)) + // Outside httpmw.Auth (the updater sends no Authorization header) and + // outside the WEB_PASSWORD gate (the script must be installable either + // way). The path segment carries the token instead. + mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", userscript.Handler(cfg.Token, cfg.UserscriptPath)) - h := &bookmarkHandler{store: store} + h := &api.Handler{Store: s} protected := http.NewServeMux() - protected.HandleFunc("GET /bookmarks", h.list) - protected.HandleFunc("PUT /bookmarks/{key}", h.put) - protected.HandleFunc("DELETE /bookmarks/{key}", h.delete) + protected.HandleFunc("GET /bookmarks", h.List) + protected.HandleFunc("PUT /bookmarks/{key}", h.Put) + protected.HandleFunc("DELETE /bookmarks/{key}", h.Delete) - auth := withAuth(cfg.Token, protected) + auth := httpmw.Auth(cfg.Token, protected) mux.Handle("/bookmarks", auth) mux.Handle("/bookmarks/", auth) @@ -172,14 +179,14 @@ func newRouter(store *Store, cfg Config) http.Handler { // deployment that forgets WEB_PASSWORD exposes nothing rather than // exposing an unprotected list. if cfg.WebPassword != "" { - web, err := newWebHandler(store, cfg) + wh, err := web.New(s, cfg.Token, cfg.WebPassword) if err != nil { log.Fatalf("web handler: %v", err) } - web.register(mux) + wh.Register(mux) } - return withCORS(cfg.AllowedOrigins, withGzip(guardEmptyUserscriptToken(mux))) + return httpmw.CORS(cfg.AllowedOrigins, httpmw.Gzip(guardEmptyUserscriptToken(mux))) } // guardEmptyUserscriptToken heads off ServeMux's own path-cleaning redirect: @@ -203,22 +210,22 @@ func main() { log.Fatal("API_TOKEN is required") } - store, err := OpenStore(cfg.DBPath) + s, err := store.Open(cfg.DBPath) if err != nil { log.Fatalf("open store: %v", err) } - defer store.Close() + defer s.Close() // The poller is off the request path entirely: if it cannot start, the // service still serves bookmarks and the userscript still captures latest // chapters on its own. pollCtx, stopPoll := context.WithCancel(context.Background()) defer stopPoll() - startLatestPoller(pollCtx, store, cfg.LatestPoll) + startLatestPoller(pollCtx, s, cfg.LatestPoll) srv := &http.Server{ Addr: ":" + cfg.Port, - Handler: newRouter(store, cfg), + Handler: newRouter(s, cfg), ReadHeaderTimeout: 10 * time.Second, } @@ -248,24 +255,39 @@ func main() { // HTTP client cannot be built. Any problem here is logged and skipped: this // feature going missing degrades the service to userscript-only latest-chapter // tracking, which is exactly how it behaved before. -func startLatestPoller(ctx context.Context, store *Store, cfg LatestPoll) { +func startLatestPoller(ctx context.Context, s *store.Store, cfg LatestPoll) { if !cfg.Enabled { log.Println("latest-chapter poller: disabled by config") return } - f, err := newTLSFetcher() + f, err := latest.NewTLSFetcher() if err != nil { log.Printf("latest-chapter poller: disabled, cannot build client: %v", err) return } - p := &latestPoller{ - store: store, - fetch: f, - now: time.Now, - cooldown: cfg.Cooldown, - interval: cfg.Interval, - stagger: cfg.Stagger, - batch: cfg.Batch, + p := &latest.Poller{ + Store: s, + Fetch: f, + Now: time.Now, + Cooldown: cfg.Cooldown, + Interval: cfg.Interval, + Stagger: cfg.Stagger, + Batch: cfg.Batch, } + + // Optional: without it, sites behind a JavaScript challenge are simply not + // polled, and their latest_chapter comes from the userscript alone — which + // is how the service behaved before the sidecar existed. + if ws := strings.TrimSpace(os.Getenv("BROWSER_WS_URL")); ws != "" { + bf, err := latest.NewBrowserFetcher(ws) + if err != nil { + log.Printf("latest-chapter poller: browser fetcher disabled: %v", err) + } else { + p.BrowserFetch = bf + context.AfterFunc(ctx, bf.Close) + log.Printf("latest-chapter poller: browser fetcher at %s", ws) + } + } + go p.Run(ctx) } diff --git a/backend/main_test.go b/backend/main_test.go index 1d89df6..66eaa29 100644 --- a/backend/main_test.go +++ b/backend/main_test.go @@ -10,6 +10,8 @@ import ( "strings" "testing" "time" + + "mangabm/backend/internal/store" ) func TestLoadLatestPollDefaults(t *testing.T) { @@ -148,7 +150,7 @@ func TestPutStatusValidation(t *testing.T) { if tc.want != http.StatusOK { return } - var got Bookmark + var got store.Bookmark if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil { t.Fatalf("decode: %v", err) } @@ -187,7 +189,7 @@ func TestPutOmittedStatusPreservesArchivedAndAppliesProgress(t *testing.T) { t.Fatalf("status = %d, want 200 (body %s)", rr.Code, rr.Body.String()) } - var got Bookmark + var got store.Bookmark if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil { t.Fatalf("decode: %v", err) } diff --git a/backend/web_test.go b/backend/web_test.go index 44f90fa..4a9a048 100644 --- a/backend/web_test.go +++ b/backend/web_test.go @@ -10,6 +10,10 @@ import ( "strings" "testing" "time" + + "mangabm/backend/internal/session" + "mangabm/backend/internal/store" + "mangabm/backend/internal/web" ) const testPassword = "hunter2" @@ -22,22 +26,22 @@ func webConfig() Config { // newWebTestServer returns the full router plus the store behind it, so tests // can seed rows and assert on what the handlers wrote back. -func newWebTestServer(t *testing.T, cfg Config) (http.Handler, *Store) { +func newWebTestServer(t *testing.T, cfg Config) (http.Handler, *store.Store) { t.Helper() - store, err := OpenStore(filepath.Join(t.TempDir(), "test.db")) + st, err := store.Open(filepath.Join(t.TempDir(), "test.db")) if err != nil { - t.Fatalf("OpenStore: %v", err) + t.Fatalf("store.Open: %v", err) } - t.Cleanup(func() { store.Close() }) - return newRouter(store, cfg), store + t.Cleanup(func() { st.Close() }) + return newRouter(st, cfg), st } // sessionCookie returns a cookie a handler will accept for cfg's API token. func sessionCookie(t *testing.T, cfg Config) *http.Cookie { t.Helper() return &http.Cookie{ - Name: sessionCookieName, - Value: signSession(sessionKey(cfg.Token, cfg.WebPassword), time.Now().Add(time.Hour).UnixMilli()), + Name: session.CookieName, + Value: session.Sign(session.Key(cfg.Token, cfg.WebPassword), time.Now().Add(time.Hour).UnixMilli()), } } @@ -56,8 +60,8 @@ func TestIndexWithoutSessionShowsLogin(t *testing.T) { func TestIndexWithSessionShowsList(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - if _, err := store.Upsert(Bookmark{ + srv, st := newWebTestServer(t, cfg) + if _, err := st.Upsert(store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45", LastChapterNum: 45, UpdatedAt: time.Now().UnixMilli(), @@ -90,8 +94,8 @@ func TestLoginSuccessSetsCookie(t *testing.T) { t.Fatalf("POST /login status = %d, want 303", rr.Code) } cookies := rr.Result().Cookies() - if len(cookies) != 1 || cookies[0].Name != sessionCookieName || cookies[0].Value == "" { - t.Fatalf("POST /login cookies = %+v, want one non-empty %s", cookies, sessionCookieName) + if len(cookies) != 1 || cookies[0].Name != session.CookieName || cookies[0].Value == "" { + t.Fatalf("POST /login cookies = %+v, want one non-empty %s", cookies, session.CookieName) } } @@ -122,14 +126,14 @@ func TestLoginRateLimited(t *testing.T) { srv.ServeHTTP(rr, req) return rr } - for i := 0; i < loginMaxFailures; i++ { + for i := 0; i < session.MaxFailures; i++ { if code := post().Code; code != http.StatusUnauthorized { t.Fatalf("attempt %d status = %d, want 401", i+1, code) } } rr := post() if rr.Code != http.StatusTooManyRequests { - t.Fatalf("attempt %d status = %d, want 429", loginMaxFailures+1, rr.Code) + t.Fatalf("attempt %d status = %d, want 429", session.MaxFailures+1, rr.Code) } if after := rr.Header().Get("Retry-After"); after == "" { t.Fatal("429 response has no Retry-After header") @@ -202,9 +206,9 @@ func TestStaticAssetsServed(t *testing.T) { } // seed inserts one bookmark and returns it as stored. -func seed(t *testing.T, store *Store, b Bookmark) Bookmark { +func seed(t *testing.T, st *store.Store, b store.Bookmark) store.Bookmark { t.Helper() - stored, err := store.Upsert(b) + stored, err := st.Upsert(b) if err != nil { t.Fatalf("Upsert: %v", err) } @@ -245,8 +249,8 @@ func TestUIRoutesRequireSession(t *testing.T) { func TestFavoriteTogglesWithoutReordering(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - before := seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + before := seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45", LastChapterNum: 45, UpdatedAt: 1_000_000, @@ -258,7 +262,7 @@ func TestFavoriteTogglesWithoutReordering(t *testing.T) { t.Fatalf("favorite status = %d, want 200", rr.Code) } - after, ok, err := store.Get("asura:solo") + after, ok, err := st.Get("asura:solo") if err != nil || !ok { t.Fatalf("Get after favorite: %v ok=%v", err, ok) } @@ -276,7 +280,7 @@ func TestFavoriteTogglesWithoutReordering(t *testing.T) { // Toggling again turns it back off. rr = httptest.NewRecorder() srv.ServeHTTP(rr, uiRequest(t, cfg, http.MethodPost, "/ui/bookmarks/asura:solo/favorite", nil)) - back, _, _ := store.Get("asura:solo") + back, _, _ := st.Get("asura:solo") if back.Favorite { t.Fatal("Favorite = true after a second toggle, want false") } @@ -294,8 +298,8 @@ func TestFavoriteTogglesWithoutReordering(t *testing.T) { // selector, just a regression guard against reintroducing the bare-id form. func TestCardHxTargetIsValidSelectorForColonKey(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45", LastChapterNum: 45, UpdatedAt: 1_000_000, @@ -320,8 +324,8 @@ func TestCardHxTargetIsValidSelectorForColonKey(t *testing.T) { func TestChapterOverrideMovesUpdatedAt(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - before := seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + before := seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45", LastChapterNum: 45, LastChapterURL: "https://example.test/ch/45", SeriesURL: "https://example.test/solo", @@ -335,7 +339,7 @@ func TestChapterOverrideMovesUpdatedAt(t *testing.T) { t.Fatalf("chapter override status = %d, want 200", rr.Code) } - after, ok, err := store.Get("asura:solo") + after, ok, err := st.Get("asura:solo") if err != nil || !ok { t.Fatalf("Get after override: %v ok=%v", err, ok) } @@ -355,8 +359,8 @@ func TestChapterOverrideMovesUpdatedAt(t *testing.T) { func TestChapterOverrideNoOpPreservesURLAndUpdatedAt(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - before := seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + before := seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapter: "45.0", LastChapterNum: 45, LastChapterURL: "https://example.test/ch/45", SeriesURL: "https://example.test/solo", @@ -375,7 +379,7 @@ func TestChapterOverrideNoOpPreservesURLAndUpdatedAt(t *testing.T) { t.Fatalf("chapter no-op status = %d, want 200", rr.Code) } - after, ok, err := store.Get("asura:solo") + after, ok, err := st.Get("asura:solo") if err != nil || !ok { t.Fatalf("Get after no-op override: %v ok=%v", err, ok) } @@ -395,8 +399,8 @@ func TestChapterOverrideNoOpPreservesURLAndUpdatedAt(t *testing.T) { func TestChapterOverrideRejectsBadInput(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapterNum: 45, UpdatedAt: 1_000_000, }) @@ -409,7 +413,7 @@ func TestChapterOverrideRejectsBadInput(t *testing.T) { if rr.Code != http.StatusBadRequest { t.Fatalf("status = %d, want 400", rr.Code) } - after, _, _ := store.Get("asura:solo") + after, _, _ := st.Get("asura:solo") if after.LastChapterNum != 45 { t.Fatalf("chapter changed to %v on invalid input", after.LastChapterNum) } @@ -440,8 +444,8 @@ func TestMutationsOnMissingKey(t *testing.T) { func TestUIDeleteRemovesRow(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", UpdatedAt: 1_000_000, }) @@ -460,19 +464,19 @@ func TestUIDeleteRemovesRow(t *testing.T) { if !strings.Contains(body, `id="new-count" hx-swap-oob="true"`) { t.Fatalf("delete body = %q, want the out-of-band badge", body) } - if _, ok, _ := store.Get("asura:solo"); ok { + if _, ok, _ := st.Get("asura:solo"); ok { t.Fatal("row still present after delete") } } func TestUIListFavouritesTab(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", Favorite: true, UpdatedAt: 2_000_000, }) - seed(t, store, Bookmark{ + seed(t, st, store.Bookmark{ Key: "demonic:tower", Site: "demonic", SeriesID: "tower", Title: "Tower of God", Favorite: false, UpdatedAt: 1_000_000, }) @@ -493,14 +497,14 @@ func TestUIListFavouritesTab(t *testing.T) { func TestUIListNewTab(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", LastChapterNum: 10, LatestChapter: "Chapter 12", LatestChapterNum: floatPtr(12), UpdatedAt: 2_000_000, }) - seed(t, store, Bookmark{ + seed(t, st, store.Bookmark{ Key: "demonic:tower", Site: "demonic", SeriesID: "tower", Title: "Tower of God", LastChapterNum: 5, LatestChapter: "Chapter 5", LatestChapterNum: floatPtr(5), @@ -524,22 +528,22 @@ func TestUIListNewTab(t *testing.T) { // seedStatusRows puts one series in each bucket, the archived one also // favourited and with a new chapter out, so a leak into any reading-bucket tab // shows up as a failure rather than passing by accident. -func seedStatusRows(t *testing.T, store *Store) { +func seedStatusRows(t *testing.T, st *store.Store) { t.Helper() // floatPtr already exists in store_test.go — same package, reuse it. - rows := []Bookmark{ + rows := []store.Bookmark{ {Key: "asura:reading", Site: "asura", SeriesID: "reading", Title: "ReadingOne", - Status: statusReading, LastChapterNum: 10, Favorite: true, + Status: store.StatusReading, LastChapterNum: 10, Favorite: true, LatestChapter: "11", LatestChapterNum: floatPtr(11)}, {Key: "asura:archived", Site: "asura", SeriesID: "archived", Title: "ArchivedOne", - Status: statusArchived, LastChapterNum: 5, Favorite: true, + Status: store.StatusArchived, LastChapterNum: 5, Favorite: true, LatestChapter: "99", LatestChapterNum: floatPtr(99)}, {Key: "asura:finished", Site: "asura", SeriesID: "finished", Title: "FinishedOne", - Status: statusFinished, LastChapterNum: 200, Favorite: true}, + Status: store.StatusFinished, LastChapterNum: 200, Favorite: true}, } for _, b := range rows { b.UpdatedAt = time.Now().UnixMilli() - if _, err := store.Upsert(b); err != nil { + if _, err := st.Upsert(b); err != nil { t.Fatalf("seed %s: %v", b.Key, err) } } @@ -547,8 +551,8 @@ func seedStatusRows(t *testing.T, store *Store) { func TestTabsShowOnlyTheirBucket(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) cases := []struct { tab string @@ -604,16 +608,16 @@ func stripOf(t *testing.T, srv http.Handler, cfg Config, tab string) string { // updated_at-ordered list below it does not already say — and only on All. func TestRecentStripCarriesUnreadOnlyAndOnlyOnAll(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) // ReadingOne is at 10 with 11 out; the rest are not reading + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) // ReadingOne is at 10 with 11 out; the rest are not reading // A reading series that is caught up has nothing waiting, so it stays out. - caught := Bookmark{ + caught := store.Bookmark{ Key: "asura:caught", Site: "asura", SeriesID: "caught", Title: "CaughtUpOne", - Status: statusReading, LastChapterNum: 40, LatestChapter: "40", + Status: store.StatusReading, LastChapterNum: 40, LatestChapter: "40", LatestChapterNum: floatPtr(40), UpdatedAt: time.Now().UnixMilli(), } - if _, err := store.Upsert(caught); err != nil { + if _, err := st.Upsert(caught); err != nil { t.Fatalf("seed %s: %v", caught.Key, err) } @@ -633,12 +637,12 @@ func TestRecentStripCarriesUnreadOnlyAndOnlyOnAll(t *testing.T) { } // Nothing new anywhere: the strip has nothing to say and does not render. - reading, _, err := store.Get("asura:reading") + reading, _, err := st.Get("asura:reading") if err != nil { t.Fatalf("Get: %v", err) } reading.LatestChapterNum = floatPtr(reading.LastChapterNum) - if _, err := store.Upsert(reading); err != nil { + if _, err := st.Upsert(reading); err != nil { t.Fatalf("Upsert: %v", err) } // The section still ships (an out-of-band swap needs the id to exist) but @@ -652,23 +656,23 @@ func TestRecentStripCarriesUnreadOnlyAndOnlyOnAll(t *testing.T) { } } -// The strip never grows past recentCount, however many series are waiting. +// The strip never grows past web.RecentCount, however many series are waiting. func TestRecentStripCapped(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - for i := 0; i <= recentCount; i++ { - b := Bookmark{ + srv, st := newWebTestServer(t, cfg) + for i := 0; i <= web.RecentCount; i++ { + b := store.Bookmark{ Key: fmt.Sprintf("asura:new%d", i), Site: "asura", SeriesID: fmt.Sprintf("new%d", i), Title: fmt.Sprintf("Waiting%d", i), - Status: statusReading, LastChapterNum: 1, LatestChapter: "2", + Status: store.StatusReading, LastChapterNum: 1, LatestChapter: "2", LatestChapterNum: floatPtr(2), UpdatedAt: time.Now().UnixMilli() + int64(i), } - if _, err := store.Upsert(b); err != nil { + if _, err := st.Upsert(b); err != nil { t.Fatalf("seed %s: %v", b.Key, err) } } - if got := strings.Count(stripOf(t, srv, cfg, "all"), "recent-card"); got != recentCount { - t.Fatalf("strip rendered %d cards, want %d", got, recentCount) + if got := strings.Count(stripOf(t, srv, cfg, "all"), "recent-card"); got != web.RecentCount { + t.Fatalf("strip rendered %d cards, want %d", got, web.RecentCount) } } @@ -686,14 +690,14 @@ func postStatus(t *testing.T, srv http.Handler, cfg Config, key, status string) func TestUIStatusSetsBucket(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) - for _, want := range []string{statusArchived, statusFinished, statusReading} { + for _, want := range []string{store.StatusArchived, store.StatusFinished, store.StatusReading} { if rr := postStatus(t, srv, cfg, "asura:reading", want); rr.Code != http.StatusOK { t.Fatalf("set %s: status = %d, body %s", want, rr.Code, rr.Body.String()) } - b, ok, err := store.Get("asura:reading") + b, ok, err := st.Get("asura:reading") if err != nil || !ok { t.Fatalf("Get: ok=%v err=%v", ok, err) } @@ -705,21 +709,21 @@ func TestUIStatusSetsBucket(t *testing.T) { func TestUIStatusRejectsUnknownValue(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) if rr := postStatus(t, srv, cfg, "asura:reading", "dropped"); rr.Code != http.StatusBadRequest { t.Fatalf("status = %d, want 400", rr.Code) } - b, _, _ := store.Get("asura:reading") - if b.Status != statusReading { + b, _, _ := st.Get("asura:reading") + if b.Status != store.StatusReading { t.Fatalf("stored status = %q, want it untouched", b.Status) } } func TestUIStatusRequiresSession(t *testing.T) { - srv, store := newWebTestServer(t, webConfig()) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, webConfig()) + seedStatusRows(t, st) req := httptest.NewRequest(http.MethodPost, "/ui/bookmarks/asura:reading/status", strings.NewReader("status=archived")) @@ -734,15 +738,15 @@ func TestUIStatusRequiresSession(t *testing.T) { func TestUIStatusDoesNotReorderList(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) - before, _, _ := store.Get("asura:reading") + before, _, _ := st.Get("asura:reading") time.Sleep(2 * time.Millisecond) - if rr := postStatus(t, srv, cfg, "asura:reading", statusArchived); rr.Code != http.StatusOK { + if rr := postStatus(t, srv, cfg, "asura:reading", store.StatusArchived); rr.Code != http.StatusOK { t.Fatalf("status = %d", rr.Code) } - after, _, _ := store.Get("asura:reading") + after, _, _ := st.Get("asura:reading") if after.UpdatedAt != before.UpdatedAt { t.Fatalf("UpdatedAt moved %d -> %d", before.UpdatedAt, after.UpdatedAt) } @@ -750,8 +754,8 @@ func TestUIStatusDoesNotReorderList(t *testing.T) { func TestCardShowsStatusControls(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) cases := []struct { tab string @@ -788,8 +792,8 @@ func TestCardShowsStatusControls(t *testing.T) { func TestAppRendersNewTabs(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seedStatusRows(t, store) + srv, st := newWebTestServer(t, cfg) + seedStatusRows(t, st) req := httptest.NewRequest(http.MethodGet, "/", nil) req.AddCookie(sessionCookie(t, cfg)) @@ -807,10 +811,10 @@ func TestAppRendersNewTabs(t *testing.T) { // outside the swapped card, so nothing else would correct them. func TestMutationRefreshesChromeOutOfBand(t *testing.T) { cfg := webConfig() - srv, store := newWebTestServer(t, cfg) - seed(t, store, Bookmark{ + srv, st := newWebTestServer(t, cfg) + seed(t, st, store.Bookmark{ Key: "asura:solo", Site: "asura", SeriesID: "solo", Title: "Solo Leveling", - Status: statusReading, LastChapterNum: 10, LatestChapter: "Chapter 11", + Status: store.StatusReading, LastChapterNum: 10, LatestChapter: "Chapter 11", LatestChapterNum: floatPtr(11), UpdatedAt: time.Now().UnixMilli(), }) @@ -820,7 +824,7 @@ func TestMutationRefreshesChromeOutOfBand(t *testing.T) { } req := uiRequest(t, cfg, http.MethodPost, "/ui/bookmarks/asura:solo/status", - url.Values{"status": {statusArchived}}) + url.Values{"status": {store.StatusArchived}}) req.Header.Set("HX-Current-URL", "http://localhost/?tab=all") rr := httptest.NewRecorder() srv.ServeHTTP(rr, req) diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml index 78bc45b..e58e286 100644 --- a/docker-compose.prod.yml +++ b/docker-compose.prod.yml @@ -17,8 +17,19 @@ services: manga-api: # Traffic arrives over the Traefik network, not a published port. ports: !reset [] + environment: + # Must be an IP, not the DNS name — see the base file's comment on this + # same key: Chrome's DevTools HTTP handler 500s any Host header that + # isn't an IP or "localhost". + BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://172.28.0.10:9222} + depends_on: + - headless-shell + # `networks:` here replaces the base file's list entirely, so both must be + # named: `proxy` for Traefik routing, `browser` (defined in the base file) + # to keep reaching headless-shell without putting it on `proxy` too. networks: - proxy + - browser labels: - "traefik.enable=true" - "traefik.docker.network=${PROXY_NETWORK:-proxy}" @@ -36,6 +47,11 @@ services: - "traefik.http.routers.mangaweb.tls.certresolver=${TRAEFIK_CERTRESOLVER:-le}" - "traefik.http.routers.mangaweb.service=mangabm" + # headless-shell is untouched here: it keeps its `browser` network membership + # from the base file and must never join `proxy` — that network is shared + # with whatever else sits behind Traefik on this host, and an exposed + # CDP endpoint on it would be remote code execution for any of them. + networks: proxy: external: true diff --git a/docker-compose.yml b/docker-compose.yml index c8adc6b..a819f16 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -15,7 +15,7 @@ services: environment: # API_TOKEN is required — compose refuses to start without it. API_TOKEN: ${API_TOKEN:?set API_TOKEN in .env} - ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-https://asuracomic.net,https://asurascans.com,https://demonicscans.org} + ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to} DB_PATH: /data/bookmarks.db PORT: "8080" # Gates the browser UI. Unset means the web routes are not served at all. @@ -29,6 +29,17 @@ services: LATEST_CHAPTER_POLL_INTERVAL: ${LATEST_CHAPTER_POLL_INTERVAL:-10m} LATEST_CHAPTER_POLL_BATCH: ${LATEST_CHAPTER_POLL_BATCH:-14} LATEST_CHAPTER_POLL_STAGGER: ${LATEST_CHAPTER_POLL_STAGGER:-20s} + # CDP endpoint for sites behind a JavaScript challenge (kagane). Unset + # disables browser polling for those sites; the userscript still covers them. + # Must be an IP, not the "headless-shell" DNS name: Chrome's DevTools HTTP + # handler rejects the discovery request (GET /json/version) with a 500 + # unless the Host header is an IP address or "localhost" — confirmed + # 2026-08-03 against chromedp/headless-shell:stable, independent of + # chromedp's own dial logic. The sidecar's static address below exists so + # this URL survives container recreation. + BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://172.28.0.10:9222} + depends_on: + - headless-shell volumes: - bookmarks-data:/data # The userscript is served from here, read fresh on every request. Editing @@ -40,6 +51,44 @@ services: # the public internet does not. ports: - "127.0.0.1:8080:8080" + networks: + - browser + + headless-shell: + image: chromedp/headless-shell:stable + restart: unless-stopped + # Chrome allocates shared memory per tab and dies on Docker's 64MB default. + shm_size: '1gb' + # Reaps zombie renderer processes, which otherwise accumulate for the + # container's lifetime. + init: true + # Deliberately no `ports:` — an exposed CDP endpoint is remote code + # execution. Only manga-api, via the `browser` network below, may reach it. + # Don't pass --remote-debugging-address/--remote-debugging-port here: the + # image's own entrypoint (/headless-shell/run.sh) already starts Chrome on + # 127.0.0.1:9223 and fronts it with a socat proxy listening on 0.0.0.0:9222. + # Redeclaring the port flag here overrides Chrome's, so it binds 9222 + # directly (IPv6 loopback only) instead of 9223 — collides with socat's own + # bind on 9222 and leaves nothing listening on 9223, so every external + # connection to headless-shell:9222 fails with EOF. Only pass flags the + # entrypoint doesn't already set. + command: + - --disable-gpu + - --no-sandbox + networks: + browser: + # Pinned so BROWSER_WS_URL can name an IP (required, see above) that + # survives `docker compose up` recreating this container. + ipv4_address: 172.28.0.10 volumes: bookmarks-data: + +networks: + # Not `internal: true`: headless Chrome still needs outbound access to reach + # kagane.to. Isolation here comes from membership (only manga-api and + # headless-shell join it), not from cutting egress. + browser: + ipam: + config: + - subnet: 172.28.0.0/24 diff --git a/docs/design-system.md b/docs/design-system.md index a19e051..9aed761 100644 --- a/docs/design-system.md +++ b/docs/design-system.md @@ -1,16 +1,16 @@ # Cinder — mangaBookmark design system -Source of truth: the Claude Design doc **Cinder Sheet** -(`cfa39183-8874-4f76-987c-afef14dceebb`, files `Cinder Sheet.dc.html` for the -static spec and `Cinder Sheet App.dc.html` for the interactive one). This file -records the rules that got implemented so a future agent can extend the UI -without re-reading the design. +Source of truth: the Claude Design project **mangaBookmark Web UI** +(`969ac210-fe02-4c01-ae1b-9a271dcc779a`, `index.html` + siblings +`archived.html`/`fav.html`/`finished.html`/`new.html`/`login.html`/`mobile.html`, +`style.css`, `filter.js`). This file records the rules that got implemented so +a future agent can extend the UI without re-reading the design. Implemented in: | Surface | Files | | --- | --- | -| Web UI (login, list, card, empty, errors) | `backend/static/style.css`, `backend/templates/{app,card,list,login,icons}.html`, `backend/static/filter.js` | +| Web UI (login, list, card, empty, errors) | `backend/static/style.css`, `backend/templates/{app,card,list,login,chrome,icons}.html`, `backend/static/filter.js` | | Userscript panel (Shadow DOM) | `userscript/manga-bookmark.user.js` — `TEMPLATE` and `CSS` at the bottom of the IIFE | ## 1. The one idea @@ -19,9 +19,12 @@ Implemented in: allowed to be crimson: its title turns `--paper-hot` and sits on a 1px ember underline sized to the text, its cover gains a 3px ember rule at the foot, and its `Ch N out` meta and play icon go ember. Everything else — favourites, -status, chrome — stays cool. If a new feature wants to be noticed, it does *not* -get to borrow the ember; find a typographic answer (weight, italic, a rule) or -use brass, which is already spoken for by favourites. +status, chrome, destruction — stays off that one colour. Destruction gets its +own token (`--danger`, a duller oxblood) precisely so a remove confirm is +never mistaken across the room for an unread chapter. If a new feature wants +to be noticed, it does *not* get to borrow the ember; find a typographic +answer (weight, italic, a rule) or reach for one of the named action accents +(§2). Corollaries: @@ -34,7 +37,7 @@ Corollaries: - **Three type roles, never mixed.** Display serif for anything a human reads as a name (brand, titles, tabs, primary buttons, empty-state headings). Mono small-caps for machine facts (site, chapter numbers, labels, status, badges, - ghost buttons). Sans for prose only (empty-state body, hints). + ghost buttons, the action key). Sans for prose only (empty-state body, hints). ## 2. Tokens @@ -44,7 +47,7 @@ Defined once in `backend/static/style.css` `:root`, mirrored in the userscript's | Token | Dark | Light | Use | | --- | --- | --- | --- | | `--ink` | `#100f0e` | `#f7f4ef` | page | -| `--ash` | `#161413` | `#efeae3` | recessed panel (chapter form, toast) | +| `--ash` | `#161413` | `#efeae3` | recessed panel (chapter form) | | `--dim` | `#0d0c0b` | `#f1ede7` | archived / finished row background | | `--rule` | `#221f1d` | `#e0dad2` | hairline between sheets, button borders | | `--rule-soft` | `#1a1817` | `#e8e3dc` | the measure's own side edges | @@ -54,23 +57,34 @@ Defined once in `backend/static/style.css` `:root`, mirrored in the userscript's | `--paper-hot` | `#f0d3cb` | `#a33018` | title of a series with a new chapter | | `--paper-dim` | `#ddd5cb` | `#191715` | resting title | | `--mute` | `#8d857c` | `#6b645d` | secondary text, idle icons | -| `--mute-2` | `#5a5450` | `#857d75` | eyebrow labels, hints | +| `--mute-2` | `#877f76` | `#6c655e` | eyebrow labels, hints (must clear 4.5:1 on both `--ink` and `--ash`) | | `--faint` | `#3a3733` | `#c9c2ba` | the `/` separators in a meta line | | `--faint-2` | `#57504b` | `#a8a098` | cover monogram | | `--ember` | `#e0452c` | `#c23a22` | heat — see §1 | -| `--ember-wash` | `#1a1211` | `#fbeee9` | ember-tinted surface (confirm, error) | +| `--ember-wash` | `#1a1211` | `#fbeee9` | ember-tinted surface | | `--ember-ink` | `#150907` | `#fff` | text on solid ember | | `--ember-soft` | `#eda798` | `#8d2c17` | text on ember wash | -| `--brass` | `#b8912f` | `#8a681c` | favourites, and only favourites | -| `--trash` | `#6b5450` | `#a98276` | remove, at rest | +| `--danger` | `#cf5c4d` | `#97362a` | destruction — remove confirm, never the same as `--ember` | +| `--danger-wash` | `#211311` | `#fbe9e5` | remove-confirm surface | +| `--danger-ink` | `#150808` | `#fff` | text on solid danger | +| `--danger-soft` | `#e2aaa1` | `#7c2c22` | text on danger wash | +| `--brass` | `#b8912f` | `#8a681c` | favourite — a cooler second metal | +| `--slate` | `#7fa0c0` | `#3f6689` | archive accent | +| `--moss` | `#7fae86` | `#3d6c46` | finished accent | +| `--clay` | `#b5906f` | `#7c5533` | set-chapter accent | +| `--trash` | `#977671` | `#8c6558` | remove, at rest — icons need 3:1, not 4.5:1 | +| `--play-hot-line` | `#3a1d18` | `#f0cfc6` | desktop cell border, play when `.is-new` | +| `--fav-line` | `#332b14` | `#e3d3a4` | desktop cell border, favourite when on | | `--asura` | `#7d93a5` | `#4f6b80` | site tag | | `--demonic` | `#a98a78` | `#8a6a55` | site tag | | `--hatch` / `--hatch-dim` | 135° 5px stripe | paper stripe | missing-cover slot | -Dark is the default (`color-scheme: dark light`); light is a -`@media (prefers-color-scheme: light)` override of the same names. **Any new -colour must be added in both branches** — light is not a filter over dark, the -hues are re-tuned. +`--slate`/`--moss`/`--clay`/`--brass` are held at the same weight deliberately: +one accent per action, so a press says which lane it belongs to, with none of +them competing with ember. Dark is the default (`color-scheme: dark light`); +light is a `@media (prefers-color-scheme: light)` override of the same names. +**Any new colour must be added in both branches** — light is not a filter over +dark, the hues are re-tuned. ## 3. Type @@ -83,11 +97,10 @@ hues are re-tuned. The web UI **self-hosts** all three: five latin-subset woff2 files in `backend/static/fonts/` (~120 KB total), declared by the `@font-face` block at the top of `style.css` and embedded in the binary by the existing -`//go:embed static`. There is no request to Google — this UI is read in Bromite, -where `fonts.googleapis.com` is routinely blocked, and over a LAN with no -internet route. `staticHandler()` in `web.go` registers the `.woff2` MIME type -because Go's built-in table lacks it and the scratch image has no -`/etc/mime.types`. +`//go:embed static`. There is no request to Google — this UI needs to survive +on a LAN with no internet route. `staticHandler()` in `web.go` registers the +`.woff2` MIME type because Go's built-in table lacks it and the scratch image +has no `/etc/mime.types`. Adding a weight means adding a file: grab the *latin* `@font-face` block from `https://fonts.googleapis.com/css2?...` **with a browser User-Agent** (Google @@ -102,37 +115,57 @@ root is at the mercy of the host site's CSP. Recurring specs (copy these rather than inventing sizes): -- Brand: `400 26px/1 display`, with `<em>` in ember italic — `manga<em>Bookmark</em>`. -- Row title: `400 19px/1.2 display` (21px ≥720px). -- Tab: `400 17px display` (18px ≥720px), active gets `border-bottom: 2px` in +- Brand: `400 26px/1 display` (`30px` ≥720px), inline SVG mark (§4) + `<em>` in + ember italic — `manga<em>Bookmark</em>`. +- Row title: `400 21px/1.2 display` (`22px` ≥720px). +- Tab: `400 17px display` (`18px` ≥720px), active gets `border-bottom: 2px` in `--paper` (`--ember` for Updated) plus `margin-bottom: -1px` so it lands on the row's own hairline. -- Meta / label / badge: `500 10px mono`, `letter-spacing: .12em`, - `text-transform: uppercase`. Eyebrows ("CONTINUE READING") use `.2em`. +- Meta / label / badge / action key: `500 10–11px mono`, `letter-spacing: + .04em`–`.2em`, `text-transform: uppercase`. Eyebrows use the widest tracking. - Empty-state heading: `400 20px display`; body `400 14px/1.6 sans`, `max-width: 44ch`. -- Primary button: `--paper` fill, `--ink` text, `400 17px display`, no border radius. +- Primary button: `--paper` fill, `--ink` text, `400 17–19px display`, no border radius. - Ghost button: mono small-caps, transparent, `border-bottom: 1px --field-line`. ## 4. Components (web UI) ``` .sheet - .topbar .brand + .ghost (log out) + .topbar .brand (mark + wordmark) + .ghost (log out) .chrome .searchbar + nav.tabs (column on phone, row ≥720px via order:) + .keyrow one-line action key: Read / Fav / Chapter / Archive / Done / Delete .recent h2 eyebrow + .recent-strip > a.recent-card main#list article.card … | .empty ``` +**Brand mark**: an inline `<svg class="mark">` (`viewBox="0 0 200 172"`), +defined once in `chrome.html`'s `mark` template and reused by `app.html` and +`login.html` so it takes the page's `--ink`/`currentColor`/`--ember` rather +than shipping as a static asset. The blade at its centre strokes +`var(--logo-blade, var(--ember))` — override that custom property, don't +duplicate the SVG, if a surface ever needs a different blade colour. Drawn at +a 5px stroke on a 200-unit grid; at brand size that thins out, so `.brand .mark +g` nudges `stroke-width` up to `6.5` rather than scaling the artwork down. + +**Action key** (`.keyrow`): one permanent line under the tabs naming what +every icon in `.actions` does — Read / Fav / Chapter / Archive / Done / +Delete — so the icon strip on a card is never a guess. On a phone each pair +stacks icon-over-word (`flex-direction: column`) so the word gets the full +cell width and can stay in long form; ≥720px it lays out icon-beside-word and +switches the `.short`/`.full` label pair. `.pair.brass` and `.pair.trash` +carry their icon's resting accent so the key itself teaches the colour +vocabulary in §1/§2. + `article.card` — the row, and the only per-series component: ``` article.card[.is-new|.is-dim]#card-<key>[data-title] .row - a.cover img | span.monogram, + span.foot-rule[.brass] + a.cover[tabindex="-1" aria-hidden] img | span.monogram, + span.foot-rule[.brass] .body .title-line (h3.title + svg.fav-mark) , p.meta - .actions play, favourite, chapter, archive|restore, finish, remove - form.chapter-form[hidden] .hint + .field(input + Save) - .confirm-row[hidden] span + (Remove, Cancel) + .actions play, favourite, chapter | lifecycle: archive/restore, finish, remove + form.chapter-form[hidden] .hint + .field(input + Save) + .hint (latest known) + .confirm-row[.calm][hidden] × one per lifecycle action, span + (go/danger-solid, Cancel) p.error-inline[hidden] ``` @@ -143,9 +176,29 @@ Rules that are easy to break: dim rule is a descendant selector off those two classes, so a new sub-element inherits the state for free. - `.actions` is `flex: 1 0 100%` inside `.row`, which is what makes it a - full-width strip under the row on a phone and a group of 40px squares beside + full-width strip under the row on a phone and a group of 44px squares beside the row at ≥720px. Cells are 46px tall on phone (thumb target) and divided by `border-right: 1px var(--rule)`, last child none. +- Three clusters by consequence, in this order: navigate (`.play`) | organize + (`.fav`, `.pencil`) | lifecycle (`.box`/`.restore`, `.finish`, `.remove`, + each carrying the `.lifecycle` class). Lifecycle cells sit on a recessed + `--ash` ground so the thumb reads "this one moves the series" before it + reads which icon it landed on; ≥720px they separate by a 10px gap instead of + the phone's inset hairline. +- Every lifecycle button that moves a series out of the list is + **confirm-gated**: it opens its own `.confirm-row` (`archive`, `finish`, + `remove` — `toggleConfirmRow(key, kind)` in `filter.js`). Archive and finish + ask in `.calm` grey since they're reversible; remove alone gets the + `--danger-wash` treatment and names the series in its question. Restore + fires instantly — no confirm — because it's the reversal. +- Per-action hover/press accent: `.fav` → `--brass`, `.pencil` → `--clay`, + `.box` → `--slate`, `.finish` → `--moss`. `.play` stays paper/ember (ember + only when `.is-new`). `.remove` stays `--trash` at rest, `--danger` on + hover. Desktop cell borders follow the same accent on hover + (`border-color: currentColor`); the two coloured *resting* states + (`.is-new .play`, `.fav.on`) get their own dim border tokens + (`--play-hot-line`, `--fav-line`) instead of the full accent, since a + resting border needs less contrast than a hover one. - Icons are `<use href="#i-…">` against the sprite in `templates/icons.html`, included once by `app.html`. htmx-swapped card fragments reference the page's sprite, so a card never inlines a path. New icon → add a `<symbol>` @@ -155,11 +208,15 @@ Rules that are easy to break: - Cover foot rule: ember when new, brass when favourite-and-not-new. Never both. - `[hidden] { display: none !important; }` is load-bearing — every disclosure panel is a flex container, and `display` beats `hidden`. -- Busy state is `.card.htmx-request::before`, a 1px ember bar sliding across the +- Busy state is `.card.htmx-request::before`, a 1px grey bar sliding across the top hairline (`barSlide`), plus the action strip at `opacity: .5`. Never a - spinner. -- `.open` on the pencil / trash cell marks which panel is showing; `filter.js` - `togglePanel()` owns that class alongside `hidden`. + spinner, and deliberately `--mute` not `--ember` — on a list screen ember + means "new chapter" and nothing else, so a system state can't borrow it. +- `.open` on the pencil / lifecycle cell marks which panel is showing; + `filter.js` `togglePanel()`/`toggleConfirmRow()` own that class alongside + `hidden`. An open lifecycle cell needs the next surface step up from + `--hover` (`--rule`) to stay legible as the panel's owner, since the panel + itself already sits on `--ash`. ## 5. Components (userscript panel) @@ -167,9 +224,9 @@ Same tokens, same heat rule, structure unchanged from before the revamp (`#fab`/`#hit`, `#panel`, `#nav` chips, `#context`, `#tabs`, `#list` of `.item`). Cinder-specific: `.item.hot` (new chapter) and `.item.dim` (archived) mirror `.is-new` / `.is-dim`; chips and `.btn`s are mono small-caps with hairline -borders instead of pills; loading is the same sliding ember hairline (`.spinner` -is now a 1px bar, not a rotating ring); toasts are `--ash` with a 2px left rule, -ember-washed when `.err`. +borders instead of pills; loading is the same sliding hairline (`.spinner` +is a 1px bar, not a rotating ring); toasts are `--ash` with a 2px left rule, +`--danger`-washed when `.err`. **Do not touch** the FAB geometry while restyling: `#fab` keeps `touch-action: none`, must not regain `overflow: hidden`, and `#hit` keeps the @@ -179,36 +236,48 @@ ember-washed when `.err`. ## 6. Motion Three animations, all ≤ 1.15s and all disabled under -`prefers-reduced-motion: reduce`: +`prefers-reduced-motion: reduce` (pseudo-elements need naming explicitly in +that query — `*` does not match `::before`/`::after`, so the busy bar and +error dot are listed by name and fall back to their static drawn form): - `sheetIn` — 180ms fade + 4px rise, on a row and on each disclosure panel. -- `barSlide` — the burning hairline, for any busy state. -- `emberPulse` — the 5px dot on `.error-inline`. +- `barSlide` — the sliding hairline, for any busy state. +- `mutePulse` — the 5px dot on `.error-inline`. No transforms on hover, no scale, no easing curves beyond `ease-out`/`linear`. ## 7. Accessibility floor (not negotiable) -- Touch targets on the phone layout are 44–46px; the 40px desktop cells are +- Touch targets on the phone layout are 44–46px; the 44px desktop cells are pointer-only (≥720px). - Every icon-only control keeps `title` + `aria-label`; the SVG inside is - `aria-hidden`. + `aria-hidden`. Lifecycle buttons also carry `aria-expanded` + + `aria-controls` pointing at their `.confirm-row`. - The cover link is `tabindex="-1" aria-hidden="true"` because the title link and the play cell already reach the same URL — do not make it a third tab stop. - Tabs keep `role="tab"` / `role="tablist"`; the active one is marked by class, and `setActiveTab()` in `filter.js` maintains it after an htmx swap. +- `.confirm-row` and `.error-inline` are `role="group"`/`role="status"` with + `aria-live="polite"` so a disclosure opening is announced. - Light and dark are both first-class. Check any new colour in both. ## 8. Adding something new — checklist 1. Can it be a hairline, a small-caps label, or a serif line instead of a new component? Prefer that. -2. Tokens only, both colour branches. +2. Tokens only, both colour branches. A new action gets its own named accent + (like `--slate`/`--moss`/`--clay`) at the same weight as the existing set — + never reuse `--ember` or `--danger` for anything but their one meaning. 3. If it is per-series, hang it off `.is-new` / `.is-dim` rather than adding a third state class. -4. Icon → `templates/icons.html`; nothing inlines SVG paths. -5. Phone first (44px targets, single column), then the ≥720px block. -6. Verify: `cd backend && go test ./...`, then run the binary and screenshot +4. If it removes a series from the current view (archive/finish/remove-shaped), + it is confirm-gated via its own `.confirm-row` — no exceptions, restore is + the only instant action because it's the one that's reversible by nature. +5. Icon → `templates/icons.html`; nothing inlines SVG paths. Brand mark stays + the one exception (`chrome.html`'s `mark` template), since it takes + page-level custom properties the sprite can't carry per-instance. +6. Phone first (44px targets, single column), then the ≥720px block. +7. Verify: `cd backend && go test ./...`, then run the binary and screenshot both widths and both colour schemes (Playwright: `emulateMedia`, `setViewportSize`; disable the browser cache — `/static/*` is served with `max-age=3600`, and templates are `go:embed`ed so the binary must be rebuilt diff --git a/userscript/manga-bookmark.user.js b/userscript/manga-bookmark.user.js index 9392de1..02b31d5 100644 --- a/userscript/manga-bookmark.user.js +++ b/userscript/manga-bookmark.user.js @@ -1,14 +1,16 @@ // ==UserScript== // @name Manga Bookmark Sync // @namespace mangabm -// @version 1.5.0 -// @description Track read progress on Asura & Demonic and sync to a self-hosted backend. Bromite-compatible (no GM_* APIs). +// @version 1.6.0 +// @description Track read progress on Asura, Demonic, Comix & Kagane and sync to a self-hosted backend. Bromite-compatible (no GM_* APIs). // @author you // @downloadURL https://manga-api.violetcrown.my.id/u/40d79969b5442f90df4fe306a092c7c50e7b4a7a98099f98cc398f4fb374b1df/manga-bookmark.user.js // @updateURL https://manga-api.violetcrown.my.id/u/40d79969b5442f90df4fe306a092c7c50e7b4a7a98099f98cc398f4fb374b1df/manga-bookmark.user.js // @match https://asuracomic.net/* // @match https://asurascans.com/* // @match https://demonicscans.org/* +// @match https://comix.to/* +// @match https://kagane.to/* // @run-at document-idle // @noframes // ==/UserScript== @@ -80,6 +82,14 @@ return slug.replace(/-[0-9a-f]{8}$/, ""); } + // comix path segments are "<id>-<slug>", where the slug is a rendering of the + // current title and changes when a series is renamed. Only the id is the + // identity; seriesUrl keeps the full segment because navigation needs it. + function comixSeriesId(segment) { + const i = segment.indexOf("-"); + return i === -1 ? segment : segment.slice(0, i); + } + const asura = { site: "asura", // asuracomic.net deep links 301 to the asurascans.com *root*, dropping the @@ -208,7 +218,162 @@ }, }; - const ADAPTERS = [asura, demonic]; + const comix = { + site: "comix", + matches: (loc) => /(^|\.)comix\.to$/.test(loc.hostname), + detect(loc) { + const path = loc.pathname; + // /title/<id>-<slug>/<uploadId>-chapter-<n>. Several uploads (different + // groups or languages) share one chapter number; the number is the + // progress identity, the upload id is not. + let m = path.match(/^\/title\/([^/]+)\/[^/]*-chapter-([\d.]+)/); + if (m) { + const num = parseFloat(m[2]); + return { + type: "chapter", + site: this.site, + seriesId: comixSeriesId(m[1]), + title: cleanTitle(meta("og:title")), + cover: coverFromPage(), + seriesUrl: loc.origin + "/title/" + m[1], + chapterLabel: "Chapter " + m[2], + chapterNum: isNaN(num) ? null : num, + chapterUrl: loc.href, + }; + } + // /title/<id>-<slug> + m = path.match(/^\/title\/([^/?#]+)\/?$/); + if (m) { + return { + type: "series", + site: this.site, + seriesId: comixSeriesId(m[1]), + title: cleanTitle(meta("og:title")), + cover: coverFromPage(), + seriesUrl: loc.origin + "/title/" + m[1], + chapterLabel: null, + chapterNum: null, + chapterUrl: null, + }; + } + return { type: "other" }; + + // comix chapter og:title is "<Title> · Ch.<n>"; series is clean. + function cleanTitle(t) { + if (!t) return ""; + return t.replace(/\s*·\s*Ch\.[\d.]+\s*$/i, "").trim(); + } + + // comix serves no og:image, so this is the one adapter that has to read + // the DOM for a cover. Matching on alt rather than a class keeps it off + // the site's styling: the cover is the image whose alt is the title. + // Do not "simplify" this into meta("og:image") — that returns null. + function coverFromPage() { + const title = cleanTitle(meta("og:title")); + if (!title || !document.querySelectorAll) return ""; + for (const img of document.querySelectorAll("img[alt]")) { + if (img.getAttribute("alt") === title) return img.getAttribute("src") || ""; + } + return ""; + } + }, + // Scoped to this series' own id prefix so a recommendation strip's links + // cannot win the maximum. seriesId is passed in because the anchors alone + // do not say which series the page belongs to. + latestChapterFromAnchors(anchors, seriesId) { + let best = null; + const re = new RegExp("^/title/" + seriesId + "-[^/]*/[^/]*-chapter-([\\d.]+)"); + for (const a of anchors) { + const m = a.href.match(re); + if (!m) continue; + const num = parseFloat(m[1]); + if (isNaN(num)) continue; + if (!best || num > best.num) best = { num, label: "Chapter " + m[1] }; + } + return best; + }, + }; + + const kagane = { + site: "kagane", + matches: (loc) => /(^|\.)kagane\.to$/.test(loc.hostname), + detect(loc) { + const path = loc.pathname; + const UUID = "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"; + // /series/<uuid>/reader/<bookUuid> — no chapter number anywhere in the + // URL, so it comes out of og:title instead. + let m = path.match(new RegExp("^/series/(" + UUID + ")/reader/" + UUID)); + if (m) { + const num = chapterNumFromTitle(meta("og:title")); + return { + type: "chapter", + site: this.site, + seriesId: m[1], + title: cleanTitle(meta("og:title")), + cover: meta("og:image") || "", + seriesUrl: loc.origin + "/series/" + m[1], + chapterLabel: num === null ? null : "Chapter " + num, + chapterNum: num, + chapterUrl: loc.href, + }; + } + // /series/<uuid> + m = path.match(new RegExp("^/series/(" + UUID + ")/?$")); + if (m) { + return { + type: "series", + site: this.site, + seriesId: m[1], + title: cleanTitle(meta("og:title")), + cover: meta("og:image") || "", + seriesUrl: loc.origin + "/series/" + m[1], + chapterLabel: null, + chapterNum: null, + chapterUrl: null, + }; + } + return { type: "other" }; + + // Reader og:title is "<Title> - Chapter <n> - <episode name>". + function chapterNumFromTitle(t) { + const m = t && t.match(/\s-\sChapter\s([\d.]+)\s/); + if (!m) return null; + const num = parseFloat(m[1]); + return isNaN(num) ? null : num; + } + + function cleanTitle(t) { + if (!t) return ""; + return t.replace(/\s-\sChapter\s[\d.]+\s-\s.*$/i, "").trim(); + } + }, + // Reader hrefs are uuids with no number in them, so no maximum can be taken + // from anchors at all. latestChapterFromApi replaces this path entirely. + latestChapterFromAnchors() { + return null; + }, + // Same-origin only: the request needs the Cloudflare clearance cookie that + // this browser already holds for kagane.to. Called cross-origin it would be + // challenged and return nothing. + async latestChapterFromApi(seriesId) { + try { + const res = await fetch("/api/v2/series/" + seriesId); + if (!res.ok) return null; + const data = await res.json(); + let best = null; + for (const b of (data && data.series_books) || []) { + const num = parseFloat(b.chapter_no); + if (isNaN(num)) continue; + if (!best || num > best.num) best = { num, label: "Chapter " + b.chapter_no }; + } + return best; + } catch (e) { + return null; + } + }, + }; + + const ADAPTERS = [asura, demonic, comix, kagane]; function detect() { const loc = window.location; @@ -227,9 +392,11 @@ } // Highest chapter the site lists, or null when the markup yields nothing. - function computeLatestChapter(site, anchors) { + // seriesId is only consulted by adapters whose pages carry other series' + // chapter links; the rest ignore it. + function computeLatestChapter(site, anchors, seriesId) { const a = adapterFor(site); - return a ? a.latestChapterFromAnchors(anchors) : null; + return a ? a.latestChapterFromAnchors(anchors, seriesId) : null; } function currentSite() { @@ -671,8 +838,8 @@ title: existing.title || p.title || p.seriesId, series_url: existing.series_url || p.seriesUrl || "", cover: existing.cover || p.cover || "", - last_chapter: p.chapterLabel || "", - last_chapter_num: p.chapterNum, + last_chapter: p.chapterLabel || existing.last_chapter || "", + last_chapter_num: p.chapterNum != null ? p.chapterNum : existing.last_chapter_num, last_chapter_url: p.chapterUrl || "", updated_at: Date.now(), }); @@ -750,14 +917,15 @@ if (!existing) return; applyLatestChapterIfChanged( existing, - computeLatestChapter(p.site, anchorsFromDocument(document)) + computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId) ); } - // Everything else is only learned by fetching a series page. Same-origin - // only: these requests carry the session that gets us past the site's bot - // checks, which a request to the other site (or from a server) would not. - // One series per navigation keeps it indistinguishable from browsing. + // Everything else is only learned by fetching a series page — or, where the + // site offers one, its JSON API. Same-origin only: these requests carry the + // session that gets us past the site's bot checks, which a request to the + // other site (or from a server) would not. A few series per navigation keeps + // it indistinguishable from browsing. async function backgroundRefreshLatest() { const site = currentSite(); if (!site) return; @@ -772,15 +940,21 @@ .slice(0, LATEST_CHECK_BATCH); if (due.length === 0) return; + const adapter = adapterFor(site); for (const bm of due) { // Recorded even when the fetch fails, so a broken series is retried on // the next throttle window rather than on every single page load. checked[bm.key] = Date.now(); try { - const res = await fetch(bm.series_url, { credentials: "same-origin" }); - if (!res.ok) continue; - const html = await res.text(); - const latest = computeLatestChapter(bm.site, anchorsFromHTML(html)); + let latest; + if (adapter && adapter.latestChapterFromApi) { + latest = await adapter.latestChapterFromApi(bm.series_id); + } else { + const res = await fetch(bm.series_url, { credentials: "same-origin" }); + if (!res.ok) continue; + const html = await res.text(); + latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id); + } await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest); } catch (e) { /* offline or blocked — try again after the throttle window */ @@ -1414,6 +1588,8 @@ <a class="chip" href="${WEB_BASE}" target="_blank" rel="noopener">Web</a> <a class="chip" href="https://asurascans.com" target="_blank" rel="noopener">Asura</a> <a class="chip" href="https://demonicscans.org" target="_blank" rel="noopener">Demonic</a> + <a class="chip" href="https://comix.to" target="_blank" rel="noopener">Comix</a> + <a class="chip" href="https://kagane.to" target="_blank" rel="noopener">Kagane</a> <button id="pending" class="chip pending" hidden>⟳ 0 pending</button> </div> <section id="context"></section> @@ -1608,7 +1784,7 @@ // Exposes pure logic only — see userscript/test/logic.test.js. // ============================================================ if (typeof window === "undefined" && typeof module === "object" && module.exports) { - module.exports = { stripBuildHash, asura, demonic, anchorsFromHTML, statusOf }; + module.exports = { stripBuildHash, comixSeriesId, asura, demonic, comix, kagane, anchorsFromHTML, statusOf }; } // ============================================================ diff --git a/userscript/test/logic.test.js b/userscript/test/logic.test.js index 00990cf..f6b96c3 100644 --- a/userscript/test/logic.test.js +++ b/userscript/test/logic.test.js @@ -31,6 +31,9 @@ globalThis.location = { // og: meta tags the adapters read through meta(). Reassigned per test. let metaTags = {}; +// img[alt] elements comix's coverFromPage() scans. Reassigned per test; each +// entry is {alt, src}. +let pageImages = []; globalThis.document = { querySelector(sel) { const m = sel.match(/^meta\[property="([^"]+)"\]$/); @@ -38,14 +41,23 @@ globalThis.document = { const v = metaTags[m[1]]; return v == null ? null : { getAttribute: () => v }; }, + querySelectorAll(sel) { + if (sel !== "img[alt]") return []; + return pageImages.map((img) => ({ + getAttribute: (attr) => img[attr] ?? null, + })); + }, addEventListener() {}, body: undefined, }; const { stripBuildHash, + comixSeriesId, asura, demonic, + comix, + kagane, anchorsFromHTML, statusOf, } = require("../manga-bookmark.user.js"); @@ -166,6 +178,187 @@ test("demonic.latestChapterFromAnchors parses chaptered.php links, including &am assert.deepEqual(best, { num: 12, label: "Chapter 12" }); }); +// ============================================================ +// comix — the id prefix is the stable identity; the slug tail is a title +// rendering that changes when a series is renamed. +// ============================================================ + +test("comixSeriesId keeps only the prefix before the first dash", () => { + assert.equal(comixSeriesId("n8we-dungeons-and-crayons"), "n8we"); + assert.equal(comixSeriesId("20xzd-the-baddest-villainess-is-back"), "20xzd"); +}); + +test("comixSeriesId leaves a bare id untouched", () => { + assert.equal(comixSeriesId("n8we"), "n8we"); +}); + +test("comix detects a series page", () => { + metaTags = { "og:title": "Dungeons and Crayons" }; + const p = comix.detect(loc("https://comix.to/title/n8we-dungeons-and-crayons")); + assert.equal(p.type, "series"); + assert.equal(p.site, "comix"); + assert.equal(p.seriesId, "n8we"); + assert.equal(p.title, "Dungeons and Crayons"); + assert.equal(p.seriesUrl, "https://comix.to/title/n8we-dungeons-and-crayons"); + assert.equal(p.chapterNum, null); +}); + +test("comix detects a chapter page and strips the Ch. suffix from the title", () => { + metaTags = { "og:title": "Dungeons and Crayons · Ch.80" }; + const p = comix.detect( + loc("https://comix.to/title/n8we-dungeons-and-crayons/11139891-chapter-80") + ); + assert.equal(p.type, "chapter"); + assert.equal(p.seriesId, "n8we"); + assert.equal(p.title, "Dungeons and Crayons"); + assert.equal(p.chapterNum, 80); + assert.equal(p.chapterLabel, "Chapter 80"); + assert.equal(p.seriesUrl, "https://comix.to/title/n8we-dungeons-and-crayons"); +}); + +test("comix parses decimal chapter numbers", () => { + metaTags = { "og:title": "Dungeons and Crayons · Ch.80.5" }; + const p = comix.detect( + loc("https://comix.to/title/n8we-dungeons-and-crayons/11139891-chapter-80.5") + ); + assert.equal(p.chapterNum, 80.5); +}); + +test("comix.detect reads the cover from an img whose alt matches the cleaned title", () => { + metaTags = { "og:title": "Dungeons and Crayons · Ch.80" }; + pageImages = [ + { alt: "Some Other Series", src: "https://cdn.example/other.jpg" }, + { alt: "Dungeons and Crayons", src: "https://cdn.example/cover.jpg" }, + ]; + const p = comix.detect( + loc("https://comix.to/title/n8we-dungeons-and-crayons/11139891-chapter-80") + ); + assert.equal(p.cover, "https://cdn.example/cover.jpg"); +}); + +test("comix.detect leaves cover empty when no img alt matches the title", () => { + metaTags = { "og:title": "Dungeons and Crayons" }; + pageImages = [{ alt: "Some Other Series", src: "https://cdn.example/other.jpg" }]; + const p = comix.detect(loc("https://comix.to/title/n8we-dungeons-and-crayons")); + assert.equal(p.cover, ""); + pageImages = []; +}); + +test("comix ignores unrelated paths", () => { + assert.equal(comix.detect(loc("https://comix.to/browse")).type, "other"); +}); + +test("comix takes the max chapter and ignores other series' links", () => { + const anchors = anchorsFromHTML(` + <a href="/title/n8we-dungeons-and-crayons/11123327-chapter-79">Chapter 79</a> + <a href="/title/n8we-dungeons-and-crayons/11139891-chapter-80">Chapter 80</a> + <a href="/title/n8we-dungeons-and-crayons/10794753-chapter-78">Chapter 78</a> + <a href="/title/qqwrm-full-time-awakening/99999999-chapter-999">Chapter 999</a> + `); + assert.deepEqual(comix.latestChapterFromAnchors(anchors, "n8we"), { + num: 80, + label: "Chapter 80", + }); +}); + +test("comix latest returns null when no chapter links are present", () => { + assert.equal(comix.latestChapterFromAnchors(anchorsFromHTML("<a href='/browse'>x</a>"), "n8we"), null); +}); + +test("asura and demonic ignore the seriesId argument", () => { + const asuraAnchors = anchorsFromHTML( + `<a href="/comics/x-aabbccdd/chapter/12"><span>Chapter 12</span></a>` + ); + assert.deepEqual(asura.latestChapterFromAnchors(asuraAnchors, "ignored"), { + num: 12, + label: "Chapter 12", + }); +}); + +// ============================================================ +// kagane — reader URLs carry uuids and no chapter number, so the number has to +// come out of og:title. When that fails, chapterNum is null and the existing +// progress logic records the current chapter rather than guessing. +// ============================================================ + +const KAGANE_SERIES = "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"; +const KAGANE_BOOK = "019fa2e0-6dbd-73ca-b40b-fe06ab75eb0e"; + +test("kagane detects a series page", () => { + metaTags = { + "og:title": "Infinite Decryption: The Strongest Level 0", + "og:image": "https://kagane.to/api/v2/image/abc/compressed", + }; + const p = kagane.detect(loc("https://kagane.to/series/" + KAGANE_SERIES)); + assert.equal(p.type, "series"); + assert.equal(p.site, "kagane"); + assert.equal(p.seriesId, KAGANE_SERIES); + assert.equal(p.title, "Infinite Decryption: The Strongest Level 0"); + assert.equal(p.cover, "https://kagane.to/api/v2/image/abc/compressed"); + assert.equal(p.seriesUrl, "https://kagane.to/series/" + KAGANE_SERIES); +}); + +test("kagane reads the chapter number out of og:title", () => { + metaTags = { + "og:title": "Infinite Decryption: The Strongest Level 0 - Chapter 41 - Episode 41", + "og:image": "https://kagane.to/api/v2/image/abc/compressed", + }; + const p = kagane.detect( + loc("https://kagane.to/series/" + KAGANE_SERIES + "/reader/" + KAGANE_BOOK) + ); + assert.equal(p.type, "chapter"); + assert.equal(p.seriesId, KAGANE_SERIES); + assert.equal(p.title, "Infinite Decryption: The Strongest Level 0"); + assert.equal(p.chapterNum, 41); + assert.equal(p.chapterLabel, "Chapter 41"); + assert.equal(p.seriesUrl, "https://kagane.to/series/" + KAGANE_SERIES); +}); + +test("kagane yields a null chapterNum when og:title has no chapter", () => { + metaTags = { "og:title": "Infinite Decryption: The Strongest Level 0" }; + const p = kagane.detect( + loc("https://kagane.to/series/" + KAGANE_SERIES + "/reader/" + KAGANE_BOOK) + ); + assert.equal(p.type, "chapter"); + assert.equal(p.chapterNum, null); +}); + +test("kagane ignores unrelated paths", () => { + assert.equal(kagane.detect(loc("https://kagane.to/search")).type, "other"); +}); + +test("kagane anchor scanning is structurally impossible and returns null", () => { + const anchors = anchorsFromHTML( + `<a href="/series/${KAGANE_SERIES}/reader/${KAGANE_BOOK}">Chapter 41</a>` + ); + assert.equal(kagane.latestChapterFromAnchors(anchors, KAGANE_SERIES), null); +}); + +test("kagane latestChapterFromApi takes the max chapter_no", async () => { + globalThis.fetch = async (url) => { + assert.equal(url, "/api/v2/series/" + KAGANE_SERIES); + return { + ok: true, + json: async () => ({ + series_books: [ + { chapter_no: "1", title: "Episode 1" }, + { chapter_no: "41", title: "Episode 41" }, + { chapter_no: "40.5", title: "Episode 40.5" }, + ], + }), + }; + }; + assert.deepEqual(await kagane.latestChapterFromApi(KAGANE_SERIES), { + num: 41, + label: "Chapter 41", + }); +}); + +test("kagane latestChapterFromApi returns null on a challenge or error", async () => { + globalThis.fetch = async () => ({ ok: false, status: 403 }); + assert.equal(await kagane.latestChapterFromApi(KAGANE_SERIES), null); +}); + // ============================================================ // Shared helpers // ============================================================