Files
mangaBookmark/backend/AGENTS.md
T
sulthan ec74559ca5 feat(web): proxy kagane cover images so the UI can render them
kagane serves cover images from behind the same Cloudflare challenge as
its pages and with cross-origin-resource-policy: same-origin. The second
header is the decisive one: no <img> on the web UI's origin can load a
kagane cover even from a browser that already holds the clearance cookie,
verified 2026-08-08 by loading one from a foreign origin with and without
a referrer. Hot-linking cannot be made to work, so every kagane series
rendered the monogram placeholder.

Bookmark.CoverURL rewrites a stored kagane og:image to /img/kagane/{id}
and returns every other cover untouched; the templates render .CoverURL
in place of .Cover. The endpoint is session-gated like every other UI
route, and hands the id to the shared headless browser, whose fetch is
same-origin with kagane and therefore satisfies both the challenge and
the CORP header. Results are memoised in-process, so a cover costs one
navigation per deployment lifetime.

The id is matched against a UUID regex before it reaches the browser.
That gate is load-bearing rather than tidiness: the cover is a stored
client-supplied string, so an unvalidated one turns the endpoint into an
SSRF primitive aimed at the deployment's own network. ServeMux
path-cleans a traversal into a redirect before the handler runs, but the
handler does not rely on that, and a test pins it.

With BROWSER_WS_URL unset there is no browser and the endpoint answers
404 rather than reaching for a nil fetcher - the same degrade-to-
userscript behaviour the poller already has for these sites.
2026-08-08 23:03:58 +07:00

12 KiB

Guidance for OpenCode (and Claude Code) working under backend/. See root AGENTS.md for the project-wide architecture diagram, hard constraints, and design system.

  • Backend (backend/): stdlib net/http (handful routes, no framework) + Postgres over jackc/pgx/v5 (pure Go, CGO_ENABLED=0 -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain :8080. Single binary, split into packages under backend/internal/: store (Bookmark type, Postgres persistence, migration runner), latest (background poller, site parsers, TLS fetcher), session (cookie signing, login rate limiter), httpmw (Auth/Gzip/CORS middleware), api (JSON bookmark handlers), userscript (userscript-serving handler), web (browser UI handler + templates/ + static/, go:embed-ed). backend/main.go is the composition root — the only place that wires packages together into newRouter. Root-level *_test.go hold integration tests that exercise the full router; unit tests for a package live beside it under internal/.
  • Schema is migration-owned. internal/store/migrations/*.sql is go:embed-ed and applied on every start by store.migrate: one numbered file per change, one transaction each, versions recorded in schema_migrations. Files are append-only — editing an applied one changes nothing on a database that already ran it. No column probing, no data-fixup migrations: both were SQLite-era machinery and are gone.
  • Tests need Docker. internal/pgtest starts one postgres:17-alpine container per test binary (TestMain -> pgtest.Main) and hands each test its own database (pgtest.URL(t)). A package whose tests touch the store must have that TestMain.
  • Reader-owned store, four tables. readers is keyed by Discord user ID and carries the SHA-256 of the Reader's userscript credential plus a token_epoch (issue #24). Credentials are derived, never stored: token.Token(TOKEN_KEY, discord_id, epoch) (HMAC, internal/token), and only its SHA-256 sits in readers.token_sha256, so install URLs can be rebuilt after any restart while a database leak yields nothing but hashes. The seed creates the owner row at startup; its epoch-0 hash is refreshed on every start only while the row has never been rotated, so a restart can never resurrect a rotated-away credential. Every other row is created by that Reader's own first login (Store.EnsureReader, idempotent on discord_id, and it never rewrites an existing row's hash). Rotation is Store.RotateToken (epoch bump + hash rewrite in one transaction), driven by the web UI. series keyed (site, series_id) (asura|demonic|comix|kagane|novelfull|lightnovelworld) owns the shared facts — title, cover, canonical URL, kind (manga|novel), Latest Chapter, latest_checked_at — and bookmarks holds only what differs between readers: progress, favourite, lifecycle bucket, updated_at. A bookmark is keyed (reader_id, site, series_id) — no surrogate id; the wire key is derived as site:series_id on read — and every store read/write is scoped to the reader it names. Auth resolves the acting Reader from the presented credential (httpmw.Auth) and nothing else — there is no unauthenticated-by-Reader route and no global token; the reader id travels in the request context. Sync last-write-wins; the wire format stays flat (ADR-0004). Store.Upsert decomposes one flat body across two tables and enforces the ownership rule: client title/series_url/cover are written only when the series row is new (ADR-0003).
  • Endpoints: GET /bookmarks, PUT /bookmarks/{key} (upsert; see updated_at rule below), DELETE /bookmarks/{key}, GET /healthz (no auth).
  • Web UI: same binary serve the browser UI on a second hostname — GET / (list, or login page when no session), GET /auth/discord + GET /auth/discord/callback (Discord OAuth, ADR-0002), POST /logout, GET /static/*, htmx fragment endpoints under /ui/*. Templates + assets go:embed-ed under backend/internal/web/, so backend/Dockerfile must copy the whole internal/ tree, not just *.go. Sessions are rows in the sessions table: the cookie carries only an opaque id, looked up (and expiry- checked) on every request, and deleting the row revokes the session. Guild membership is registration (issue #27): discordCallback gates on membership (and DISCORD_REQUIRED_ROLE when set) and then calls Store.EnsureReader, so a refusal creates nothing and a returning Reader reuses their row. The owner is the only Reader with administrative reach: POST /readers/{id}/revoke (404 for anyone else) drops that Reader's sessions, and the readers panel renders only on the owner's page. A Reader with no bookmarks at all sees listView.Fresh, whose empty state offers both install links instead of describing a filter. UI mutations read-modify-write through Store.Get + Store.Upsert so updated_at rule stays one place. See docs/superpowers/specs/2026-07-25-web-ui-design.md. Design-tool caveat: templates link /static/style.css root-absolutely (correct — served from /), but impeccable detector resolves stylesheet href with path.resolve(fileDir, href), drops directory on leading / and silently skip file. Relative href don't help either: template's directory isn't its served path. So detect.mjs backend/internal/web/templates reports false clean — always pass backend/internal/web/static too. One finding there, overused-font on "Instrument Serif", deliberate identity choice, not debt.
  • Every action that moves series out of list is confirm-gated. Archive, finish, remove each open own .confirm-row disclosure (toggleConfirmRow(key, kind) in filter.js, kind ∈ archive|finish|remove); restore fire instantly since it's the reversal. Remove's row wear ember wash, two reversible ones wear .calm grey. --ember stay reserved for new-chapter signal: busy bar and inline error use --mute.
  • Latest-chapter poller: ticker goroutine in same binary re-check each bookmarked series' newest published chapter from backend's own network access, so latest_chapter stay fresh when user not browsing. Second, parallel signal — userscript keep own maybeCaptureLatestOnSeriesPage/backgroundRefreshLatest logic unchanged. Two independent clocks: per-series cooldown (series.latest_checked_at, enforced by Store.DueForLatestCheck's WHERE clause) and wake interval. The poller walks Series, not Bookmarks — a series referenced by several bookmarks is fetched once per cycle, and the due queue orders reader_count DESC, latest_checked_at ASC (ADR-0003). Series row stamped before fetch so broken series wait out full cooldown instead of retrying every tick; found chapter written straight to the series row via Store.SetLatestChapter, so a bookmark's updated_at — and the list order — is never touched. Fetches use bogdanfinn/tls-client with Chrome profile as defence in depth against fingerprint-based blocking; any failure log and skip. kagane and novelfull sit behind Cloudflare JavaScript challenges the TLS client can't clear, so they are browser-only: fetched over CDP via BROWSER_WS_URL, and simply not polled when that's unset. See docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md. The poller's series write is a single-column UPDATE (Store.SetLatestChapter), not a read-modify-write of the whole bookmark: it cannot revert read progress or move updated_at, so the old stale-re-read race is gone with the Get+Upsert flow.
  • updated_at drives list order, so moves only on real reading progress: server apply its timestamp when row new or last_chapter_num changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. PUT therefore returns row as stored, clients must adopt that response rather than own payload. See plans/2026-07-25-bookmark-list-favorites-design.md §4.
  • Lifecycle buckets: status on each bookmark is reading | archived | finished, orthogonal to favorite. Archived and finished appear only in own tab — not in All, Updated, Favourites, or recent strip. Poller keeps checking archived series and skip finished ones. finished settable only from web UI; PUT /bookmarks/{key} reject it with 400. Empty incoming status means "keep stored one" — resolved on the VALUES side of Store.Upsert, not conflict clause, since excluded.* is post-evaluation row and default applied there would wipe bucket on every PUT from client that predates column. See docs/superpowers/specs/2026-07-27-status-buckets-design.md.
  • Config via env: TOKEN_KEY (derives every Reader's userscript credential; required), OWNER_DISCORD_ID (seeds the owner Reader — the administrator and the owner of every pre-registration bookmark; required), ALLOWED_ORIGINS (comma list), DATABASE_URL (Postgres connection URL, required — no default), PORT (default 8080), DISCORD_CLIENT_ID/_CLIENT_SECRET/_GUILD_ID/ _REDIRECT_URI (required; Discord OAuth for the browser UI), DISCORD_REQUIRED_ROLE (optional role gate, empty by default), DISCORD_API_BASE (default https://discord.com/api/v10), LATEST_CHAPTER_POLL_ENABLED/_COOLDOWN/_INTERVAL/_BATCH/_STAGGER (background latest-chapter poller; defaults on, 1h/10m/14/20s). USERSCRIPT_PATH and NOVEL_USERSCRIPT_PATH (files served at /u/{token}/manga-bookmark.user.js and /u/{token}/novel-bookmark.user.js, defaults /userscript/manga-bookmark.user.js and /userscript/novel-bookmark.user.js, both supplied by bindmount; the __API_TOKEN__ placeholder inside them is substituted with the requesting Reader's credential at serve time). BROWSER_WS_URL (CDP endpoint of the chrome/ sidecar, used by the poller for kagane and novelfull and by the web UI's kagane cover proxy; unset disables browser polling and serves 404 from the proxy, leaving those sites to the userscript alone).
  • kagane covers are proxied, not hot-linked: kagane serves cover images behind the same challenge as its pages and with cross-origin-resource-policy: same-origin, so no <img> on the web UI's origin can load one — not even from a browser holding the clearance cookie (verified 2026-08-08). Bookmark.CoverURL rewrites a stored kagane og:image to /img/kagane/{id}, served by internal/web/cover.go through latest.BrowserFetcher.Image and memoised in-process. The templates render .CoverURL, never .Cover. The id is matched against a UUID regex before it reaches the browser: the stored value is client-supplied, so an unchecked one is an SSRF primitive pointed at the deployment's own network.
  • Web UI also owns: session-gated GET /install/{manga,novel}-bookmark.user.js (renders the bindmounted script with the acting Reader's derived credential substituted in — the credential never appears in page markup, the address bar, or a redirect; ?download=1 adds Content-Disposition: attachment for mobile Violentmonkey, which ignores a .user.js navigation) and POST /rotate-token (atomic epoch bump + hash rewrite; invalidates every installed copy, so the panel warns to reinstall on all devices). Owner-only POST /readers/{id}/revoke (drops one Reader's session rows and re-renders the readers panel; 404 for any non-owner) is the only route that reaches across Readers.