mangaBookmark/backend/AGENTS.md

Guidance for OpenCode (and Claude Code) working under backend/. See root AGENTS.md for the project-wide architecture diagram, hard constraints, and design system.

• Backend (backend/): stdlib net/http (handful routes, no framework) + Postgres over jackc/pgx/v5 (pure Go, CGO_ENABLED=0 -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain :8080. Single binary, split into packages under backend/internal/: store (Bookmark type, Postgres persistence, migration runner), latest (background poller, site parsers, TLS fetcher), session (cookie signing, login rate limiter), httpmw (Auth/Gzip/CORS middleware), api (JSON bookmark handlers), userscript (userscript-serving handler), web (browser UI handler + templates/ + static/, go:embed-ed). backend/main.go is the composition root — the only place that wires packages together into newRouter. Root-level *_test.go hold integration tests that exercise the full router; unit tests for a package live beside it under internal/.
• Schema is migration-owned. internal/store/migrations/*.sql is go:embed-ed and applied on every start by store.migrate: one numbered file per change, one transaction each, versions recorded in schema_migrations. Files are append-only — editing an applied one changes nothing on a database that already ran it. No column probing, no data-fixup migrations: both were SQLite-era machinery and are gone.
• Tests need Docker. internal/pgtest starts one postgres:17-alpine container per test binary (TestMain -> pgtest.Main) and hands each test its own database (pgtest.URL(t)). A package whose tests touch the store must have that TestMain.
• Reader-owned store, four tables. readers is keyed by Discord user ID and carries the SHA-256 of the Reader's userscript credential plus a token_epoch (issue #24). Credentials are derived, never stored: token.Token(TOKEN_KEY, discord_id, epoch) (HMAC, internal/token), and only its SHA-256 sits in readers.token_sha256, so install URLs can be rebuilt after any restart while a database leak yields nothing but hashes. The seed creates the owner row at startup; its epoch-0 hash is refreshed on every start only while the row has never been rotated, so a restart can never resurrect a rotated-away credential. Every other row is created by that Reader's own first login (Store.EnsureReader, idempotent on discord_id, and it never rewrites an existing row's hash). Rotation is Store.RotateToken (epoch bump + hash rewrite in one transaction), driven by the web UI. series keyed (site, series_id) (asura|demonic|comix|kagane|novelfull|lightnovelworld) owns the shared facts — title, cover, canonical URL, kind (manga|novel), Latest Chapter, latest_checked_at — and bookmarks holds only what differs between readers: progress, favourite, lifecycle bucket, updated_at. A bookmark is keyed (reader_id, site, series_id) — no surrogate id; the wire key is derived as site:series_id on read — and every store read/write is scoped to the reader it names. Auth resolves the acting Reader from the presented credential (httpmw.Auth) and nothing else — there is no unauthenticated-by-Reader route and no global token; the reader id travels in the request context. Sync last-write-wins; the wire format stays flat (ADR-0004). Store.Upsert decomposes one flat body across two tables and enforces the ownership rule: client title/series_url/cover are written only when the series row is new (ADR-0003).
• Endpoints: GET /bookmarks, PUT /bookmarks/{key} (upsert; see updated_at rule below), DELETE /bookmarks/{key}, GET /healthz (no auth).
• Web UI: same binary serve the browser UI on a second hostname — GET / (list, or login page when no session), GET /auth/discord + GET /auth/discord/callback (Discord OAuth, ADR-0002), POST /logout, GET /static/*, htmx fragment endpoints under /ui/*. Templates + assets go:embed-ed under backend/internal/web/, so backend/Dockerfile must copy the whole internal/ tree, not just *.go. Sessions are rows in the sessions table: the cookie carries only an opaque id, looked up (and expiry- checked) on every request, and deleting the row revokes the session. Guild membership is registration (issue #27): discordCallback gates on membership (and DISCORD_REQUIRED_ROLE when set) and then calls Store.EnsureReader, so a refusal creates nothing and a returning Reader reuses their row. The owner is the only Reader with administrative reach: POST /readers/{id}/revoke (404 for anyone else) drops that Reader's sessions, and the readers panel renders only on the owner's page. A Reader with no bookmarks at all sees listView.Fresh, whose empty state offers both install links instead of describing a filter. UI mutations read-modify-write through Store.Get + Store.Upsert so updated_at rule stays one place. See docs/superpowers/specs/2026-07-25-web-ui-design.md. Design-tool caveat: templates link /static/style.css root-absolutely (correct — served from /), but impeccable detector resolves stylesheet href with path.resolve(fileDir, href), drops directory on leading / and silently skip file. Relative href don't help either: template's directory isn't its served path. So detect.mjs backend/internal/web/templates reports false clean — always pass backend/internal/web/static too. One finding there, overused-font on "Instrument Serif", deliberate identity choice, not debt.
• Every action that moves series out of list is confirm-gated. Archive, finish, remove each open own .confirm-row disclosure (toggleConfirmRow(key, kind) in filter.js, kind ∈ archive|finish|remove); restore fire instantly since it's the reversal. Remove's row wear ember wash, two reversible ones wear .calm grey. --ember stay reserved for new-chapter signal: busy bar and inline error use --mute.
• Latest-chapter poller: ticker goroutine in same binary re-check each bookmarked series' newest published chapter from backend's own network access, so latest_chapter stay fresh when user not browsing. Second, parallel signal — userscript keep own maybeCaptureLatestOnSeriesPage/backgroundRefreshLatest logic unchanged. Two independent clocks: per-series cooldown (series.latest_checked_at, enforced by Store.DueForLatestCheck's WHERE clause) and wake interval. The poller walks Series, not Bookmarks — a series referenced by several bookmarks is fetched once per cycle, and the due queue orders reader_count DESC, latest_checked_at ASC (ADR-0003). Series row stamped before fetch so broken series wait out full cooldown instead of retrying every tick; found chapter written straight to the series row via Store.SetLatestChapter, so a bookmark's updated_at — and the list order — is never touched. Fetches use bogdanfinn/tls-client with Chrome profile as defence in depth against fingerprint-based blocking; any failure log and skip. kagane and novelfull sit behind Cloudflare JavaScript challenges the TLS client can't clear, so they are fetched over CDP via BROWSER_WS_URL; kagane is simply not polled when that's unset, while novelfull falls back to a plain-TLS attempt — its challenge is a live time-varying fact, and its cover bytes never need the browser. See docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md. The poller's series write is a single-column UPDATE (Store.SetLatestChapter), not a read-modify-write of the whole bookmark: it cannot revert read progress or move updated_at, so the old stale-re-read race is gone with the Get+Upsert flow.
• Covers are acquired at creation, then served from our own origin (ADR-0007): the first Bookmark of a Series fires Store.OnSeriesCreated, which latest.Acquirer turns into one series-page fetch yielding both the Latest Chapter and the cover URL; the bytes then go through latest.CoverBytesFetcher into Store.SetSeriesCover. It runs in a goroutine — the Reader's PUT must neither block on a Site nor fail with one — and every failure is logged and dropped, leaving the Bookmark intact. The wire's cover is the absolute PUBLIC_BASE_URL + /covers/{sha256} once bytes exist and "" before, never an address that 404s. GET /covers/{addr} is public and uncredentialed: the userscript renders it on a Site's origin, where no cookie or token of ours travels. A client-sent cover is decoded and discarded, permanently (ADR-0004 compatibility). Browser-backed Sites join the same pipeline (issue #62): kagane pages and cover bytes go through the browser sidecar (nothing falls back to a plain fetch, which would only retrieve a challenge page), while novelfull needs the browser only for its HTML — the cover URL comes out of the browser-fetched page and the bytes go over plain TLS. With no browser configured, kagane Covers are simply absent; novelfull still gets one — at creation and on the poll — when its page body happens to answer a plain request (the challenge is a live time-varying fact).
• updated_at drives list order, so moves only on real reading progress: server apply its timestamp when row new or last_chapter_num changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. PUT therefore returns row as stored, clients must adopt that response rather than own payload. See plans/2026-07-25-bookmark-list-favorites-design.md §4.
• Lifecycle buckets: status on each bookmark is reading | archived | finished, orthogonal to favorite. Archived and finished appear only in own tab — not in All, Updated, Favourites, or recent strip. Poller keeps checking archived series and skip finished ones. finished settable only from web UI; PUT /bookmarks/{key} reject it with 400. Empty incoming status means "keep stored one" — resolved on the VALUES side of Store.Upsert, not conflict clause, since excluded.* is post-evaluation row and default applied there would wipe bucket on every PUT from client that predates column. See docs/superpowers/specs/2026-07-27-status-buckets-design.md.
• Config via env: TOKEN_KEY (derives every Reader's userscript credential; required), OWNER_DISCORD_ID (seeds the owner Reader — the administrator and the owner of every pre-registration bookmark; required), ALLOWED_ORIGINS (comma list), DATABASE_URL (Postgres connection URL, required — no default), COVER_DIR (required filesystem volume for content-addressed Cover bytes), PUBLIC_BASE_URL (required origin this deployment answers on, trailing slash trimmed; every Cover URL on the wire is built from it, absolute because the userscript renders on a Site's origin — ADR-0007), PORT (default 8080), DISCORD_CLIENT_ID/_CLIENT_SECRET/_GUILD_ID/ _REDIRECT_URI (required; Discord OAuth for the browser UI), DISCORD_REQUIRED_ROLE (optional role gate, empty by default), DISCORD_API_BASE (default https://discord.com/api/v10), LATEST_CHAPTER_POLL_ENABLED/_COOLDOWN/_BROWSER_COOLDOWN/_INTERVAL/ _BATCH/_STAGGER (background latest-chapter poller; defaults on, 1h plain-TLS cooldown, 6h browser cooldown, 10m/14/20s; both cooldowns have a 15m floor). USERSCRIPT_PATH and NOVEL_USERSCRIPT_PATH (files served at /u/{token}/manga-bookmark.user.js and /u/{token}/novel-bookmark.user.js, defaults /userscript/manga-bookmark.user.js and /userscript/novel-bookmark.user.js, both supplied by bindmount; the __API_TOKEN__ placeholder inside them is substituted with the requesting Reader's credential at serve time). BROWSER_WS_URL (CDP endpoint of the browser, which runs on a separate machine and is reached over the tailnet — ADR-0006, chrome/docker-compose.yml. Used by the poller for kagane and novelfull and by the web UI's kagane cover proxy; unset — the default — disables browser polling and serves 404 for covers not already stored, leaving those sites to the userscript alone. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s /json/version for any Host that isn't an IP or localhost).
• kagane covers are proxied, not hot-linked: kagane serves cover images behind the same challenge as its pages and with cross-origin-resource-policy: same-origin, so no <img> on the web UI's origin can load one — not even from a browser holding the clearance cookie og:image to /img/kagane/{id}. internal/web/cover.go reads the persistent covers table first, then fetches a miss through latest.BrowserFetcher.Image. The templates render .CoverURL, never .Cover. The id is matched against a UUID regex before it reaches the browser: the stored value is client-supplied, so an unchecked one is an SSRF primitive pointed at the deployment's own network.
• Web UI also owns: session-gated GET /install/{manga,novel}-bookmark.user.js (renders the bindmounted script with the acting Reader's derived credential substituted in — the credential never appears in page markup, the address bar, or a redirect; ?download=1 adds Content-Disposition: attachment for mobile Violentmonkey, which ignores a .user.js navigation) and POST /rotate-token (atomic epoch bump + hash rewrite; invalidates every installed copy, so the panel warns to reinstall on all devices). Owner-only POST /readers/{id}/revoke (drops one Reader's session rows and re-renders the readers panel; 404 for any non-owner) is the only route that reaches across Readers.