mangaBookmark/backend/AGENTS.md

Guidance for OpenCode (and Claude Code) working under backend/. See root AGENTS.md for the project-wide architecture diagram, hard constraints, and design system.

• Backend (backend/): stdlib net/http (handful routes, no framework) + Postgres over jackc/pgx/v5 (pure Go, CGO_ENABLED=0 -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain :8080. Single binary, split into packages under backend/internal/: store (Bookmark type, Postgres persistence, migration runner), latest (background poller, site parsers, TLS fetcher), session (cookie signing, login rate limiter), httpmw (Auth/Gzip/CORS middleware), api (JSON bookmark handlers), userscript (userscript-serving handler), web (browser UI handler + templates/ + static/, go:embed-ed). backend/main.go is the composition root — the only place that wires packages together into newRouter. Root-level *_test.go hold integration tests that exercise the full router; unit tests for a package live beside it under internal/.
• Schema is migration-owned. internal/store/migrations/*.sql is go:embed-ed and applied on every start by store.migrate: one numbered file per change, one transaction each, versions recorded in schema_migrations. Files are append-only — editing an applied one changes nothing on a database that already ran it. No column probing, no data-fixup migrations: both were SQLite-era machinery and are gone.
• Tests need Docker. internal/pgtest starts one postgres:17-alpine container per test binary (TestMain -> pgtest.Main) and hands each test its own database (pgtest.URL(t)). A package whose tests touch the store must have that TestMain.
• Reader-owned store, four tables. readers is keyed by Discord user ID and carries the SHA-256 of the Reader's userscript credential plus a token_epoch (issue #24). Credentials are derived, never stored: token.Token(TOKEN_KEY, discord_id, epoch) (HMAC, internal/token), and only its SHA-256 sits in readers.token_sha256, so install URLs can be rebuilt after any restart while a database leak yields nothing but hashes. The seed creates the owner row at startup; its epoch-0 hash is refreshed on every start only while the row has never been rotated, so a restart can never resurrect a rotated-away credential. Every other row is created by that Reader's own first login (Store.EnsureReader, idempotent on discord_id, and it never rewrites an existing row's hash). Rotation is Store.RotateToken (epoch bump + hash rewrite in one transaction), driven by the web UI. series keyed (site, series_id) (asura|demonic|comix|kagane|novelfull|lightnovelworld) owns the shared facts — title, cover, canonical URL, kind (manga|novel), Latest Chapter, latest_checked_at — and bookmarks holds only what differs between readers: progress, favourite, lifecycle bucket, updated_at. A bookmark is keyed (reader_id, site, series_id) — no surrogate id; the wire key is derived as site:series_id on read — and every store read/write is scoped to the reader it names. Auth resolves the acting Reader from the presented credential (httpmw.Auth) and nothing else — there is no unauthenticated-by-Reader route and no global token; the reader id travels in the request context. Sync last-write-wins; the wire format stays flat (ADR-0004). Store.Upsert decomposes one flat body across two tables and enforces the ownership rule: client title/series_url/cover are written only when the series row is new (ADR-0003).
• Endpoints: GET /bookmarks, PUT /bookmarks/{key} (upsert; see updated_at rule below), DELETE /bookmarks/{key}, GET /healthz (no auth).
• Web UI: same binary serve the browser UI on a second hostname — GET / (list, or login page when no session), GET /auth/discord + GET /auth/discord/callback (Discord OAuth, ADR-0002), POST /logout, GET /static/*, htmx fragment endpoints under /ui/*. Templates + assets go:embed-ed under backend/internal/web/, so backend/Dockerfile must copy the whole internal/ tree, not just *.go. Sessions are rows in the sessions table: the cookie carries only an opaque id, looked up (and expiry- checked) on every request, and deleting the row revokes the session. Guild membership is registration (issue #27): discordCallback gates on membership (and DISCORD_REQUIRED_ROLE when set) and then calls Store.EnsureReader, so a refusal creates nothing and a returning Reader reuses their row. The owner is the only Reader with administrative reach: POST /readers/{id}/revoke (404 for anyone else) drops that Reader's sessions, and the readers panel renders only on the owner's page. A Reader with no bookmarks at all sees listView.Fresh, whose empty state offers both install links instead of describing a filter. UI mutations read-modify-write through Store.Get + Store.Upsert so updated_at rule stays one place. See docs/superpowers/specs/2026-07-25-web-ui-design.md. Design-tool caveat: templates link /static/style.css root-absolutely (correct — served from /), but impeccable detector resolves stylesheet href with path.resolve(fileDir, href), drops directory on leading / and silently skip file. Relative href don't help either: template's directory isn't its served path. So detect.mjs backend/internal/web/templates reports false clean — always pass backend/internal/web/static too. One finding there, overused-font on "Instrument Serif", deliberate identity choice, not debt.
• Every action that moves series out of list is confirm-gated. Archive, finish, remove each open own .confirm-row disclosure (toggleConfirmRow(key, kind) in filter.js, kind ∈ archive|finish|remove); restore fire instantly since it's the reversal. Remove's row wear ember wash, two reversible ones wear .calm grey. --ember stay reserved for new-chapter signal: busy bar and inline error use --mute.
• Latest-chapter poller: ticker goroutine in same binary re-check each bookmarked series' newest published chapter from backend's own network access, so latest_chapter stay fresh when user not browsing. Second, parallel signal — userscript keep own maybeCaptureLatestOnSeriesPage/backgroundRefreshLatest logic unchanged. Two independent clocks: per-series cooldown (series.latest_checked_at, enforced by Store.DueForLatestCheck's WHERE clause) and wake interval. The poller walks Series, not Bookmarks — a series referenced by several bookmarks is fetched once per cycle, and the due queue orders reader_count DESC, latest_checked_at ASC (ADR-0003). Series row stamped before fetch so broken series wait out full cooldown instead of retrying every tick; found chapter written straight to the series row via Store.SetLatestChapter, so a bookmark's updated_at — and the list order — is never touched. Fetches use bogdanfinn/tls-client with Chrome profile as defence in depth against fingerprint-based blocking; any failure log and skip. kagane and novelfull sit behind Cloudflare JavaScript challenges the TLS client can't clear, so they are fetched over CDP via BROWSER_WS_URL; kagane is simply not polled when that's unset, while novelfull falls back to a plain-TLS attempt — its challenge is a live time-varying fact, and its cover bytes never need the browser. See docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md. The poller's series write is a single-column UPDATE (Store.SetLatestChapter), not a read-modify-write of the whole bookmark: it cannot revert read progress or move updated_at, so the old stale-re-read race is gone with the Get+Upsert flow.
• Covers are acquired at creation, then served from our own origin (ADR-0007): the first Bookmark of a Series fires Store.OnSeriesCreated, which latest.Acquirer turns into one series-page fetch yielding both the Latest Chapter and the cover URL; the bytes then go through latest.CoverBytesFetcher into Store.SetSeriesCover. It runs in a goroutine — the Reader's PUT must neither block on a Site nor fail with one — and every failure is logged and dropped, leaving the Bookmark intact. The wire's cover is the absolute PUBLIC_BASE_URL + /covers/{sha256} once bytes exist and "" before, never an address that 404s. GET /covers/{addr} is public and uncredentialed: the userscript renders it on a Site's origin, where no cookie or token of ours travels. A client-sent cover is decoded and discarded, permanently (ADR-0004 compatibility). Browser-backed Sites join the same pipeline (issue #62): kagane pages and cover bytes go through the browser sidecar (nothing falls back to a plain fetch, which would only retrieve a challenge page), while novelfull needs the browser only for its HTML — the cover URL comes out of the browser-fetched page and the bytes go over plain TLS. With no browser configured, kagane Covers are simply absent; novelfull still gets one — at creation and on the poll — when its page body happens to answer a plain request (the challenge is a live time-varying fact). The old kagane-only serving path (/img/kagane/{id}, template rewrite, CoverFetcher) is gone (issue #63): the one public route serves every Site.
• updated_at drives list order, so moves only on real reading progress: server apply its timestamp when row new or last_chapter_num changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. PUT therefore returns row as stored, clients must adopt that response rather than own payload. See plans/2026-07-25-bookmark-list-favorites-design.md §4.
• Lifecycle buckets: status on each bookmark is reading | archived | finished, orthogonal to favorite. Archived and finished appear only in own tab — not in All, Updated, Favourites, or recent strip. Poller keeps checking archived series and skip finished ones. finished settable only from web UI; PUT /bookmarks/{key} reject it with 400. Empty incoming status means "keep stored one" — resolved on the VALUES side of Store.Upsert, not conflict clause, since excluded.* is post-evaluation row and default applied there would wipe bucket on every PUT from client that predates column. See docs/superpowers/specs/2026-07-27-status-buckets-design.md.
• Config via env: TOKEN_KEY (derives every Reader's userscript credential; required), OWNER_DISCORD_ID (seeds the owner Reader — the administrator and the owner of every pre-registration bookmark; required), ALLOWED_ORIGINS (comma list), DATABASE_URL (Postgres connection URL, required — no default), COVER_DIR (required filesystem volume for content-addressed Cover bytes), PUBLIC_BASE_URL (required origin this deployment answers on, trailing slash trimmed; every Cover URL on the wire is built from it, absolute because the userscript renders on a Site's origin — ADR-0007), PORT (default 8080), DISCORD_CLIENT_ID/_CLIENT_SECRET/_GUILD_ID/ _REDIRECT_URI (required; Discord OAuth for the browser UI), DISCORD_REQUIRED_ROLE (optional role gate, empty by default), DISCORD_API_BASE (default https://discord.com/api/v10), LATEST_CHAPTER_POLL_ENABLED/_COOLDOWN/_BROWSER_COOLDOWN/_INTERVAL/ _BATCH/_STAGGER (background latest-chapter poller; defaults on, 1h plain-TLS cooldown, 6h browser cooldown, 10m/14/20s; both cooldowns have a 15m floor). The browser cooldown is longer for cost, not for safety: a challenged page costs seconds of a serialized single-tab browser, while a plain read costs one request. It buys no documented reduction in challenge risk — free-plan zones have no bot score and no published per-IP rate input, and cf_clearance expires in 30 minutes so every cadence at or above 1h re-solves anyway — docs/research/cloudflare-bot-scoring-and-poll-cadence.md. USERSCRIPT_PATH and NOVEL_USERSCRIPT_PATH (files served at /u/{token}/manga-bookmark.user.js and /u/{token}/novel-bookmark.user.js, defaults /userscript/manga-bookmark.user.js and /userscript/novel-bookmark.user.js, both supplied by bindmount; the __API_TOKEN__ placeholder inside them is substituted with the requesting Reader's credential at serve time). BROWSER_WS_URL (CDP endpoint of the browser, which runs on a separate machine and is reached over the tailnet — ADR-0006, chrome/docker-compose.yml. Used by the poller for kagane and novelfull page fetches and by the cover pipeline for kagane's image bytes (the browser is the only route that clears the challenge kagane serves its covers behind); unset — the default — disables browser polling and leaves kagane Covers blank until stored bytes exist. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s /json/version for any Host that isn't an IP or localhost).
• No per-Site cover path (issue #63): every Cover — all six Sites — is served by the one public GET /covers/{addr} route from content-addressed bytes. There is no proxy, no per-Site rewrite, no second place that decides a Cover's renderable address: the wire cover is it. The only place a Site name still appears in cover code is the extraction module (latest), where kagane's image URLs are claimed by browserOnlyCoverURL — they answer a plain fetch with a challenge and cross-origin-resource-policy: same-origin; every other Site's CDN answers plain TLS. Templates render .Cover — the wire value — never anything else.
• Web UI also owns: session-gated GET /install/{manga,novel}-bookmark.user.js (renders the bindmounted script with the acting Reader's derived credential substituted in — the credential never appears in page markup, the address bar, or a redirect; ?download=1 adds Content-Disposition: attachment for mobile Violentmonkey, which ignores a .user.js navigation) and POST /rotate-token (atomic epoch bump + hash rewrite; invalidates every installed copy, so the panel warns to reinstall on all devices). Owner-only POST /readers/{id}/revoke (drops one Reader's session rows and re-renders the readers panel; 404 for any non-owner) is the only route that reaches across Readers.