mangaBookmark/backend/AGENTS.md

Scope: backend/.

Each entry names the code that holds the truth — read that for what it does. The prose here is only what code cannot tell you: rationale, rejected alternatives, dated measurements, and invariants a plausible refactor would silently break.

Layout

backend/main.go → newRouter is the composition root, the only place packages are wired. Packages under backend/internal/: store, latest, session, httpmw, api, userscript, web, token, pgtest. Root-level *_test.go exercise the full router; unit tests live beside their package.

Not visible from any single file: stdlib net/http with no framework, Postgres over jackc/pgx/v5, CGO_ENABLED=0 static binary into a distroless image, TLS terminated by the reverse proxy so the service listens plain :8080.

Schema — internal/store/migrations/*.sql, run by store.migrate

• Migration files are append-only. Editing an applied one changes nothing on a database that already recorded its version in schema_migrations, so the fix silently applies to new deployments only.
• No column probing, no data-fixup migrations. Both were SQLite-era machinery and were removed deliberately — don't reintroduce either.

Tests need Docker — internal/pgtest

pgtest.Main from TestMain starts one postgres:17-alpine per test binary; pgtest.URL hands each test its own database. A package whose tests touch the store must have that TestMain or it has no database at all.

Reader-owned store — internal/store, internal/token

• The Reader-owned tables are readers, bookmarks, series, and sessions; auxiliary covers, poll_lanes, and poll_passes are also defined in the migrations.

• Credentials are derived, never stored. token.Token(TOKEN_KEY, discord_id, epoch) is an HMAC; only its SHA-256 reaches readers.token_sha256. So install URLs can be rebuilt after any restart, and a database leak yields nothing usable.

• The owner's epoch-0 hash is refreshed at startup only while the row has never been rotated. Drop that condition and a restart resurrects a rotated-away credential.

• Store.EnsureReader never rewrites an existing row's hash — a returning Reader's login must not invalidate their installed scripts.

• Every read and write is scoped to the acting Reader, resolved from the presented credential by httpmw.Auth and carried in the request context. There is no unauthenticated-by-Reader route and no global token.

• series holds what readers share, bookmarks only what differs. A bookmark key is (reader_id, site, series_id) with no surrogate id; the wire key is derived as site:series_id on read.

• Store.Upsert splits one flat body across both tables and enforces the ownership rule: client title/series_url/cover are written only when the series row is new, so one reader cannot retitle a shared series.

• Sync is last-write-wins and the wire format stays flat — clients depend on both; neither is an implementation detail to tidy up.

Web UI — internal/web

Routes, templates and assets are all in that package; AdminPatterns() and adminRoutes() enumerate the privileged ones.

• backend/Dockerfile must copy the whole internal/ tree, not just *.go: templates and static assets are go:embed-ed from internal/web/.
• Guild membership is registration. discordCallback gates on membership (plus DISCORD_REQUIRED_ROLE when set) and only then calls Store.EnsureReader, so a refusal creates nothing.
• Sessions are rows, not signatures: the cookie carries an opaque id and expiry is checked on lookup, which is what makes deleting the row an instant revocation.
• UI mutations go through Store.Get + Store.Upsert so the updated_at rule below stays in exactly one place.
• listView.Fresh exists because a Reader with no bookmarks at all needs install links, not an empty-filter message.
• Design-tool caveat: detect.mjs backend/internal/web/templates reports a false clean. Templates link /static/style.css root-absolutely (correct — it is served from /), but the detector resolves hrefs with path.resolve(fileDir, href), which drops the directory on a leading / and skips the file silently; a relative href doesn't help either, since a template's directory isn't its served path. Always pass backend/internal/web/static too. The one finding there, overused-font on "Instrument Serif", is a deliberate identity choice, not debt.

Confirm gating — internal/web/static/filter.js, toggleConfirmRow(key, kind)

Every action that pulls a series out of the list (archive|finish|remove) opens its own .confirm-row; restore fires instantly because it is the reversal. Remove wears the ember wash, the two reversible ones wear .calm grey. --ember is reserved for the new-chapter signal — the busy bar and inline errors must use --mute, or the one colour that means "something to read" stops meaning it.

Latest-chapter poller — internal/latest, Site registry in sites.go

One goroutine per Site (a Poll Lane) re-checks that Site's bookmarked series from the backend's own network position, so latest_chapter stays fresh while nobody is browsing. The userscript's reportLatestChapter is a second, parallel signal — it PUTs every read, unchanged numbers included, because an unchanged read is exactly the Sighting worth deferring a Poll on.

• Pace lives in the Site registry, not config. Two clocks: per-series rest (series.latest_checked_at, enforced in Store.DueForLatestCheck's WHERE) and per-Lane gap (effectiveGap). The five env knobs that used to size one shared pace are gone; don't add them back.
• The poller walks Series, not Bookmarks — a series several readers hold is fetched once per cycle, and the due queue orders reader_count DESC, latest_checked_at ASC so the widely-read ones win contention.
• The series row is stamped before the fetch, so a permanently broken series waits out its rest instead of being retried every tick.
• Store.SetLatestChapter is a single-column UPDATE, deliberately not a read-modify-write of the bookmark: it therefore cannot revert read progress or move updated_at. The old stale-re-read race died with the Get+Upsert flow — don't restore one here.

Sightings (Store.RecordSighting, the due query's HAVING clause, latest.checkOne) let a Reader's own page read defer a Poll.

• Recorded by the PUT handler before the Upsert, because the raise test needs the row as it stands.
• A Series is deferred only while it has exactly one Bookmark, was sighted within one Rest, and is under sightingCeilingRests since its last Poll — so a shared Series is never deferred and nothing goes six hours unpolled whatever arrives.
• A higher report clears the attribution rather than crediting it: the value the Poll then stores is its own, so a later retraction isn't the Reader's fault.
• store.SightingDisagreementLimit contradictions stop a Reader deferring — their reports still write the Latest Chapter — and store.SightingAgreementsToClear agreements forgive them, as does the owner's clear-marks control.
• Deferral is recomputed from live facts each round, so nothing needs invalidating when a Series gains a second Bookmark. The one input read earlier is the Reader's marks, so crossing or clearing a threshold takes effect from their next Sighting and the standing already bought lasts out its rest.

Refusals and browser loss are Lane-local. Two errChallengeHeld in a pass stop that Site for RefuseBackoff while other Lanes continue. An errBrowserInterrupted (remote Chrome restarted) sets a shared Poller flag so the other browser Lanes skip their passes for the same window — otherwise a restarting Chrome stamps one Series per Lane per pass, burning rests on failures. The flag decays and they probe again.

• isInterstitial matches the orchestration path /cdn-cgi/challenge-platform/h/, never the bare prefix. Cloudflare injects /cdn-cgi/challenge-platform/scripts/jsd/main.js into ordinary 200 pages once a zone turns JS detections on, which demonic did on 2026-08-16: the prefix match read every real demonic page as a refusal and parked the Lane in backoff while plain TLS was returning full series pages.
• Fetches use bogdanfinn/tls-client with a Chrome profile as defence in depth against fingerprint blocking; any failure logs and skips.
• kagane, comix and novelfull sit behind Cloudflare JS challenges the TLS client can't clear, so they go over CDP (BROWSER_WS_URL). kagane and comix are simply not polled when it's unset — a plain fetch would only retrieve a challenge page — while novelfull still attempts plain TLS, because its challenge is a live time-varying fact and its cover bytes never need a browser.
• comix's browser read is an in-tab fetch() of the Series URL, not a DOM render. It is an SPA: rendering cost ~65 requests for the same server-rendered HTML one fetch returns (measured 2026-08-12).
• Browser Lanes wake Chrome only when 5+ Series are due or one has waited 15m, and cover work runs in the background so a slow CDN can't eat a Lane's gap.

Covers — Store.OnSeriesCreated, latest.Acquirer, latest.CoverBytesFetcher, Store.SetSeriesCover

Acquired once when the first Bookmark of a Series is created, then served from our own origin by the public GET /covers/{addr}.

• Acquisition runs in a goroutine: the Reader's PUT must neither block on a Site nor fail with one. Every failure is logged and dropped, leaving the Bookmark intact.
• The wire cover is the absolute PUBLIC_BASE_URL + /covers/{sha256} once bytes exist and "" before — never an address that 404s. Absolute because the userscript renders it on a Site's origin.
• GET /covers/{addr} is public and uncredentialed by design: no cookie or token of ours may travel to a Site's origin.
• A client-sent cover is decoded and discarded, permanently — wire compatibility, not an oversight.
• One route serves all six Sites. No proxy, no per-Site rewrite, no second place that decides a renderable address: the wire cover is it. Templates render .Cover and nothing else. The old kagane-only serving path (/img/kagane/{id} plus a template rewrite) is gone; don't reintroduce a per-Site route because one Site's CDN misbehaves.
• The only Site names left in cover code are in browserOnlyCoverURL (internal/latest): kagane answers a plain fetch with a challenge and cross-origin-resource-policy: same-origin, and static.comix.to answers with the same challenge its pages serve. Every other Site's CDN answers plain TLS.
• comix cover bytes must arrive by direct navigation, not an in-page fetch: its Series page sets cross-origin-embedder-policy: require-corp, which fails a page-context fetch of static.comix.to.
• With no browser configured, kagane and comix Covers are simply absent; novelfull still gets one whenever its page answers a plain request.

updated_at drives list order — Store.Upsert

The server applies its own timestamp only when the row is new or last_chapter_num changes, else it keeps the stored value. Favouriting a series, or a newly published chapter arriving, must not reorder the list — only real reading progress moves a row. Consequently PUT returns the row as stored and clients must adopt that response rather than their own payload.

Lifecycle buckets — status on each bookmark

reading | archived | finished, orthogonal to favorite. Archived and finished appear only in their own tab, never in All, Updated, Favourites or the recent strip. The poller keeps checking archived series and skips finished ones.

• finished is settable only from the web UI; PUT /bookmarks/{key} rejects it with 400.
• An empty incoming status means "keep the stored one", and it is resolved on the VALUES side of Store.Upsert, not in the conflict clause: excluded.* is the post-evaluation row, so a default applied there would wipe the bucket on every PUT from a client predating the column.

Config — Config / loadConfig / loadLatestPoll in backend/main.go

That function is the complete list of env vars, their defaults, and which are required. What it can't tell you:

• PUBLIC_BASE_URL must be an absolute origin because every Cover URL on the wire is built from it and the userscript renders on a Site's origin.
• BROWSER_WS_URL must be a tailnet IP, never a hostname — Chrome's DevTools handler 500s /json/version for any Host that isn't an IP or localhost. Unset (the default) disables browser polling.
• USERSCRIPT_PATH / NOVEL_USERSCRIPT_PATH are bindmounted files; the __API_TOKEN__ placeholder inside them is substituted with the requesting Reader's credential at serve time.
• Pace is per Site in the registry, not env. The _COOLDOWN/_BROWSER_COOLDOWN/_INTERVAL/_BATCH/_STAGGER knobs are gone on purpose.
• The 1h rest for browser Sites is safe on documented grounds: a challenged page costs seconds of a serialized single-tab browser, free-plan zones carry no bot score and no published per-IP rate input, and cf_clearance expires in 30 minutes, so every cadence at or above 1h re-solves anyway.

Userscript install & rotation — internal/userscript, internal/token

Session-gated GET /install/{manga,novel}-bookmark.user.js renders the bindmounted script with the acting Reader's derived credential substituted in, so the credential never appears in page markup, the address bar, or a redirect. ?download=1 adds Content-Disposition: attachment for mobile Violentmonkey, which ignores a .user.js navigation. POST /rotate-token is an atomic epoch bump plus hash rewrite and invalidates every installed copy — the panel must keep warning to reinstall on all devices.

Owner-only admin — internal/web/admin.go

• Every route reaching past the acting Reader is listed in adminRoutes() and wrapped in requireOwner at registration — add it there, not as a check inside a handler; web.AdminPatterns() is what the gate test walks. A non-owner gets 404, never 403.
• The one owner comparison left outside the gate is in index (view.Owner = readerID == h.store.OwnerID()): it gates a link, not an endpoint, so it is a rendering decision a registration-time wrapper cannot express. Do not "unify" it into the gate.
• The Lanes page reads the pass log, never a running poller: lanesView() in admin_lanes.go projects store.LatestLanePasses() and store.LanePassOutcomes() (ADR-0012), so a restart answers the instant the database is up. Browser configuration is a config fact and reachability is derived from recent browser-Site passes inside latest.RefuseBackoff — no reporter interface exists to fake.
• A pass that returns before computing figures (refusal backoff, sidecar down) carries the previous pass's numbers forward rather than recording zeroes.
• Checked next to Due is what separates a stopped Lane from a quiet one, so neither may be dropped from the row.
• Due-without-Checked is not by itself a stall: a browser Lane under both wake thresholds records its pass with the SkipAsleep skip and renders "browser asleep", and that never counts toward Attention. It is the commonest healthy state for kagane, comix and novelfull, so spending the stall mark on it would train the owner to ignore the mark that matters.