Closes #59. Part of spec #55, and the ticket that fixes the reported bug #47. Architecture: `docs/adr/0007-backend-hosts-cover-bytes.md`. Does not close #47 or #55. ## What changed A Reader bookmarks a Series nobody holds yet — the exact case in #47 — and within seconds the list shows its artwork instead of a broken image. The first Bookmark to create a Series fires `Store.OnSeriesCreated` after commit, and the new `latest.Acquirer` turns that into **one** series-page fetch that yields both the Latest Chapter and the cover URL. The bytes go through the gated cover fetcher from #57 and are stored content-addressed through #56, so the wire carries an absolute URL on this deployment's own origin — never a third-party address, and never one that 404s. ### Store - Migration `0009_series_cover_address.sql` adds `series.cover_address`. The two facts are now split: `series.cover` is the third-party source address the bytes came from (the acquisition path's dedupe key), `series.cover_address` is the SHA-256 they are stored under. An empty `cover_address` is precisely what "no Cover yet" means, which is the distinction both the API and the UI depend on. - `SetSeriesCover` writes the address only after the bytes are on disk, so the wire can never name an object that is not there. - `CoverWireURL` builds `PUBLIC_BASE_URL + /covers/<sha256>` for every scanned row, and returns `""` for a blank address. - The cover columns are gone from `Upsert`'s `INSERT` and its `DO UPDATE`. A client-supplied cover cannot reach the shared Series row on any path, not just the creation path. - `Open` now rejects a base URL that is not an absolute `http(s)` origin: `PUBLIC_BASE_URL=bookmarks.example.com` would otherwise start cleanly and emit addresses no browser can load. ### Acquisition - `internal/latest/acquire.go`: one fetch, gated by the poller's own `fetchableSeriesURL` (a `series_url` arrives in a client-supplied PUT body, so without the gate a token-holder chooses what the server fetches from its own network position). - Asynchronous and log-and-drop. The Bookmark, its progress and its Latest Chapter are already committed; a Site that is down or a cover that cannot be produced disturbs none of them. - Bounded by a two-slot semaphore. A bulk sync creating N Series would otherwise fire N simultaneous requests from one IP — the traffic shape the poller's stagger exists to avoid. - Cancelled at shutdown (shares the poller's context) and stamps `latest_checked_at`, so the poller does not refetch the same page a tick later. - Browser-backed Sites (kagane, novelfull) are deliberately skipped: their pages only yield a Cloudflare challenge to the TLS client, so the request would be spent for nothing. They arrive in #62. ### Wire and route - `GET /covers/{address}` serves the bytes publicly and uncredentialed with `Cache-Control: public, max-age=604800, immutable`. The address is gated by a `^[0-9a-f]{64}$` pattern and cross-checked against a pure function of itself before any filesystem read, so no request shaped like a traversal reaches disk. - `PUT /bookmarks/{key}` still accepts a `cover` field and discards it, permanently. Rejecting it would break every installed userscript the moment this deploys, and ADR-0004's compatibility argument depends on those scripts continuing to work. The decode site says so in place of a TODO nobody intends to keep. - `store.CoverContentType` canonicalises comix's non-standard `image/jpg` to `image/jpeg`, so one image cannot land under two spellings. This one was found by the live smoke test, not by reading. ### Config `PUBLIC_BASE_URL` is new and required (cover URLs must go out absolute — the userscript renders them on third-party origins, where a relative path resolves against the Site). Documented in `.env.example`, `docker-compose.yml` (`:?` so compose fails too), `DEPLOY.md` and `backend/AGENTS.md`. ## Acceptance criteria All twelve of #59's criteria are met; the checklist on the issue is ticked with the evidence. ## Verification - `go test ./...` green (Docker-backed Postgres suite). - Live smoke against a real backend + Postgres: bookmarking `comix:n8we-dungeons-and-crayons` produced `"cover": "http://127.0.0.1:8099/covers/8ce74d80…"` and `"latest_chapter": "Chapter 81"` within seconds of the PUT; `curl` on that address returned `200`, `Content-Type: image/jpeg`, `Cache-Control: public, max-age=604800, immutable`, and a 280x420 JPEG. That run is what surfaced the `image/jpg` content type. - Mutation-checked the asynchrony test: removing the `go` from `Acquire` turns `TestAcquireDoesNotBlockTheWrite` red. ## Reviewed Both axes of `/code-review` were run against this diff before commit. Their findings that were actionable here are folded in: the concurrency bound, the shutdown tie, the `PUBLIC_BASE_URL` validation, the missing `latest_checked_at` stamp, and a test that could not fail. ## Known sequencing A kagane/novelfull Series created between this deploy and #62 has no cover source at all: the acquisition skips those Sites and `Upsert` no longer persists the userscript-scraped address. This is #59's stated boundary rather than a defect, but it is a user-visible gap on two Sites and should order #62 accordingly. Reviewed-on: #68 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
Manga Bookmark
Track manga read-progress on asurascans.com (a.k.a. asuracomic.net), demonicscans.org, comix.to, and kagane.to from a phone (Bromite / mobile Chromium), synced to a self-hosted Go backend so bookmarks unify across all four sites and all devices.
Two parts:
backend/— tiny Go (net/http+ Postgres via pure-Gopgx) sync service. 4 routes, static binary, distroless container.userscript/manga-bookmark.user.js— single Bromite-compatible userscript (noGM_*APIs) that injects an on-page bookmark UI and syncs viafetch().
Bromite userscript (isolated world, Shadow DOM UI, localStorage cache)
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> Postgres (volume)
|
| CDP over the tailnet
v
headless Chrome, on-demand,
on a separate machine
(chrome/, ADR-0006)
Kagane and novelfull sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the poller reaches those two through a real Chrome over CDP. That browser is not part of the API stack: it is its own compose unit on a second machine, spawned on the first connection and reaped when idle. The API needs it only to discover new chapters and to fetch a kagane cover once — covers are stored, so the library renders in full with the browser switched off.
1. Backend
Config (env)
| Var | Default | Notes |
|---|---|---|
TOKEN_KEY |
(required) | Secret every Reader's userscript credential is derived from (issue #24); only SHA-256 hashes of credentials are stored. |
OWNER_DISCORD_ID |
(required) | Discord user ID of the owner: seeded as the first Reader, owns every pre-registration bookmark, and is the only Reader who can revoke another's sessions. |
ALLOWED_ORIGINS |
Asura + Demonic + Comix + Kagane origins | Comma-separated CORS allowlist. |
DATABASE_URL |
(required) | Postgres connection URL, e.g. postgres://bookmarks:…@postgres:5432/bookmarks?sslmode=disable. Compose builds it from POSTGRES_PASSWORD. |
COVER_DIR |
(required) | Filesystem volume for immutable, content-addressed Cover bytes. Compose builds the image and mounts cover-data at this path; standalone runs may choose another writable durable path. |
PORT |
8080 |
Plain HTTP; TLS terminated by the proxy. |
BROWSER_WS_URL |
empty | CDP endpoint of the remote browser (ws://<tailnet IP>:9222), used to poll Kagane/Novelfull past their JS challenge and to fetch uncached Kagane covers. Must be an IP or localhost — Chrome's DevTools handler 500s any other Host header, MagicDNS names included. Unset disables both; stored covers still serve. |
DISCORD_CLIENT_ID |
(required) | Discord application credentials for the browser sign-in (ADR-0002). |
DISCORD_CLIENT_SECRET |
(required) | As above. Never logged, never echoed in an error. |
DISCORD_GUILD_ID |
(required) | The one guild whose membership gates sign-in, checked at login only. Membership is registration: any member becomes a Reader on first login. |
DISCORD_REDIRECT_URI |
(required) | Exact callback URL; Discord matches it verbatim against the registered redirect. |
DISCORD_REQUIRED_ROLE |
empty | Role snowflake a member must additionally hold. Empty means guild membership alone suffices. |
DISCORD_API_BASE |
https://discord.com/api/v10 |
Test seam — tests point it at a local stub so the real token exchange runs. |
USERSCRIPT_PATH |
/userscript/manga-bookmark.user.js |
Bindmounted file served at /u/{token}/manga-bookmark.user.js. |
NOVEL_USERSCRIPT_PATH |
/userscript/novel-bookmark.user.js |
Same, for the novel library. |
LATEST_CHAPTER_POLL_ENABLED |
1 |
0 turns the poller off entirely. |
LATEST_CHAPTER_POLL_COOLDOWN |
1h |
Rest between checks of one plain-TLS series; floor 15m. |
LATEST_CHAPTER_POLL_BROWSER_COOLDOWN |
6h |
Rest between checks of one browser-backed series; floor 15m. |
LATEST_CHAPTER_POLL_INTERVAL |
10m |
How often the poller wakes. Cannot shorten either cooldown. |
LATEST_CHAPTER_POLL_BATCH |
14 |
Series per wake. Keep BATCH × STAGGER under INTERVAL. |
LATEST_CHAPTER_POLL_STAGGER |
20s |
Delay between fetches in a batch — this is the outbound request rate. |
Compose reads a few more from the same .env that the backend never sees:
POSTGRES_PASSWORD (required — DATABASE_URL is built from it, and Postgres
only applies it while postgres-data is empty), BOOKMARK_API_HOST and
BOOKMARK_WEB_HOST (required by the prod override), and the optional
PROXY_NETWORK / TRAEFIK_ENTRYPOINT / TRAEFIK_CERTRESOLVER. The browser
unit has its own chrome/.env on its own machine — BROWSER_BIND_ADDR
(required, the tailnet IP the CDP port is published on) and the optional
BROWSER_TZ. Full commentary is in .env.example and chrome/.env.example;
deployment order is DEPLOY.md.
Endpoints
| Method | Path | Auth | Description |
|---|---|---|---|
GET |
/bookmarks |
Bearer | All bookmarks of the acting Reader. |
PUT |
/bookmarks/{key} |
Bearer | Upsert one series; returns the row as stored. |
DELETE |
/bookmarks/{key} |
Bearer | Remove one. |
GET |
/healthz |
none | 200 ok. |
GET |
/u/{token}/manga-bookmark.user.js |
credential in path | Serves the userscript with the requesting Reader's credential substituted in and an mtime-derived @version. |
key is <site>:<series_id> — e.g. asura:trash-of-the-counts-family-f886a8af,
demonic:Infinite-Level-Up-in-Murim, comix:12345, or
kagane:3fa85f64-5717-4562-b3fc-2c963f66afa6. Sync is last-write-wins.
updated_at orders the bookmark list, so it moves only on real reading
progress: the server applies its timestamp when the row is new or
last_chapter_num changes, and otherwise keeps the stored one. Favouriting a
series or recording a newly published chapter therefore leaves the order alone.
Because the timestamp a client sends is only a candidate, PUT echoes the row
as stored and clients adopt that rather than their own payload.
Develop / test
cd backend
go test ./... # unit + handler tests
CGO_ENABLED=0 go build # static binary
go test ./... requires Docker. The store talks to a real Postgres, so
each test package starts a throwaway postgres:17-alpine container and gives
every test its own database inside it (internal/pgtest). Nothing is stubbed
and nothing reaches the network beyond the local Docker daemon.
Run the stack
cp .env.example .env
# edit .env: set TOKEN_KEY (openssl rand -hex 32) and
# POSTGRES_PASSWORD (openssl rand -hex 24)
docker compose up -d --build # binds 127.0.0.1:8080
That brings up two services — the API and Postgres. The browser is deliberately
not one of them; without BROWSER_WS_URL the poller logs and skips kagane and
novelfull, and everything else works. To run one locally, publish it on the
Docker bridge gateway so the API container can name it by IP:
cd chrome
echo 'BROWSER_BIND_ADDR=172.17.0.1' > .env
docker compose up -d --build
# then in the repo's own .env: BROWSER_WS_URL=ws://172.17.0.1:9222
Bind it to 127.0.0.1 instead if you only want to drive it from the host, e.g.
SMOKE_BROWSER_WS_URL=ws://127.0.0.1:9222 go test -run TestSmokeKagane ./internal/latest.
In production that address is the home machine's tailnet IP and nothing else —
see DEPLOY.md §7 and ADR-0006.
Smoke test:
# The credential is per Reader and derived, so there is no token in .env to
# grep. Take yours from the Userscripts panel's install link after signing in,
# or read it out of an installed script's API_TOKEN constant.
TOKEN=<your Reader credential>
curl -s localhost:8080/healthz # ok
curl -s localhost:8080/bookmarks # 401
curl -s -H "Authorization: Bearer $TOKEN" localhost:8080/bookmarks # []
curl -s -X PUT -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d '{"title":"Test","last_chapter":"Chapter 1","last_chapter_num":1}' \
localhost:8080/bookmarks/asura:test-1
curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \
-H 'Access-Control-Request-Method: PUT' \
localhost:8080/bookmarks/asura:test-1 | grep -i access-control # 204 + CORS headers
Deploy behind your reverse proxy
Route https://bookmark-api.<domain> → the service on :8080 (TLS at the proxy).
-
Host proxy (nginx/Caddy on the host): the base compose already binds
127.0.0.1:8080; point the proxyproxy_pass http://127.0.0.1:8080;. -
Docker proxy (Traefik/nginx in a container on its own network): use the override, which drops the published port and joins the shared network:
docker network create proxy # once, if it doesn't exist docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --buildSet
PROXY_NETWORKin.envif your network isn't namedproxy.
Verify: https://bookmark-api.<domain>/healthz returns ok over valid TLS (no
mixed-content), and an OPTIONS preflight from a real site origin returns the
CORS headers.
2. Userscript
Install
Sign in to the web UI and open the Userscripts panel: it offers one
install link per library. Each link serves a script rendered with your own
credential already inside it — you never see, type or copy a credential. The
served script carries @downloadURL/@updateURL pointing at its
credential-bearing path, so Violentmonkey keeps auto-updating it.
The bindmounted files carry __API_TOKEN__ placeholders; the backend
substitutes the requesting Reader's credential at serve time, so no real
credential is ever committed. Rotating the credential (same panel) invalidates
every installed copy immediately — reinstall on all devices.
Install on Bromite (mobile)
Bromite runs Chromium's native userscript engine (no Tampermonkey needed):
- Bromite → Settings → User scripts → enable user scripts (allow the permission prompt).
- Open the install link from the web UI — Bromite detects the
.user.jsand offers to install it. - Confirm the install; the
@matchlist covers both sites. - Open a series on either site — a 📑 button appears bottom-right.
Exact menu wording varies by Bromite build; if "User scripts" is absent, update Bromite or use a build with userscript support.
Desktop iteration (optional)
The script is GM_*-free, so it also runs in Tampermonkey/Violentmonkey on
desktop for faster testing — install the same file unchanged.
Use
- Bookmark: on a series or chapter page, open the panel → + Bookmark this.
- Auto-progress: opening a chapter of a bookmarked series records it when the chapter number ≥ the stored one (re-reading older chapters never regresses progress; unparseable numbers set the current chapter).
- Manual override: panel → Edit on any row forces a specific chapter.
- Continue: jumps to the last-read chapter (or the series page).
- Latest chapter: rows read
Read: … · Latest: …once the newest published chapter is known and it is ahead of your progress. See below for how that is found. - Favourites: the ☆ on any row toggles it; the ★ Favourites tab narrows the list. Favourited series still appear under All. The flag syncs, so it follows you across devices; the chosen tab does not persist.
- Archive: the Archive button on any row parks a series — it drops out of All and ★ Favourites and moves to the Archived tab. The server keeps checking it for new chapters, so it is worth coming back to. Archiving does not touch read progress, and reading an archived series leaves it archived.
- Finished: series you have completed live in a Finished tab in the web UI only. It is set there and nowhere else — the API rejects the value — and finished series are hidden from every userscript tab and are no longer polled for new chapters.
- Bookmarks made on Asura appear when the panel is opened on Demonic, and vice versa — the backend is the shared store.
Neither favouriting nor learning a new chapter reorders the list — only reading progress does.
Offline / backend down: changes are cached in localStorage and retried on the
next successful load (last-write-wins).
How "latest chapter" is found
Only a series page lists every chapter (a reader page links just its neighbours), and the backend cannot fetch either site — Cloudflare blocks server-side requests, and neither site offers an API or feed to poll. So the userscript does the looking, from your own browser session:
- Opening a bookmarked series page records its newest chapter directly.
- Otherwise it fetches series pages in the background — same-origin only, so
browsing Asura refreshes Asura bookmarks and Demonic refreshes Demonic. One
series per navigation, and at most one check per series every 4 hours
(
LATEST_CHECK_BATCH/LATEST_CHECK_THROTTLE_MS). Failures are silent and simply retried after the window.
Freshness is tracked per device in localStorage under bmgr:manga:lastchecked
and is deliberately not synced, since each device checks on its own.
This means a bookmark is as current as its last check — not the moment a chapter drops. Nothing can be instant here: neither site offers push, feeds, or an API.
Adapter reference (verified live 2026-07-24)
The site adapters key everything off URL regex, with title/cover from
og:title / og:image. Confirmed against live pages via Playwright:
| Site | Series URL | Chapter URL | series_id |
|---|---|---|---|
Asura (asurascans.com) |
/comics/<slug-hash> |
/comics/<slug-hash>/chapter/<n> |
<slug-hash> |
Demonic (demonicscans.org) |
/manga/<slug> |
/title/<slug>/chapter/<n>/<page> (chaptered.php?manga=<id>&chapter=<n> 301s here) |
<slug> |
Comix (comix.to) |
/title/<id>-<slug> |
/title/<id>-<slug>/<uploadId>-chapter-<n> |
<id> |
Kagane (kagane.to) |
/series/<uuid> |
/series/<uuid>/reader/<bookUuid> |
<uuid> |
Notes:
asuracomic.netdeep links are dead (re-checked 2026-07-25). They 301 to theasurascans.comroot, discarding the path, at the edge — before the userscript gets a document — so nothing client-side can rescue them. Reach series throughasurascans.com. The host stays matched in case the redirect starts preserving paths again.- Asura
og:titlecarries aChapter N - Read Online \| Asura Scanssuffix that the adapter strips; Demonic chapterog:titleis<Title> Chapter N. - Demonic's
<slug>is identical on/manga/…and the canonical/title/…reader, so a bookmark set from the series page and the auto-update from the reader resolve to the same key. - Asura showed no Next.js markers on the live site, so navigation uses a framework-agnostic watcher (history patch + polling) rather than a Next-only hook — works for client-routed and full-reload sites alike.
If either site changes its URL shape, update the regex in the matching adapter
in userscript/manga-bookmark.user.js and re-verify.