mangaBookmark/README.md

# Manga Bookmark

Track manga read-progress on **asurascans.com**,
**demonicscans.org**, **comix.to**, and **kagane.to** from a phone (Bromite /
mobile Chromium), synced to a self-hosted Go backend so bookmarks unify across
all four sites and all devices.

Two parts:

- **`backend/`** — tiny Go (`net/http` + Postgres via pure-Go `pgx`) sync service. 4 routes,
  static binary, distroless container.
- **`userscript/manga-bookmark.user.js`** — single Bromite-compatible userscript
  (no `GM_*` APIs) that injects an on-page bookmark UI and syncs via `fetch()`.

```
Bromite userscript (isolated world, Shadow DOM UI, localStorage cache)
    -- fetch() HTTPS -->  reverse proxy (TLS + CORS)  -->  Go net/http  -->  Postgres (volume)
                                                              |
                                                              | CDP over the tailnet
                                                              v
                                                    headless Chrome, on-demand,
                                                    on a separate machine
                                                    (chrome/, ADR-0006)
```

Kagane and novelfull sit behind a Cloudflare JavaScript challenge no TLS
fingerprint clears, so the poller reaches those two through a real Chrome over
CDP. That browser is **not** part of the API stack: it is its own compose unit
on a second machine, spawned on the first connection and reaped when idle. The
API needs it only to discover new chapters and to fetch a kagane cover once —
covers are stored, so the library renders in full with the browser switched off.

---

## 1. Backend

### Config (env)

| Var | Default | Notes |
|-----|---------|-------|
| `TOKEN_KEY` | *(required)* | Secret every Reader's userscript credential is derived from (issue #24); only SHA-256 hashes of credentials are stored. |
| `OWNER_DISCORD_ID` | *(required)* | Discord user ID of the owner: seeded as the first Reader, owns every pre-registration bookmark, and is the only Reader who can revoke another's sessions. |
| `ALLOWED_ORIGINS` | Asura + Demonic + Comix + Kagane origins | Comma-separated CORS allowlist. |
| `DATABASE_URL` | *(required)* | Postgres connection URL, e.g. `postgres://bookmarks:…@postgres:5432/bookmarks?sslmode=disable`. Compose builds it from `POSTGRES_PASSWORD`. |
| `COVER_DIR` | *(required)* | Filesystem volume for immutable, content-addressed Cover bytes. Compose builds the image and mounts `cover-data` at this path; standalone runs may choose another writable durable path. |
| `PORT` | `8080` | Plain HTTP; TLS terminated by the proxy. |
| `BROWSER_WS_URL` | empty | CDP endpoint of the remote browser (`ws://<tailnet IP>:9222`), used to poll Kagane/Novelfull past their JS challenge and to fetch uncached Kagane covers. Must be an IP or `localhost` — Chrome's DevTools handler 500s any other Host header, MagicDNS names included. Unset disables both; stored covers still serve. |
| `DISCORD_CLIENT_ID` | *(required)* | Discord application credentials for the browser sign-in (ADR-0002). |
| `DISCORD_CLIENT_SECRET` | *(required)* | As above. Never logged, never echoed in an error. |
| `DISCORD_GUILD_ID` | *(required)* | The one guild whose membership gates sign-in, checked at login only. Membership *is* registration: any member becomes a Reader on first login. |
| `DISCORD_REDIRECT_URI` | *(required)* | Exact callback URL; Discord matches it verbatim against the registered redirect. |
| `DISCORD_REQUIRED_ROLE` | empty | Role snowflake a member must additionally hold. Empty means guild membership alone suffices. |
| `DISCORD_API_BASE` | `https://discord.com/api/v10` | Test seam — tests point it at a local stub so the real token exchange runs. |
| `USERSCRIPT_PATH` | `/userscript/manga-bookmark.user.js` | Bindmounted file served at `/u/{token}/manga-bookmark.user.js`. |
| `NOVEL_USERSCRIPT_PATH` | `/userscript/novel-bookmark.user.js` | Same, for the novel library. |
| `LATEST_CHAPTER_POLL_ENABLED` | `1` | `0` turns the poller off entirely. Pace is per Site in the registry — one Poll Lane per Site, each with its own rest and gap (issue #100) — so no other knobs exist. |

Compose reads a few more from the same `.env` that the backend never sees:
`POSTGRES_PASSWORD` (required — `DATABASE_URL` is built from it, and Postgres
only applies it while `postgres-data` is empty), `BOOKMARK_API_HOST` and
`BOOKMARK_WEB_HOST` (required by the prod override), and the optional
`PROXY_NETWORK` / `TRAEFIK_ENTRYPOINT` / `TRAEFIK_CERTRESOLVER`. The browser
unit has its own `chrome/.env` on its own machine — `BROWSER_BIND_ADDR`
(required, the tailnet IP the CDP port is published on) and the optional
`BROWSER_TZ`. Full commentary is in `.env.example` and `chrome/.env.example`;
deployment order is `DEPLOY.md`.

### Endpoints

| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/bookmarks` | Bearer | All bookmarks of the acting Reader. |
| `PUT` | `/bookmarks/{key}` | Bearer | Upsert one series; returns the row as stored. |
| `DELETE` | `/bookmarks/{key}` | Bearer | Remove one. |
| `GET` | `/healthz` | none | `200 ok`. |
| `GET` | `/u/{token}/manga-bookmark.user.js` | credential in path | Serves the userscript with the requesting Reader's credential substituted in and an mtime-derived `@version`. |

`key` is `<site>:<series_id>` — e.g. `asura:trash-of-the-counts-family-f886a8af`,
`demonic:Infinite-Level-Up-in-Murim`, `comix:12345`, or
`kagane:3fa85f64-5717-4562-b3fc-2c963f66afa6`. Sync is last-write-wins.

`updated_at` orders the bookmark list, so it moves only on real reading
progress: the server applies its timestamp when the row is new or
`last_chapter_num` changes, and otherwise keeps the stored one. Favouriting a
series or recording a newly published chapter therefore leaves the order alone.
Because the timestamp a client sends is only a candidate, `PUT` echoes the row
**as stored** and clients adopt that rather than their own payload.

### Develop / test

```bash
cd backend
go test ./...                          # unit + handler tests
CGO_ENABLED=0 go build                 # static binary
```

**`go test ./...` requires Docker.** The store talks to a real Postgres, so
each test package starts a throwaway `postgres:17-alpine` container and gives
every test its own database inside it (`internal/pgtest`). Nothing is stubbed
and nothing reaches the network beyond the local Docker daemon.

### Run the stack

```bash
cp .env.example .env
# edit .env: set TOKEN_KEY (openssl rand -hex 32) and
#            POSTGRES_PASSWORD (openssl rand -hex 24)

docker compose up -d --build           # binds 127.0.0.1:8080
```

That brings up two services — the API and Postgres. The browser is deliberately
not one of them; without `BROWSER_WS_URL` the poller logs and skips kagane and
novelfull, and everything else works. To run one locally, publish it on the
Docker bridge gateway so the API container can name it by IP:

```bash
cd chrome
echo 'BROWSER_BIND_ADDR=172.17.0.1' > .env
docker compose up -d --build
# then in the repo's own .env:  BROWSER_WS_URL=ws://172.17.0.1:9222
```

Bind it to `127.0.0.1` instead if you only want to drive it from the host, e.g.
`SMOKE_BROWSER_WS_URL=ws://127.0.0.1:9222 go test -run TestSmokeKagane ./internal/latest`.
In production that address is the home machine's tailnet IP and nothing else —
see `DEPLOY.md` §7 and ADR-0006.

Smoke test:

```bash
# The credential is per Reader and derived, so there is no token in .env to
# grep. Take yours from the Userscripts panel's install link after signing in,
# or read it out of an installed script's API_TOKEN constant.
TOKEN=<your Reader credential>
curl -s localhost:8080/healthz                                    # ok
curl -s localhost:8080/bookmarks                                  # 401
curl -s -H "Authorization: Bearer $TOKEN" localhost:8080/bookmarks # []
curl -s -X PUT -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"title":"Test","last_chapter":"Chapter 1","last_chapter_num":1}' \
  localhost:8080/bookmarks/asura:test-1
curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \
  -H 'Access-Control-Request-Method: PUT' \
  localhost:8080/bookmarks/asura:test-1 | grep -i access-control   # 204 + CORS headers
```

### Deploy behind your reverse proxy

Route `https://bookmark-api.<domain>` → the service on `:8080` (TLS at the proxy).

- **Host proxy** (nginx/Caddy on the host): the base compose already binds
  `127.0.0.1:8080`; point the proxy `proxy_pass http://127.0.0.1:8080;`.
- **Docker proxy** (Traefik/nginx in a container on its own network): use the
  override, which drops the published port and joins the shared network:

  ```bash
  docker network create proxy            # once, if it doesn't exist
  docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
  ```
  Set `PROXY_NETWORK` in `.env` if your network isn't named `proxy`.

Verify: `https://bookmark-api.<domain>/healthz` returns `ok` over valid TLS (no
mixed-content), and an `OPTIONS` preflight from a real site origin returns the
CORS headers.

---

## 2. Userscript

### Install

Sign in to the web UI and open the **Userscripts** panel: it offers one
install link per library. Each link serves a script rendered with your own
credential already inside it — you never see, type or copy a credential. The
served script carries `@downloadURL`/`@updateURL` pointing at its
credential-bearing path, so Violentmonkey keeps auto-updating it.

The bindmounted files carry `__API_TOKEN__` placeholders; the backend
substitutes the requesting Reader's credential at serve time, so no real
credential is ever committed. Rotating the credential (same panel) invalidates
every installed copy immediately — reinstall on all devices.

### Install on Bromite (mobile)

Bromite runs Chromium's native userscript engine (no Tampermonkey needed):

1. Bromite → **Settings → User scripts** → enable user scripts (allow the
   permission prompt).
2. Open the install link from the web UI — Bromite detects the `.user.js` and
   offers to install it.
3. Confirm the install; the `@match` list covers both sites.
4. Open a series on either site — a 📑 button appears bottom-right.

> Exact menu wording varies by Bromite build; if "User scripts" is absent,
> update Bromite or use a build with userscript support.

### Desktop iteration (optional)

The script is `GM_*`-free, so it also runs in Tampermonkey/Violentmonkey on
desktop for faster testing — install the same file unchanged.

### Use

- **Bookmark**: on a series or chapter page, open the panel → **+ Bookmark this**.
- **Auto-progress**: opening a chapter of a bookmarked series records it when the
  chapter number ≥ the stored one (re-reading older chapters never regresses
  progress; unparseable numbers set the current chapter).
- **Manual override**: panel → **Edit** on any row forces a specific chapter.
- **Continue**: jumps to the last-read chapter (or the series page).
- **Latest chapter**: rows read `Read: … · Latest: …` once the newest published
  chapter is known and it is ahead of your progress. See below for how that is
  found.
- **Favourites**: the ☆ on any row toggles it; the **★ Favourites** tab narrows
  the list. Favourited series still appear under **All**. The flag syncs, so it
  follows you across devices; the chosen tab does not persist.
- **Archive**: the **Archive** button on any row parks a series — it drops out
  of **All** and **★ Favourites** and moves to the **Archived** tab. The server
  keeps checking it for new chapters, so it is worth coming back to. Archiving
  does not touch read progress, and reading an archived series leaves it
  archived.
- **Finished**: the owner marks a Series finished from its detail page; the
  backend stops polling it, and every Reader sees a read-only label. It is a
  fact about the Series, not a Reader's bucket: old `Finished` bookmarks were
  folded into Archived in the cutover, so there is no Finished tab.
- Bookmarks made on Asura appear when the panel is opened on Demonic, and vice
  versa — the backend is the shared store.

Neither favouriting nor learning a new chapter reorders the list — only reading
progress does.

Offline / backend down: changes are cached in `localStorage` and retried on the
next successful load (last-write-wins).

#### How "latest chapter" is found

Only a series page lists every chapter (a reader page links just its
neighbours), and the backend cannot fetch either site — Cloudflare blocks
server-side requests, and neither site offers an API or feed to poll. So the
userscript does the looking, from your own browser session:

- Opening a bookmarked series page records its newest chapter directly.
- Otherwise it fetches series pages in the background — **same-origin only**, so
  browsing Asura refreshes Asura bookmarks and Demonic refreshes Demonic. One
  series per navigation, and at most one check per series every 4 hours
  (`LATEST_CHECK_BATCH` / `LATEST_CHECK_THROTTLE_MS`). Failures are silent and
  simply retried after the window.

Freshness is tracked per device in `localStorage` under `bmgr:manga:lastchecked`
and is deliberately not synced, since each device checks on its own.

This means a bookmark is as current as its last check — not the moment a
chapter drops. Nothing can be instant here: neither site offers push, feeds, or
an API.

---

## Adapter reference (verified live 2026-07-24)

The site adapters key everything off URL regex, with `title` from `og:title`
(or the page's own heading where a site ships none). No adapter reads a cover:
the backend acquires, stores and serves every Cover from its own origin
(ADR-0007). Confirmed against live pages via Playwright:

| Site | Series URL | Chapter URL | `series_id` |
|------|-----------|-------------|-------------|
| **Asura** (`asurascans.com`) | `/comics/<slug-hash>` | `/comics/<slug-hash>/chapter/<n>` | `<slug-hash>` |
| **Demonic** (`demonicscans.org`) | `/manga/<slug>` | `/title/<slug>/chapter/<n>/<page>` (`chaptered.php?manga=<id>&chapter=<n>` 301s here) | `<slug>` |
| **Comix** (`comix.to`) | `/title/<id>-<slug>` | `/title/<id>-<slug>/<uploadId>-chapter-<n>` | `<id>` |
| **Kagane** (`kagane.to`) | `/series/<uuid>` | `/series/<uuid>/reader/<bookUuid>` | `<uuid>` |

Notes:
- **`asuracomic.net` is no longer matched (deep links dead, re-checked
  2026-07-25).** They 301 to the `asurascans.com` **root**, discarding the path,
  at the edge — before the userscript gets a document — so nothing client-side
  can rescue them. The backend rejects stored addresses on that host too, since
  the poller pins each Site to one hostname. Reach series through
  `asurascans.com`.
- Asura `og:title` carries a `Chapter N - Read Online \| Asura Scans` suffix that
  the adapter strips; Demonic chapter `og:title` is `<Title> Chapter N`.
- Demonic's `<slug>` is identical on `/manga/…` and the canonical `/title/…`
  reader, so a bookmark set from the series page and the auto-update from the
  reader resolve to the **same key**.
- Asura showed no Next.js markers on the live site, so navigation uses a
  framework-agnostic watcher (history patch + polling) rather than a Next-only
  hook — works for client-routed and full-reload sites alike.

If either site changes its URL shape, update the regex in the matching adapter
in `userscript/manga-bookmark.user.js` and re-verify.