sulthan 2cc1e69f5d Prove the import against a copy of the real library (#25) (#33)
Closes #25.

Retires the biggest risk in #18 — losing the owner's reading history — on a copy, before production is anywhere near it.

## What was run

A throwaway generator (python3 stdlib `sqlite3`, ~20 lines, **not committed**) read a copy of `bookmarks-20260807-213515.db` and emitted plain SQL: 29 distinct Series first, then 29 Bookmarks referencing them, each `INSERT ... SELECT id FROM owner` so the reader id is resolved rather than hardcoded. The target was a scratch Postgres whose schema and owner Reader were built by the real binary (`go run .` against a throwaway container), not by hand-written DDL. Production was not touched.

## Verified

| check | result |
|---|---|
| Bookmarks total | 29 |
| reading / archived / other | 18 / 11 / 0 |
| Series | 29, equal to the distinct `(site, series_id)` count in the source |
| Readers | 1; Bookmarks not owned by the owner: 0 |
| Field-by-field diff, all 29 rows x 15 columns | 0 differences |
| `GET /bookmarks` over the real read path | 29 rows, values match source |
| `TRUNCATE bookmarks, series;` then re-apply | clean, 29 again |

The spot-check the ticket asked for was widened to a full row-by-row comparison — 29 rows is small enough that sampling was the more expensive option.

## What is committed

`CUTOVER.md` only, plus two cross-links from `REDEPLOY.md`. The generator stays out of the repository: its output is the owner's reading history, and it reads SQLite, which the backend module dropped in ADR-0001. So the runbook specifies the transformation — column mapping, ordering, nullability, quoting, the temp-table ownership trick — rather than shipping a script. `backend/go.mod` gains nothing.

## Review

Two-axis review ran on the diff; six findings applied, all in the runbook:

- Six source columns (`title`, `series_url`, `cover`, `last_chapter`, `last_chapter_url`, `last_chapter_num`) are nullable in SQLite but `NOT NULL` in Postgres and must be coalesced — the opposite of `latest_chapter_num`, the one column where `NULL` is meaningful. The 2026-08-07 export had none; a fresh one is not promised the same.
- `CREATE TEMP TABLE ... ON COMMIT DROP` must sit *inside* the transaction, or psql's autocommit drops it instantly.
- The ownership check now resolves the Reader by Discord id; comparing against `ORDER BY id LIMIT 1` was true by construction and could never fail.
- The spot-check now samples archived and favourite rows explicitly instead of hoping they fall inside `ORDER BY updated_at DESC LIMIT 5`.
- `git pull --ff-only` before `up -d --build`, or a pre-cutover server rebuilds the SQLite image.
- `python3` and `jq` named as prerequisites; column count corrected to sixteen.

Every query in the runbook was executed against the scratch database as written.

`go vet`, `CGO_ENABLED=0 go build ./...` and `go test ./...` all pass — no Go code changed.

Reviewed-on: #33
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 15:15:33 +07:00
2026-07-24 16:23:24 +07:00
2026-07-24 16:23:24 +07:00

Manga Bookmark

Track manga read-progress on asurascans.com (a.k.a. asuracomic.net), demonicscans.org, comix.to, and kagane.to from a phone (Bromite / mobile Chromium), synced to a self-hosted Go backend so bookmarks unify across all four sites and all devices.

Two parts:

  • backend/ — tiny Go (net/http + Postgres via pure-Go pgx) sync service. 4 routes, static binary, distroless container.
  • userscript/manga-bookmark.user.js — single Bromite-compatible userscript (no GM_* APIs) that injects an on-page bookmark UI and syncs via fetch().
Bromite userscript (isolated world, Shadow DOM UI, localStorage cache)
    -- fetch() HTTPS -->  reverse proxy (TLS + CORS)  -->  Go net/http  -->  Postgres (volume)

1. Backend

Config (env)

Var Default Notes
TOKEN_KEY (required) Secret every Reader's userscript credential is derived from (issue #24); only SHA-256 hashes of credentials are stored.
API_TOKEN (retired) Global credential, honoured only until API_TOKEN_GRACE_UNTIL for already-installed scripts; remove both after the window.
API_TOKEN_GRACE_UNTIL unset Moment the retired credential stops resolving to the owner (YYYY-MM-DD or RFC3339), enforced in code.
OWNER_DISCORD_ID (required) Discord user ID of the owner; seeds the one Reader all bookmarks belong to.
ALLOWED_ORIGINS Asura + Demonic + Comix + Kagane origins Comma-separated CORS allowlist.
DATABASE_URL (required) Postgres connection URL, e.g. postgres://bookmarks:…@postgres:5432/bookmarks?sslmode=disable. Compose builds it from POSTGRES_PASSWORD.
PORT 8080 Plain HTTP; TLS terminated by the proxy.
BROWSER_WS_URL ws://172.28.0.10:9222 Headless-shell CDP endpoint used to poll Kagane past its JS challenge. Must be an IP or localhost — Chrome's DevTools handler 500s any other Host header.

Endpoints

Method Path Auth Description
GET /bookmarks Bearer All bookmarks of the acting Reader.
PUT /bookmarks/{key} Bearer Upsert one series; returns the row as stored.
DELETE /bookmarks/{key} Bearer Remove one.
GET /healthz none 200 ok.
GET /u/{token}/manga-bookmark.user.js credential in path Serves the userscript with the requesting Reader's credential substituted in and an mtime-derived @version.

key is <site>:<series_id> — e.g. asura:trash-of-the-counts-family-f886a8af, demonic:Infinite-Level-Up-in-Murim, comix:12345, or kagane:3fa85f64-5717-4562-b3fc-2c963f66afa6. Sync is last-write-wins.

updated_at orders the bookmark list, so it moves only on real reading progress: the server applies its timestamp when the row is new or last_chapter_num changes, and otherwise keeps the stored one. Favouriting a series or recording a newly published chapter therefore leaves the order alone. Because the timestamp a client sends is only a candidate, PUT echoes the row as stored and clients adopt that rather than their own payload.

Develop / test

cd backend
go test ./...                          # unit + handler tests
CGO_ENABLED=0 go build                 # static binary

go test ./... requires Docker. The store talks to a real Postgres, so each test package starts a throwaway postgres:17-alpine container and gives every test its own database inside it (internal/pgtest). Nothing is stubbed and nothing reaches the network beyond the local Docker daemon.

Run the stack

cp .env.example .env
# edit .env: set TOKEN_KEY (openssl rand -hex 32) and
#            POSTGRES_PASSWORD (openssl rand -hex 24)

docker compose up -d --build           # binds 127.0.0.1:8080

Smoke test:

TOKEN=$(grep '^API_TOKEN=' .env | cut -d= -f2)
curl -s localhost:8080/healthz                                    # ok
curl -s localhost:8080/bookmarks                                  # 401
curl -s -H "Authorization: Bearer $TOKEN" localhost:8080/bookmarks # []
curl -s -X PUT -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"title":"Test","last_chapter":"Chapter 1","last_chapter_num":1}' \
  localhost:8080/bookmarks/asura:test-1
curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \
  -H 'Access-Control-Request-Method: PUT' \
  localhost:8080/bookmarks/asura:test-1 | grep -i access-control   # 204 + CORS headers

Deploy behind your reverse proxy

Route https://bookmark-api.<domain> → the service on :8080 (TLS at the proxy).

  • Host proxy (nginx/Caddy on the host): the base compose already binds 127.0.0.1:8080; point the proxy proxy_pass http://127.0.0.1:8080;.

  • Docker proxy (Traefik/nginx in a container on its own network): use the override, which drops the published port and joins the shared network:

    docker network create proxy            # once, if it doesn't exist
    docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
    

    Set PROXY_NETWORK in .env if your network isn't named proxy.

Verify: https://bookmark-api.<domain>/healthz returns ok over valid TLS (no mixed-content), and an OPTIONS preflight from a real site origin returns the CORS headers.


2. Userscript

Install

Sign in to the web UI and open the Userscripts panel: it offers one install link per library. Each link serves a script rendered with your own credential already inside it — you never see, type or copy a credential. The served script carries @downloadURL/@updateURL pointing at its credential-bearing path, so Violentmonkey keeps auto-updating it.

The bindmounted files carry __API_TOKEN__ placeholders; the backend substitutes the requesting Reader's credential at serve time, so no real credential is ever committed. Rotating the credential (same panel) invalidates every installed copy immediately — reinstall on all devices.

Install on Bromite (mobile)

Bromite runs Chromium's native userscript engine (no Tampermonkey needed):

  1. Bromite → Settings → User scripts → enable user scripts (allow the permission prompt).
  2. Open the install link from the web UI — Bromite detects the .user.js and offers to install it.
  3. Confirm the install; the @match list covers both sites.
  4. Open a series on either site — a 📑 button appears bottom-right.

Exact menu wording varies by Bromite build; if "User scripts" is absent, update Bromite or use a build with userscript support.

Desktop iteration (optional)

The script is GM_*-free, so it also runs in Tampermonkey/Violentmonkey on desktop for faster testing — install the same file unchanged.

Use

  • Bookmark: on a series or chapter page, open the panel → + Bookmark this.
  • Auto-progress: opening a chapter of a bookmarked series records it when the chapter number ≥ the stored one (re-reading older chapters never regresses progress; unparseable numbers set the current chapter).
  • Manual override: panel → Edit on any row forces a specific chapter.
  • Continue: jumps to the last-read chapter (or the series page).
  • Latest chapter: rows read Read: … · Latest: … once the newest published chapter is known and it is ahead of your progress. See below for how that is found.
  • Favourites: the ☆ on any row toggles it; the ★ Favourites tab narrows the list. Favourited series still appear under All. The flag syncs, so it follows you across devices; the chosen tab does not persist.
  • Archive: the Archive button on any row parks a series — it drops out of All and ★ Favourites and moves to the Archived tab. The server keeps checking it for new chapters, so it is worth coming back to. Archiving does not touch read progress, and reading an archived series leaves it archived.
  • Finished: series you have completed live in a Finished tab in the web UI only. It is set there and nowhere else — the API rejects the value — and finished series are hidden from every userscript tab and are no longer polled for new chapters.
  • Bookmarks made on Asura appear when the panel is opened on Demonic, and vice versa — the backend is the shared store.

Neither favouriting nor learning a new chapter reorders the list — only reading progress does.

Offline / backend down: changes are cached in localStorage and retried on the next successful load (last-write-wins).

How "latest chapter" is found

Only a series page lists every chapter (a reader page links just its neighbours), and the backend cannot fetch either site — Cloudflare blocks server-side requests, and neither site offers an API or feed to poll. So the userscript does the looking, from your own browser session:

  • Opening a bookmarked series page records its newest chapter directly.
  • Otherwise it fetches series pages in the background — same-origin only, so browsing Asura refreshes Asura bookmarks and Demonic refreshes Demonic. One series per navigation, and at most one check per series every 4 hours (LATEST_CHECK_BATCH / LATEST_CHECK_THROTTLE_MS). Failures are silent and simply retried after the window.

Freshness is tracked per device in localStorage under bmgr:manga:lastchecked and is deliberately not synced, since each device checks on its own.

This means a bookmark is as current as its last check — not the moment a chapter drops. Nothing can be instant here: neither site offers push, feeds, or an API.


Adapter reference (verified live 2026-07-24)

The site adapters key everything off URL regex, with title/cover from og:title / og:image. Confirmed against live pages via Playwright:

Site Series URL Chapter URL series_id
Asura (asurascans.com) /comics/<slug-hash> /comics/<slug-hash>/chapter/<n> <slug-hash>
Demonic (demonicscans.org) /manga/<slug> /title/<slug>/chapter/<n>/<page> (chaptered.php?manga=<id>&chapter=<n> 301s here) <slug>
Comix (comix.to) /title/<id>-<slug> /title/<id>-<slug>/<uploadId>-chapter-<n> <id>
Kagane (kagane.to) /series/<uuid> /series/<uuid>/reader/<bookUuid> <uuid>

Notes:

  • asuracomic.net deep links are dead (re-checked 2026-07-25). They 301 to the asurascans.com root, discarding the path, at the edge — before the userscript gets a document — so nothing client-side can rescue them. Reach series through asurascans.com. The host stays matched in case the redirect starts preserving paths again.
  • Asura og:title carries a Chapter N - Read Online \| Asura Scans suffix that the adapter strips; Demonic chapter og:title is <Title> Chapter N.
  • Demonic's <slug> is identical on /manga/… and the canonical /title/… reader, so a bookmark set from the series page and the auto-update from the reader resolve to the same key.
  • Asura showed no Next.js markers on the live site, so navigation uses a framework-agnostic watcher (history patch + polling) rather than a Next-only hook — works for client-routed and full-reload sites alike.

If either site changes its URL shape, update the regex in the matching adapter in userscript/manga-bookmark.user.js and re-verify.

S
Description
No description provided
Readme 13 MiB
Languages
Go 75.9%
JavaScript 15.1%
CSS 4.6%
HTML 3.6%
Shell 0.5%
Other 0.3%