08749df050
Swap modernc.org/sqlite for jackc/pgx/v5 with no observable change: same endpoints, same wire format, same updated_at ordering rule. The schema now comes from numbered SQL embedded in the binary and applied on startup, one transaction each, recorded in schema_migrations. That replaces two pieces of SQLite-era machinery, both deleted rather than ported: the column probing (Postgres has ADD COLUMN IF NOT EXISTS, and there is no legacy database left to probe) and the Asura key rewrite, which has run clean on every start for months now that the userscripts strip build hashes before writing. Its regexp survives as latest.asuraBuildHash, where the poller still needs it to scope chapter links to a series whose slug carries a rotating hash. Types get real: favorite is a boolean, chapter numbers double precision, timestamps stay unix-ms bigint. SQLite's null-safe IS NOT becomes IS DISTINCT FROM, which is what implements the rule that only reading progress reorders a list. Inside COALESCE/NULLIF the status and kind parameters need an explicit ::text -- there is no target column to infer from and Postgres refuses to guess. Tests lose their free t.TempDir() database, so Docker is now a hard prerequisite for `go test ./...`: internal/pgtest starts one postgres:17-alpine per test binary and hands each test a database of its own. Also lands CONTEXT.md and the four ADRs written while scoping #18. BREAKING CHANGE: DB_PATH is retired for DATABASE_URL, which is required and has no default. Compose gains a postgres service on an internal network with its own volume; POSTGRES_PASSWORD joins .env. The old bookmarks-data volume is deliberately left undeclared so `docker compose down -v` cannot take the pre-migration database with it. main is not deployable until #25 and #26 land. Closes #20 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
231 lines
10 KiB
Markdown
231 lines
10 KiB
Markdown
# Manga Bookmark
|
|
|
|
Track manga read-progress on **asurascans.com** (a.k.a. asuracomic.net),
|
|
**demonicscans.org**, **comix.to**, and **kagane.to** from a phone (Bromite /
|
|
mobile Chromium), synced to a self-hosted Go backend so bookmarks unify across
|
|
all four sites and all devices.
|
|
|
|
Two parts:
|
|
|
|
- **`backend/`** — tiny Go (`net/http` + Postgres via pure-Go `pgx`) sync service. 4 routes,
|
|
static binary, distroless container.
|
|
- **`userscript/manga-bookmark.user.js`** — single Bromite-compatible userscript
|
|
(no `GM_*` APIs) that injects an on-page bookmark UI and syncs via `fetch()`.
|
|
|
|
```
|
|
Bromite userscript (isolated world, Shadow DOM UI, localStorage cache)
|
|
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> Postgres (volume)
|
|
```
|
|
|
|
---
|
|
|
|
## 1. Backend
|
|
|
|
### Config (env)
|
|
|
|
| Var | Default | Notes |
|
|
|-----|---------|-------|
|
|
| `API_TOKEN` | *(required)* | Bearer token shared with the userscript. |
|
|
| `ALLOWED_ORIGINS` | Asura + Demonic + Comix + Kagane origins | Comma-separated CORS allowlist. |
|
|
| `DATABASE_URL` | *(required)* | Postgres connection URL, e.g. `postgres://bookmarks:…@postgres:5432/bookmarks?sslmode=disable`. Compose builds it from `POSTGRES_PASSWORD`. |
|
|
| `PORT` | `8080` | Plain HTTP; TLS terminated by the proxy. |
|
|
| `BROWSER_WS_URL` | `ws://172.28.0.10:9222` | Headless-shell CDP endpoint used to poll Kagane past its JS challenge. Must be an IP or `localhost` — Chrome's DevTools handler 500s any other Host header. |
|
|
|
|
### Endpoints
|
|
|
|
| Method | Path | Auth | Description |
|
|
|--------|------|------|-------------|
|
|
| `GET` | `/bookmarks` | Bearer | All bookmarks (single-user). |
|
|
| `PUT` | `/bookmarks/{key}` | Bearer | Upsert one series; returns the row as stored. |
|
|
| `DELETE` | `/bookmarks/{key}` | Bearer | Remove one. |
|
|
| `GET` | `/healthz` | none | `200 ok`. |
|
|
| `GET` | `/u/{token}/manga-bookmark.user.js` | token in path | Serves the userscript with an mtime-derived `@version`. |
|
|
|
|
`key` is `<site>:<series_id>` — e.g. `asura:trash-of-the-counts-family-f886a8af`,
|
|
`demonic:Infinite-Level-Up-in-Murim`, `comix:12345`, or
|
|
`kagane:3fa85f64-5717-4562-b3fc-2c963f66afa6`. Sync is last-write-wins.
|
|
|
|
`updated_at` orders the bookmark list, so it moves only on real reading
|
|
progress: the server applies its timestamp when the row is new or
|
|
`last_chapter_num` changes, and otherwise keeps the stored one. Favouriting a
|
|
series or recording a newly published chapter therefore leaves the order alone.
|
|
Because the timestamp a client sends is only a candidate, `PUT` echoes the row
|
|
**as stored** and clients adopt that rather than their own payload.
|
|
|
|
### Develop / test
|
|
|
|
```bash
|
|
cd backend
|
|
go test ./... # unit + handler tests
|
|
CGO_ENABLED=0 go build # static binary
|
|
```
|
|
|
|
**`go test ./...` requires Docker.** The store talks to a real Postgres, so
|
|
each test package starts a throwaway `postgres:17-alpine` container and gives
|
|
every test its own database inside it (`internal/pgtest`). Nothing is stubbed
|
|
and nothing reaches the network beyond the local Docker daemon.
|
|
|
|
### Run the stack
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# edit .env: set API_TOKEN (openssl rand -hex 32) and
|
|
# POSTGRES_PASSWORD (openssl rand -hex 24)
|
|
|
|
docker compose up -d --build # binds 127.0.0.1:8080
|
|
```
|
|
|
|
Smoke test:
|
|
|
|
```bash
|
|
TOKEN=$(grep '^API_TOKEN=' .env | cut -d= -f2)
|
|
curl -s localhost:8080/healthz # ok
|
|
curl -s localhost:8080/bookmarks # 401
|
|
curl -s -H "Authorization: Bearer $TOKEN" localhost:8080/bookmarks # []
|
|
curl -s -X PUT -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
|
|
-d '{"title":"Test","last_chapter":"Chapter 1","last_chapter_num":1}' \
|
|
localhost:8080/bookmarks/asura:test-1
|
|
curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \
|
|
-H 'Access-Control-Request-Method: PUT' \
|
|
localhost:8080/bookmarks/asura:test-1 | grep -i access-control # 204 + CORS headers
|
|
```
|
|
|
|
### Deploy behind your reverse proxy
|
|
|
|
Route `https://bookmark-api.<domain>` → the service on `:8080` (TLS at the proxy).
|
|
|
|
- **Host proxy** (nginx/Caddy on the host): the base compose already binds
|
|
`127.0.0.1:8080`; point the proxy `proxy_pass http://127.0.0.1:8080;`.
|
|
- **Docker proxy** (Traefik/nginx in a container on its own network): use the
|
|
override, which drops the published port and joins the shared network:
|
|
|
|
```bash
|
|
docker network create proxy # once, if it doesn't exist
|
|
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
|
|
```
|
|
Set `PROXY_NETWORK` in `.env` if your network isn't named `proxy`.
|
|
|
|
Verify: `https://bookmark-api.<domain>/healthz` returns `ok` over valid TLS (no
|
|
mixed-content), and an `OPTIONS` preflight from a real site origin returns the
|
|
CORS headers.
|
|
|
|
---
|
|
|
|
## 2. Userscript
|
|
|
|
### Configure
|
|
|
|
Edit the config block at the top of `userscript/manga-bookmark.user.js`:
|
|
|
|
```js
|
|
const API_BASE = "https://bookmark-api.<domain>"; // no trailing slash
|
|
const API_TOKEN = "<same token as backend>";
|
|
```
|
|
|
|
The token lives in the userscript's **isolated world** — the manga sites' own
|
|
JS cannot read it.
|
|
|
|
### Install on Bromite (mobile)
|
|
|
|
Bromite runs Chromium's native userscript engine (no Tampermonkey needed):
|
|
|
|
1. Bromite → **Settings → User scripts** → enable user scripts (allow the
|
|
permission prompt).
|
|
2. Save the configured `manga-bookmark.user.js` to the device (or open its raw
|
|
URL). Bromite detects the `.user.js` and offers to install it.
|
|
3. Confirm the install; the `@match` list covers both sites.
|
|
4. Open a series on either site — a 📑 button appears bottom-right.
|
|
|
|
> Exact menu wording varies by Bromite build; if "User scripts" is absent,
|
|
> update Bromite or use a build with userscript support.
|
|
|
|
### Desktop iteration (optional)
|
|
|
|
The script is `GM_*`-free, so it also runs in Tampermonkey/Violentmonkey on
|
|
desktop for faster testing — install the same file unchanged.
|
|
|
|
### Use
|
|
|
|
- **Bookmark**: on a series or chapter page, open the panel → **+ Bookmark this**.
|
|
- **Auto-progress**: opening a chapter of a bookmarked series records it when the
|
|
chapter number ≥ the stored one (re-reading older chapters never regresses
|
|
progress; unparseable numbers set the current chapter).
|
|
- **Manual override**: panel → **Edit** on any row forces a specific chapter.
|
|
- **Continue**: jumps to the last-read chapter (or the series page).
|
|
- **Latest chapter**: rows read `Read: … · Latest: …` once the newest published
|
|
chapter is known and it is ahead of your progress. See below for how that is
|
|
found.
|
|
- **Favourites**: the ☆ on any row toggles it; the **★ Favourites** tab narrows
|
|
the list. Favourited series still appear under **All**. The flag syncs, so it
|
|
follows you across devices; the chosen tab does not persist.
|
|
- **Archive**: the **Archive** button on any row parks a series — it drops out
|
|
of **All** and **★ Favourites** and moves to the **Archived** tab. The server
|
|
keeps checking it for new chapters, so it is worth coming back to. Archiving
|
|
does not touch read progress, and reading an archived series leaves it
|
|
archived.
|
|
- **Finished**: series you have completed live in a **Finished** tab in the web
|
|
UI only. It is set there and nowhere else — the API rejects the value — and
|
|
finished series are hidden from every userscript tab and are no longer polled
|
|
for new chapters.
|
|
- Bookmarks made on Asura appear when the panel is opened on Demonic, and vice
|
|
versa — the backend is the shared store.
|
|
|
|
Neither favouriting nor learning a new chapter reorders the list — only reading
|
|
progress does.
|
|
|
|
Offline / backend down: changes are cached in `localStorage` and retried on the
|
|
next successful load (last-write-wins).
|
|
|
|
#### How "latest chapter" is found
|
|
|
|
Only a series page lists every chapter (a reader page links just its
|
|
neighbours), and the backend cannot fetch either site — Cloudflare blocks
|
|
server-side requests, and neither site offers an API or feed to poll. So the
|
|
userscript does the looking, from your own browser session:
|
|
|
|
- Opening a bookmarked series page records its newest chapter directly.
|
|
- Otherwise it fetches series pages in the background — **same-origin only**, so
|
|
browsing Asura refreshes Asura bookmarks and Demonic refreshes Demonic. One
|
|
series per navigation, and at most one check per series every 4 hours
|
|
(`LATEST_CHECK_BATCH` / `LATEST_CHECK_THROTTLE_MS`). Failures are silent and
|
|
simply retried after the window.
|
|
|
|
Freshness is tracked per device in `localStorage` under `bmgr:manga:lastchecked`
|
|
and is deliberately not synced, since each device checks on its own.
|
|
|
|
This means a bookmark is as current as its last check — not the moment a
|
|
chapter drops. Nothing can be instant here: neither site offers push, feeds, or
|
|
an API.
|
|
|
|
---
|
|
|
|
## Adapter reference (verified live 2026-07-24)
|
|
|
|
The site adapters key everything off URL regex, with `title`/`cover` from
|
|
`og:title` / `og:image`. Confirmed against live pages via Playwright:
|
|
|
|
| Site | Series URL | Chapter URL | `series_id` |
|
|
|------|-----------|-------------|-------------|
|
|
| **Asura** (`asurascans.com`) | `/comics/<slug-hash>` | `/comics/<slug-hash>/chapter/<n>` | `<slug-hash>` |
|
|
| **Demonic** (`demonicscans.org`) | `/manga/<slug>` | `/title/<slug>/chapter/<n>/<page>` (`chaptered.php?manga=<id>&chapter=<n>` 301s here) | `<slug>` |
|
|
| **Comix** (`comix.to`) | `/title/<id>-<slug>` | `/title/<id>-<slug>/<uploadId>-chapter-<n>` | `<id>` |
|
|
| **Kagane** (`kagane.to`) | `/series/<uuid>` | `/series/<uuid>/reader/<bookUuid>` | `<uuid>` |
|
|
|
|
Notes:
|
|
- **`asuracomic.net` deep links are dead (re-checked 2026-07-25).** They 301 to
|
|
the `asurascans.com` **root**, discarding the path, at the edge — before the
|
|
userscript gets a document — so nothing client-side can rescue them. Reach
|
|
series through `asurascans.com`. The host stays matched in case the redirect
|
|
starts preserving paths again.
|
|
- Asura `og:title` carries a `Chapter N - Read Online \| Asura Scans` suffix that
|
|
the adapter strips; Demonic chapter `og:title` is `<Title> Chapter N`.
|
|
- Demonic's `<slug>` is identical on `/manga/…` and the canonical `/title/…`
|
|
reader, so a bookmark set from the series page and the auto-update from the
|
|
reader resolve to the **same key**.
|
|
- Asura showed no Next.js markers on the live site, so navigation uses a
|
|
framework-agnostic watcher (history patch + polling) rather than a Next-only
|
|
hook — works for client-routed and full-reload sites alike.
|
|
|
|
If either site changes its URL shape, update the regex in the matching adapter
|
|
in `userscript/manga-bookmark.user.js` and re-verify.
|