docs: initial wiki

2026-07-26 20:39:31 +07:00
parent 1bedb56a30
commit 5f3fbc0d9c
8 changed files with 518 additions and 1 deletions
+65
@@ -0,0 +1,65 @@
# Architecture
```
Bromite userscript (isolated world, per-site adapters, localStorage cache)
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume)
```
## Why this shape
Bromite runs Chromium's **native** userscript engine, not Tampermonkey. That
rules out `GM_*` APIs entirely:
- No `GM_setValue`/`GM_getValue` → page `localStorage` instead.
- No `GM_registerMenuCommand` → on-page UI (floating button + slide-in panel)
instead of a browser menu.
- No `GM_xmlhttpRequest` for cross-origin → plain `fetch()`, which only works
against a CORS-enabled backend.
Manga sites are `https://`, so the backend **must** be HTTPS or the browser
blocks the `fetch()` as mixed content.
Asura and Demonic are separate origins with separate `localStorage` — a
shared remote store (the backend) is the only way to unify bookmarks across
them. Cloud sync is required, not a nice-to-have.
The userscript runs in an **isolated world**, so the embedded API token is
invisible to the site's own JS.
## Components
- **Backend** (`backend/`) — stdlib `net/http`, no framework, `modernc.org/sqlite`
(pure Go, `CGO_ENABLED=0` → static binary → distroless image). Listens plain
`:8080`; the reverse proxy terminates TLS.
- **Store** — one `bookmarks` table, keyed `<site>:<series_id>`. Single-user.
Sync is last-write-wins.
- **Web UI** — same binary, second hostname, password-gated. See [[Web-UI]].
- **Latest-chapter poller** — background goroutine that re-checks each
bookmarked series' newest chapter on its own schedule, independent of the
userscript. Details in [[Backend-API]] and `plans/2026-07-26-server-latest-chapter-polling.md`.
- **Userscript** — one file, per-site adapters, Shadow DOM UI. See [[Userscript]].
## The `updated_at` rule
`updated_at` drives list order, so it must move **only** on real reading
progress — not on favouriting, not on the poller learning a new chapter.
The server applies its own timestamp when a row is new or `last_chapter_num`
changed; otherwise it keeps the stored value. `PUT` returns the row **as
stored**, and every client (userscript, web UI) adopts that response instead
of its own payload. This is the single place that rule lives — see
`plans/2026-07-25-bookmark-list-favorites-design.md` §4.
## Known accepted limitation
The poller's read-modify-write (`Store.Get` + `Store.Upsert`) is not wrapped
in a transaction. A userscript `PUT` that lands between the poller's read and
write can be overwritten by the poller's stale re-read — reverting progress
and reordering the list. Accepted for a single-user deployment; not scheduled
to be fixed.
## Skills used for infra work
`multi-stage-dockerfile` and `docker-compose-orchestration` (see repo
`CLAUDE.md`) cover the container patterns behind `backend/Dockerfile` and the
compose files.
+96
@@ -0,0 +1,96 @@
# Backend API
## Config (env)
| Var | Default | Notes |
|-----|---------|-------|
| `API_TOKEN` | *(required)* | Bearer token shared with the userscript. |
| `ALLOWED_ORIGINS` | Asura + Demonic origins | Comma-separated CORS allowlist. |
| `DB_PATH` | `/data/bookmarks.db` | SQLite file location. |
| `PORT` | `8080` | Plain HTTP; TLS terminated by the proxy. |
| `WEB_PASSWORD` | unset | Gates the browser UI. Unset = web routes not registered at all. |
| `LATEST_CHAPTER_POLL_ENABLED` | `true` | Background latest-chapter poller on/off. |
| `LATEST_CHAPTER_POLL_COOLDOWN` | `1h` | Minimum time between checks of the same series. |
| `LATEST_CHAPTER_POLL_INTERVAL` | `10m` | Wake interval of the poller ticker. |
| `LATEST_CHAPTER_POLL_BATCH` | `14` | Series checked per wake. |
| `LATEST_CHAPTER_POLL_STAGGER` | `20s` | Delay between fetches within a batch. |
## Endpoints
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/bookmarks` | Bearer | All bookmarks (single-user). |
| `PUT` | `/bookmarks/{key}` | Bearer | Upsert one series; returns the row **as stored**. |
| `DELETE` | `/bookmarks/{key}` | Bearer | Remove one. |
| `GET` | `/healthz` | none | `200 ok`. |
`key` is `<site>:<series_id>` — e.g. `asura:trash-of-the-counts-family-f886a8af`
or `demonic:Infinite-Level-Up-in-Murim`.
Web UI routes (only when `WEB_PASSWORD` set) are separate — see [[Web-UI]].
## Auth & CORS (security invariants)
- `/bookmarks*` requires `Authorization: Bearer <API_TOKEN>`, checked with a
**constant-time compare**; `401` otherwise.
- CORS reflects `Origin` only when it's in `ALLOWED_ORIGINS`; allows
`GET,PUT,DELETE,OPTIONS` and headers `Authorization,Content-Type`; answers
preflight `OPTIONS` with `204`.
- CORS is the outermost middleware layer, so preflight short-circuits before
auth ever runs. `/healthz` is public, everything under `/bookmarks` is not.
## Sync model
Single-user, single `bookmarks` table, last-write-wins. Whichever device
`PUT`s last for a given key overrides the stored row.
## The `updated_at` rule
List order is driven by `updated_at`, and it only advances on real reading
progress:
- Row is new, **or**
- `last_chapter_num` changed from what's stored
Otherwise the server keeps the existing `updated_at` — so toggling a
favourite or the poller discovering a new published chapter never reorders
the list. Because of this, `PUT` returns the row as the server actually
stored it, and every client must adopt that response rather than trust its
own request payload. Full rationale: `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
## Latest-chapter poller
A ticker goroutine in the same binary, independent of the userscript's own
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic — two
parallel clocks feeding the same `latest_chapter` field:
- Per-bookmark cooldown via the `latest_checked_at` column
(`Store.DueForLatestCheck`'s `WHERE` clause).
- A wake interval that picks a batch of due series each tick.
The row is stamped **before** the fetch, so a broken series waits out a full
cooldown instead of retrying every tick. Fetches use `bogdanfinn/tls-client`
with a Chrome TLS profile as defence in depth against fingerprint-based
blocking. Any failure logs and skips — this is an enhancement, not on the
request path, and never blocks bookmark sync. Design doc:
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
## Develop / test
```bash
cd backend
go test ./... # unit + handler tests
CGO_ENABLED=0 go build # static binary
```
## Smoke test
```bash
TOKEN=$(grep '^API_TOKEN=' .env | cut -d= -f2)
curl -s localhost:8080/healthz # ok
curl -s localhost:8080/bookmarks # 401
curl -s -H "Authorization: Bearer $TOKEN" localhost:8080/bookmarks # []
curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \
-H 'Access-Control-Request-Method: PUT' \
localhost:8080/bookmarks/asura:test-1 | grep -i access-control # 204 + CORS headers
```
+96
@@ -0,0 +1,96 @@
# Deployment
Docker + Traefik. Assumes Traefik already runs in Docker with a working HTTPS
entrypoint/cert resolver, and you control a domain. Full detail:
`DEPLOY.md` in the main repo — this page is the condensed path.
## 0. Prerequisites
- Docker + Docker Compose on the server.
- Traefik watching a Docker network (default assumed name: `proxy`).
- DNS `A`/`AAAA` records for `manga-api.<domain>` (API) and, if using the web
UI, `manga.<domain>` — both pointing at the server.
- Repo copied to the server (needs `backend/`, `docker-compose.yml`,
`docker-compose.prod.yml`, `.env.example`).
```bash
docker network ls | grep proxy || docker network create proxy
```
## 1. Configure `.env`
```bash
cp .env.example .env
sed -i "s|^API_TOKEN=.*|API_TOKEN=$(openssl rand -hex 32)|" .env
```
Required vars: `API_TOKEN`, `ALLOWED_ORIGINS`, `MANGA_API_HOST`,
`MANGA_WEB_HOST` (needed even if the web UI stays off — its Traefik label has
no fallback). Full var table: [[Backend-API]].
## 1b. Web UI (optional)
```ini
MANGA_WEB_HOST=manga.<domain>
WEB_PASSWORD=<openssl rand -base64 18>
```
Leaving `WEB_PASSWORD` unset is safe — web routes never register, `/` 404s,
the userscript API is unaffected. See [[Web-UI]] for what the password gates.
Rotating `API_TOKEN` or `WEB_PASSWORD` logs every browser session out
(sessions are stateless, keyed off both).
## 2. Build + start
```bash
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
```
Always pass both `-f` flags — the prod override alone is not standalone (it
drops the published port and adds Traefik labels).
```bash
docker compose -f docker-compose.yml -f docker-compose.prod.yml ps
docker logs manga-api --tail 20 # expect: "listening on :8080 ..."
```
## 3. Verify over HTTPS
```bash
curl -s https://manga-api.<domain>/healthz # -> ok
curl -s -o /dev/null -w '%{http_code}\n' https://manga-api.<domain>/bookmarks # -> 401
TOKEN=$(grep -E '^API_TOKEN=' .env | cut -d= -f2)
curl -s -H "Authorization: Bearer $TOKEN" https://manga-api.<domain>/bookmarks # -> []
curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \
-H 'Access-Control-Request-Method: PUT' \
https://manga-api.<domain>/bookmarks/x | grep -i access-control
```
All must pass — valid TLS is non-negotiable (mixed content blocks the
userscript's `fetch()` otherwise).
## 4–5. Configure and install the userscript
See [[Userscript]] — set `API_BASE`/`API_TOKEN` in the file, install on
Bromite.
## 6. Smoke-test the full loop
Bookmark on Asura → confirm via `curl /bookmarks` on the server → open a
chapter, confirm progress updates → open Demonic, confirm the same bookmark
shows there.
## Updating
```bash
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
```
SQLite data persists in the named volume `bookmarks-data` across rebuilds.
## Troubleshooting
See [[Troubleshooting]].
+47
@@ -0,0 +1,47 @@
# Development
## Backend
```bash
cd backend
go test ./... # unit + handler tests
go test -run TestName ./... # single test
CGO_ENABLED=0 go build # static binary (needed for the distroless image)
```
## Local stack
```bash
docker compose up # named volume mounted at /data, restart: unless-stopped
```
Uses `docker-compose.yml` only (no Traefik override, no host domain needed) —
binds `127.0.0.1:8080`.
## Userscript iteration
The file is `GM_*`-free, so install the same unmodified file in desktop
Tampermonkey/Violentmonkey for faster iteration than round-tripping through a
phone. Point `API_BASE` at a local/dev backend while testing.
## Relevant skills
`multi-stage-dockerfile` and `docker-compose-orchestration` cover the
container patterns used in `backend/Dockerfile` and the compose files.
## Design docs
Each non-trivial feature has a plan/spec written before implementation:
- `plans/mangaBookmark.md` — original spec, still the source of truth for
scope and constraints.
- `plans/2026-07-25-bookmark-implementation-plan.md`
- `plans/2026-07-25-bookmark-list-favorites-design.md` — the `updated_at`
ordering rule, in full.
- `plans/2026-07-25-web-ui-implementation-plan.md`
- `plans/2026-07-26-server-latest-chapter-polling.md`
- `docs/superpowers/specs/` — design docs for web UI and the latest-chapter
poller.
Read the relevant plan before changing behaviour it describes — it usually
records *why*, not just *what*.
+33 -1
@@ -1 +1,33 @@
Welcome to the Wiki.
# Manga Bookmark Wiki
Manga read-progress tracker for **asurascans.com** and **demonicscans.org**,
read from **Bromite** (mobile Chromium). A userscript injects on-page UI and
syncs progress to a self-hosted Go backend, so bookmarks unify across both
sites and all devices.
```
Bromite userscript (isolated world, Shadow DOM UI, localStorage cache)
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume)
```
## Pages
- [[Architecture]] — components, data flow, why each design choice exists
- [[Backend-API]] — config vars, endpoints, the `updated_at` ordering rule
- [[Web-UI]] — password-gated browser UI, sessions, routes
- [[Userscript]] — site adapters, config, install, day-to-day use
- [[Deployment]] — Docker Compose + Traefik, step by step
- [[Development]] — running tests, building, local dev loop
- [[Troubleshooting]] — common failures and fixes
## Repo layout
| Path | What |
|------|------|
| `backend/` | Go service: `net/http` + pure-Go SQLite (`modernc.org/sqlite`), static binary |
| `userscript/manga-bookmark.user.js` | single-file Bromite/Tampermonkey userscript |
| `docker-compose.yml` / `docker-compose.prod.yml` | base stack / Traefik override |
| `plans/` | design docs for each feature (source of truth for *why*) |
Full source lives in the main repo; this wiki is the map, not a copy — when in
doubt, the code and `plans/*.md` win.
+15
@@ -0,0 +1,15 @@
# Troubleshooting
| Symptom | Likely cause / fix |
|---------|--------------------|
| No cert / TLS error at the domain | `TRAEFIK_ENTRYPOINT` or `TRAEFIK_CERTRESOLVER` name wrong, or DNS not resolving yet. Check `docker logs <traefik>`. |
| 404 from Traefik | Service not on the `proxy` network, or `MANGA_API_HOST` mismatch. Confirm `docker network inspect proxy` lists `manga-api`. |
| `fetch` fails in the userscript, `curl` works | Origin missing from `ALLOWED_ORIGINS`, or mixed content (backend not HTTPS). |
| 401 with the right token | Trailing space/newline in `API_TOKEN`; regenerate and restart. |
| Panel button absent | URL didn't match an adapter, or user scripts disabled in Bromite. |
| `compose ... config` errors about `API_TOKEN` | Run compose from the dir with `.env`, or export the vars. |
| Web UI logs everyone out unexpectedly | `API_TOKEN` or `WEB_PASSWORD` was rotated — sessions are stateless and keyed off both, so this is expected, not a bug. |
| curl to asurascans.com/demonicscans.org gets challenged | Cloudflare's block is IP-reputation-based, not universal or permanent — re-check live rather than assume; see [[Architecture]] / repo `CLAUDE.md`. |
| A manga site changed its URL shape, adapter stops matching | Update the regex in that site's adapter in `manga-bookmark.user.js`, re-verify against a live page (Playwright or a direct probe) before trusting it. |
Backend config reference and endpoint list: [[Backend-API]].
+105
@@ -0,0 +1,105 @@
# Userscript
Single file: `userscript/manga-bookmark.user.js`. One IIFE, no `GM_*` APIs, so
it runs unchanged in Bromite's native engine **and** in desktop
Tampermonkey/Violentmonkey for fast iteration.
## Structure
1. **Site adapters** — one per host. `detect(location, document)` returns
page `type` + IDs, keyed off **URL regex** (most stable signal).
`title`/`cover` come from `og:title`/`og:image` meta tags, not CSS classes.
2. **API client** — `apiGet/apiPut/apiDelete` with bearer header.
`localStorage` key `mangabm:cache` gives instant render + offline fallback.
3. **Progress logic** — auto-upserts `last_chapter` only when
`chapterNum >= stored last_chapter_num` (re-reading old chapters never
regresses progress; an unparseable number just sets the current chapter).
Manual panel override forces any value regardless.
4. **UI** — rendered inside a **Shadow DOM** root, isolating it from site CSS
(critical on mobile where site styles are unpredictable).
5. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages:
the script patches `history.pushState`/`replaceState` and listens
`popstate`, re-running `detect()` on URL change so auto-update fires
without a full reload. Demonic does classic reloads, so an initial
`document-idle` run is enough there.
## Config
Edit the block at the top of the file:
```js
const API_BASE = "https://manga-api.<domain>"; // no trailing slash
const API_TOKEN = "<same token as backend's API_TOKEN>";
```
The token lives in the userscript's **isolated world** — the manga sites'
own JS cannot read it.
## Site adapter reference (verified live 2026-07-24)
| Site | Series URL | Chapter URL | `series_id` |
|------|-----------|-------------|-------------|
| **Asura** (`asurascans.com`) | `/comics/<slug-hash>` | `/comics/<slug-hash>/chapter/<n>` | `<slug-hash>` |
| **Demonic** (`demonicscans.org`) | `/manga/<slug>` | `/title/<slug>/chapter/<n>/<page>` (`chaptered.php?manga=<id>&chapter=<n>` 301s here) | `<slug>` |
Notes:
- `asuracomic.net` deep links 301 to the **root** of `asurascans.com`,
discarding the path — dead end, use `asurascans.com` directly. The host
stays in `@match` in case the redirect starts preserving paths again.
- Asura `og:title` has a `Chapter N - Read Online | Asura Scans` suffix the
adapter strips; Demonic chapter `og:title` is `<Title> Chapter N`.
- Demonic's `<slug>` is identical on `/manga/…` and the canonical `/title/…`
reader, so a bookmark set from the series page and an auto-update from the
reader resolve to the same key.
- If either site changes its URL shape, update the regex in that site's
adapter and re-verify against a live page before trusting it.
## Install on Bromite (mobile)
1. Bromite → **Settings → User scripts** → enable (accept the permission
prompt).
2. Save the configured `manga-bookmark.user.js` to the device (or open its
raw URL) — Bromite detects `.user.js` and offers to install.
3. Confirm install; the `@match` list covers both sites.
4. Open a series on either site — a 📑 button appears bottom-right.
> Menu wording varies by Bromite build. If "User scripts" is absent, update
> Bromite or use a build that has it.
## Day-to-day use
- **Bookmark**: panel → **+ Bookmark this**.
- **Auto-progress**: opening a chapter of a bookmarked series records it when
the chapter number is ≥ the stored one.
- **Manual override**: panel → **Edit** on any row forces a specific chapter.
- **Continue**: jumps to the last-read chapter (or the series page).
- **Latest chapter**: rows show `Read: … · Latest: …` once the newest
published chapter is known and ahead of your progress.
- **Favourites**: ☆ toggles per row; **★ Favourites** tab narrows the list.
Favourited series still show under **All**. The flag syncs across devices;
the chosen tab does not.
- Bookmarks made on one site appear when the panel opens on the other — the
backend is the shared store.
- Neither favouriting nor a newly-learned chapter reorders the list — only
reading progress does (see [[Backend-API]]).
- Offline / backend down: changes cache in `localStorage`, retried on the
next successful load (last-write-wins).
## How "latest chapter" is found
Only a series page lists every chapter; the backend can't fetch either site
itself (Cloudflare blocks server-side requests, no API/feed exists). So the
userscript does the looking, from your own browser session:
- Opening a bookmarked series page records its newest chapter directly.
- Otherwise it background-fetches series pages — **same-origin only**, so
browsing Asura refreshes Asura bookmarks and vice versa. At most one series
check per navigation, at most one check per series every 4 hours
(`LATEST_CHECK_BATCH` / `LATEST_CHECK_THROTTLE_MS`). Failures are silent,
retried after the window.
Freshness (`mangabm:lastchecked` in `localStorage`) is per-device and
deliberately not synced — each device checks on its own. The backend also
runs its own independent poller (see [[Backend-API]]) so `latest_chapter`
stays fresh even when you aren't browsing.
+61
@@ -0,0 +1,61 @@
# Web UI
The same backend binary serves a password-gated browser UI on a **second
hostname**, so you can view/edit bookmarks from any browser, not just via the
userscript.
## Enable / disable
Controlled entirely by `WEB_PASSWORD`:
- **Set** → web routes registered, UI reachable, gated by the password.
- **Unset** → web routes are **not registered at all**; `/` returns `404`.
The userscript's `/bookmarks*` API is unaffected either way.
## Routes
| Method | Path | Auth | Purpose |
|--------|------|------|---------|
| `GET` | `/` | session or none | List (or login page if no session) |
| `POST` | `/login` | password | Sets session cookie |
| `POST` | `/logout` | session | Clears cookie |
| `GET` | `/static/*` | none | CSS/JS assets (`go:embed`) |
| `GET` | `/ui/list` | session | htmx fragment: bookmark list |
| `POST` | `/ui/bookmarks/{key}/favorite` | session | Toggle favourite |
| `POST` | `/ui/bookmarks/{key}/chapter` | session | Manual chapter override |
| `DELETE` | `/ui/bookmarks/{key}` | session | Remove a bookmark |
Templates (`templates/*.html`) and assets (`static/*`) are `go:embed`-ed into
the binary — `backend/Dockerfile` copies `templates/` and `static/` alongside
the `*.go` files.
## Sessions
Stateless — no session table. The cookie is:
```
<expiryMs>.<base64url HMAC-SHA256(expiryMs)>
```
signed with a key derived from **both** `API_TOKEN` and `WEB_PASSWORD`
(SHA-256 of `apiToken + "\x00" + webPassword + "mangabm-web-session-v1"`).
Rotating either secret invalidates every outstanding session at once —
there's nothing to revoke individually, so this is the way to force a logout
everywhere.
- TTL: **60 days** — long enough a phone stays logged in between reading
sessions.
- Verification order: shape → expiry → HMAC (constant-time compare last, so
earlier cheap checks leak no timing information about the signature).
- Login is rate-limited (see `newLoginLimiter()` in `session.go`).
## Mutations stay consistent with the API
Every UI mutation (favourite, chapter override, delete) goes through the same
`Store.Get` + `Store.Upsert` path the `/bookmarks` API uses — so the
`updated_at` ordering rule (see [[Backend-API]]) applies identically whether
the change came from the userscript or the web UI.
## Design doc
Full design rationale: `docs/superpowers/specs/2026-07-25-web-ui-design.md`.