diff --git a/Architecture.md b/Architecture.md new file mode 100644 index 0000000..ec14b39 --- /dev/null +++ b/Architecture.md @@ -0,0 +1,65 @@ +# Architecture + +``` +Bromite userscript (isolated world, per-site adapters, localStorage cache) + -- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume) +``` + +## Why this shape + +Bromite runs Chromium's **native** userscript engine, not Tampermonkey. That +rules out `GM_*` APIs entirely: + +- No `GM_setValue`/`GM_getValue` → page `localStorage` instead. +- No `GM_registerMenuCommand` → on-page UI (floating button + slide-in panel) + instead of a browser menu. +- No `GM_xmlhttpRequest` for cross-origin → plain `fetch()`, which only works + against a CORS-enabled backend. + +Manga sites are `https://`, so the backend **must** be HTTPS or the browser +blocks the `fetch()` as mixed content. + +Asura and Demonic are separate origins with separate `localStorage` — a +shared remote store (the backend) is the only way to unify bookmarks across +them. Cloud sync is required, not a nice-to-have. + +The userscript runs in an **isolated world**, so the embedded API token is +invisible to the site's own JS. + +## Components + +- **Backend** (`backend/`) — stdlib `net/http`, no framework, `modernc.org/sqlite` + (pure Go, `CGO_ENABLED=0` → static binary → distroless image). Listens plain + `:8080`; the reverse proxy terminates TLS. +- **Store** — one `bookmarks` table, keyed `:`. Single-user. + Sync is last-write-wins. +- **Web UI** — same binary, second hostname, password-gated. See [[Web-UI]]. +- **Latest-chapter poller** — background goroutine that re-checks each + bookmarked series' newest chapter on its own schedule, independent of the + userscript. Details in [[Backend-API]] and `plans/2026-07-26-server-latest-chapter-polling.md`. +- **Userscript** — one file, per-site adapters, Shadow DOM UI. See [[Userscript]]. + +## The `updated_at` rule + +`updated_at` drives list order, so it must move **only** on real reading +progress — not on favouriting, not on the poller learning a new chapter. + +The server applies its own timestamp when a row is new or `last_chapter_num` +changed; otherwise it keeps the stored value. `PUT` returns the row **as +stored**, and every client (userscript, web UI) adopts that response instead +of its own payload. This is the single place that rule lives — see +`plans/2026-07-25-bookmark-list-favorites-design.md` §4. + +## Known accepted limitation + +The poller's read-modify-write (`Store.Get` + `Store.Upsert`) is not wrapped +in a transaction. A userscript `PUT` that lands between the poller's read and +write can be overwritten by the poller's stale re-read — reverting progress +and reordering the list. Accepted for a single-user deployment; not scheduled +to be fixed. + +## Skills used for infra work + +`multi-stage-dockerfile` and `docker-compose-orchestration` (see repo +`CLAUDE.md`) cover the container patterns behind `backend/Dockerfile` and the +compose files. diff --git a/Backend-API.md b/Backend-API.md new file mode 100644 index 0000000..445f049 --- /dev/null +++ b/Backend-API.md @@ -0,0 +1,96 @@ +# Backend API + +## Config (env) + +| Var | Default | Notes | +|-----|---------|-------| +| `API_TOKEN` | *(required)* | Bearer token shared with the userscript. | +| `ALLOWED_ORIGINS` | Asura + Demonic origins | Comma-separated CORS allowlist. | +| `DB_PATH` | `/data/bookmarks.db` | SQLite file location. | +| `PORT` | `8080` | Plain HTTP; TLS terminated by the proxy. | +| `WEB_PASSWORD` | unset | Gates the browser UI. Unset = web routes not registered at all. | +| `LATEST_CHAPTER_POLL_ENABLED` | `true` | Background latest-chapter poller on/off. | +| `LATEST_CHAPTER_POLL_COOLDOWN` | `1h` | Minimum time between checks of the same series. | +| `LATEST_CHAPTER_POLL_INTERVAL` | `10m` | Wake interval of the poller ticker. | +| `LATEST_CHAPTER_POLL_BATCH` | `14` | Series checked per wake. | +| `LATEST_CHAPTER_POLL_STAGGER` | `20s` | Delay between fetches within a batch. | + +## Endpoints + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| `GET` | `/bookmarks` | Bearer | All bookmarks (single-user). | +| `PUT` | `/bookmarks/{key}` | Bearer | Upsert one series; returns the row **as stored**. | +| `DELETE` | `/bookmarks/{key}` | Bearer | Remove one. | +| `GET` | `/healthz` | none | `200 ok`. | + +`key` is `:` — e.g. `asura:trash-of-the-counts-family-f886a8af` +or `demonic:Infinite-Level-Up-in-Murim`. + +Web UI routes (only when `WEB_PASSWORD` set) are separate — see [[Web-UI]]. + +## Auth & CORS (security invariants) + +- `/bookmarks*` requires `Authorization: Bearer `, checked with a + **constant-time compare**; `401` otherwise. +- CORS reflects `Origin` only when it's in `ALLOWED_ORIGINS`; allows + `GET,PUT,DELETE,OPTIONS` and headers `Authorization,Content-Type`; answers + preflight `OPTIONS` with `204`. +- CORS is the outermost middleware layer, so preflight short-circuits before + auth ever runs. `/healthz` is public, everything under `/bookmarks` is not. + +## Sync model + +Single-user, single `bookmarks` table, last-write-wins. Whichever device +`PUT`s last for a given key overrides the stored row. + +## The `updated_at` rule + +List order is driven by `updated_at`, and it only advances on real reading +progress: + +- Row is new, **or** +- `last_chapter_num` changed from what's stored + +Otherwise the server keeps the existing `updated_at` — so toggling a +favourite or the poller discovering a new published chapter never reorders +the list. Because of this, `PUT` returns the row as the server actually +stored it, and every client must adopt that response rather than trust its +own request payload. Full rationale: `plans/2026-07-25-bookmark-list-favorites-design.md` §4. + +## Latest-chapter poller + +A ticker goroutine in the same binary, independent of the userscript's own +`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic — two +parallel clocks feeding the same `latest_chapter` field: + +- Per-bookmark cooldown via the `latest_checked_at` column + (`Store.DueForLatestCheck`'s `WHERE` clause). +- A wake interval that picks a batch of due series each tick. + +The row is stamped **before** the fetch, so a broken series waits out a full +cooldown instead of retrying every tick. Fetches use `bogdanfinn/tls-client` +with a Chrome TLS profile as defence in depth against fingerprint-based +blocking. Any failure logs and skips — this is an enhancement, not on the +request path, and never blocks bookmark sync. Design doc: +`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`. + +## Develop / test + +```bash +cd backend +go test ./... # unit + handler tests +CGO_ENABLED=0 go build # static binary +``` + +## Smoke test + +```bash +TOKEN=$(grep '^API_TOKEN=' .env | cut -d= -f2) +curl -s localhost:8080/healthz # ok +curl -s localhost:8080/bookmarks # 401 +curl -s -H "Authorization: Bearer $TOKEN" localhost:8080/bookmarks # [] +curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \ + -H 'Access-Control-Request-Method: PUT' \ + localhost:8080/bookmarks/asura:test-1 | grep -i access-control # 204 + CORS headers +``` diff --git a/Deployment.md b/Deployment.md new file mode 100644 index 0000000..3cc5333 --- /dev/null +++ b/Deployment.md @@ -0,0 +1,96 @@ +# Deployment + +Docker + Traefik. Assumes Traefik already runs in Docker with a working HTTPS +entrypoint/cert resolver, and you control a domain. Full detail: +`DEPLOY.md` in the main repo — this page is the condensed path. + +## 0. Prerequisites + +- Docker + Docker Compose on the server. +- Traefik watching a Docker network (default assumed name: `proxy`). +- DNS `A`/`AAAA` records for `manga-api.` (API) and, if using the web + UI, `manga.` — both pointing at the server. +- Repo copied to the server (needs `backend/`, `docker-compose.yml`, + `docker-compose.prod.yml`, `.env.example`). + +```bash +docker network ls | grep proxy || docker network create proxy +``` + +## 1. Configure `.env` + +```bash +cp .env.example .env +sed -i "s|^API_TOKEN=.*|API_TOKEN=$(openssl rand -hex 32)|" .env +``` + +Required vars: `API_TOKEN`, `ALLOWED_ORIGINS`, `MANGA_API_HOST`, +`MANGA_WEB_HOST` (needed even if the web UI stays off — its Traefik label has +no fallback). Full var table: [[Backend-API]]. + +## 1b. Web UI (optional) + +```ini +MANGA_WEB_HOST=manga. +WEB_PASSWORD= +``` + +Leaving `WEB_PASSWORD` unset is safe — web routes never register, `/` 404s, +the userscript API is unaffected. See [[Web-UI]] for what the password gates. + +Rotating `API_TOKEN` or `WEB_PASSWORD` logs every browser session out +(sessions are stateless, keyed off both). + +## 2. Build + start + +```bash +docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build +``` + +Always pass both `-f` flags — the prod override alone is not standalone (it +drops the published port and adds Traefik labels). + +```bash +docker compose -f docker-compose.yml -f docker-compose.prod.yml ps +docker logs manga-api --tail 20 # expect: "listening on :8080 ..." +``` + +## 3. Verify over HTTPS + +```bash +curl -s https://manga-api./healthz # -> ok +curl -s -o /dev/null -w '%{http_code}\n' https://manga-api./bookmarks # -> 401 + +TOKEN=$(grep -E '^API_TOKEN=' .env | cut -d= -f2) +curl -s -H "Authorization: Bearer $TOKEN" https://manga-api./bookmarks # -> [] + +curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \ + -H 'Access-Control-Request-Method: PUT' \ + https://manga-api./bookmarks/x | grep -i access-control +``` + +All must pass — valid TLS is non-negotiable (mixed content blocks the +userscript's `fetch()` otherwise). + +## 4–5. Configure and install the userscript + +See [[Userscript]] — set `API_BASE`/`API_TOKEN` in the file, install on +Bromite. + +## 6. Smoke-test the full loop + +Bookmark on Asura → confirm via `curl /bookmarks` on the server → open a +chapter, confirm progress updates → open Demonic, confirm the same bookmark +shows there. + +## Updating + +```bash +docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build +``` + +SQLite data persists in the named volume `bookmarks-data` across rebuilds. + +## Troubleshooting + +See [[Troubleshooting]]. diff --git a/Development.md b/Development.md new file mode 100644 index 0000000..ce98693 --- /dev/null +++ b/Development.md @@ -0,0 +1,47 @@ +# Development + +## Backend + +```bash +cd backend +go test ./... # unit + handler tests +go test -run TestName ./... # single test +CGO_ENABLED=0 go build # static binary (needed for the distroless image) +``` + +## Local stack + +```bash +docker compose up # named volume mounted at /data, restart: unless-stopped +``` + +Uses `docker-compose.yml` only (no Traefik override, no host domain needed) — +binds `127.0.0.1:8080`. + +## Userscript iteration + +The file is `GM_*`-free, so install the same unmodified file in desktop +Tampermonkey/Violentmonkey for faster iteration than round-tripping through a +phone. Point `API_BASE` at a local/dev backend while testing. + +## Relevant skills + +`multi-stage-dockerfile` and `docker-compose-orchestration` cover the +container patterns used in `backend/Dockerfile` and the compose files. + +## Design docs + +Each non-trivial feature has a plan/spec written before implementation: + +- `plans/mangaBookmark.md` — original spec, still the source of truth for + scope and constraints. +- `plans/2026-07-25-bookmark-implementation-plan.md` +- `plans/2026-07-25-bookmark-list-favorites-design.md` — the `updated_at` + ordering rule, in full. +- `plans/2026-07-25-web-ui-implementation-plan.md` +- `plans/2026-07-26-server-latest-chapter-polling.md` +- `docs/superpowers/specs/` — design docs for web UI and the latest-chapter + poller. + +Read the relevant plan before changing behaviour it describes — it usually +records *why*, not just *what*. diff --git a/Home.md b/Home.md index 5d08b7b..93caabd 100644 --- a/Home.md +++ b/Home.md @@ -1 +1,33 @@ -Welcome to the Wiki. \ No newline at end of file +# Manga Bookmark Wiki + +Manga read-progress tracker for **asurascans.com** and **demonicscans.org**, +read from **Bromite** (mobile Chromium). A userscript injects on-page UI and +syncs progress to a self-hosted Go backend, so bookmarks unify across both +sites and all devices. + +``` +Bromite userscript (isolated world, Shadow DOM UI, localStorage cache) + -- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume) +``` + +## Pages + +- [[Architecture]] — components, data flow, why each design choice exists +- [[Backend-API]] — config vars, endpoints, the `updated_at` ordering rule +- [[Web-UI]] — password-gated browser UI, sessions, routes +- [[Userscript]] — site adapters, config, install, day-to-day use +- [[Deployment]] — Docker Compose + Traefik, step by step +- [[Development]] — running tests, building, local dev loop +- [[Troubleshooting]] — common failures and fixes + +## Repo layout + +| Path | What | +|------|------| +| `backend/` | Go service: `net/http` + pure-Go SQLite (`modernc.org/sqlite`), static binary | +| `userscript/manga-bookmark.user.js` | single-file Bromite/Tampermonkey userscript | +| `docker-compose.yml` / `docker-compose.prod.yml` | base stack / Traefik override | +| `plans/` | design docs for each feature (source of truth for *why*) | + +Full source lives in the main repo; this wiki is the map, not a copy — when in +doubt, the code and `plans/*.md` win. diff --git a/Troubleshooting.md b/Troubleshooting.md new file mode 100644 index 0000000..dcb73e9 --- /dev/null +++ b/Troubleshooting.md @@ -0,0 +1,15 @@ +# Troubleshooting + +| Symptom | Likely cause / fix | +|---------|--------------------| +| No cert / TLS error at the domain | `TRAEFIK_ENTRYPOINT` or `TRAEFIK_CERTRESOLVER` name wrong, or DNS not resolving yet. Check `docker logs `. | +| 404 from Traefik | Service not on the `proxy` network, or `MANGA_API_HOST` mismatch. Confirm `docker network inspect proxy` lists `manga-api`. | +| `fetch` fails in the userscript, `curl` works | Origin missing from `ALLOWED_ORIGINS`, or mixed content (backend not HTTPS). | +| 401 with the right token | Trailing space/newline in `API_TOKEN`; regenerate and restart. | +| Panel button absent | URL didn't match an adapter, or user scripts disabled in Bromite. | +| `compose ... config` errors about `API_TOKEN` | Run compose from the dir with `.env`, or export the vars. | +| Web UI logs everyone out unexpectedly | `API_TOKEN` or `WEB_PASSWORD` was rotated — sessions are stateless and keyed off both, so this is expected, not a bug. | +| curl to asurascans.com/demonicscans.org gets challenged | Cloudflare's block is IP-reputation-based, not universal or permanent — re-check live rather than assume; see [[Architecture]] / repo `CLAUDE.md`. | +| A manga site changed its URL shape, adapter stops matching | Update the regex in that site's adapter in `manga-bookmark.user.js`, re-verify against a live page (Playwright or a direct probe) before trusting it. | + +Backend config reference and endpoint list: [[Backend-API]]. diff --git a/Userscript.md b/Userscript.md new file mode 100644 index 0000000..1c6d51a --- /dev/null +++ b/Userscript.md @@ -0,0 +1,105 @@ +# Userscript + +Single file: `userscript/manga-bookmark.user.js`. One IIFE, no `GM_*` APIs, so +it runs unchanged in Bromite's native engine **and** in desktop +Tampermonkey/Violentmonkey for fast iteration. + +## Structure + +1. **Site adapters** — one per host. `detect(location, document)` returns + page `type` + IDs, keyed off **URL regex** (most stable signal). + `title`/`cover` come from `og:title`/`og:image` meta tags, not CSS classes. +2. **API client** — `apiGet/apiPut/apiDelete` with bearer header. + `localStorage` key `mangabm:cache` gives instant render + offline fallback. +3. **Progress logic** — auto-upserts `last_chapter` only when + `chapterNum >= stored last_chapter_num` (re-reading old chapters never + regresses progress; an unparseable number just sets the current chapter). + Manual panel override forces any value regardless. +4. **UI** — rendered inside a **Shadow DOM** root, isolating it from site CSS + (critical on mobile where site styles are unpredictable). +5. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: + the script patches `history.pushState`/`replaceState` and listens + `popstate`, re-running `detect()` on URL change so auto-update fires + without a full reload. Demonic does classic reloads, so an initial + `document-idle` run is enough there. + +## Config + +Edit the block at the top of the file: + +```js +const API_BASE = "https://manga-api."; // no trailing slash +const API_TOKEN = ""; +``` + +The token lives in the userscript's **isolated world** — the manga sites' +own JS cannot read it. + +## Site adapter reference (verified live 2026-07-24) + +| Site | Series URL | Chapter URL | `series_id` | +|------|-----------|-------------|-------------| +| **Asura** (`asurascans.com`) | `/comics/` | `/comics//chapter/` | `` | +| **Demonic** (`demonicscans.org`) | `/manga/` | `/title//chapter//` (`chaptered.php?manga=&chapter=` 301s here) | `` | + +Notes: + +- `asuracomic.net` deep links 301 to the **root** of `asurascans.com`, + discarding the path — dead end, use `asurascans.com` directly. The host + stays in `@match` in case the redirect starts preserving paths again. +- Asura `og:title` has a `Chapter N - Read Online | Asura Scans` suffix the + adapter strips; Demonic chapter `og:title` is ` Chapter N`. +- Demonic's `<slug>` is identical on `/manga/…` and the canonical `/title/…` + reader, so a bookmark set from the series page and an auto-update from the + reader resolve to the same key. +- If either site changes its URL shape, update the regex in that site's + adapter and re-verify against a live page before trusting it. + +## Install on Bromite (mobile) + +1. Bromite → **Settings → User scripts** → enable (accept the permission + prompt). +2. Save the configured `manga-bookmark.user.js` to the device (or open its + raw URL) — Bromite detects `.user.js` and offers to install. +3. Confirm install; the `@match` list covers both sites. +4. Open a series on either site — a 📑 button appears bottom-right. + +> Menu wording varies by Bromite build. If "User scripts" is absent, update +> Bromite or use a build that has it. + +## Day-to-day use + +- **Bookmark**: panel → **+ Bookmark this**. +- **Auto-progress**: opening a chapter of a bookmarked series records it when + the chapter number is ≥ the stored one. +- **Manual override**: panel → **Edit** on any row forces a specific chapter. +- **Continue**: jumps to the last-read chapter (or the series page). +- **Latest chapter**: rows show `Read: … · Latest: …` once the newest + published chapter is known and ahead of your progress. +- **Favourites**: ☆ toggles per row; **★ Favourites** tab narrows the list. + Favourited series still show under **All**. The flag syncs across devices; + the chosen tab does not. +- Bookmarks made on one site appear when the panel opens on the other — the + backend is the shared store. +- Neither favouriting nor a newly-learned chapter reorders the list — only + reading progress does (see [[Backend-API]]). +- Offline / backend down: changes cache in `localStorage`, retried on the + next successful load (last-write-wins). + +## How "latest chapter" is found + +Only a series page lists every chapter; the backend can't fetch either site +itself (Cloudflare blocks server-side requests, no API/feed exists). So the +userscript does the looking, from your own browser session: + +- Opening a bookmarked series page records its newest chapter directly. +- Otherwise it background-fetches series pages — **same-origin only**, so + browsing Asura refreshes Asura bookmarks and vice versa. At most one series + check per navigation, at most one check per series every 4 hours + (`LATEST_CHECK_BATCH` / `LATEST_CHECK_THROTTLE_MS`). Failures are silent, + retried after the window. + +Freshness (`mangabm:lastchecked` in `localStorage`) is per-device and +deliberately not synced — each device checks on its own. The backend also +runs its own independent poller (see [[Backend-API]]) so `latest_chapter` +stays fresh even when you aren't browsing. diff --git a/Web-UI.md b/Web-UI.md new file mode 100644 index 0000000..7c1a676 --- /dev/null +++ b/Web-UI.md @@ -0,0 +1,61 @@ +# Web UI + +The same backend binary serves a password-gated browser UI on a **second +hostname**, so you can view/edit bookmarks from any browser, not just via the +userscript. + +## Enable / disable + +Controlled entirely by `WEB_PASSWORD`: + +- **Set** → web routes registered, UI reachable, gated by the password. +- **Unset** → web routes are **not registered at all**; `/` returns `404`. + The userscript's `/bookmarks*` API is unaffected either way. + +## Routes + +| Method | Path | Auth | Purpose | +|--------|------|------|---------| +| `GET` | `/` | session or none | List (or login page if no session) | +| `POST` | `/login` | password | Sets session cookie | +| `POST` | `/logout` | session | Clears cookie | +| `GET` | `/static/*` | none | CSS/JS assets (`go:embed`) | +| `GET` | `/ui/list` | session | htmx fragment: bookmark list | +| `POST` | `/ui/bookmarks/{key}/favorite` | session | Toggle favourite | +| `POST` | `/ui/bookmarks/{key}/chapter` | session | Manual chapter override | +| `DELETE` | `/ui/bookmarks/{key}` | session | Remove a bookmark | + +Templates (`templates/*.html`) and assets (`static/*`) are `go:embed`-ed into +the binary — `backend/Dockerfile` copies `templates/` and `static/` alongside +the `*.go` files. + +## Sessions + +Stateless — no session table. The cookie is: + +``` +<expiryMs>.<base64url HMAC-SHA256(expiryMs)> +``` + +signed with a key derived from **both** `API_TOKEN` and `WEB_PASSWORD` +(SHA-256 of `apiToken + "\x00" + webPassword + "mangabm-web-session-v1"`). +Rotating either secret invalidates every outstanding session at once — +there's nothing to revoke individually, so this is the way to force a logout +everywhere. + +- TTL: **60 days** — long enough a phone stays logged in between reading + sessions. +- Verification order: shape → expiry → HMAC (constant-time compare last, so + earlier cheap checks leak no timing information about the signature). +- Login is rate-limited (see `newLoginLimiter()` in `session.go`). + +## Mutations stay consistent with the API + +Every UI mutation (favourite, chapter override, delete) goes through the same +`Store.Get` + `Store.Upsert` path the `/bookmarks` API uses — so the +`updated_at` ordering rule (see [[Backend-API]]) applies identically whether +the change came from the userscript or the web UI. + +## Design doc + +Full design rationale: `docs/superpowers/specs/2026-07-25-web-ui-design.md`.