180ee78b1f
Tracks read progress on comix.to and kagane.to alongside asura and demonic, in both the userscript and the backend. Implements `docs/superpowers/plans/2026-08-03-comix-kagane-support.md`. ## Userscript - `comix` adapter — `/title/<id>-<slug>`; only the id prefix is identity (the slug follows the title). No `og:image`, so the cover is matched by `alt`. - `kagane` adapter — reader URLs are uuids with no chapter number, so it comes out of `og:title`; anchor scanning is structurally impossible, replaced by `latestChapterFromApi` against kagane's same-origin JSON API. - `seriesId` threaded through `latestChapterFromAnchors` so comix can scope its scan to its own series and a recommendation strip cannot win the maximum. - `@match` for both hosts, panel chips, v1.6.0. ## Backend - `latestChapterFrom` cases: comix parses the SSR JSON state blob (`latestChapterUrl`, scoped to the series id); kagane parses API JSON (`chapter_no`). - Poller allowlist extended; `Poller.BrowserFetch` with `fetcherFor(site)` routes kagane to a browser fetcher. Nil means kagane is not polled at all — never a fallback to the TLS fetcher, which would only ever retrieve a challenge page. - `BrowserFetcher`: chromedp against a `headless-shell` sidecar. kagane sits behind a Cloudflare JS challenge that no TLS fingerprint clears, and the request is made inside the page rather than by replaying `cf_clearance`. - `BROWSER_WS_URL` wiring, sidecar in both compose files (no `ports:`, dedicated non-external network), Dockerfile on `golang:1.26-alpine` — chromedp requires go 1.26. - Web UI `--comix` / `--kagane` tokens in both colour branches. ## Notes for review - `series_url` is client-supplied and a headless browser is a strong SSRF primitive, so kagane's host is pinned twice: in `fetchableSeriesURL` and again in `kaganeAPIURL`. - Three chained defects found during verification made the browser path dead under Compose (sidecar flag collision, Chrome's Host-header DNS-rebinding check, the wrong chromedp option). Fixed; the compose comments record the wrong configurations too, so they don't get "simplified" back. - `ALLOWED_ORIGINS` now includes both new origins. Without it every write from comix/kagane silently fails CORS preflight, parks in the retry queue, and drops at the cap. ## Verification 221 backend tests, 32 userscript tests, static `CGO_ENABLED=0` build, both compose configs. Two gaps, both real: 1. The userscript on live pages via Violentmonkey needs a human browser profile — not run. Check: comix series page (title/cover, no chapter), comix chapter page (records the number; an *older* chapter must not regress it), comix SPA navigation without reload, kagane series page (og:image cover), kagane reader (number from `og:title`), both chips opening the right sites. 2. The kagane browser path has not completed end-to-end anywhere. Dial/navigate/fetch is confirmed, but Cloudflare 403'd headless-shell's Chrome on every attempt from the dev sandbox, and comix's poll-through-Docker was blocked by that environment's TLS interception. Both environment-dependent rather than branch defects — the first real deploy is the actual verification. Reviewed-on: #13 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
273 lines
9.3 KiB
Markdown
273 lines
9.3 KiB
Markdown
# Deployment
|
|
|
|
Step-by-step for the backend (Docker + Traefik) and the Bromite userscript.
|
|
Assumes you already run Traefik in Docker with a working HTTPS entrypoint and an
|
|
ACME/cert resolver, and control a domain.
|
|
|
|
---
|
|
|
|
## 0. Prerequisites
|
|
|
|
- Docker + Docker Compose on the server.
|
|
- A Traefik instance watching a Docker network (default name assumed: `proxy`).
|
|
- DNS: an `A`/`AAAA` record for `manga-api.<yourdomain>` pointing at the server.
|
|
- The repo copied to the server, e.g. `/opt/mangabm/` (needs `backend/`,
|
|
`docker-compose.yml`, `docker-compose.prod.yml`, `.env.example`).
|
|
|
|
Confirm the Traefik network exists (create if not):
|
|
|
|
```bash
|
|
docker network ls | grep proxy || docker network create proxy
|
|
```
|
|
|
|
---
|
|
|
|
## 1. Configure `.env`
|
|
|
|
```bash
|
|
cd /opt/mangabm
|
|
cp .env.example .env
|
|
```
|
|
|
|
Edit `.env`:
|
|
|
|
```ini
|
|
# Required — long random secret, also goes in the userscript.
|
|
API_TOKEN=<paste output of: openssl rand -hex 32>
|
|
|
|
# CORS allowlist — leave as-is unless a site changes hostname.
|
|
ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to
|
|
|
|
# Required for the Traefik override. Both have no fallback — compose refuses
|
|
# to start without them. MANGA_WEB_HOST is required even if you never set
|
|
# WEB_PASSWORD; see 1b.
|
|
MANGA_API_HOST=manga-api.violetcrown.my.id
|
|
MANGA_WEB_HOST=manga.violetcrown.my.id
|
|
|
|
# Only if your Traefik setup differs from these defaults:
|
|
# PROXY_NETWORK=proxy
|
|
# TRAEFIK_ENTRYPOINT=websecure
|
|
# TRAEFIK_CERTRESOLVER=le
|
|
```
|
|
|
|
Generate + insert the token in one line:
|
|
|
|
```bash
|
|
sed -i "s|^API_TOKEN=.*|API_TOKEN=$(openssl rand -hex 32)|" .env
|
|
grep -E '^API_TOKEN=' .env # copy this — the userscript needs the same value
|
|
```
|
|
|
|
> Match `TRAEFIK_ENTRYPOINT` / `TRAEFIK_CERTRESOLVER` to your Traefik's actual
|
|
> names (check your Traefik static config — common alternatives: `https`,
|
|
> `myresolver`, `cloudflare`). Wrong names = no certificate issued.
|
|
|
|
---
|
|
|
|
## 1b. Web UI
|
|
|
|
The browser UI is served by the same container on a second hostname.
|
|
|
|
1. Add a DNS `A`/`AAAA` record for `manga.<yourdomain>` pointing at the server —
|
|
the same address as `manga-api.<yourdomain>`.
|
|
|
|
2. Set both variables in `.env`:
|
|
|
|
```ini
|
|
MANGA_WEB_HOST=manga.violetcrown.my.id
|
|
WEB_PASSWORD=<paste output of: openssl rand -base64 18>
|
|
```
|
|
|
|
Generate and insert in one line:
|
|
|
|
```bash
|
|
sed -i "s|^WEB_PASSWORD=.*|WEB_PASSWORD=$(openssl rand -base64 18)|" .env
|
|
grep -E '^WEB_PASSWORD=' .env # this is what you type into the site
|
|
```
|
|
|
|
3. Redeploy and check:
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
|
|
curl -s -o /dev/null -w '%{http_code}\n' https://manga.violetcrown.my.id/
|
|
```
|
|
|
|
Expected `200`, serving the login page.
|
|
|
|
Leaving `WEB_PASSWORD` unset is safe: the web routes are not registered and `/`
|
|
returns 404. The userscript's API on `MANGA_API_HOST` is unaffected either way.
|
|
|
|
`MANGA_WEB_HOST` itself is required by the prod override regardless — like
|
|
`MANGA_API_HOST`, its Traefik label has no fallback, so `docker compose up`
|
|
refuses to start without it even if `WEB_PASSWORD` is unset and the web UI is
|
|
otherwise dormant.
|
|
|
|
Sessions are signed with a key derived from `API_TOKEN` and `WEB_PASSWORD`, so
|
|
rotating either one logs every browser out. The session cookie lasts 60 days.
|
|
|
|
---
|
|
|
|
## 2. Build + start
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
|
|
```
|
|
|
|
This merges the base file (build/image/env/volume) with the prod override
|
|
(no host port, Traefik network + router labels). Always pass **both** `-f`
|
|
flags — the prod file is not standalone.
|
|
|
|
Two services come up: `manga-api` (the backend) and `headless-shell`, a CDP
|
|
sidecar the poller uses to fetch kagane (behind a Cloudflare JS challenge).
|
|
It has no published port — only `manga-api` can reach it, over
|
|
`BROWSER_WS_URL`. Missing or unreachable, the poller just skips kagane and
|
|
logs it; nothing else is affected.
|
|
|
|
Check it's up and healthy:
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml -f docker-compose.prod.yml ps
|
|
docker logs manga-api --tail 20 # expect: "listening on :8080 ..."
|
|
```
|
|
|
|
---
|
|
|
|
## 3. Verify over HTTPS
|
|
|
|
Give Traefik a few seconds to issue the cert, then:
|
|
|
|
```bash
|
|
# Health (no auth) — must be valid TLS, no cert warning.
|
|
curl -s https://manga-api.violetcrown.my.id/healthz # -> ok
|
|
|
|
# Auth enforced.
|
|
curl -s -o /dev/null -w '%{http_code}\n' \
|
|
https://manga-api.violetcrown.my.id/bookmarks # -> 401
|
|
|
|
TOKEN=$(grep -E '^API_TOKEN=' .env | cut -d= -f2)
|
|
curl -s -H "Authorization: Bearer $TOKEN" \
|
|
https://manga-api.violetcrown.my.id/bookmarks # -> []
|
|
|
|
# CORS preflight from a real site origin.
|
|
curl -s -i -X OPTIONS \
|
|
-H 'Origin: https://asurascans.com' \
|
|
-H 'Access-Control-Request-Method: PUT' \
|
|
https://manga-api.violetcrown.my.id/bookmarks/x | grep -i access-control
|
|
# -> Access-Control-Allow-Origin: https://asurascans.com (+ Methods/Headers)
|
|
```
|
|
|
|
All four must pass. Valid TLS is non-negotiable — the manga sites are HTTPS, so
|
|
a bad cert makes the browser block the userscript's `fetch()` (mixed content).
|
|
|
|
---
|
|
|
|
## 4. Configure the userscript
|
|
|
|
Edit the config block at the top of `userscript/manga-bookmark.user.js`:
|
|
|
|
```js
|
|
const API_BASE = "https://manga-api.yourdomain.com"; // no trailing slash
|
|
const API_TOKEN = "<same token as .env>";
|
|
```
|
|
|
|
The token sits in the userscript's isolated world — the manga sites' JS can't
|
|
read it.
|
|
|
|
Also edit the `@downloadURL`/`@updateURL` metadata lines near the top of the
|
|
file — they ship hardcoded to this deployment's domain and token, so a
|
|
deployer who skips them ends up auto-updating from someone else's backend.
|
|
See "Installing / updating the userscript" below for how those two lines are
|
|
used.
|
|
|
|
---
|
|
|
|
## 5. Install on Bromite
|
|
|
|
1. Bromite → **Settings → User scripts** → enable (accept the permission prompt).
|
|
2. Put the edited `manga-bookmark.user.js` on the device (save the file, or open
|
|
its raw URL). Bromite detects `.user.js` and offers to install.
|
|
3. Confirm install — the `@match` list covers both sites.
|
|
4. Open a series on asurascans.com or demonicscans.org → a 📑 button appears
|
|
bottom-right → tap → **+ Bookmark this**.
|
|
|
|
Optional desktop test: the script is `GM_*`-free, so the same file installs in
|
|
Tampermonkey/Violentmonkey for quick checks before going mobile.
|
|
|
|
---
|
|
|
|
## 6. Smoke-test the full loop
|
|
|
|
1. Bookmark a series on Asura.
|
|
2. `curl -s -H "Authorization: Bearer $TOKEN" https://manga-api.yourdomain.com/bookmarks`
|
|
on the server — the series should appear.
|
|
3. Open a chapter of that series — reopen the panel; last-read updates to that
|
|
chapter (auto, never regresses on older chapters).
|
|
4. Open Demonic, open the panel — the Asura bookmark shows there too (shared
|
|
store, cross-site unified list).
|
|
|
|
---
|
|
|
|
## Updating
|
|
|
|
Pull new code, then rebuild:
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
|
|
```
|
|
|
|
SQLite data persists in the named volume `bookmarks-data` across rebuilds.
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
| Symptom | Likely cause / fix |
|
|
|---------|--------------------|
|
|
| No cert / TLS error at the domain | `TRAEFIK_ENTRYPOINT` or `TRAEFIK_CERTRESOLVER` name wrong; or DNS not resolving yet. Check `docker logs <traefik>`. |
|
|
| 404 from Traefik | Service not on the `proxy` network, or `MANGA_API_HOST` mismatch. Confirm `docker network inspect proxy` lists `manga-api`. |
|
|
| `fetch` fails in the userscript, `curl` works | Origin missing from `ALLOWED_ORIGINS`, or mixed content (backend not HTTPS). |
|
|
| 401 with the right token | Trailing space/newline in `API_TOKEN`; regenerate and restart. |
|
|
| Panel button absent | URL didn't match an adapter, or user scripts disabled in Bromite. |
|
|
| `compose ... config` errors about `API_TOKEN` | Run compose from the dir with `.env`, or export the vars. |
|
|
|
|
Backend config reference and endpoint list: see `README.md`.
|
|
|
|
---
|
|
|
|
## Installing / updating the userscript
|
|
|
|
The backend serves the script itself, so Violentmonkey can auto-update it.
|
|
Complements §4 above — that step points `API_BASE`/`API_TOKEN` at your
|
|
backend; this one points `@downloadURL`/`@updateURL` at the same place so
|
|
auto-updates come from it too.
|
|
|
|
Install once, on the phone (Cromite + Violentmonkey):
|
|
|
|
```
|
|
https://manga-api.<your-domain>/u/<API_TOKEN>/manga-bookmark.user.js
|
|
```
|
|
|
|
Open that URL in Cromite; Violentmonkey offers to install it. The token is in
|
|
the path because Violentmonkey's update poll sends no `Authorization` header,
|
|
and the script embeds `API_TOKEN` in plain text — an open URL would leak it. A
|
|
wrong token answers 404.
|
|
|
|
Updating, without a redeploy:
|
|
|
|
```bash
|
|
vi userscript/manga-bookmark.user.js # on the VPS, in this checkout
|
|
```
|
|
|
|
`./userscript` is bindmounted read-only into the container and read fresh on
|
|
every request, so the edit is live immediately. The served `@version` is derived
|
|
from the file's mtime (`YYYY.MM.DD.HHMM`, UTC), not from the `@version` in the
|
|
file, so any edit outranks the installed copy and Violentmonkey pulls it on its
|
|
next check. The `@version` in the repo is a human marker only.
|
|
|
|
Updating via redeploy: `git pull` overwrites the file with the committed
|
|
version, which is the intended behaviour — a deploy always ships the repo's
|
|
script. Note that `git pull` sets mtime to checkout time, so even a rollback
|
|
serves a *higher* version and is adopted.
|
|
|
|
If the mount is missing, the endpoint answers 404 and logs it; bookmark sync is
|
|
unaffected.
|