180ee78b1f
Tracks read progress on comix.to and kagane.to alongside asura and demonic, in both the userscript and the backend. Implements `docs/superpowers/plans/2026-08-03-comix-kagane-support.md`. ## Userscript - `comix` adapter — `/title/<id>-<slug>`; only the id prefix is identity (the slug follows the title). No `og:image`, so the cover is matched by `alt`. - `kagane` adapter — reader URLs are uuids with no chapter number, so it comes out of `og:title`; anchor scanning is structurally impossible, replaced by `latestChapterFromApi` against kagane's same-origin JSON API. - `seriesId` threaded through `latestChapterFromAnchors` so comix can scope its scan to its own series and a recommendation strip cannot win the maximum. - `@match` for both hosts, panel chips, v1.6.0. ## Backend - `latestChapterFrom` cases: comix parses the SSR JSON state blob (`latestChapterUrl`, scoped to the series id); kagane parses API JSON (`chapter_no`). - Poller allowlist extended; `Poller.BrowserFetch` with `fetcherFor(site)` routes kagane to a browser fetcher. Nil means kagane is not polled at all — never a fallback to the TLS fetcher, which would only ever retrieve a challenge page. - `BrowserFetcher`: chromedp against a `headless-shell` sidecar. kagane sits behind a Cloudflare JS challenge that no TLS fingerprint clears, and the request is made inside the page rather than by replaying `cf_clearance`. - `BROWSER_WS_URL` wiring, sidecar in both compose files (no `ports:`, dedicated non-external network), Dockerfile on `golang:1.26-alpine` — chromedp requires go 1.26. - Web UI `--comix` / `--kagane` tokens in both colour branches. ## Notes for review - `series_url` is client-supplied and a headless browser is a strong SSRF primitive, so kagane's host is pinned twice: in `fetchableSeriesURL` and again in `kaganeAPIURL`. - Three chained defects found during verification made the browser path dead under Compose (sidecar flag collision, Chrome's Host-header DNS-rebinding check, the wrong chromedp option). Fixed; the compose comments record the wrong configurations too, so they don't get "simplified" back. - `ALLOWED_ORIGINS` now includes both new origins. Without it every write from comix/kagane silently fails CORS preflight, parks in the retry queue, and drops at the cap. ## Verification 221 backend tests, 32 userscript tests, static `CGO_ENABLED=0` build, both compose configs. Two gaps, both real: 1. The userscript on live pages via Violentmonkey needs a human browser profile — not run. Check: comix series page (title/cover, no chapter), comix chapter page (records the number; an *older* chapter must not regress it), comix SPA navigation without reload, kagane series page (og:image cover), kagane reader (number from `og:title`), both chips opening the right sites. 2. The kagane browser path has not completed end-to-end anywhere. Dial/navigate/fetch is confirmed, but Cloudflare 403'd headless-shell's Chrome on every attempt from the dev sandbox, and comix's poll-through-Docker was blocked by that environment's TLS interception. Both environment-dependent rather than branch defects — the first real deploy is the actual verification. Reviewed-on: #13 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
95 lines
4.4 KiB
YAML
95 lines
4.4 KiB
YAML
# Base stack — works standalone for local smoke testing (`docker compose up`).
|
|
# The service binds 127.0.0.1:8080; a host reverse proxy (nginx/Caddy/Traefik)
|
|
# terminates TLS for manga-api.<domain> and forwards to it.
|
|
#
|
|
# If your proxy runs in Docker on its own network, use the prod override which
|
|
# attaches to that network instead of publishing a port:
|
|
# docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
|
|
services:
|
|
manga-api:
|
|
build: ./backend
|
|
image: mangabm-backend:latest
|
|
container_name: manga-api
|
|
restart: unless-stopped
|
|
environment:
|
|
# API_TOKEN is required — compose refuses to start without it.
|
|
API_TOKEN: ${API_TOKEN:?set API_TOKEN in .env}
|
|
ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to}
|
|
DB_PATH: /data/bookmarks.db
|
|
PORT: "8080"
|
|
# Gates the browser UI. Unset means the web routes are not served at all.
|
|
WEB_PASSWORD: ${WEB_PASSWORD:-}
|
|
# Path inside the container; matches the bindmount above.
|
|
USERSCRIPT_PATH: ${USERSCRIPT_PATH:-/userscript/manga-bookmark.user.js}
|
|
# Latest-chapter poller. LATEST_CHAPTER_POLL_ENABLED=0 in .env is the kill
|
|
# switch; it only takes effect because these are listed here.
|
|
LATEST_CHAPTER_POLL_ENABLED: ${LATEST_CHAPTER_POLL_ENABLED:-1}
|
|
LATEST_CHAPTER_POLL_COOLDOWN: ${LATEST_CHAPTER_POLL_COOLDOWN:-1h}
|
|
LATEST_CHAPTER_POLL_INTERVAL: ${LATEST_CHAPTER_POLL_INTERVAL:-10m}
|
|
LATEST_CHAPTER_POLL_BATCH: ${LATEST_CHAPTER_POLL_BATCH:-14}
|
|
LATEST_CHAPTER_POLL_STAGGER: ${LATEST_CHAPTER_POLL_STAGGER:-20s}
|
|
# CDP endpoint for sites behind a JavaScript challenge (kagane). Unset
|
|
# disables browser polling for those sites; the userscript still covers them.
|
|
# Must be an IP, not the "headless-shell" DNS name: Chrome's DevTools HTTP
|
|
# handler rejects the discovery request (GET /json/version) with a 500
|
|
# unless the Host header is an IP address or "localhost" — confirmed
|
|
# 2026-08-03 against chromedp/headless-shell:stable, independent of
|
|
# chromedp's own dial logic. The sidecar's static address below exists so
|
|
# this URL survives container recreation.
|
|
BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://172.28.0.10:9222}
|
|
depends_on:
|
|
- headless-shell
|
|
volumes:
|
|
- bookmarks-data:/data
|
|
# The userscript is served from here, read fresh on every request. Editing
|
|
# the file in this checkout takes effect on the next Violentmonkey poll —
|
|
# no rebuild, no restart. `git pull` restores the committed version, which
|
|
# is why a redeploy always ships the repo's script.
|
|
- ./userscript:/userscript:ro
|
|
# Bound to loopback only: the proxy (or curl during smoke test) reaches it,
|
|
# the public internet does not.
|
|
ports:
|
|
- "127.0.0.1:8080:8080"
|
|
networks:
|
|
- browser
|
|
|
|
headless-shell:
|
|
image: chromedp/headless-shell:stable
|
|
restart: unless-stopped
|
|
# Chrome allocates shared memory per tab and dies on Docker's 64MB default.
|
|
shm_size: '1gb'
|
|
# Reaps zombie renderer processes, which otherwise accumulate for the
|
|
# container's lifetime.
|
|
init: true
|
|
# Deliberately no `ports:` — an exposed CDP endpoint is remote code
|
|
# execution. Only manga-api, via the `browser` network below, may reach it.
|
|
# Don't pass --remote-debugging-address/--remote-debugging-port here: the
|
|
# image's own entrypoint (/headless-shell/run.sh) already starts Chrome on
|
|
# 127.0.0.1:9223 and fronts it with a socat proxy listening on 0.0.0.0:9222.
|
|
# Redeclaring the port flag here overrides Chrome's, so it binds 9222
|
|
# directly (IPv6 loopback only) instead of 9223 — collides with socat's own
|
|
# bind on 9222 and leaves nothing listening on 9223, so every external
|
|
# connection to headless-shell:9222 fails with EOF. Only pass flags the
|
|
# entrypoint doesn't already set.
|
|
command:
|
|
- --disable-gpu
|
|
- --no-sandbox
|
|
networks:
|
|
browser:
|
|
# Pinned so BROWSER_WS_URL can name an IP (required, see above) that
|
|
# survives `docker compose up` recreating this container.
|
|
ipv4_address: 172.28.0.10
|
|
|
|
volumes:
|
|
bookmarks-data:
|
|
|
|
networks:
|
|
# Not `internal: true`: headless Chrome still needs outbound access to reach
|
|
# kagane.to. Isolation here comes from membership (only manga-api and
|
|
# headless-shell join it), not from cutting egress.
|
|
browser:
|
|
ipam:
|
|
config:
|
|
- subnet: 172.28.0.0/24
|