sulthan
180ee78b1f
Tracks read progress on comix.to and kagane.to alongside asura and demonic, in both the userscript and the backend. Implements `docs/superpowers/plans/2026-08-03-comix-kagane-support.md`. ## Userscript - `comix` adapter — `/title/<id>-<slug>`; only the id prefix is identity (the slug follows the title). No `og:image`, so the cover is matched by `alt`. - `kagane` adapter — reader URLs are uuids with no chapter number, so it comes out of `og:title`; anchor scanning is structurally impossible, replaced by `latestChapterFromApi` against kagane's same-origin JSON API. - `seriesId` threaded through `latestChapterFromAnchors` so comix can scope its scan to its own series and a recommendation strip cannot win the maximum. - `@match` for both hosts, panel chips, v1.6.0. ## Backend - `latestChapterFrom` cases: comix parses the SSR JSON state blob (`latestChapterUrl`, scoped to the series id); kagane parses API JSON (`chapter_no`). - Poller allowlist extended; `Poller.BrowserFetch` with `fetcherFor(site)` routes kagane to a browser fetcher. Nil means kagane is not polled at all — never a fallback to the TLS fetcher, which would only ever retrieve a challenge page. - `BrowserFetcher`: chromedp against a `headless-shell` sidecar. kagane sits behind a Cloudflare JS challenge that no TLS fingerprint clears, and the request is made inside the page rather than by replaying `cf_clearance`. - `BROWSER_WS_URL` wiring, sidecar in both compose files (no `ports:`, dedicated non-external network), Dockerfile on `golang:1.26-alpine` — chromedp requires go 1.26. - Web UI `--comix` / `--kagane` tokens in both colour branches. ## Notes for review - `series_url` is client-supplied and a headless browser is a strong SSRF primitive, so kagane's host is pinned twice: in `fetchableSeriesURL` and again in `kaganeAPIURL`. - Three chained defects found during verification made the browser path dead under Compose (sidecar flag collision, Chrome's Host-header DNS-rebinding check, the wrong chromedp option). Fixed; the compose comments record the wrong configurations too, so they don't get "simplified" back. - `ALLOWED_ORIGINS` now includes both new origins. Without it every write from comix/kagane silently fails CORS preflight, parks in the retry queue, and drops at the cap. ## Verification 221 backend tests, 32 userscript tests, static `CGO_ENABLED=0` build, both compose configs. Two gaps, both real: 1. The userscript on live pages via Violentmonkey needs a human browser profile — not run. Check: comix series page (title/cover, no chapter), comix chapter page (records the number; an *older* chapter must not regress it), comix SPA navigation without reload, kagane series page (og:image cover), kagane reader (number from `og:title`), both chips opening the right sites. 2. The kagane browser path has not completed end-to-end anywhere. Dial/navigate/fetch is confirmed, but Cloudflare 403'd headless-shell's Chrome on every attempt from the dev sandbox, and comix's poll-through-Docker was blocked by that environment's TLS interception. Both environment-dependent rather than branch defects — the first real deploy is the actual verification. Reviewed-on: #13 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
63 lines
3.1 KiB
Bash
63 lines
3.1 KiB
Bash
# Copy to .env and fill in. Never commit the real .env.
|
|
|
|
# Long random secret shared with the userscript's API_TOKEN. Generate one:
|
|
# openssl rand -hex 32
|
|
API_TOKEN=changeme-generate-a-long-random-token
|
|
|
|
# Comma-separated origins allowed to call the API (CORS). Both Asura domains
|
|
# plus Demonic, Comix, and Kagane. Add/remove as the sites' hostnames change.
|
|
ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to
|
|
|
|
# --- Prod override (Traefik) only ---
|
|
# Subdomain Traefik routes to this service (required by the prod override).
|
|
# MANGA_API_HOST=manga-api.example.com
|
|
# Traefik's docker network name, if not "proxy".
|
|
# PROXY_NETWORK=proxy
|
|
# Traefik HTTPS entrypoint + cert resolver names, if yours differ from these.
|
|
# TRAEFIK_ENTRYPOINT=websecure
|
|
# TRAEFIK_CERTRESOLVER=le
|
|
|
|
# --- Web UI ---
|
|
# Password for the browser UI at https://$MANGA_WEB_HOST. Leave unset to
|
|
# disable the web UI entirely (the routes are not registered at all).
|
|
# Generate one: openssl rand -base64 18
|
|
WEB_PASSWORD=
|
|
|
|
# Subdomain Traefik routes to the browser UI (required by the prod override,
|
|
# whether or not WEB_PASSWORD is set). Left commented on purpose: an example
|
|
# value here would be a silent wrong-hostname fallback, and Traefik would
|
|
# publish the UI router on a domain you do not own. The same container also
|
|
# answers on MANGA_API_HOST for the userscript's API.
|
|
# MANGA_WEB_HOST=manga.example.com
|
|
|
|
# --- Latest-chapter poller ---
|
|
# The backend re-checks each bookmarked series' newest published chapter on its
|
|
# own schedule, so latest_chapter stays fresh even when you never open the manga
|
|
# sites. This runs in parallel with the userscript's own in-browser check.
|
|
# Set to 0 to turn it off entirely.
|
|
# LATEST_CHAPTER_POLL_ENABLED=1
|
|
#
|
|
# Two independent clocks. COOLDOWN is how long one series rests between checks;
|
|
# INTERVAL is how often the poller wakes up and looks for series past that
|
|
# cooldown. Shortening INTERVAL cannot shorten a COOLDOWN.
|
|
# LATEST_CHAPTER_POLL_COOLDOWN=1h # per series, floor 15m
|
|
# LATEST_CHAPTER_POLL_INTERVAL=10m # how often to wake
|
|
# LATEST_CHAPTER_POLL_BATCH=14 # series per wake
|
|
# LATEST_CHAPTER_POLL_STAGGER=20s # delay between fetches in a batch
|
|
#
|
|
# Uses a ticker, not an immediate first run: the first poll happens one
|
|
# INTERVAL after startup, not at startup. A container restarting more often
|
|
# than INTERVAL never polls.
|
|
#
|
|
# BATCH x (COOLDOWN / INTERVAL) series hold the cooldown cadence — 84 with these
|
|
# defaults. Beyond that the cadence stretches uniformly rather than breaking;
|
|
# raise BATCH or lower INTERVAL. Keep BATCH x STAGGER under INTERVAL.
|
|
|
|
# Headless-shell CDP endpoint for sites behind a JavaScript challenge (kagane).
|
|
# Unset disables browser polling; those sites then rely on the userscript alone.
|
|
# Leave commented — the compose files' own default (ws://172.28.0.10:9222) is
|
|
# correct. Do NOT set this to the "headless-shell" DNS name: Chrome's DevTools
|
|
# HTTP handler 500s any /json/version request whose Host header isn't an IP or
|
|
# "localhost", which silently breaks every kagane poll.
|
|
# BROWSER_WS_URL=ws://172.28.0.10:9222
|