5713e3d04a
chromedp/headless-shell cannot clear kagane.to's managed challenge. It is
a stripped Chrome build, and the tells are structural rather than a
header: navigator.webdriver is true, the plugin list is empty, and the
client hints are Chromium- rather than Chrome-branded. Overriding
webdriver through CDP was tried on its own and changed nothing.
Everything below was measured on 2026-08-08 from a single IP, against the
same kagane cover, so the comparisons are like for like:
chromedp/headless-shell:stable never cleared (90s)
zenika/alpine-chrome never cleared - ships Chrome 124, old
enough that Cloudflare refuses it and
old enough to break chromedp's CDP structs
google-chrome, default UA never cleared (60s) - --headless=new
advertises "HeadlessChrome"
google-chrome, stock UA, UTC never cleared (90s)
google-chrome, stock UA, TZ set cleared in ~4s
So both remaining tells are load-bearing, and each was tested in
isolation. chrome/ is a Debian image with google-chrome-stable, a UA
whose version is read back out of the binary at startup (a hardcoded one
would drift out of step with the Sec-CH-UA hints on the next Chrome
update and become a fresh tell), and no --enable-automation.
The timezone matters because Cloudflare scores a browser whose clock zone
disagrees with its egress IP's country as a proxy. Note that the usual
`-v /etc/localtime:/etc/localtime:ro` does not work here: Chrome resolves
the zone through ICU, which takes the name from that path's symlink
target and ignores the file's contents, so glibc reports the host zone
while Chrome still reports UTC. /etc/timezone carries the name and is
mounted instead; BROWSER_TZ overrides it for a host whose clock is UTC in
a country that is not.
Chrome also binds its DevTools port to loopback and silently ignores
--remote-debugging-address, which is why headless-shell fronted it with
socat. This image does the same, so it stays a drop-in: the compose
service keeps the headless-shell name and its pinned address, and
BROWSER_WS_URL is unchanged.
Deploying needs `docker compose build headless-shell`.
150 lines
7.5 KiB
YAML
150 lines
7.5 KiB
YAML
# Base stack — works standalone for local smoke testing (`docker compose up`).
|
|
# The service binds 127.0.0.1:8080; a host reverse proxy (nginx/Caddy/Traefik)
|
|
# terminates TLS for bookmark-api.<domain> and forwards to it.
|
|
#
|
|
# If your proxy runs in Docker on its own network, use the prod override which
|
|
# attaches to that network instead of publishing a port:
|
|
# docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
|
|
services:
|
|
bookmark-api:
|
|
build: ./backend
|
|
image: bookmarkmanager-backend:latest
|
|
container_name: bookmark-api
|
|
restart: unless-stopped
|
|
environment:
|
|
# TOKEN_KEY derives every Reader's userscript credential (issue #24) —
|
|
# compose refuses to start without it.
|
|
TOKEN_KEY: ${TOKEN_KEY:?set TOKEN_KEY in .env}
|
|
# Owner's Discord user ID — required. Seeds the owner Reader (the
|
|
# administrator); every other Reader registers on their first login.
|
|
OWNER_DISCORD_ID: ${OWNER_DISCORD_ID:?set OWNER_DISCORD_ID in .env}
|
|
ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to,https://novelfull.com,https://lightnovelworld.net}
|
|
# The bookmarks database. Host is the compose service name; the password
|
|
# comes from .env so it is never committed.
|
|
DATABASE_URL: ${DATABASE_URL:-postgres://bookmarks:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@postgres:5432/bookmarks?sslmode=disable}
|
|
PORT: "8080"
|
|
# Discord OAuth for the browser UI (ADR-0002). The first four are
|
|
# required; DISCORD_REQUIRED_ROLE is optional and empty by default.
|
|
# Guild membership is the whole gate: any member becomes a Reader.
|
|
DISCORD_CLIENT_ID: ${DISCORD_CLIENT_ID:?set DISCORD_CLIENT_ID in .env}
|
|
DISCORD_CLIENT_SECRET: ${DISCORD_CLIENT_SECRET:?set DISCORD_CLIENT_SECRET in .env}
|
|
DISCORD_GUILD_ID: ${DISCORD_GUILD_ID:?set DISCORD_GUILD_ID in .env}
|
|
DISCORD_REQUIRED_ROLE: ${DISCORD_REQUIRED_ROLE:-}
|
|
DISCORD_API_BASE: ${DISCORD_API_BASE:-https://discord.com/api/v10}
|
|
DISCORD_REDIRECT_URI: ${DISCORD_REDIRECT_URI:?set DISCORD_REDIRECT_URI in .env}
|
|
# Path inside the container; matches the bindmount above.
|
|
USERSCRIPT_PATH: ${USERSCRIPT_PATH:-/userscript/manga-bookmark.user.js}
|
|
# Second script from the same bindmount; the novel library is a separate
|
|
# Violentmonkey install.
|
|
NOVEL_USERSCRIPT_PATH: ${NOVEL_USERSCRIPT_PATH:-/userscript/novel-bookmark.user.js}
|
|
# Latest-chapter poller. LATEST_CHAPTER_POLL_ENABLED=0 in .env is the kill
|
|
# switch; it only takes effect because these are listed here.
|
|
LATEST_CHAPTER_POLL_ENABLED: ${LATEST_CHAPTER_POLL_ENABLED:-1}
|
|
LATEST_CHAPTER_POLL_COOLDOWN: ${LATEST_CHAPTER_POLL_COOLDOWN:-1h}
|
|
LATEST_CHAPTER_POLL_INTERVAL: ${LATEST_CHAPTER_POLL_INTERVAL:-10m}
|
|
LATEST_CHAPTER_POLL_BATCH: ${LATEST_CHAPTER_POLL_BATCH:-14}
|
|
LATEST_CHAPTER_POLL_STAGGER: ${LATEST_CHAPTER_POLL_STAGGER:-20s}
|
|
# CDP endpoint for sites behind a JavaScript challenge (kagane). Unset
|
|
# disables browser polling for those sites; the userscript still covers them.
|
|
# Must be an IP, not the "headless-shell" DNS name: Chrome's DevTools HTTP
|
|
# handler rejects the discovery request (GET /json/version) with a 500
|
|
# unless the Host header is an IP address or "localhost" — confirmed
|
|
# 2026-08-03 against chromedp/headless-shell:stable, independent of
|
|
# chromedp's own dial logic. The sidecar's static address below exists so
|
|
# this URL survives container recreation.
|
|
BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://172.28.0.10:9222}
|
|
depends_on:
|
|
headless-shell:
|
|
condition: service_started
|
|
# The migration runner is the first thing the binary does, so a Postgres
|
|
# that is still initialising means a crash-loop until it is not.
|
|
postgres:
|
|
condition: service_healthy
|
|
volumes:
|
|
# The userscript is served from here, read fresh on every request. Editing
|
|
# the file in this checkout takes effect on the next Violentmonkey poll —
|
|
# no rebuild, no restart. `git pull` restores the committed version, which
|
|
# is why a redeploy always ships the repo's script.
|
|
- ./userscript:/userscript:ro
|
|
# Bound to loopback only: the proxy (or curl during smoke test) reaches it,
|
|
# the public internet does not.
|
|
ports:
|
|
- "127.0.0.1:8080:8080"
|
|
networks:
|
|
- browser
|
|
- db
|
|
|
|
postgres:
|
|
image: postgres:17-alpine
|
|
restart: unless-stopped
|
|
environment:
|
|
POSTGRES_DB: bookmarks
|
|
POSTGRES_USER: bookmarks
|
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U bookmarks -d bookmarks"]
|
|
interval: 5s
|
|
timeout: 3s
|
|
retries: 10
|
|
volumes:
|
|
- postgres-data:/var/lib/postgresql/data
|
|
# Deliberately no `ports:` — only bookmark-api, over the `db` network,
|
|
# reaches it. Use `docker compose exec postgres psql` for a shell.
|
|
networks:
|
|
- db
|
|
|
|
headless-shell:
|
|
# Real Google Chrome, not chromedp/headless-shell — see chrome/Dockerfile.
|
|
# The service name is kept so existing overrides and BROWSER_WS_URL stay put.
|
|
build: ./chrome
|
|
image: bookmarkmanager-chrome:latest
|
|
restart: unless-stopped
|
|
environment:
|
|
# Cloudflare scores a browser whose clock zone disagrees with its egress
|
|
# IP's country as a proxy, and kagane's challenge then never clears
|
|
# (measured 2026-08-08: identical container, UTC never cleared in 90s,
|
|
# Asia/Bangkok cleared in 4s from a Thai IP). Unset falls back to the
|
|
# host's /etc/timezone below, which is right whenever the host clock is
|
|
# set to local time; set BROWSER_TZ when the host runs UTC somewhere that
|
|
# isn't, since it is the IP's country that has to match, not the clock's.
|
|
TZ: ${BROWSER_TZ:-}
|
|
volumes:
|
|
# The zone *name*, which is what Chrome's ICU needs — see chrome/entrypoint.sh.
|
|
# Absent on a non-Debian host, which the entrypoint handles by falling back to UTC.
|
|
- /etc/timezone:/etc/timezone:ro
|
|
# Chrome allocates shared memory per tab and dies on Docker's 64MB default.
|
|
shm_size: '1gb'
|
|
# Reaps zombie renderer processes, which otherwise accumulate for the
|
|
# container's lifetime.
|
|
init: true
|
|
# Deliberately no `ports:` — an exposed CDP endpoint is remote code
|
|
# execution. Only bookmark-api, via the `browser` network below, may reach it.
|
|
# No `command:` either: every flag this browser needs is in its entrypoint,
|
|
# and the UA override there is load-bearing for the challenge.
|
|
networks:
|
|
browser:
|
|
# Pinned so BROWSER_WS_URL can name an IP (required, see above) that
|
|
# survives `docker compose up` recreating this container.
|
|
ipv4_address: 172.28.0.10
|
|
|
|
volumes:
|
|
postgres-data:
|
|
# The pre-Postgres SQLite volume (bookmarks-data) is deliberately no longer
|
|
# declared here: undeclared means `docker compose down -v` cannot take it
|
|
# with the rest, so the old database survives the cutover until someone
|
|
# removes it by hand.
|
|
|
|
networks:
|
|
# Not `internal: true`: headless Chrome still needs outbound access to reach
|
|
# kagane.to. Isolation here comes from membership (only bookmark-api and
|
|
# headless-shell join it), not from cutting egress.
|
|
browser:
|
|
ipam:
|
|
config:
|
|
- subnet: 172.28.0.0/24
|
|
# Postgres needs no egress and nothing outside bookmark-api needs to reach
|
|
# it, so this one really can be cut off from the outside world.
|
|
db:
|
|
internal: true
|