fix(docker): correct the timezone claim — UTC is the tell, not a country mismatch
The previous commit documented the clock-zone requirement as "the zone must match the egress IP's country". Re-measuring against the deployment case shows that is wrong. The original inference came from reading the host's /etc/timezone (Asia/Bangkok) and assuming the egress IP was Thai. It is not: this host egresses from an Indonesian IP. Asia/Bangkok cleared the challenge not because it matched a country but because it is simply not UTC, and the two share +07, which hid the distinction. Measured 2026-08-08, identical container, one Indonesian egress IP: TZ=UTC never cleared (60s, twice) TZ=Asia/Jakarta cleared in 4s TZ=America/New_York cleared in 4s America/New_York matches neither the country nor the offset nor the hemisphere, and clears just as fast. So a UTC clock is itself the bot signal - Cloudflare scores it as the datacenter default - and any real zone satisfies the check. This makes the knob considerably less fragile than documented: BROWSER_TZ needs a plausible zone, not a geolocated one, and a deployment that moves region does not have to keep it in sync. Comments in chrome/entrypoint.sh and docker-compose.yml, the hard constraint in AGENTS.md, and the PR description are corrected accordingly. BROWSER_TZ is also documented in .env.example for the first time, which is the file an operator actually copies - the setting decides whether kagane works at all, and a UTC server (the common case) is exactly the one that fails with it unset. Re-verified against the shipped image with TZ=Asia/Jakarta: TestSmokeKaganeImage PASS (5.00s, 56710 bytes of image/webp), TestSmokeKaganeGet PASS (1.24s, status 200).
This commit is contained in:
@@ -90,3 +90,16 @@ DISCORD_REDIRECT_URI=
|
|||||||
# HTTP handler 500s any /json/version request whose Host header isn't an IP or
|
# HTTP handler 500s any /json/version request whose Host header isn't an IP or
|
||||||
# "localhost", which silently breaks every kagane poll.
|
# "localhost", which silently breaks every kagane poll.
|
||||||
# BROWSER_WS_URL=ws://172.28.0.10:9222
|
# BROWSER_WS_URL=ws://172.28.0.10:9222
|
||||||
|
|
||||||
|
# Clock zone the headless browser reports. A UTC clock is itself the bot
|
||||||
|
# signal — Cloudflare treats it as the datacenter default — and kagane's
|
||||||
|
# challenge then never clears. Measured 2026-08-08, identical container, one
|
||||||
|
# Indonesian egress IP: UTC never cleared in 60s (twice); Asia/Jakarta and
|
||||||
|
# America/New_York both cleared in 4s. So any real zone works; it does not
|
||||||
|
# have to match the IP's country, it just must not be UTC.
|
||||||
|
#
|
||||||
|
# Unset falls back to the host's /etc/timezone, which is a real zone whenever
|
||||||
|
# the host clock is set to local time. Set this when the host runs UTC — a UTC
|
||||||
|
# server is exactly the case that fails. Only the browser sidecar reads it;
|
||||||
|
# the backend keeps its UTC clock.
|
||||||
|
# BROWSER_TZ=Asia/Jakarta
|
||||||
|
|||||||
@@ -20,7 +20,8 @@ Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free
|
|||||||
- Userscript run in **isolated world**, so embedded API token safe from site's JS.
|
- Userscript run in **isolated world**, so embedded API token safe from site's JS.
|
||||||
- Cloudflare's block on manga sites **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — Cloudflare's bot scoring can flip previously-clean IP without notice. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test.
|
- Cloudflare's block on manga sites **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — Cloudflare's bot scoring can flip previously-clean IP without notice. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test.
|
||||||
- **kagane.to and novelfull.com are the exception to the above** — both sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`) and skips them entirely when that's unset. The four other sites poll fine over plain TLS.
|
- **kagane.to and novelfull.com are the exception to the above** — both sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`) and skips them entirely when that's unset. The four other sites poll fine over plain TLS.
|
||||||
- **The CDP sidecar must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a clock zone matching the egress IP's country cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead.
|
- **The CDP sidecar must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a non-UTC clock zone cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, ignoring the file's contents, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead.
|
||||||
|
- **UTC is the tell, not a country mismatch.** A UTC clock is the datacenter default, so Cloudflare scores it as one; any real zone clears. Measured 2026-08-08, identical container, one Indonesian egress IP: UTC never cleared in 60s (twice), while `Asia/Jakarta` **and** `America/New_York` both cleared in 4s. An earlier note here claimed the zone had to match the egress IP's country — that was wrong, inferred from the host clock (`Asia/Bangkok`) rather than the measured egress. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one.
|
||||||
- **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives.
|
- **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives.
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|||||||
@@ -1,11 +1,12 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
# Cloudflare scores a browser whose clock zone disagrees with its egress IP's
|
# A UTC clock is itself the bot signal — Cloudflare treats it as the datacenter
|
||||||
# country as a proxy, and kagane's challenge then never clears (measured
|
# default — and kagane's challenge then never clears. Measured 2026-08-08 with
|
||||||
# 2026-08-08 from a Thai IP: identical container, UTC never cleared in 90s,
|
# an identical container on one Indonesian egress IP: UTC never cleared in 60s
|
||||||
# Asia/Bangkok cleared in 4s). So the zone has to be right, and it has to be
|
# (twice), while Asia/Jakarta and America/New_York both cleared in 4s. Any real
|
||||||
# right the way Chrome reads it.
|
# zone will do; the zone does not have to match the IP's country, it just must
|
||||||
|
# not be UTC. It does have to be right the way Chrome reads it.
|
||||||
#
|
#
|
||||||
# TZ must carry the zone *name*. Chrome resolves the zone through ICU, which
|
# TZ must carry the zone *name*. Chrome resolves the zone through ICU, which
|
||||||
# takes the name from /etc/localtime's symlink target and ignores the file's
|
# takes the name from /etc/localtime's symlink target and ignores the file's
|
||||||
|
|||||||
+8
-7
@@ -101,13 +101,14 @@ services:
|
|||||||
image: bookmarkmanager-chrome:latest
|
image: bookmarkmanager-chrome:latest
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
environment:
|
environment:
|
||||||
# Cloudflare scores a browser whose clock zone disagrees with its egress
|
# A UTC clock is itself the bot signal: Cloudflare treats it as the
|
||||||
# IP's country as a proxy, and kagane's challenge then never clears
|
# datacenter default, and kagane's challenge then never clears. Measured
|
||||||
# (measured 2026-08-08: identical container, UTC never cleared in 90s,
|
# 2026-08-08, identical container, one Indonesian egress IP: UTC never
|
||||||
# Asia/Bangkok cleared in 4s from a Thai IP). Unset falls back to the
|
# cleared in 60s (twice); Asia/Jakarta and America/New_York both cleared
|
||||||
# host's /etc/timezone below, which is right whenever the host clock is
|
# in 4s. So any real zone works and it need not match the IP's country —
|
||||||
# set to local time; set BROWSER_TZ when the host runs UTC somewhere that
|
# only UTC fails. Unset falls back to the host's /etc/timezone below,
|
||||||
# isn't, since it is the IP's country that has to match, not the clock's.
|
# which is a real zone whenever the host clock is set to local time; set
|
||||||
|
# BROWSER_TZ when the host runs UTC.
|
||||||
TZ: ${BROWSER_TZ:-}
|
TZ: ${BROWSER_TZ:-}
|
||||||
volumes:
|
volumes:
|
||||||
# The zone *name*, which is what Chrome's ICU needs — see chrome/entrypoint.sh.
|
# The zone *name*, which is what Chrome's ICU needs — see chrome/entrypoint.sh.
|
||||||
|
|||||||
Reference in New Issue
Block a user