From 1552dd15da941ed4cff04a0358f8ed400ec59904 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Sat, 8 Aug 2026 23:16:39 +0700 Subject: [PATCH] =?UTF-8?q?fix(docker):=20correct=20the=20timezone=20claim?= =?UTF-8?q?=20=E2=80=94=20UTC=20is=20the=20tell,=20not=20a=20country=20mis?= =?UTF-8?q?match?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit documented the clock-zone requirement as "the zone must match the egress IP's country". Re-measuring against the deployment case shows that is wrong. The original inference came from reading the host's /etc/timezone (Asia/Bangkok) and assuming the egress IP was Thai. It is not: this host egresses from an Indonesian IP. Asia/Bangkok cleared the challenge not because it matched a country but because it is simply not UTC, and the two share +07, which hid the distinction. Measured 2026-08-08, identical container, one Indonesian egress IP: TZ=UTC never cleared (60s, twice) TZ=Asia/Jakarta cleared in 4s TZ=America/New_York cleared in 4s America/New_York matches neither the country nor the offset nor the hemisphere, and clears just as fast. So a UTC clock is itself the bot signal - Cloudflare scores it as the datacenter default - and any real zone satisfies the check. This makes the knob considerably less fragile than documented: BROWSER_TZ needs a plausible zone, not a geolocated one, and a deployment that moves region does not have to keep it in sync. Comments in chrome/entrypoint.sh and docker-compose.yml, the hard constraint in AGENTS.md, and the PR description are corrected accordingly. BROWSER_TZ is also documented in .env.example for the first time, which is the file an operator actually copies - the setting decides whether kagane works at all, and a UTC server (the common case) is exactly the one that fails with it unset. Re-verified against the shipped image with TZ=Asia/Jakarta: TestSmokeKaganeImage PASS (5.00s, 56710 bytes of image/webp), TestSmokeKaganeGet PASS (1.24s, status 200). --- .env.example | 13 +++++++++++++ AGENTS.md | 3 ++- chrome/entrypoint.sh | 11 ++++++----- docker-compose.yml | 15 ++++++++------- 4 files changed, 29 insertions(+), 13 deletions(-) diff --git a/.env.example b/.env.example index 7f1e213..77d1bee 100644 --- a/.env.example +++ b/.env.example @@ -90,3 +90,16 @@ DISCORD_REDIRECT_URI= # HTTP handler 500s any /json/version request whose Host header isn't an IP or # "localhost", which silently breaks every kagane poll. # BROWSER_WS_URL=ws://172.28.0.10:9222 + +# Clock zone the headless browser reports. A UTC clock is itself the bot +# signal — Cloudflare treats it as the datacenter default — and kagane's +# challenge then never clears. Measured 2026-08-08, identical container, one +# Indonesian egress IP: UTC never cleared in 60s (twice); Asia/Jakarta and +# America/New_York both cleared in 4s. So any real zone works; it does not +# have to match the IP's country, it just must not be UTC. +# +# Unset falls back to the host's /etc/timezone, which is a real zone whenever +# the host clock is set to local time. Set this when the host runs UTC — a UTC +# server is exactly the case that fails. Only the browser sidecar reads it; +# the backend keeps its UTC clock. +# BROWSER_TZ=Asia/Jakarta diff --git a/AGENTS.md b/AGENTS.md index dea9c68..98d94bc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,7 +20,8 @@ Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free - Userscript run in **isolated world**, so embedded API token safe from site's JS. - Cloudflare's block on manga sites **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — Cloudflare's bot scoring can flip previously-clean IP without notice. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test. - **kagane.to and novelfull.com are the exception to the above** — both sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`) and skips them entirely when that's unset. The four other sites poll fine over plain TLS. -- **The CDP sidecar must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a clock zone matching the egress IP's country cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead. +- **The CDP sidecar must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a non-UTC clock zone cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, ignoring the file's contents, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead. +- **UTC is the tell, not a country mismatch.** A UTC clock is the datacenter default, so Cloudflare scores it as one; any real zone clears. Measured 2026-08-08, identical container, one Indonesian egress IP: UTC never cleared in 60s (twice), while `Asia/Jakarta` **and** `America/New_York` both cleared in 4s. An earlier note here claimed the zone had to match the egress IP's country — that was wrong, inferred from the host clock (`Asia/Bangkok`) rather than the measured egress. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one. - **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives. ## Architecture diff --git a/chrome/entrypoint.sh b/chrome/entrypoint.sh index cc0dbad..fefde7d 100755 --- a/chrome/entrypoint.sh +++ b/chrome/entrypoint.sh @@ -1,11 +1,12 @@ #!/bin/sh set -eu -# Cloudflare scores a browser whose clock zone disagrees with its egress IP's -# country as a proxy, and kagane's challenge then never clears (measured -# 2026-08-08 from a Thai IP: identical container, UTC never cleared in 90s, -# Asia/Bangkok cleared in 4s). So the zone has to be right, and it has to be -# right the way Chrome reads it. +# A UTC clock is itself the bot signal — Cloudflare treats it as the datacenter +# default — and kagane's challenge then never clears. Measured 2026-08-08 with +# an identical container on one Indonesian egress IP: UTC never cleared in 60s +# (twice), while Asia/Jakarta and America/New_York both cleared in 4s. Any real +# zone will do; the zone does not have to match the IP's country, it just must +# not be UTC. It does have to be right the way Chrome reads it. # # TZ must carry the zone *name*. Chrome resolves the zone through ICU, which # takes the name from /etc/localtime's symlink target and ignores the file's diff --git a/docker-compose.yml b/docker-compose.yml index 78cc2e9..eb124f4 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -101,13 +101,14 @@ services: image: bookmarkmanager-chrome:latest restart: unless-stopped environment: - # Cloudflare scores a browser whose clock zone disagrees with its egress - # IP's country as a proxy, and kagane's challenge then never clears - # (measured 2026-08-08: identical container, UTC never cleared in 90s, - # Asia/Bangkok cleared in 4s from a Thai IP). Unset falls back to the - # host's /etc/timezone below, which is right whenever the host clock is - # set to local time; set BROWSER_TZ when the host runs UTC somewhere that - # isn't, since it is the IP's country that has to match, not the clock's. + # A UTC clock is itself the bot signal: Cloudflare treats it as the + # datacenter default, and kagane's challenge then never clears. Measured + # 2026-08-08, identical container, one Indonesian egress IP: UTC never + # cleared in 60s (twice); Asia/Jakarta and America/New_York both cleared + # in 4s. So any real zone works and it need not match the IP's country — + # only UTC fails. Unset falls back to the host's /etc/timezone below, + # which is a real zone whenever the host clock is set to local time; set + # BROWSER_TZ when the host runs UTC. TZ: ${BROWSER_TZ:-} volumes: # The zone *name*, which is what Chrome's ICU needs — see chrome/entrypoint.sh.