Closes #46 once deployed. The headless browser leaves the API stack and becomes its own compose unit (`chrome/docker-compose.yml`) intended for the home machine, reached over the tailnet. No fallback sidecar is left on the VPS. The backend needs no code change — `BROWSER_WS_URL` was already the only coupling. Its default is now empty rather than a pinned Docker IP, so an unconfigured or unreachable browser degrades exactly as it always has: plain-TLS libraries unaffected, kagane/novelfull logged and skipped, stored covers still served. ### What shipped - `chrome/docker-compose.yml` + `chrome/.env.example` — the browser unit, with the CDP port bound to `${BROWSER_BIND_ADDR}` (no default) and the resource limits from the epic: 512 MiB / 1 GiB memory+swap, `oom_score_adj 800`, halved CPU weight, shm 1 GiB -> 128 MiB. - API stack drops the service, its `depends_on` and the `browser` network. - `bookmark-api` gains the `default` network. Dropping `browser` had left it on `db` alone, which is `internal: true` — no published port and, worse, no egress for the poller at all. Caught by actually bringing the stack up. - ADR-0006 for the topology; `DEPLOY.md` §7 for first-time setup of the browser machine; `REDEPLOY.md` §8 for its independent update cadence; architecture diagrams, config tables and troubleshooting rows across README/AGENTS/env. ### Verified locally - Browser unit builds and runs: Chrome 151, UA carries no `HeadlessChrome`, all limits applied as declared. - **Live smoke passes through the new unit**: `TestSmokeKaganeImage` fetched 56710 bytes of `image/webp`, `TestSmokeKaganeGet` got a 200 with a real chapter list. The challenge cleared under the reduced 128 MiB shm. - Bind isolation proven: refused on the host's non-loopback address, accepted on the configured one. - 321 MiB peak of the 512 MiB cap after a full solve; 0 restarts, no OOM kill. - API stack comes up clean, `/healthz` 200; egress confirmed present on `default` and absent on `db`. - `go test ./...`, `go vet`, `gofmt` clean. ### Left to the operator Provisioning the home machine, the Tailscale ACL, setting `BROWSER_WS_URL` in production, and observing acceptance criteria 5-7 (covers with the machine off, several days of zero OOM/restarts, VPS memory improvement). `DEPLOY.md` §7 now carries the before/after `free -m` reading those need. Reviewed-on: #52 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #52.
This commit is contained in:
+21
-58
@@ -5,6 +5,9 @@
|
||||
# If your proxy runs in Docker on its own network, use the prod override which
|
||||
# attaches to that network instead of publishing a port:
|
||||
# docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
||||
#
|
||||
# The browser is not here. It is its own unit on the home machine —
|
||||
# chrome/docker-compose.yml — reached over the tailnet via BROWSER_WS_URL.
|
||||
|
||||
services:
|
||||
bookmark-api:
|
||||
@@ -27,7 +30,7 @@ services:
|
||||
# Log timestamps only. Go's `log` stamps lines in local time, and this
|
||||
# service has no other use for a zone: bookmark timestamps are unix ms
|
||||
# and the two real time columns are timestamptz, both absolute instants.
|
||||
# Purely so these lines read on the same clock as the sidecar's. Named
|
||||
# Purely so these lines read on the same clock as the browser's. Named
|
||||
# API_TZ rather than TZ so an operator's exported shell TZ cannot leak
|
||||
# in; distroless already carries tzdata, so the name just resolves.
|
||||
TZ: ${API_TZ:-Asia/Jakarta}
|
||||
@@ -53,18 +56,20 @@ services:
|
||||
LATEST_CHAPTER_POLL_INTERVAL: ${LATEST_CHAPTER_POLL_INTERVAL:-10m}
|
||||
LATEST_CHAPTER_POLL_BATCH: ${LATEST_CHAPTER_POLL_BATCH:-14}
|
||||
LATEST_CHAPTER_POLL_STAGGER: ${LATEST_CHAPTER_POLL_STAGGER:-20s}
|
||||
# CDP endpoint for sites behind a JavaScript challenge (kagane). Unset
|
||||
# disables browser polling for those sites; the userscript still covers them.
|
||||
# Must be an IP, not the "headless-shell" DNS name: Chrome's DevTools HTTP
|
||||
# handler rejects the discovery request (GET /json/version) with a 500
|
||||
# unless the Host header is an IP address or "localhost" — confirmed
|
||||
# 2026-08-03 against chromedp/headless-shell:stable, independent of
|
||||
# chromedp's own dial logic. The sidecar's static address below exists so
|
||||
# this URL survives container recreation.
|
||||
BROWSER_WS_URL: ${BROWSER_WS_URL:-ws://172.28.0.10:9222}
|
||||
# CDP endpoint for sites behind a JavaScript challenge (kagane,
|
||||
# novelfull). The browser is not part of this stack — it runs on the home
|
||||
# machine as its own unit (chrome/docker-compose.yml) and is reached over
|
||||
# the tailnet. Unset disables browser polling for those sites and serves
|
||||
# 404 from the cover proxy for covers not already stored; the userscript
|
||||
# still covers them. Set it in .env to ws://<home machine tailnet IP>:9222.
|
||||
#
|
||||
# Must be an IP, not a MagicDNS hostname: Chrome's DevTools HTTP handler
|
||||
# rejects the discovery request (GET /json/version) with a 500 unless the
|
||||
# Host header is an IP address or "localhost" — confirmed 2026-08-03,
|
||||
# independent of chromedp's own dial logic. The same trap that used to
|
||||
# force a pinned Docker IP now forbids the tailnet name.
|
||||
BROWSER_WS_URL: ${BROWSER_WS_URL:-}
|
||||
depends_on:
|
||||
headless-shell:
|
||||
condition: service_started
|
||||
# The migration runner is the first thing the binary does, so a Postgres
|
||||
# that is still initialising means a crash-loop until it is not.
|
||||
postgres:
|
||||
@@ -79,8 +84,11 @@ services:
|
||||
# the public internet does not.
|
||||
ports:
|
||||
- "127.0.0.1:8080:8080"
|
||||
# `default` is not decoration: `db` is `internal: true`, and a container on
|
||||
# nothing but an internal network gets neither a published port nor egress
|
||||
# — which would silently kill every poller fetch.
|
||||
networks:
|
||||
- browser
|
||||
- default
|
||||
- db
|
||||
|
||||
postgres:
|
||||
@@ -102,59 +110,14 @@ services:
|
||||
networks:
|
||||
- db
|
||||
|
||||
headless-shell:
|
||||
# Real Google Chrome, not chromedp/headless-shell — see chrome/Dockerfile.
|
||||
# The service name is kept so existing overrides and BROWSER_WS_URL stay put.
|
||||
build: ./chrome
|
||||
image: bookmarkmanager-chrome:latest
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
# A UTC clock is itself the bot signal: Cloudflare treats it as the
|
||||
# datacenter default, and kagane's challenge then never clears. Measured
|
||||
# 2026-08-08, identical container, one Indonesian egress IP: UTC never
|
||||
# cleared in 60s (twice); Asia/Jakarta and America/New_York both cleared
|
||||
# in 4s. So any real zone works and it need not match the IP's country —
|
||||
# only UTC fails. Unset falls back to the host's /etc/timezone below,
|
||||
# which is a real zone whenever the host clock is set to local time; set
|
||||
# BROWSER_TZ when the host runs UTC.
|
||||
TZ: ${BROWSER_TZ:-}
|
||||
volumes:
|
||||
# The zone *name*, which is what Chrome's ICU needs — see chrome/entrypoint.sh.
|
||||
# Absent on a non-Debian host, which the entrypoint handles by falling back to UTC.
|
||||
- /etc/timezone:/etc/timezone:ro
|
||||
# Cloudflare clearance must survive Chrome reaping and image recreation.
|
||||
- chrome-profile:/home/chrome/profile
|
||||
# Chrome allocates shared memory per tab and dies on Docker's 64MB default.
|
||||
shm_size: '1gb'
|
||||
# Reaps zombie renderer processes, which otherwise accumulate for the
|
||||
# container's lifetime.
|
||||
init: true
|
||||
# Deliberately no `ports:` — an exposed CDP endpoint is remote code
|
||||
# execution. Only bookmark-api, via the `browser` network below, may reach it.
|
||||
# No `command:` either: every flag this browser needs is in its entrypoint,
|
||||
# and the UA override there is load-bearing for the challenge.
|
||||
networks:
|
||||
browser:
|
||||
# Pinned so BROWSER_WS_URL can name an IP (required, see above) that
|
||||
# survives `docker compose up` recreating this container.
|
||||
ipv4_address: 172.28.0.10
|
||||
|
||||
volumes:
|
||||
postgres-data:
|
||||
# The pre-Postgres SQLite volume (bookmarks-data) is deliberately no longer
|
||||
# declared here: undeclared means `docker compose down -v` cannot take it
|
||||
# with the rest, so the old database survives the cutover until someone
|
||||
# removes it by hand.
|
||||
chrome-profile:
|
||||
|
||||
networks:
|
||||
# Not `internal: true`: headless Chrome still needs outbound access to reach
|
||||
# kagane.to. Isolation here comes from membership (only bookmark-api and
|
||||
# headless-shell join it), not from cutting egress.
|
||||
browser:
|
||||
ipam:
|
||||
config:
|
||||
- subnet: 172.28.0.0/24
|
||||
# Postgres needs no egress and nothing outside bookmark-api needs to reach
|
||||
# it, so this one really can be cut off from the outside world.
|
||||
db:
|
||||
|
||||
Reference in New Issue
Block a user