Poll comix.to through the browser sidecar (#98) #105

2026-08-16T12:08:58+07:00

sulthan commented

2026-08-16 12:08:58 +07:00

Closes #98.

comix.to began answering plain-TLS fetches with a Cloudflare JavaScript
challenge on 2026-08-12, so every poll got a 403 interstitial. Its cover host
static.comix.to is gated the same way. comix therefore joins kagane and
novelfull as a browser-backed Site.

What changed

Registry (internal/latest/sites.go): comix gains a Browser entry —
comixRead, Done: body != "" && !isInterstitial(body), Fallback: false.
Skip-when-no-browser falls out of the existing routing; no site-string compare
was added anywhere.
Read shape (internal/latest/browser.go): an in-tab fetch() of the
Series URL, not a DOM render. comix is an SPA — rendering it costs ~65
requests for the same server-rendered HTML one fetch returns (24.5 KB,
~480 ms measured). comixSeriesPageURL pins scheme + host + /title/<slug>
and rebuilds the address, so a client-supplied series_url cannot aim the
browser anywhere else.
Cover bytes: comixImageURLRe pins https://static.comix.to/<path>.<ext>;
BrowserFetcher.Image now gates on browserOnlyCoverURL rather than a
kagane-only regex, so both Sites' image URLs route through the one path.
Bytes come from direct navigation, not a page-context fetch — comix's Series
page sets cross-origin-embedder-policy: require-corp, which fails one.
Parsers and stored Series identity: untouched. The in-tab body is the same
server-rendered HTML the existing fixtures were cut from.

Verification

go test ./... green (needs Docker).
New seam tests: comix routes to the browser when one is configured, and is
not fetched at all when none is (TestComixUsesBrowserFetcher,
TestComixSkippedWhenNoBrowserFetcher); URL-pin and cover-gate table tests.
Live proof against the real browser unit, TestSmokeComix (env-gated):
page 24793 bytes in one in-tab fetch, chapter 53, cover accepted by the pin,
26862 bytes of image/jpg retrieved.
Two-axis review run; findings were stale comments on BrowserFetcher, Get
and the Fallback field, fixed in f000cc7.

Docs updated: root AGENTS.md (constraint + smoke command, including the note
that this dev machine's ISP DNS-hijacks comix.to), backend/AGENTS.md
(poller, cover pipeline, BROWSER_WS_URL), REDEPLOY.md §8 degrade note.

Closes #98. comix.to began answering plain-TLS fetches with a Cloudflare JavaScript challenge on 2026-08-12, so every poll got a 403 interstitial. Its cover host `static.comix.to` is gated the same way. comix therefore joins kagane and novelfull as a browser-backed Site. ## What changed - **Registry** (`internal/latest/sites.go`): comix gains a `Browser` entry — `comixRead`, `Done: body != "" && !isInterstitial(body)`, `Fallback: false`. Skip-when-no-browser falls out of the existing routing; no site-string compare was added anywhere. - **Read shape** (`internal/latest/browser.go`): an in-tab `fetch()` of the Series URL, not a DOM render. comix is an SPA — rendering it costs ~65 requests for the same server-rendered HTML one fetch returns (24.5 KB, ~480 ms measured). `comixSeriesPageURL` pins scheme + host + `/title/<slug>` and rebuilds the address, so a client-supplied `series_url` cannot aim the browser anywhere else. - **Cover bytes**: `comixImageURLRe` pins `https://static.comix.to/<path>.<ext>`; `BrowserFetcher.Image` now gates on `browserOnlyCoverURL` rather than a kagane-only regex, so both Sites' image URLs route through the one path. Bytes come from direct navigation, not a page-context fetch — comix's Series page sets `cross-origin-embedder-policy: require-corp`, which fails one. - **Parsers and stored Series identity: untouched.** The in-tab body is the same server-rendered HTML the existing fixtures were cut from. ## Verification - `go test ./...` green (needs Docker). - New seam tests: comix routes to the browser when one is configured, and is not fetched at all when none is (`TestComixUsesBrowserFetcher`, `TestComixSkippedWhenNoBrowserFetcher`); URL-pin and cover-gate table tests. - Live proof against the real browser unit, `TestSmokeComix` (env-gated): page 24793 bytes in one in-tab fetch, chapter 53, cover accepted by the pin, 26862 bytes of `image/jpg` retrieved. - Two-axis review run; findings were stale comments on `BrowserFetcher`, `Get` and the `Fallback` field, fixed in f000cc7. Docs updated: root `AGENTS.md` (constraint + smoke command, including the note that this dev machine's ISP DNS-hijacks `comix.to`), `backend/AGENTS.md` (poller, cover pipeline, `BROWSER_WS_URL`), `REDEPLOY.md` §8 degrade note.

sulthan added 3 commits 2026-08-16 12:08:59 +07:00

feat(latest): poll comix through the browser sidecar (#98 ) 86160c164a

comix.to began answering plain-TLS fetches with a Cloudflare JavaScript
challenge on 2026-08-12, so every poll got a 403 interstitial and its cover
host static.comix.to is gated the same way. comix joins kagane and novelfull
as a browser Site: one registry entry, no plain-TLS fallback, and cover bytes
routed through the browser's image path behind a fully pinned URL pattern.

The read is an in-tab fetch of the Series URL, not a DOM render: comix is an
SPA, so rendering costs ~65 requests for the same server-rendered HTML one
fetch returns (24.5 KB, ~480 ms). Parsers and stored Series identity are
untouched.

Verified live against the real browser unit: page 24793 bytes in one fetch,
chapter 53, cover accepted by the pin and 26862 image bytes retrieved by
direct navigation (comix's Series page sets cross-origin-embedder-policy:
require-corp, so an in-page fetch of the cover host cannot work).

docs(latest): correct the browser-site comments the comix change made stale f000cc7eaa

Review findings from #98: BrowserFetcher's doc still described two sites and
two read shapes, Get's doc enumerated them a second time, and the Fallback
field doc omitted comix. The comix fixture comment now records the live
in-tab body it was re-checked against.

chore: refresh the code graph for the comix browser path 82a11f255b

sulthan added 1 commit 2026-08-16 12:11:35 +07:00

docs: make the comix DNS-hijack note resolver-conditional, not machine-local 55ecb29b81

The note read as a repo-wide fact ('this dev machine', 'here'), which is
meaningless in a clone elsewhere and invites adding --add-host
unconditionally. Now: symptom (ERR_CERT_COMMON_NAME_INVALID), the check
(getent hosts inside the container), the workaround, and an explicit warning
not to bake the hosts into chrome/docker-compose.yml.

sulthan merged commit ddbd57070d into main

2026-08-16 12:14:00 +07:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sulthan/mangaBookmark#105