Browser-backed Sites join the Cover pipeline #62
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
#62 Browser-backed Sites join the Cover pipeline
Parent
Spec: #55. Originating bug: #47. Architecture and rejected alternatives:
docs/adr/0007-backend-hosts-cover-bytes.md. Domain vocabulary: CONTEXT.md.
Do not close #47 or #55 from this ticket.
What to build
kagane and novelfull Series get their Covers through the same pipeline as
everything else, so that the panel shows a kagane Cover — the second symptom
reported in #47 — and novelfull Covers stop depending on a client scrape
that no longer exists.
The two Sites need the browser for different reasons, and conflating them
wastes the scarcest resource in the stack:
• kagane needs the browser for its bytes. It serves cover images
on another origin can load one and why the bytes must
with cross-origin-resource-policy: same-origin behind a JavaScript challenge,
which is why no
come through the sidecar that already clears that challenge.
• novelfull needs the browser only for its HTML. Its pages answer
cf-mitigated: challenge to a plain fetch, but its image paths do not — a
plain fetch of a novelfull cover answers 200 with access-control-allow-
origin: * (measured 2026-08-09). Its bytes therefore go over plain TLS
through the ordinary gated fetcher. Routing them through the browser would
be needless work on a resource that exists for seconds at a time.
When the browser sidecar is unconfigured, kagane Covers are simply
unavailable — consistent with how kagane chapter polling already degrades.
There is no fallback to a plain fetch, which would only ever retrieve a
challenge page.
Acceptance criteria
[x] kagane cover bytes are fetched through the browser sidecar and stored in
the content-addressed store
[x] novelfull cover URLs are extracted from the browser-fetched HTML, and
its bytes are fetched over plain TLS
[x] With no browser sidecar configured, kagane Covers are absent and nothing
falls back to a plain fetch
[x] With no browser sidecar configured, novelfull Covers still work if its
page body is available
[x] Manually verified on-device: a kagane Series shows its Cover in the
panel, not a broken-image glyph
[x] go test ./... is green, with live-network checks gated behind an
environment variable as the existing kagane image smoke test is
Blocked by
• #59 — it reuses that ticket's acquisition and wire path
Implemented in PR #72 (branch feat/62-browser-sites-join-cover-pipeline).
Done and verified by automated tests (all green, go test ./...):
Not done here: the on-device manual verification (kagane Cover renders in the panel) — a separate agent is running it against a mocked scenario, no prod data. Will report back separately.
PR references Fixes #62, so this issue auto-closes when the PR merges.
Update: code review (standards + spec axes) found one hard issue — the Poller and Acquirer encoded opposite novelfull policies in duplicated fetcherFor methods. Fixed in
a66491a: one shared fetcherFor for both paths (novelfull now also falls back to plain TLS on the poll, so pre-existing client-scraped rows get healed, not just Series created after this change), a byte-level no-fallback test for kagane (TestAcquireKaganeBytesNeverFallBackToPlainTLS), and the Acquirer is now wired even when the TLS client fails (kagane needs only the sidecar). Full go test ./... green. PR #72 updated.Manual verification of criterion 5 — mocked, on-device
Verdict: PASS (mocked scenario; no real kagane.to traffic, no prod stack, no prod data).
Code under test: branch
feat/62-browser-sites-join-cover-pipeline, commit40ce68b. Evidence saved under/tmp/manual-verify-62/on the dev box.What was mocked (and what was real)
docker compose up -d --build), Postgres 17, cover volume.env— randomTOKEN_KEY,OWNER_DISCORD_ID=1046923170000000000,PUBLIC_BASE_URL=http://localhost:8080, placeholderDISCORD_*GET /u/{token}/manga-bookmark.user.jsAPI_BASEconstant patched from the prod origin tohttp://localhost:8080(one line; nothing else touched)store.SetSeriesCover— real repo code, real DB rows, real content-addressed file/tmp, and a fake source URLhttps://kagane.to/api/v2/image/<uuid>/compressed(never fetched)renderItemcover block running in a real Chromium page on originhttps://kagane.to<script>— fulfilled by Playwright route interception; every other network egress abortedBROWSER_WS_URLwas empty for the whole run (docker inspect bookmark-api→BROWSER_WS_URL=), i.e. no sidecar browser.1. Stack up
2. No browser configured → kagane cover is absent, nothing falls back to a plain fetch
coveris""— verified. Backend log line at the same instant:That is the designed no-browser behaviour: the cover is skipped, not fetched over plain TLS.
3. Mocked stored cover → wire URL, bytes, and a real
<img>Throwaway Go program (module
bookmarkmanager/backend/tmpseed,replaceontobackend/, builtCGO_ENABLED=0, run in a scratch container as uid 65532 on the composedbnetwork with thecover-datavolume mounted) calling the repo's own store:The address is exactly
sha256("https://kagane.to/api/v2/image/9c2f18d0-.../compressed")— checked withprintf %s <url> | sha256sum→9c6c7108d813.... Absolute, onPUBLIC_BASE_URL, as ADR-0007 requires.Byte-identical round trip.
4. The panel, end to end in a browser
Playwright: every request intercepted.
https://kagane.to/series/3f1c9a52-...fulfilled with a local mock HTML page,https://kagane.to/__mock__/manga-bookmark.user.jsfulfilled with the script the backend served (so the page origin is genuinelyhttps://kagane.toand the real@matchorigin semantics hold);http://localhost:8080/**proxied through Playwright's request context with CORS headers (Chrome will not let anhttps://page reachhttp://localhostdirectly — a mock artifact, prod is https→https). One cover URL was deliberatelyroute.abort('failed')ed to exercise theonerrorpath.The userscript booted with zero page errors, built its shadow-root panel, and
GET /bookmarksreturned 3 items. After clicking#fab:<img class="cover">,naturalWidth=60,naturalHeight=90,complete=true, painted at 52×70. The screenshot shows the actual checkerboard image in the panel.div.cover.phplaceholder, never an<img>, so no glyph is possible.onerrorhandler (#47) had already replaced the<img>withdiv.cover.ph— the DOM contains no<img>at all afterwards.Console for the whole run: 3 errors, all of them
net::ERR_FAILEDfor the URL I deliberately aborted; the rest are Chrome mixed-content warnings caused by the httpPUBLIC_BASE_URLin the mock.Screenshot:
/tmp/manual-verify-62/panel-kagane.png(panel open, one real cover + two placeholders).Verified vs inferred
cover:""on a browser-less kagane PUT, the absolute wire URL, the 200/image/png/234-byte cover serve, the byte-identical hash, the renderednaturalWidth/naturalHeight, the placeholder substitution in both cover-less and failed-cover cases, the screenshot.BROWSER_WS_URLand live kagane, which was explicitly out of scope here. This run proves everything downstream ofSetSeriesCover: once bytes are stored under the kagane series, the wire URL, the cover endpoint and the panel all behave. The sidecar-side fetch remains covered by the unit +SMOKE_*tests (not re-run here; the implementing agent reportedgo test ./...green).store.Openwith a zeroTokenHash, which re-seeded the owner row'stoken_sha256; I restored it with apsqlUPDATE. That is my program's bug, not the backend's.Cleanup
Throwaway Go program deleted (
/tmp/seedcover), local static server stopped. The local compose stack and its mock.envare left up for inspection; no tracked repo file was modified.Manual verification (criterion 5) — PASS, run by a separate verification agent against a fully mocked scenario (no prod stack, no prod data, no real kagane.to traffic):
Evidence: /tmp/manual-verify-62/ (screenshots, curl outputs, seed logs). Full report posted by the verification agent (prior comment).