Compare commits

...

4 Commits

Author SHA1 Message Date
sulthan 55ecb29b81 docs: make the comix DNS-hijack note resolver-conditional, not machine-local
The note read as a repo-wide fact ('this dev machine', 'here'), which is
meaningless in a clone elsewhere and invites adding --add-host
unconditionally. Now: symptom (ERR_CERT_COMMON_NAME_INVALID), the check
(getent hosts inside the container), the workaround, and an explicit warning
not to bake the hosts into chrome/docker-compose.yml.
2026-08-16 12:11:29 +07:00
sulthan 82a11f255b chore: refresh the code graph for the comix browser path 2026-08-16 12:08:39 +07:00
sulthan f000cc7eaa docs(latest): correct the browser-site comments the comix change made stale
Review findings from #98: BrowserFetcher's doc still described two sites and
two read shapes, Get's doc enumerated them a second time, and the Fallback
field doc omitted comix. The comix fixture comment now records the live
in-tab body it was re-checked against.
2026-08-16 12:08:28 +07:00
sulthan 86160c164a feat(latest): poll comix through the browser sidecar (#98)
comix.to began answering plain-TLS fetches with a Cloudflare JavaScript
challenge on 2026-08-12, so every poll got a 403 interstitial and its cover
host static.comix.to is gated the same way. comix joins kagane and novelfull
as a browser Site: one registry entry, no plain-TLS fallback, and cover bytes
routed through the browser's image path behind a fully pinned URL pattern.

The read is an in-tab fetch of the Series URL, not a DOM render: comix is an
SPA, so rendering costs ~65 requests for the same server-rendered HTML one
fetch returns (24.5 KB, ~480 ms). Parsers and stored Series identity are
untouched.

Verified live against the real browser unit: page 24793 bytes in one fetch,
chapter 53, cover accepted by the pin and 26862 image bytes retrieved by
direct navigation (comix's Series page sets cross-origin-embedder-policy:
require-corp, so an in-page fetch of the cover host cannot work).
2026-08-16 12:05:11 +07:00
16 changed files with 1265 additions and 1006 deletions
+13 -5
View File
@@ -19,9 +19,9 @@ Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free
- Every site is its **own origin with its own `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync required, not optional. - Every site is its **own origin with its own `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync required, not optional.
- Userscript run in **isolated world**, so embedded API token safe from site's JS. - Userscript run in **isolated world**, so embedded API token safe from site's JS.
- Cloudflare's block on manga sites is **per-zone configuration plus request fingerprint, not IP reputation — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — a Site can turn its protection on overnight, which is exactly what comix.to did on 2026-08-12. An earlier version of this line blamed "Cloudflare's bot scoring"; that was wrong. The 1-99 bot score is Enterprise Bot Management only and does not exist for a free-plan zone, and no per-IP request rate is documented as an input to challenge issuance — `docs/research/cloudflare-bot-scoring-and-poll-cadence.md`. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test. - Cloudflare's block on manga sites is **per-zone configuration plus request fingerprint, not IP reputation — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — a Site can turn its protection on overnight, which is exactly what comix.to did on 2026-08-12. An earlier version of this line blamed "Cloudflare's bot scoring"; that was wrong. The 1-99 bot score is Enterprise Bot Management only and does not exist for a free-plan zone, and no per-IP request rate is documented as an input to challenge issuance — `docs/research/cloudflare-bot-scoring-and-poll-cadence.md`. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test.
- **kagane.to and novelfull.com are the exception to the above** — both sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`). When that's unset, kagane is skipped entirely (a plain fetch would only retrieve a challenge page) while novelfull pages are still attempted over plain TLS — its challenge is a live time-varying fact and its cover bytes never need the browser. The four other sites poll fine over plain TLS. - **kagane.to, comix.to and novelfull.com are the exception to the above** — all three sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`). When that's unset, kagane and comix are skipped entirely (a plain fetch would only retrieve a challenge page) while novelfull pages are still attempted over plain TLS — its challenge is a live time-varying fact and its cover bytes never need the browser. comix turned hostile on 2026-08-12 (#98): its cover host `static.comix.to` is gated too, so its cover bytes go through the browser as well, and its page is read as an in-tab `fetch()` of the series URL rather than a rendered DOM — comix is an SPA, and rendering costs ~65 requests for the same server-rendered HTML one fetch returns. The three other sites poll fine over plain TLS.
- **The CDP browser must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a non-UTC clock zone cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, ignoring the file's contents, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead. - **The CDP browser must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a non-UTC clock zone cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, ignoring the file's contents, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead.
- **The browser is not in the API stack and must not be put back.** It's its own compose unit (`chrome/docker-compose.yml`) on a second machine, reached over the tailnet — it held 471 MiB on a 1974 MiB swapless VPS, and a residential egress avoids the cloud-hosting-IP signature Bot Fight Mode documentedly challenges (ADR-0006; not a better "score" — free-plan zones have no score). Consequences that constrain code: `BROWSER_WS_URL` must be a tailnet **IP** (a MagicDNS name 500s at `/json/version`, same trap as the old Docker service name); the CDP port binds to the tailnet address only, since CDP authenticates nothing and that host has a real LAN; and the browser is on-demand (ADR-0005), so an unreachable or asleep one must degrade exactly as an unset `BROWSER_WS_URL` — plain-TLS libraries unaffected, kagane/novelfull logged and skipped, stored covers still served. Never add `chromedp.NoModifyURL`: discovery per fetch is what makes a restarted Chrome invisible. - **The browser is not in the API stack and must not be put back.** It's its own compose unit (`chrome/docker-compose.yml`) on a second machine, reached over the tailnet — it held 471 MiB on a 1974 MiB swapless VPS, and a residential egress avoids the cloud-hosting-IP signature Bot Fight Mode documentedly challenges (ADR-0006; not a better "score" — free-plan zones have no score). Consequences that constrain code: `BROWSER_WS_URL` must be a tailnet **IP** (a MagicDNS name 500s at `/json/version`, same trap as the old Docker service name); the CDP port binds to the tailnet address only, since CDP authenticates nothing and that host has a real LAN; and the browser is on-demand (ADR-0005), so an unreachable or asleep one must degrade exactly as an unset `BROWSER_WS_URL` — plain-TLS libraries unaffected, kagane/comix logged and skipped, stored covers still served. Never add `chromedp.NoModifyURL`: discovery per fetch is what makes a restarted Chrome invisible.
- **UTC is the tell, not a country mismatch.** A UTC clock is the datacenter default, and the challenge refuses it; any real zone clears. Measured 2026-08-08, identical container, one Indonesian egress IP: UTC never cleared in 60s (twice), while `Asia/Jakarta` **and** `America/New_York` both cleared in 4s. An earlier note here claimed the zone had to match the egress IP's country — that was wrong, inferred from the host clock (`Asia/Bangkok`) rather than the measured egress. A second earlier claim, that Cloudflare "scores" a UTC clock, was also wrong: the measurement is real but the mechanism is not documented anywhere — Cloudflare publishes no timezone signal, and free-plan zones carry no score at all. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one. - **UTC is the tell, not a country mismatch.** A UTC clock is the datacenter default, and the challenge refuses it; any real zone clears. Measured 2026-08-08, identical container, one Indonesian egress IP: UTC never cleared in 60s (twice), while `Asia/Jakarta` **and** `America/New_York` both cleared in 4s. An earlier note here claimed the zone had to match the egress IP's country — that was wrong, inferred from the host clock (`Asia/Bangkok`) rather than the measured egress. A second earlier claim, that Cloudflare "scores" a UTC clock, was also wrong: the measurement is real but the mechanism is not documented anywhere — Cloudflare publishes no timezone signal, and free-plan zones carry no score at all. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one.
- **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives. - **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives.
@@ -45,12 +45,20 @@ Backend (`cd backend`):
- Single test: `go test -run TestName ./...` - Single test: `go test -run TestName ./...`
- Build static binary: `CGO_ENABLED=0 go build` - Build static binary: `CGO_ENABLED=0 go build`
Local stack: `docker compose up` (bookmark-api + postgres only; `postgres-data` named volume, `restart: unless-stopped`). No browser — without `BROWSER_WS_URL` the poller logs and skips kagane and novelfull. To run one: `cd chrome && BROWSER_BIND_ADDR=172.17.0.1 docker compose up -d --build`, then `BROWSER_WS_URL=ws://172.17.0.1:9222` in the root `.env` (bridge gateway, so the API container can name it by IP). Local stack: `docker compose up` (bookmark-api + postgres only; `postgres-data` named volume, `restart: unless-stopped`). No browser — without `BROWSER_WS_URL` the poller logs and skips kagane and comix. To run one: `cd chrome && BROWSER_BIND_ADDR=172.17.0.1 docker compose up -d --build`, then `BROWSER_WS_URL=ws://172.17.0.1:9222` in the root `.env` (bridge gateway, so the API container can name it by IP).
Live CDP proof (needs that browser and network, skipped otherwise): Live CDP proof (needs that browser and network, skipped otherwise):
`SMOKE_BROWSER_WS_URL=ws://<ip>:<port> go test -run TestSmokeKagane ./internal/latest` `SMOKE_BROWSER_WS_URL=ws://<ip>:<port> go test -run 'TestSmokeKagane|TestSmokeComix' ./internal/latest`
— fetches a real kagane cover and chapter list. A red run means the challenge is — fetches a real kagane and comix cover and chapter list. A red run means the challenge is
not clearing from this IP, which is a live fact to re-check, not necessarily a defect. not clearing from this IP, which is a live fact to re-check, not necessarily a defect.
A red `TestSmokeComix` reporting `ERR_CERT_COMMON_NAME_INVALID` is not the
challenge: it means the resolver the browser container uses hijacks `comix.to`.
Observed 2026-08-16 on one Indonesian ISP, which CNAMEs it to a block page
(`aduankonten.id`). Check with `docker exec <browser> getent hosts comix.to`,
and if it is hijacked, run the container with
`--add-host comix.to:<ip> --add-host static.comix.to:<ip>` from a DoH lookup
(`curl -H 'accept: application/dns-json' 'https://1.1.1.1/dns-query?name=comix.to&type=A'`).
Machine-local, so don't put those hosts in `chrome/docker-compose.yml`.
Smoke test: `curl` endpoints with `Authorization: Bearer <token>`; confirm `OPTIONS` preflight return CORS headers and `/healthz` return 200. Smoke test: `curl` endpoints with `Authorization: Bearer <token>`; confirm `OPTIONS` preflight return CORS headers and `/healthz` return 200.
+2 -2
View File
@@ -436,8 +436,8 @@ free -m # the Gitea runner should still have its headroom
Nothing here needs doing during an API redeploy. The API stack does not Nothing here needs doing during an API redeploy. The API stack does not
`depends_on` the browser, and an unreachable one degrades exactly as an unset `depends_on` the browser, and an unreachable one degrades exactly as an unset
`BROWSER_WS_URL`: plain-TLS libraries unaffected, kagane and novelfull logged `BROWSER_WS_URL`: plain-TLS libraries unaffected, kagane and comix logged
and skipped, stored covers still served. and skipped, novelfull attempted over plain TLS, stored covers still served.
--- ---
+29 -20
View File
@@ -89,12 +89,15 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
`Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list `Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
order — is never touched. order — is never touched.
Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth
against fingerprint-based blocking; any failure log and skip. kagane and against fingerprint-based blocking; any failure log and skip. kagane, comix
novelfull sit behind Cloudflare JavaScript challenges the TLS client can't and novelfull sit behind Cloudflare JavaScript challenges the TLS client
clear, so they are fetched over CDP via `BROWSER_WS_URL`; kagane is simply can't clear, so they are fetched over CDP via `BROWSER_WS_URL`; kagane and
not polled when that's unset, while novelfull falls back to a plain-TLS comix are simply not polled when that's unset, while novelfull falls back to
attempt — its challenge is a live time-varying fact, and its cover bytes a plain-TLS attempt — its challenge is a live time-varying fact, and its
never need the browser. See cover bytes never need the browser. comix's browser read is an in-tab
`fetch()` of the Series URL, not a DOM render: it is an SPA, so rendering
costs ~65 requests for the same server-rendered HTML one fetch returns
(measured 2026-08-12, issue #98). See
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`. `docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
The poller's series write is a single-column UPDATE The poller's series write is a single-column UPDATE
(`Store.SetLatestChapter`), not a read-modify-write of the whole bookmark: (`Store.SetLatestChapter`), not a read-modify-write of the whole bookmark:
@@ -112,14 +115,17 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
is public and uncredentialed: the userscript renders it on a Site's origin, is public and uncredentialed: the userscript renders it on a Site's origin,
where no cookie or token of ours travels. A client-sent `cover` is decoded where no cookie or token of ours travels. A client-sent `cover` is decoded
and discarded, permanently (ADR-0004 compatibility). and discarded, permanently (ADR-0004 compatibility).
Browser-backed Sites join the same pipeline (issue #62): kagane pages *and* Browser-backed Sites join the same pipeline (issue #62, extended to comix by
cover bytes go through the browser sidecar (nothing falls back to a plain #98): kagane and comix pages *and* cover bytes go through the browser sidecar
fetch, which would only retrieve a challenge page), while novelfull needs (nothing falls back to a plain fetch, which would only retrieve a challenge
the browser only for its HTML — the cover URL comes out of the page), while novelfull needs the browser only for its HTML — the cover URL
browser-fetched page and the bytes go over plain TLS. With no browser comes out of the browser-fetched page and the bytes go over plain TLS. With
configured, kagane Covers are simply absent; novelfull still gets one — at no browser configured, kagane and comix Covers are simply absent; novelfull
creation and on the poll — when its page body happens to answer a plain still gets one — at creation and on the poll — when its page body happens to
request (the challenge is a live time-varying fact). The old kagane-only answer a plain request (the challenge is a live time-varying fact). comix
cover bytes must arrive by direct navigation, not an in-page fetch: its
Series page sets `cross-origin-embedder-policy: require-corp`, which fails a
page-context fetch of `static.comix.to`. The old kagane-only
serving path (`/img/kagane/{id}`, template rewrite, `CoverFetcher`) is gone serving path (`/img/kagane/{id}`, template rewrite, `CoverFetcher`) is gone
(issue #63): the one public route serves every Site. (issue #63): the one public route serves every Site.
- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4. - **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
@@ -164,10 +170,11 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
Reader's credential at serve time). Reader's credential at serve time).
`BROWSER_WS_URL` (CDP endpoint of the browser, which runs on a **separate `BROWSER_WS_URL` (CDP endpoint of the browser, which runs on a **separate
machine** and is reached over the tailnet — ADR-0006, `chrome/docker-compose.yml`. machine** and is reached over the tailnet — ADR-0006, `chrome/docker-compose.yml`.
Used by the poller for kagane and novelfull page fetches and by the cover Used by the poller for kagane, comix and novelfull page fetches and by the
pipeline for kagane's image bytes (the browser is the only route that clears cover pipeline for kagane's and comix's image bytes (the browser is the only
the challenge kagane serves its covers behind); unset — the default — route that clears the challenge those two serve their covers behind); unset —
disables browser polling and leaves kagane Covers blank until stored bytes the default — disables browser polling and leaves kagane and comix Covers
blank until stored bytes
exist. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s exist. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s
`/json/version` for any Host that isn't an IP or `localhost`). `/json/version` for any Host that isn't an IP or `localhost`).
- **No per-Site cover path (issue #63):** every Cover — all six Sites — is - **No per-Site cover path (issue #63):** every Cover — all six Sites — is
@@ -175,8 +182,10 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
bytes. There is no proxy, no per-Site rewrite, no second place that decides bytes. There is no proxy, no per-Site rewrite, no second place that decides
a Cover's renderable address: the wire `cover` is it. The only place a Site a Cover's renderable address: the wire `cover` is it. The only place a Site
name still appears in cover code is the extraction module (`latest`), where name still appears in cover code is the extraction module (`latest`), where
kagane's image URLs are claimed by `browserOnlyCoverURL` — they answer a kagane's and comix's image URLs are claimed by `browserOnlyCoverURL` — kagane
plain fetch with a challenge and `cross-origin-resource-policy: same-origin`; answers a plain fetch with a challenge and
`cross-origin-resource-policy: same-origin`, and `static.comix.to` answers
one with the same Cloudflare challenge its pages serve;
every other Site's CDN answers plain TLS. Templates render `.Cover` — the every other Site's CDN answers plain TLS. Templates render `.Cover` — the
wire value — never anything else. wire value — never anything else.
- **Web UI also owns:** session-gated `GET /install/{manga,novel}-bookmark.user.js` - **Web UI also owns:** session-gated `GET /install/{manga,novel}-bookmark.user.js`
+60 -25
View File
@@ -24,12 +24,16 @@ const challengeTimeout = 45 * time.Second
var kaganeSeriesRe = regexp.MustCompile(`^/series/([0-9a-f-]{36})/?$`) var kaganeSeriesRe = regexp.MustCompile(`^/series/([0-9a-f-]{36})/?$`)
// comixSeriesPathRe matches the one path shape comixRead will open: a Series
// page, "/title/<id>-<slug>". Verified live 2026-08-12.
var comixSeriesPathRe = regexp.MustCompile(`^/title/[^/?#]+/?$`)
// BrowserFetcher retrieves pages through a remote headless Chrome over the // BrowserFetcher retrieves pages through a remote headless Chrome over the
// DevTools Protocol. // DevTools Protocol.
// //
// It exists for one reason: kagane.to and novelfull.com sit behind a // It exists for one reason: kagane.to, novelfull.com and comix.to sit behind a
// Cloudflare JavaScript challenge. Verified 2026-08-03 (kagane) and 2026-08-05 // Cloudflare JavaScript challenge. Verified 2026-08-03 (kagane), 2026-08-05
// (novelfull) from the deployment host, plain HTTP and bogdanfinn/tls-client // (novelfull) and 2026-08-12 (comix), plain HTTP and bogdanfinn/tls-client
// with a Chrome_133 profile both get 403 with cf-mitigated: challenge on every // with a Chrome_133 profile both get 403 with cf-mitigated: challenge on every
// path, including the API, robots.txt and images. Clearing it requires // path, including the API, robots.txt and images. Clearing it requires
// executing the challenge script, which only a real browser does. // executing the challenge script, which only a real browser does.
@@ -40,10 +44,11 @@ var kaganeSeriesRe = regexp.MustCompile(`^/series/([0-9a-f-]{36})/?$`)
// sync that break silently and separately. The browser's own cookie jar // sync that break silently and separately. The browser's own cookie jar
// persists across polls, so the challenge is solved once every few hours. // persists across polls, so the challenge is solved once every few hours.
// //
// The two sites differ in how the chapter list is read: kagane serves it from // The three sites differ in what a cleared tab is asked for: kagane fetches a
// a JSON API that must be called from inside the page (so the request carries // JSON API from inside the page (the list exists nowhere else), comix fetches
// the clearance cookie), while novelfull renders it into the HTML so the // its own Series URL from inside the page (the served HTML carries the facts,
// cleared DOM is the payload. // and rendering the SPA costs ~65 requests instead of one), and novelfull
// renders its list into the HTML so the cleared DOM is the payload.
type BrowserFetcher struct { type BrowserFetcher struct {
allocCtx context.Context allocCtx context.Context
cancel context.CancelFunc cancel context.CancelFunc
@@ -87,10 +92,9 @@ func (f *BrowserFetcher) Close() {
} }
// Get navigates to seriesURL, lets any challenge resolve, then reads the // Get navigates to seriesURL, lets any challenge resolve, then reads the
// payload the Site's registry entry describes — kagane's chapter-list API from // payload the Site's registry entry describes (the shapes are listed on
// inside the page so the request carries the clearance cookie, novelfull's // BrowserFetcher). The returned body is whatever the Site's chapter list lives
// served HTML. The returned body is whatever the Site's chapter list lives in, // in, which is what the entry's LatestChapter parse expects.
// which is what the entry's LatestChapter parse expects.
func (f *BrowserFetcher) Get(ctx context.Context, seriesURL string) (string, int, error) { func (f *BrowserFetcher) Get(ctx context.Context, seriesURL string) (string, int, error) {
var body string var body string
// Sorted order (browserBackedSites sorts) makes dispatch deterministic: // Sorted order (browserBackedSites sorts) makes dispatch deterministic:
@@ -141,29 +145,47 @@ func novelfullRead(seriesURL string, out *string) (chromedp.Action, bool) {
return chromedp.OuterHTML("html", out, chromedp.ByQuery), true return chromedp.OuterHTML("html", out, chromedp.ByQuery), true
} }
// comixRead fetches the Series page from inside the cleared tab. comix is an
// SPA: rendering the page costs ~65 requests, while one same-origin fetch of
// the same address returns the server-rendered HTML — 24.5 KB, ~480 ms,
// carrying both parser anchors (measured 2026-08-12, issue #98). So this is
// kaganeRead's shape, not novelfullRead's, even though the payload is HTML.
// Refusing any other address is the per-Site half of the SSRF gate.
func comixRead(seriesURL string, out *string) (chromedp.Action, bool) {
pageURL, ok := comixSeriesPageURL(seriesURL)
if !ok {
return nil, false
}
return chromedp.Evaluate(
`fetch(`+jsString(pageURL)+`).then(r => r.ok ? r.text() : "")`,
out, awaitPromise), true
}
// Image retrieves one cover's bytes through the browser sidecar, and its // Image retrieves one cover's bytes through the browser sidecar, and its
// content type. // content type.
// //
// It exists because kagane serves covers behind the same challenge as its // It exists because kagane and comix serve covers behind the same challenge as
// pages *and* with `cross-origin-resource-policy: same-origin`, so the bytes // their pages — kagane additionally with
// are only reachable from inside a browser that already holds the clearance // `cross-origin-resource-policy: same-origin` — so the bytes are only
// cookie (verified 2026-08-08). Acquisition through the sidecar is the only // reachable from inside a browser that already holds the clearance cookie
// route. // (verified 2026-08-08 for kagane, 2026-08-12 for comix). Acquisition through
// the sidecar is the only route.
// //
// The image URL is navigated to rather than fetched from some other kagane // The image URL is navigated to rather than fetched from another page of the
// page: the challenge only runs on a top-level navigation, and once it clears // Site: the challenge only runs on a top-level navigation, and once it clears
// the document *is* the image, so a same-origin fetch of location.href reads // the document *is* the image, so a same-origin fetch of location.href reads
// it straight back out of the cache. // it straight back out of the cache. For comix the navigation is also the only
// route that works at all — its Series page sets
// `cross-origin-embedder-policy: require-corp`, which fails a page-context
// fetch of the cover host.
// //
// The challenge is not solved by the first read: WaitReady("body") is satisfied // The challenge is not solved by the first read: WaitReady("body") is satisfied
// by the interstitial too. run holds the tab open until the in-page fetch // by the interstitial too. run holds the tab open until the in-page fetch
// succeeds, which is what gives the challenge script the seconds it needs. // succeeds, which is what gives the challenge script the seconds it needs.
func (f *BrowserFetcher) Image(ctx context.Context, imageURL string) ([]byte, string, error) { func (f *BrowserFetcher) Image(ctx context.Context, imageURL string) ([]byte, string, error) {
m := kaganeImageURLRe.FindStringSubmatch(imageURL) if !browserOnlyCoverURL(imageURL) {
if m == nil {
return nil, "", fmt.Errorf("not a browser-fetchable cover url: %q", imageURL) return nil, "", fmt.Errorf("not a browser-fetchable cover url: %q", imageURL)
} }
imageID := m[1]
var dataURL string var dataURL string
err := f.run(ctx, imageURL, err := f.run(ctx, imageURL,
chromedp.Evaluate(`fetch(location.href).then(r => r.ok chromedp.Evaluate(`fetch(location.href).then(r => r.ok
@@ -175,16 +197,16 @@ func (f *BrowserFetcher) Image(ctx context.Context, imageURL string) ([]byte, st
: "")`, &dataURL, awaitPromise), : "")`, &dataURL, awaitPromise),
func() bool { return dataURL != "" }) func() bool { return dataURL != "" })
if err != nil { if err != nil {
return nil, "", fmt.Errorf("browser image %s: %w", imageID, err) return nil, "", fmt.Errorf("browser image %s: %w", imageURL, err)
} }
// "data:image/webp;base64,<payload>". // "data:image/webp;base64,<payload>".
head, payload, ok := strings.Cut(dataURL, ";base64,") head, payload, ok := strings.Cut(dataURL, ";base64,")
if !ok { if !ok {
return nil, "", fmt.Errorf("browser image %s: not a data url", imageID) return nil, "", fmt.Errorf("browser image %s: not a data url", imageURL)
} }
raw, err := base64.StdEncoding.DecodeString(payload) raw, err := base64.StdEncoding.DecodeString(payload)
if err != nil { if err != nil {
return nil, "", fmt.Errorf("browser image %s: %w", imageID, err) return nil, "", fmt.Errorf("browser image %s: %w", imageURL, err)
} }
return raw, strings.TrimPrefix(head, "data:"), nil return raw, strings.TrimPrefix(head, "data:"), nil
} }
@@ -323,6 +345,19 @@ func novelfullSeriesURL(seriesURL string) bool {
strings.HasSuffix(u.Path, ".html") strings.HasSuffix(u.Path, ".html")
} }
// comixSeriesPageURL returns the address comixRead fetches inside the tab: the
// Series page itself, rebuilt from the pinned host and path so nothing else
// travels. Host-pinned here for the same reason kagane's is — series_url is
// client-supplied and a headless browser is a strong SSRF primitive.
func comixSeriesPageURL(seriesURL string) (string, bool) {
u, err := url.Parse(seriesURL)
if err != nil || u.Scheme != "https" || u.Hostname() != "comix.to" ||
!comixSeriesPathRe.MatchString(u.Path) {
return "", false
}
return "https://comix.to" + u.Path, true
}
// awaitPromise makes Evaluate resolve the promise rather than returning a // awaitPromise makes Evaluate resolve the promise rather than returning a
// serialised Promise object. // serialised Promise object.
func awaitPromise(p *runtime.EvaluateParams) *runtime.EvaluateParams { func awaitPromise(p *runtime.EvaluateParams) *runtime.EvaluateParams {
+56
View File
@@ -61,6 +61,62 @@ func TestNovelfullSeriesURL(t *testing.T) {
}) })
} }
} }
func TestComixSeriesPageURL(t *testing.T) {
const series = "https://comix.to/title/n8we-dungeons-and-crayons"
cases := []struct {
name string
url string
want string
}{
{"series page", series, series},
{"trailing slash kept", series + "/", series + "/"},
// Query and fragment are dropped: only the pinned path travels.
{"query dropped", series + "?tab=chapters", series},
{"foreign host", "https://evil.example/title/x", ""},
{"lookalike host", "https://comix.to.evil.example/title/x", ""},
{"not https", "http://comix.to/title/x", ""},
{"not a series path", "https://comix.to/search", ""},
{"chapter page", series + "/11139891-chapter-80", ""},
{"garbage", "://nope", ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, ok := comixSeriesPageURL(tc.url)
if ok != (tc.want != "") || got != tc.want {
t.Fatalf("comixSeriesPageURL(%q) = %q, %v; want %q", tc.url, got, ok, tc.want)
}
})
}
}
// The browser is an SSRF primitive and a cover address can originate in a
// client-supplied PUT body, so this gate decides what it may navigate to.
func TestBrowserOnlyCoverURL(t *testing.T) {
cases := []struct {
url string
want bool
}{
{"https://static.comix.to/039d/i/1/34/6a6742bf15736@280.jpg", true},
{"https://kagane.to/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed", true},
// Every other Site's CDN answers plain TLS.
{"https://gg.asuracomic.net/covers/x.webp", false},
{"http://static.comix.to/039d/x.jpg", false},
{"https://static.comix.to.evil.example/039d/x.jpg", false},
{"https://evil.example/static.comix.to/x.jpg", false},
{"https://static.comix.to/039d/x.jpg?next=http://169.254.169.254/", false},
{"https://static.comix.to/039d/x.svg", false},
{"https://static.comix.to/../etc/passwd.jpg", false},
{"https://static.comix.to/", false},
}
for _, tc := range cases {
t.Run(tc.url, func(t *testing.T) {
if got := browserOnlyCoverURL(tc.url); got != tc.want {
t.Fatalf("browserOnlyCoverURL(%q) = %v, want %v", tc.url, got, tc.want)
}
})
}
}
func TestClassifyBrowserInterruption(t *testing.T) { func TestClassifyBrowserInterruption(t *testing.T) {
if err := classifyBrowserError(context.Background(), true, context.Canceled); !errors.Is(err, errBrowserInterrupted) { if err := classifyBrowserError(context.Background(), true, context.Canceled); !errors.Is(err, errBrowserInterrupted) {
t.Fatalf("classifyBrowserError(context.Canceled) = %v, want browser interruption", err) t.Fatalf("classifyBrowserError(context.Canceled) = %v, want browser interruption", err)
+2 -1
View File
@@ -25,7 +25,8 @@ type CoverBytesFetcher interface {
// fetchCoverBytes routes a cover's byte retrieval by URL shape, not by Site // fetchCoverBytes routes a cover's byte retrieval by URL shape, not by Site
// name: the browser fetcher's module claims the addresses only it can fetch // name: the browser fetcher's module claims the addresses only it can fetch
// (kagane's image route answers a plain fetch with a challenge and // (kagane's image route answers a plain fetch with a challenge and
// `cross-origin-resource-policy: same-origin`), and everything else goes over // `cross-origin-resource-policy: same-origin`, static.comix.to answers one with
// the same challenge its pages serve), and everything else goes over
// plain TLS. Missing fetchers degrade to an error the caller logs, never a // plain TLS. Missing fetchers degrade to an error the caller logs, never a
// fallback onto a path that cannot succeed. One routing rule for the poll and // fallback onto a path that cannot succeed. One routing rule for the poll and
// the acquirer, so the two cannot drift apart. // the acquirer, so the two cannot drift apart.
+5 -5
View File
@@ -17,8 +17,8 @@ type Fetcher interface {
} }
// BrowserCoverFetcher retrieves one cover's bytes through the browser-backed // BrowserCoverFetcher retrieves one cover's bytes through the browser-backed
// path — the only route that clears the challenge kagane's image URLs answer // path — the only route that clears the challenge kagane's and comix's image
// a plain fetch with. Satisfied by BrowserFetcher. // URLs answer a plain fetch with. Satisfied by BrowserFetcher.
type BrowserCoverFetcher interface { type BrowserCoverFetcher interface {
Image(ctx context.Context, imageURL string) (body []byte, contentType string, err error) Image(ctx context.Context, imageURL string) (body []byte, contentType string, err error)
} }
@@ -119,9 +119,9 @@ func (p *Poller) storeCover(ctx context.Context, sr store.Series, sourceURL stri
// fetcherFor returns the fetcher a site's page needs, or nil when the site // fetcherFor returns the fetcher a site's page needs, or nil when the site
// cannot be fetched at all right now. A Site whose registry entry carries a // cannot be fetched at all right now. A Site whose registry entry carries a
// Browser read — kagane and novelfull, both behind a Cloudflare JavaScript // Browser read — kagane, comix and novelfull, all behind a Cloudflare
// challenge no TLS fingerprint clears — prefers the browser; when it is // JavaScript challenge no TLS fingerprint clears — prefers the browser; when it
// absent, the entry's Fallback decides whether plain TLS may take over. One // is absent, the entry's Fallback decides whether plain TLS may take over. One
// routing rule for the poll and the acquirer, so the two cannot drift apart. // routing rule for the poll and the acquirer, so the two cannot drift apart.
func fetcherFor(site string, browser, tls Fetcher) Fetcher { func fetcherFor(site string, browser, tls Fetcher) Fetcher {
s, known := sites[site] s, known := sites[site]
+80
View File
@@ -762,6 +762,86 @@ func TestKaganeUsesBrowserFetcher(t *testing.T) {
} }
} }
// comix joined kagane behind the challenge on 2026-08-12 (#98): its page goes
// to the browser, its Cover bytes go through the browser's image route because
// static.comix.to is gated the same way, and the TLS fetcher is never asked
// for either.
func TestComixUsesBrowserFetcher(t *testing.T) {
s, dbURL := newTestStore(t)
const (
key = "comix:n8we-dungeons-and-crayons"
seriesID = "n8we-dungeons-and-crayons"
seriesURL = "https://comix.to/title/n8we-dungeons-and-crayons"
coverURL = "https://static.comix.to/039d/i/1/34/6a6742bf15736@280.jpg"
)
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key, Site: "comix", SeriesID: seriesID, SeriesURL: seriesURL,
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
seedCoverSource(t, dbURL, "comix", seriesID, coverURL)
tlsF := &fakeFetcher{body: "", status: 200}
browserF := &fakeFetcher{body: comixSeriesFixture, status: 200}
covers := &fakeCoverFetcher{body: []byte("cover-bytes"), contentType: "image/jpeg"}
tlsCovers := &fakeBytesCoverFetcher{body: []byte("tls-bytes"), contentType: "image/jpeg"}
p := &Poller{
Store: s, Fetch: tlsF, BrowserFetch: browserF,
CoverFetch: covers, CoverBytesFetch: tlsCovers,
Now: func() time.Time { return time.UnixMilli(5_000_000) },
Cooldown: time.Hour, BrowserCooldown: time.Hour,
Interval: time.Hour, Batch: 10,
}
p.runOnce(context.Background())
if len(tlsF.calls) != 0 {
t.Errorf("TLS fetcher was called for comix: %v", tlsF.calls)
}
if len(browserF.calls) != 1 {
t.Fatalf("browser fetcher calls = %v, want 1", browserF.calls)
}
if got := tlsCovers.callCount(); got != 0 {
t.Errorf("TLS cover fetches = %d, want 0: static.comix.to answers a challenge", got)
}
if got := covers.callCount(); got != 1 {
t.Fatalf("browser cover fetches = %d, want 1", got)
}
got, found, err := s.Get(s.OwnerID(), key)
if err != nil || !found {
t.Fatalf("Get: %v found=%v", err, found)
}
if got.LatestChapterNum == nil || *got.LatestChapterNum != 80 {
t.Errorf("LatestChapterNum = %v, want 80", got.LatestChapterNum)
}
}
// Without a browser, comix is skipped outright rather than handed to plain
// TLS: a plain fetch retrieves only a challenge page (measured 2026-08-12).
func TestComixSkippedWhenNoBrowserFetcher(t *testing.T) {
s, _ := newTestStore(t)
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: "comix:n8we-dungeons-and-crayons", Site: "comix",
SeriesID: "n8we-dungeons-and-crayons",
SeriesURL: "https://comix.to/title/n8we-dungeons-and-crayons",
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
f := &fakeFetcher{body: comixSeriesFixture, status: 200}
p := &Poller{
Store: s, Fetch: f,
Now: func() time.Time { return time.UnixMilli(5_000_000) },
Cooldown: time.Hour, BrowserCooldown: time.Hour,
Interval: time.Hour, Batch: 10,
}
p.runOnce(context.Background())
if len(f.calls) != 0 {
t.Errorf("TLS fetcher was called for comix: %v", f.calls)
}
}
func TestRunOncePrefetchesKaganeCover(t *testing.T) { func TestRunOncePrefetchesKaganeCover(t *testing.T) {
s, dbURL := newTestStore(t) s, dbURL := newTestStore(t)
const ( const (
+28 -8
View File
@@ -47,9 +47,9 @@ type browserRead struct {
// Done reports whether the payload arrived. // Done reports whether the payload arrived.
Done func(body string) bool Done func(body string) bool
// Fallback allows the plain-TLS fetcher when no browser is configured. // Fallback allows the plain-TLS fetcher when no browser is configured.
// False skips the Site instead. kagane is false — a plain fetch would // False skips the Site instead. kagane and comix are false — a plain fetch
// only ever retrieve a challenge page — and novelfull is true, because // would only ever retrieve a challenge page — and novelfull is true,
// its challenge is a live time-varying fact (AGENTS.md). // because its challenge is a live time-varying fact (AGENTS.md).
Fallback bool Fallback bool
} }
@@ -248,14 +248,25 @@ var comixInitialDataRe = regexp.MustCompile(`(?is)<script\b[^>]*\bid\s*=\s*["']i
// supplied, and a headless browser is a strong SSRF primitive. // supplied, and a headless browser is a strong SSRF primitive.
var kaganeImageURLRe = regexp.MustCompile(`^https://kagane\.to/api/v2/image/([0-9a-f-]{36})/compressed$`) var kaganeImageURLRe = regexp.MustCompile(`^https://kagane\.to/api/v2/image/([0-9a-f-]{36})/compressed$`)
// comixImageURLRe matches comix's cover host and path shape. Pinned in full
// (scheme, host, path characters, image extension) for the same reason
// kaganeImageURLRe is: the address reaches a headless browser, and it can
// originate in a client-supplied PUT body. No dot is allowed inside the path,
// so no traversal or second extension can hide in it. Shape from a live page,
// 2026-08-10: /039d/i/1/34/6a6742bf15736@280.jpg.
var comixImageURLRe = regexp.MustCompile(`^https://static\.comix\.to/[A-Za-z0-9@/_-]+\.(?:jpg|jpeg|png|webp)$`)
// browserOnlyCoverURL reports whether the browser sidecar is the only fetcher // browserOnlyCoverURL reports whether the browser sidecar is the only fetcher
// for cover bytes at imageURL. kagane's image route answers a plain fetch with // for cover bytes at imageURL. kagane's image route answers a plain fetch with
// a challenge and `cross-origin-resource-policy: same-origin`, so a TLS fetch // a challenge and `cross-origin-resource-policy: same-origin`, and
// would only ever retrieve a challenge page and must not be attempted // static.comix.to answers one with the same Cloudflare challenge its pages
// (ADR-0007). This is the byte-fetch router's per-Site knowledge; it lives in // serve (measured 2026-08-12, issue #98), so a TLS fetch would only ever
// the extraction module, which owns kagane's URL shapes. // retrieve a challenge page and must not be attempted (ADR-0007). This is the
// byte-fetch router's per-Site knowledge; it lives in the extraction module,
// which owns those URL shapes.
func browserOnlyCoverURL(imageURL string) bool { func browserOnlyCoverURL(imageURL string) bool {
return kaganeImageURLRe.MatchString(imageURL) return kaganeImageURLRe.MatchString(imageURL) ||
comixImageURLRe.MatchString(imageURL)
} }
// kagane's browser-fetched series response publishes cover image IDs under // kagane's browser-fetched series response publishes cover image IDs under
@@ -389,6 +400,15 @@ var sites = map[string]site{
Host: "comix.to", Host: "comix.to",
LatestChapter: comixLatestChapter, LatestChapter: comixLatestChapter,
Cover: comixCoverEntry, Cover: comixCoverEntry,
Browser: &browserRead{
Read: comixRead,
// The interstitial is served in place of the page, so "arrived"
// has to exclude it explicitly, as novelfull's does.
Done: func(body string) bool { return body != "" && !isInterstitial(body) },
// Never falls back: a plain fetch of a comix page or cover
// retrieves only a challenge page (measured 2026-08-12).
Fallback: false,
},
}, },
"kagane": { "kagane": {
Host: "kagane.to", Host: "kagane.to",
+7
View File
@@ -41,6 +41,13 @@ const challengeFixture = `<!DOCTYPE html><html><head><title>Just a moment...</ti
// https://comix.to/title/n8we-dungeons-and-crayons fetched 2026-08-03. comix is // https://comix.to/title/n8we-dungeons-and-crayons fetched 2026-08-03. comix is
// an SPA: the page ships a JSON state blob rather than a list of chapter // an SPA: the page ships a JSON state blob rather than a list of chapter
// anchors, and latestChapterUrl is where the newest chapter actually lives. // anchors, and latestChapterUrl is where the newest chapter actually lives.
//
// Still the right fixture after comix moved behind the challenge (#98): the
// browser read is an in-tab fetch of the Series URL, so the body a poll parses
// is this same server-rendered HTML, not a rendered DOM. Confirmed against a
// live cleared tab 2026-08-16 (TestSmokeComix): the in-tab fetch returned
// 24793 bytes of server-rendered HTML that these same parses read a chapter
// and a cover out of.
const comixSeriesFixture = ` const comixSeriesFixture = `
{"firstChapterUrl":"/title/n8we-dungeons-and-crayons/5038739-chapter-1","latestChapterUrl":"/title/n8we-dungeons-and-crayons/11139891-chapter-80"}, {"firstChapterUrl":"/title/n8we-dungeons-and-crayons/5038739-chapter-1","latestChapterUrl":"/title/n8we-dungeons-and-crayons/11139891-chapter-80"},
{""manga","recommended","n8we",1]":{"items":[{"latestChapterUrl":"/title/qqwrm-full-time-awakening/99999999-chapter-999"}]} {""manga","recommended","n8we",1]":{"items":[{"latestChapterUrl":"/title/qqwrm-full-time-awakening/99999999-chapter-999"}]}
@@ -0,0 +1,72 @@
package latest
import (
"context"
"os"
"testing"
"time"
"bookmarkmanager/backend/internal/store"
)
// TestSmokeComix answers "is comix's challenge clearing from this browser right
// now" — a live, time-varying fact, so a red run is something to re-check
// before it is a defect. Needs the real browser unit with outbound network:
//
// cd chrome && BROWSER_BIND_ADDR=127.0.0.1 docker compose up -d --build
// SMOKE_BROWSER_WS_URL=ws://127.0.0.1:9222 go test -run TestSmokeComix ./internal/latest
//
// It walks the whole read: the in-tab page fetch, both parses, and the Cover
// bytes by direct navigation to static.comix.to. The Cover address comes out of
// the page rather than being pinned in the test, because a stored one rots.
func TestSmokeComix(t *testing.T) {
ws := os.Getenv("SMOKE_BROWSER_WS_URL")
if ws == "" {
t.Skip("SMOKE_BROWSER_WS_URL unset")
}
const seriesURL = "https://comix.to/title/m12d-classmate"
f, err := NewBrowserFetcher(ws)
if err != nil {
t.Fatalf("NewBrowserFetcher: %v", err)
}
defer f.Close()
ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
defer cancel()
body, status, err := f.Get(ctx, seriesURL)
if err != nil {
t.Fatalf("Get: %v", err)
}
t.Logf("status=%d bytes=%d", status, len(body))
if status != 200 {
t.Fatalf("status = %d, want 200 — the sidecar is not clearing the challenge", status)
}
chapter, ok := latestChapterFrom("comix", seriesURL, body)
if !ok {
t.Fatalf("no latest chapter in %d bytes — page shape changed", len(body))
}
t.Logf("latest chapter: %v %q", chapter.Num, chapter.Label)
cover, ok := coverFrom("comix", seriesURL, body)
if !ok {
t.Fatalf("no cover address in %d bytes — page shape changed", len(body))
}
t.Logf("cover: %s", cover)
if !browserOnlyCoverURL(cover) {
t.Fatalf("cover %q is not claimed by the browser gate: the pin and the live URL shape disagree", cover)
}
bytes, contentType, err := f.Image(ctx, cover)
if err != nil {
t.Fatalf("Image: %v", err)
}
if len(bytes) < 1000 {
t.Fatalf("cover is %d bytes, want a real image", len(bytes))
}
t.Logf("fetched %d bytes of %s", len(bytes), contentType)
if _, ok := store.CoverContentType(contentType); !ok {
t.Fatalf("content type %q is not storable", contentType)
}
}
+5 -17
View File
@@ -18,16 +18,18 @@
"16": "Cloudflare bot scoring and poll cadence — what is actually documented", "16": "Cloudflare bot scoring and poll cadence — what is actually documented",
"17": "Go Code Style Guide", "17": "Go Code Style Guide",
"18": "Agent Skills", "18": "Agent Skills",
"19": "Store",
"20": "I/O Performance Patterns", "20": "I/O Performance Patterns",
"21": "CPU Optimization", "21": "CPU Optimization",
"22": "Caching Patterns", "22": "Caching Patterns",
"23": "Browser Entrypoint", "23": "Browser Entrypoint",
"24": "Memory Allocation & GC", "24": "Memory Allocation & GC",
"25": "Cover Fetcher Tests", "25": "Cover Fetcher Tests",
"26": "Open",
"27": "Find Skills Guide", "27": "Find Skills Guide",
"28": "Allocation Patterns", "28": "Allocation Patterns",
"29": "Observability & Alerting", "29": "Observability & Alerting",
"31": "Memory Layout", "30": "AGENTS.md",
"32": "Repo Hard Constraints", "32": "Repo Hard Constraints",
"33": "Go Testing Guide", "33": "Go Testing Guide",
"34": "Session Store", "34": "Session Store",
@@ -37,6 +39,7 @@
"38": "novel-logic.test.js", "38": "novel-logic.test.js",
"39": "UI Critique 2026-07-26A", "39": "UI Critique 2026-07-26A",
"40": "UI Critique 2026-07-26B", "40": "UI Critique 2026-07-26B",
"41": "sessions_test.go",
"43": "Issue Tracker & Triage", "43": "Issue Tracker & Triage",
"44": "Ticket Workflow", "44": "Ticket Workflow",
"45": "Go Perf Alert Rules", "45": "Go Perf Alert Rules",
@@ -146,22 +149,11 @@
"150": "Reviewer Subagent (opencode)", "150": "Reviewer Subagent (opencode)",
"151": "Finding Severity Rubric", "151": "Finding Severity Rubric",
"152": "Spec Compliance Review", "152": "Spec Compliance Review",
"158": "T",
"161": "Why Use samber/oops", "161": "Why Use samber/oops",
"162": "singleflight Cache Stampede Prevention", "162": "singleflight Cache Stampede Prevention",
"163": "Struct Field Alignment", "163": "Struct Field Alignment",
"164": "testing/synctest Deterministic Goroutine Testing", "164": "testing/synctest Deterministic Goroutine Testing",
"165": "API Package (Bookmark JSON Handlers)",
"166": "Cover Acquisition & Serving Pipeline",
"167": "Backend AGENTS.md Guidance",
"168": "HTTP Middleware (Auth/Gzip/CORS)",
"169": "Latest Package (Site Parsers & Poller)",
"170": "Latest-Chapter Poller",
"171": "Main Composition Root",
"172": "Migration-Owned Schema",
"173": "Session Package (Cookie Signing & Rate Limit)",
"174": "updated_at List-Order Rule",
"175": "Userscript Package (Serving Handler)",
"176": "AGENTS.md",
"177": "Backend CLAUDE.md Guidance", "177": "Backend CLAUDE.md Guidance",
"178": "Graphify Knowledge Graph (graphify-out/)", "178": "Graphify Knowledge Graph (graphify-out/)",
"179": "CLAUDE.md (Symlink to AGENTS.md)", "179": "CLAUDE.md (Symlink to AGENTS.md)",
@@ -226,10 +218,6 @@
"259": "Dark-First Design Constraint", "259": "Dark-First Design Constraint",
"260": "Discord Guild Membership", "260": "Discord Guild Membership",
"261": "Reader Isolation Invariant", "261": "Reader Isolation Invariant",
"268": "Browser Unit Redeploy",
"269": "pg_dump Hot Backup",
"270": "Redeploy Runbook",
"271": "Rollback Strategy",
"273": "AGENTS.md", "273": "AGENTS.md",
"279": "Userscript CLAUDE guidance" "279": "Userscript CLAUDE guidance"
} }
+66 -62
View File
@@ -1,16 +1,16 @@
# Graph Report - mangaBookmark (2026-08-16) # Graph Report - mangaBookmark (2026-08-16)
## Corpus Check ## Corpus Check
- 111 files · ~273,061 words - 112 files · ~274,851 words
- Verdict: corpus is large enough that graph structure adds value. - Verdict: corpus is large enough that graph structure adds value.
## Summary ## Summary
- 1655 nodes · 3275 edges · 233 communities (63 shown, 170 thin omitted) - 1640 nodes · 3294 edges · 221 communities (67 shown, 154 thin omitted)
- Extraction: 90% EXTRACTED · 10% INFERRED · 0% AMBIGUOUS · INFERRED: 312 edges (avg confidence: 0.77) - Extraction: 90% EXTRACTED · 10% INFERRED · 0% AMBIGUOUS · INFERRED: 313 edges (avg confidence: 0.77)
- Token cost: 0 input · 0 output - Token cost: 0 input · 0 output
## Graph Freshness ## Graph Freshness
- Built from commit: `dc269938` - Built from commit: `f000cc7e`
- Run `git rev-parse HEAD` and compare to check if the graph is stale. - Run `git rev-parse HEAD` and compare to check if the graph is stale.
- Run `graphify update .` after code changes (no API cost). - Run `graphify update .` after code changes (no API cost).
@@ -34,16 +34,17 @@
- [[_COMMUNITY_Cloudflare bot scoring and poll cadence — what is actually documented|Cloudflare bot scoring and poll cadence — what is actually documented]] - [[_COMMUNITY_Cloudflare bot scoring and poll cadence — what is actually documented|Cloudflare bot scoring and poll cadence — what is actually documented]]
- [[_COMMUNITY_Go Code Style Guide|Go Code Style Guide]] - [[_COMMUNITY_Go Code Style Guide|Go Code Style Guide]]
- [[_COMMUNITY_Agent Skills|Agent Skills]] - [[_COMMUNITY_Agent Skills|Agent Skills]]
- [[_COMMUNITY_Store|Store]]
- [[_COMMUNITY_IO Performance Patterns|I/O Performance Patterns]] - [[_COMMUNITY_IO Performance Patterns|I/O Performance Patterns]]
- [[_COMMUNITY_CPU Optimization|CPU Optimization]] - [[_COMMUNITY_CPU Optimization|CPU Optimization]]
- [[_COMMUNITY_Caching Patterns|Caching Patterns]] - [[_COMMUNITY_Caching Patterns|Caching Patterns]]
- [[_COMMUNITY_Browser Entrypoint|Browser Entrypoint]] - [[_COMMUNITY_Browser Entrypoint|Browser Entrypoint]]
- [[_COMMUNITY_Memory Allocation & GC|Memory Allocation & GC]] - [[_COMMUNITY_Memory Allocation & GC|Memory Allocation & GC]]
- [[_COMMUNITY_Cover Fetcher Tests|Cover Fetcher Tests]] - [[_COMMUNITY_Cover Fetcher Tests|Cover Fetcher Tests]]
- [[_COMMUNITY_Open|Open]]
- [[_COMMUNITY_Find Skills Guide|Find Skills Guide]] - [[_COMMUNITY_Find Skills Guide|Find Skills Guide]]
- [[_COMMUNITY_Allocation Patterns|Allocation Patterns]] - [[_COMMUNITY_Allocation Patterns|Allocation Patterns]]
- [[_COMMUNITY_Observability & Alerting|Observability & Alerting]] - [[_COMMUNITY_Observability & Alerting|Observability & Alerting]]
- [[_COMMUNITY_Memory Layout|Memory Layout]]
- [[_COMMUNITY_Repo Hard Constraints|Repo Hard Constraints]] - [[_COMMUNITY_Repo Hard Constraints|Repo Hard Constraints]]
- [[_COMMUNITY_Go Testing Guide|Go Testing Guide]] - [[_COMMUNITY_Go Testing Guide|Go Testing Guide]]
- [[_COMMUNITY_Session Store|Session Store]] - [[_COMMUNITY_Session Store|Session Store]]
@@ -53,6 +54,7 @@
- [[_COMMUNITY_novel-logic.test.js|novel-logic.test.js]] - [[_COMMUNITY_novel-logic.test.js|novel-logic.test.js]]
- [[_COMMUNITY_UI Critique 2026-07-26A|UI Critique 2026-07-26A]] - [[_COMMUNITY_UI Critique 2026-07-26A|UI Critique 2026-07-26A]]
- [[_COMMUNITY_UI Critique 2026-07-26B|UI Critique 2026-07-26B]] - [[_COMMUNITY_UI Critique 2026-07-26B|UI Critique 2026-07-26B]]
- [[_COMMUNITY_sessions_test.go|sessions_test.go]]
- [[_COMMUNITY_Issue Tracker & Triage|Issue Tracker & Triage]] - [[_COMMUNITY_Issue Tracker & Triage|Issue Tracker & Triage]]
- [[_COMMUNITY_Ticket Workflow|Ticket Workflow]] - [[_COMMUNITY_Ticket Workflow|Ticket Workflow]]
- [[_COMMUNITY_Go Perf Alert Rules|Go Perf Alert Rules]] - [[_COMMUNITY_Go Perf Alert Rules|Go Perf Alert Rules]]
@@ -162,21 +164,11 @@
- [[_COMMUNITY_Reviewer Subagent (opencode)|Reviewer Subagent (opencode)]] - [[_COMMUNITY_Reviewer Subagent (opencode)|Reviewer Subagent (opencode)]]
- [[_COMMUNITY_Finding Severity Rubric|Finding Severity Rubric]] - [[_COMMUNITY_Finding Severity Rubric|Finding Severity Rubric]]
- [[_COMMUNITY_Spec Compliance Review|Spec Compliance Review]] - [[_COMMUNITY_Spec Compliance Review|Spec Compliance Review]]
- [[_COMMUNITY_T|T]]
- [[_COMMUNITY_Why Use samberoops|Why Use samber/oops]] - [[_COMMUNITY_Why Use samberoops|Why Use samber/oops]]
- [[_COMMUNITY_singleflight Cache Stampede Prevention|singleflight Cache Stampede Prevention]] - [[_COMMUNITY_singleflight Cache Stampede Prevention|singleflight Cache Stampede Prevention]]
- [[_COMMUNITY_Struct Field Alignment|Struct Field Alignment]] - [[_COMMUNITY_Struct Field Alignment|Struct Field Alignment]]
- [[_COMMUNITY_testingsynctest Deterministic Goroutine Testing|testing/synctest Deterministic Goroutine Testing]] - [[_COMMUNITY_testingsynctest Deterministic Goroutine Testing|testing/synctest Deterministic Goroutine Testing]]
- [[_COMMUNITY_API Package (Bookmark JSON Handlers)|API Package (Bookmark JSON Handlers)]]
- [[_COMMUNITY_Cover Acquisition & Serving Pipeline|Cover Acquisition & Serving Pipeline]]
- [[_COMMUNITY_Backend AGENTS.md Guidance|Backend AGENTS.md Guidance]]
- [[_COMMUNITY_HTTP Middleware (AuthGzipCORS)|HTTP Middleware (Auth/Gzip/CORS)]]
- [[_COMMUNITY_Latest Package (Site Parsers & Poller)|Latest Package (Site Parsers & Poller)]]
- [[_COMMUNITY_Latest-Chapter Poller|Latest-Chapter Poller]]
- [[_COMMUNITY_Main Composition Root|Main Composition Root]]
- [[_COMMUNITY_Migration-Owned Schema|Migration-Owned Schema]]
- [[_COMMUNITY_Session Package (Cookie Signing & Rate Limit)|Session Package (Cookie Signing & Rate Limit)]]
- [[_COMMUNITY_updated_at List-Order Rule|updated_at List-Order Rule]]
- [[_COMMUNITY_Userscript Package (Serving Handler)|Userscript Package (Serving Handler)]]
- [[_COMMUNITY_Backend CLAUDE.md Guidance|Backend CLAUDE.md Guidance]] - [[_COMMUNITY_Backend CLAUDE.md Guidance|Backend CLAUDE.md Guidance]]
- [[_COMMUNITY_Graphify Knowledge Graph (graphify-out)|Graphify Knowledge Graph (graphify-out/)]] - [[_COMMUNITY_Graphify Knowledge Graph (graphify-out)|Graphify Knowledge Graph (graphify-out/)]]
- [[_COMMUNITY_CLAUDE.md (Symlink to AGENTS.md)|CLAUDE.md (Symlink to AGENTS.md)]] - [[_COMMUNITY_CLAUDE.md (Symlink to AGENTS.md)|CLAUDE.md (Symlink to AGENTS.md)]]
@@ -241,18 +233,14 @@
- [[_COMMUNITY_Dark-First Design Constraint|Dark-First Design Constraint]] - [[_COMMUNITY_Dark-First Design Constraint|Dark-First Design Constraint]]
- [[_COMMUNITY_Discord Guild Membership|Discord Guild Membership]] - [[_COMMUNITY_Discord Guild Membership|Discord Guild Membership]]
- [[_COMMUNITY_Reader Isolation Invariant|Reader Isolation Invariant]] - [[_COMMUNITY_Reader Isolation Invariant|Reader Isolation Invariant]]
- [[_COMMUNITY_Browser Unit Redeploy|Browser Unit Redeploy]]
- [[_COMMUNITY_pg_dump Hot Backup|pg_dump Hot Backup]]
- [[_COMMUNITY_Redeploy Runbook|Redeploy Runbook]]
- [[_COMMUNITY_Rollback Strategy|Rollback Strategy]]
- [[_COMMUNITY_AGENTS|AGENTS.md]] - [[_COMMUNITY_AGENTS|AGENTS.md]]
- [[_COMMUNITY_Userscript CLAUDE guidance|Userscript CLAUDE guidance]] - [[_COMMUNITY_Userscript CLAUDE guidance|Userscript CLAUDE guidance]]
## God Nodes (most connected - your core abstractions) ## God Nodes (most connected - your core abstractions)
1. `testConfig()` - 53 edges 1. `testConfig()` - 53 edges
2. `newWebTestServer()` - 49 edges 2. `newWebTestServer()` - 49 edges
3. `newTestStore()` - 42 edges 3. `newTestStore()` - 43 edges
4. `newTestStore()` - 41 edges 4. `newTestStore()` - 42 edges
5. `e()` - 33 edges 5. `e()` - 33 edges
6. `Handler` - 29 edges 6. `Handler` - 29 edges
7. `ne()` - 28 edges 7. `ne()` - 28 edges
@@ -261,8 +249,6 @@
10. `se()` - 27 edges 10. `se()` - 27 edges
## Surprising Connections (you probably didn't know these) ## Surprising Connections (you probably didn't know these)
- `Browser Sidecar Service` --semantically_similar_to--> `Browser Sidecar (BROWSER_WS_URL)` [INFERRED] [semantically similar]
chrome/docker-compose.yml → backend/AGENTS.md
- `el()` --indirect_call--> `c()` [INFERRED] - `el()` --indirect_call--> `c()` [INFERRED]
userscript/manga-bookmark.user.js → backend/internal/web/static/htmx.min.js userscript/manga-bookmark.user.js → backend/internal/web/static/htmx.min.js
- `el()` --indirect_call--> `c()` [INFERRED] - `el()` --indirect_call--> `c()` [INFERRED]
@@ -271,6 +257,8 @@
userscript/manga-bookmark.user.js → backend/internal/web/static/htmx.min.js userscript/manga-bookmark.user.js → backend/internal/web/static/htmx.min.js
- `el()` --indirect_call--> `k()` [INFERRED] - `el()` --indirect_call--> `k()` [INFERRED]
userscript/manga-bookmark.user.js → backend/internal/web/static/htmx.min.js userscript/manga-bookmark.user.js → backend/internal/web/static/htmx.min.js
- `el()` --indirect_call--> `k()` [INFERRED]
userscript/novel-bookmark.user.js → backend/internal/web/static/htmx.min.js
## Import Cycles ## Import Cycles
- None detected. - None detected.
@@ -284,15 +272,15 @@
- **Headless browser infrastructure (sidecar, on-demand, home deployment)** — docs_adr_0005_on_demand_browser_headless_shell, docs_adr_0005_on_demand_browser_cdp, docs_adr_0005_on_demand_browser_on_demand_start, docs_adr_0006_browser_on_the_home_machine_home_machine_rationale [INFERRED 0.85] - **Headless browser infrastructure (sidecar, on-demand, home deployment)** — docs_adr_0005_on_demand_browser_headless_shell, docs_adr_0005_on_demand_browser_cdp, docs_adr_0005_on_demand_browser_on_demand_start, docs_adr_0006_browser_on_the_home_machine_home_machine_rationale [INFERRED 0.85]
- **lightnovelworld series-identity investigation and fix** — docs_research_lightnovelworld_chapter_vs_series_slug_issue_77, docs_research_lightnovelworld_chapter_vs_series_slug_unscoped_regex, docs_adr_0008_series_identity_is_discovered_not_derived_discovered_identity [INFERRED 0.85] - **lightnovelworld series-identity investigation and fix** — docs_research_lightnovelworld_chapter_vs_series_slug_issue_77, docs_research_lightnovelworld_chapter_vs_series_slug_unscoped_regex, docs_adr_0008_series_identity_is_discovered_not_derived_discovered_identity [INFERRED 0.85]
## Communities (233 total, 170 thin omitted) ## Communities (221 total, 154 thin omitted)
### Community 0 - "HTMX Library Internals" ### Community 0 - "HTMX Library Internals"
Cohesion: 0.08 Cohesion: 0.08
Nodes (101): A(), ae(), an(), at(), B(), be(), bn(), bt() (+93 more) Nodes (101): A(), ae(), an(), at(), B(), be(), bn(), bt() (+93 more)
### Community 1 - "Cover Fetch Test Helpers" ### Community 1 - "Cover Fetch Test Helpers"
Cohesion: 0.09 Cohesion: 0.10
Nodes (86): floatPtr(), testConfig(), Config, getCover(), Cookie, Handler, ResponseRecorder, T (+78 more) Nodes (84): floatPtr(), testConfig(), getCover(), Cookie, Handler, ResponseRecorder, T, TestListRendersAcquiredCover() (+76 more)
### Community 2 - "Manga Userscript Adapters" ### Community 2 - "Manga Userscript Adapters"
Cohesion: 0.06 Cohesion: 0.06
@@ -303,48 +291,48 @@ Cohesion: 0.06
Nodes (78): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+70 more) Nodes (78): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+70 more)
### Community 4 - "Series Acquisition Tests" ### Community 4 - "Series Acquisition Tests"
Cohesion: 0.10 Cohesion: 0.09
Nodes (67): bookmarkNewKaganeSeries(), bookmarkNewNovelfullSeries(), bookmarkNewSeries(), Context, Store, T, newAcquirer(), readBookmark() (+59 more) Nodes (69): bookmarkNewKaganeSeries(), bookmarkNewNovelfullSeries(), bookmarkNewSeries(), Context, Store, T, newAcquirer(), readBookmark() (+61 more)
### Community 5 - "Bookmarks API Tests" ### Community 5 - "Bookmarks API Tests"
Cohesion: 0.08 Cohesion: 0.08
Nodes (66): auth(), getBookmarks(), Handler, Request, Store, T, newTestServer(), newTestStore() (+58 more) Nodes (67): auth(), getBookmarks(), Handler, Request, Store, T, newTestServer(), newTestStore() (+59 more)
### Community 7 - "Cover & Acquire Internals" ### Community 7 - "Cover & Acquire Internals"
Cohesion: 0.05 Cohesion: 0.10
Nodes (60): Addr, Context, Store, defaultCoverResolver(), fetchCoverBytes(), Client, Context, NewCoverFetcher() (+52 more) Nodes (30): Addr, Context, Store, defaultCoverResolver(), fetchCoverBytes(), Client, Context, NewCoverFetcher() (+22 more)
### Community 8 - "System Architecture Concepts" ### Community 8 - "System Architecture Concepts"
Cohesion: 0.10 Cohesion: 0.13
Nodes (26): Confirm-Gated Destructive Actions, Discord OAuth & Guild-Membership Gate, Lifecycle Buckets (reading/archived/finished), Reader-Owned Store, HMAC-Derived Reader Credentials, Web Package (Browser UI + Templates), app.html — App Shell Template, Manga/Novel Library Switch (+18 more) Nodes (20): app.html — App Shell Template, Manga/Novel Library Switch, Bookmark Bucket Tabs, Confirm Row (Archive/Finish/Remove), card.html — Series Card Template, htmx /ui/* Mutation Endpoints, chrome.html — Out-of-Band Regions, Action Key (+12 more)
### Community 9 - "Session Middleware" ### Community 9 - "Session Middleware"
Cohesion: 0.08 Cohesion: 0.07
Nodes (34): ClearCookie(), ClientIP(), Duration, Mutex, Request, ResponseWriter, Time, isHTTPS() (+26 more) Nodes (35): ClearCookie(), ClientIP(), Duration, Mutex, Request, ResponseWriter, Time, isHTTPS() (+27 more)
### Community 10 - "Go Test Helpers" ### Community 10 - "Go Test Helpers"
Cohesion: 0.05 Cohesion: 0.05
Nodes (39): Test Helpers, Test Timeout, Basic Handler Test, HTTP Handler Testing, Query Parameters and Headers, Docker Compose Fixture, Integration Testing, SQL Schema Fixture (+31 more) Nodes (39): Test Helpers, Test Timeout, Basic Handler Test, HTTP Handler Testing, Query Parameters and Headers, Docker Compose Fixture, Integration Testing, SQL Schema Fixture (+31 more)
### Community 11 - "Store Tests" ### Community 11 - "Store Tests"
Cohesion: 0.07 Cohesion: 0.15
Nodes (79): M, TestMain(), M, TestMain(), M, Main(), start(), URL() (+71 more) Nodes (43): Store, T, newTestStore(), readLatestCheckedAt(), readSeries(), secondReader(), seedForCheck(), seedSecondReader() (+35 more)
### Community 12 - "Bookmarks API Handler" ### Community 12 - "Bookmarks API Handler"
Cohesion: 0.08 Cohesion: 0.11
Nodes (33): Handler, Request, ResponseWriter, Store, Healthz(), writeJSON(), Auth(), compressible() (+25 more) Nodes (24): Auth(), compressible(), CORS(), Handler, ResponseWriter, Store, Gzip(), ResolveReader() (+16 more)
### Community 13 - "Web UI Handlers" ### Community 13 - "Web UI Handlers"
Cohesion: 0.06 Cohesion: 0.14
Nodes (24): coverRelativePath(), coverSourceAddress(), displayChapter(), Store, currentLib(), currentTab(), filterBookmarks(), Client (+16 more) Nodes (18): currentLib(), currentTab(), filterBookmarks(), Client, HandlerFunc, Request, ResponseWriter, Store (+10 more)
### Community 14 - "Go Error Handling" ### Community 14 - "Go Error Handling"
Cohesion: 0.06 Cohesion: 0.06
Nodes (33): Creating Errors, Custom Error Types, Custom types that wrap other errors, Decision table: which error strategy to use, Error Creation, Error String Conventions, Errors as Values, `errors.New` — static error messages (+25 more) Nodes (33): Creating Errors, Custom Error Types, Custom types that wrap other errors, Decision table: which error strategy to use, Error Creation, Error String Conventions, Errors as Values, `errors.New` — static error messages (+25 more)
### Community 15 - "CDP Browser Client" ### Community 15 - "CDP Browser Client"
Cohesion: 0.08 Cohesion: 0.07
Nodes (30): awaitPromise(), browserConnectionLost(), classifyBrowserError(), Action, Context, Mutex, jsString(), kaganeAPIURL() (+22 more) Nodes (36): awaitPromise(), browserConnectionLost(), classifyBrowserError(), comixRead(), comixSeriesPageURL(), Action, Context, Mutex (+28 more)
### Community 16 - "Cloudflare bot scoring and poll cadence — what is actually documented" ### Community 16 - "Cloudflare bot scoring and poll cadence — what is actually documented"
Cohesion: 0.06 Cohesion: 0.06
@@ -354,6 +342,10 @@ Nodes (33): 1.1 The score itself, 1.2 The detection engines (Enterprise Bot Mana
Cohesion: 0.08 Cohesion: 0.08
Nodes (23): Code Style Details, Extract Complex Conditions, Value vs Pointer Arguments, Code Organization Within Files, Complex Conditions & Init Scope, Composite Literals, Control Flow, Cross-References (+15 more) Nodes (23): Code Style Details, Extract Complex Conditions, Value vs Pointer Arguments, Code Organization Within Files, Complex Conditions & Init Scope, Composite Literals, Control Flow, Cross-References (+15 more)
### Community 19 - "Store"
Cohesion: 0.09
Nodes (8): coverRelativePath(), coverSourceAddress(), displayChapter(), Store, scanSeries(), TestDisplayChapter(), Bookmark, ReaderSummary
### Community 20 - "I/O Performance Patterns" ### Community 20 - "I/O Performance Patterns"
Cohesion: 0.11 Cohesion: 0.11
Nodes (18): Avoid io.ReadAll for large payloads, Batch Operations, Buffered I/O, Cgo Overhead, Channel: batch processing from a stream, Concurrent Multi-Stage Pipelines, Connection pooling, Database: batch inserts over row-by-row (+10 more) Nodes (18): Avoid io.ReadAll for large payloads, Batch Operations, Buffered I/O, Cgo Overhead, Channel: batch processing from a stream, Concurrent Multi-Stage Pipelines, Connection pooling, Database: batch inserts over row-by-row (+10 more)
@@ -378,21 +370,25 @@ Nodes (15): Allocation Rate Reduction, Ballast pattern (pre-Go 1.19), Garbage Co
Cohesion: 0.33 Cohesion: 0.33
Nodes (12): coverResponse(), Request, T, TestCoverFetcherCanonicalisesJpgAlias(), TestCoverFetcherFetchesPublicHTTPSImage(), TestCoverFetcherRefusesUnsafeDestinationsBeforeRequest(), TestCoverFetcherRejectsNonImage(), TestCoverFetcherRejectsOversizedBody() (+4 more) Nodes (12): coverResponse(), Request, T, TestCoverFetcherCanonicalisesJpgAlias(), TestCoverFetcherFetchesPublicHTTPSImage(), TestCoverFetcherRefusesUnsafeDestinationsBeforeRequest(), TestCoverFetcherRejectsNonImage(), TestCoverFetcherRejectsOversizedBody() (+4 more)
### Community 26 - "Open"
Cohesion: 0.20
Nodes (17): applyMigration(), migrate(), Open(), refreshOwnerToken(), seedOwner(), TestCoverIsContentAddressedOnFilesystem(), TestCoverPersistsAcrossReopen(), TestMigration0002BackfillsExistingBookmarks() (+9 more)
### Community 27 - "Find Skills Guide" ### Community 27 - "Find Skills Guide"
Cohesion: 0.14 Cohesion: 0.14
Nodes (13): Common Skill Categories, Find Skills, How to Help Users Find Skills, Step 1: Understand What They Need, Step 2: Check the Leaderboard First, Step 3: Search for Skills, Step 4: Verify Quality Before Recommending, Step 5: Present Options to the User (+5 more) Nodes (13): Common Skill Categories, Find Skills, How to Help Users Find Skills, Step 1: Understand What They Need, Step 2: Check the Leaderboard First, Step 3: Search for Skills, Step 4: Verify Quality Before Recommending, Step 5: Present Options to the User (+5 more)
### Community 28 - "Allocation Patterns" ### Community 28 - "Allocation Patterns"
Cohesion: 0.14 Cohesion: 0.11
Nodes (14): Allocation Patterns, Backing Array Leaks, Direct indexing vs append, Eliminate redundant map lookups, Interface boxing, Map never shrinks, Map size hints, Memory Optimization (+6 more) Nodes (19): Allocation Patterns, Backing Array Leaks, Direct indexing vs append, Eliminate redundant map lookups, Interface boxing, Map never shrinks, Map of pointers for large, frequently updated structs, Map size hints (+11 more)
### Community 29 - "Observability & Alerting" ### Community 29 - "Observability & Alerting"
Cohesion: 0.22 Cohesion: 0.22
Nodes (9): Alerting rules (examples), CPU saturation, GC pressure, Goroutine leaks, Grafana Dashboards, Memory leaks, Prometheus Metrics for Go, PromQL Queries for Performance Diagnosis (+1 more) Nodes (9): Alerting rules (examples), CPU saturation, GC pressure, Goroutine leaks, Grafana Dashboards, Memory leaks, Prometheus Metrics for Go, PromQL Queries for Performance Diagnosis (+1 more)
### Community 31 - "Memory Layout" ### Community 32 - "Repo Hard Constraints"
Cohesion: 0.40 Cohesion: 0.18
Nodes (5): Map of pointers for large, frequently updated structs, Memory Layout, Pointer receivers for large structs, Struct field alignment, Zero-size field at end of struct Nodes (12): M, TestMain(), M, TestMain(), M, Main(), start(), URL() (+4 more)
### Community 33 - "Go Testing Guide" ### Community 33 - "Go Testing Guide"
Cohesion: 0.20 Cohesion: 0.20
@@ -422,6 +418,10 @@ Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations,
Cohesion: 0.29 Cohesion: 0.29
Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations, Persona Red Flags, Priority Issues, Questions to Consider Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations, Persona Red Flags, Priority Issues, Questions to Consider
### Community 41 - "sessions_test.go"
Cohesion: 0.48
Nodes (6): T, TestCreateAndGetSession(), TestDeleteSessionIsPerReader(), TestDeleteSessionRevokes(), TestExpiredSessionIsGone(), TestGetSessionUnknownID()
### Community 45 - "Go Perf Alert Rules" ### Community 45 - "Go Perf Alert Rules"
Cohesion: 0.50 Cohesion: 0.50
Nodes (4): Prometheus Alerting Rules (Go Performance), GoroutineLeak Alert, HighGCPauseTime Alert, MemoryNearLimit Alert Nodes (4): Prometheus Alerting Rules (Go Performance), GoroutineLeak Alert, HighGCPauseTime Alert, MemoryNearLimit Alert
@@ -443,12 +443,12 @@ Cohesion: 1.00
Nodes (3): Mirrored Double Bookmark Mark, Ember Flame Accent, BookmarkManager Logo Nodes (3): Mirrored Double Bookmark Mark, Ember Flame Accent, BookmarkManager Logo
### Community 103 - "bookmark-api Service" ### Community 103 - "bookmark-api Service"
Cohesion: 0.24 Cohesion: 0.32
Nodes (10): Backend Go Service (stdlib net/http), Browser Sidecar (BROWSER_WS_URL), Browser Sidecar Service, CDP Endpoint (Tailnet-Bound :9222), Persistent Chrome Profile Volume, chrome/docker-compose.yml — Browser Deployable Unit, bookmark-api Prod Override, CDP Never on Shared Proxy Network (+2 more) Nodes (8): Browser Sidecar Service, CDP Endpoint (Tailnet-Bound :9222), Persistent Chrome Profile Volume, chrome/docker-compose.yml — Browser Deployable Unit, bookmark-api Prod Override, CDP Never on Shared Proxy Network, docker-compose.prod.yml — Production Override, Traefik Reverse Proxy Labels
### Community 104 - "AGENTS.md" ### Community 104 - "AGENTS.md"
Cohesion: 0.12 Cohesion: 0.12
Nodes (15): Agent skills, AGENTS.md, Architecture, Commands, Comments, Design system, Domain docs, Forge: Gitea, not GitHub (+7 more) Nodes (14): Agent skills, Architecture, Commands, Comments, Design system, Domain docs, Forge: Gitea, not GitHub, graphify (+6 more)
### Community 105 - "reviewer.md" ### Community 105 - "reviewer.md"
Cohesion: 0.12 Cohesion: 0.12
@@ -530,29 +530,33 @@ Nodes (3): Consequence, The wire format stays flat and deliberately does not mir
Cohesion: 0.50 Cohesion: 0.50
Nodes (3): ADR-0005: On-demand browser sidecar, Constraints, Decision Nodes (3): ADR-0005: On-demand browser sidecar, Constraints, Decision
### Community 158 - "T"
Cohesion: 0.08
Nodes (38): Handler, Request, ResponseWriter, Store, Healthz(), writeJSON(), Request, ReaderID() (+30 more)
### Community 273 - "AGENTS.md" ### Community 273 - "AGENTS.md"
Cohesion: 0.50 Cohesion: 0.50
Nodes (3): Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust), Second script: `novel-bookmark.user.js`, Userscript structure (single IIFE, `manga-bookmark.user.js`) Nodes (3): Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust), Second script: `novel-bookmark.user.js`, Userscript structure (single IIFE, `manga-bookmark.user.js`)
## Knowledge Gaps ## Knowledge Gaps
- **533 isolated node(s):** `bookmarkmanager/backend`, `ctxKey`, `loginView`, `ctxKey`, `test` (+528 more) - **520 isolated node(s):** `bookmarkmanager/backend`, `ctxKey`, `loginView`, `ctxKey`, `test` (+515 more)
These have ≤1 connection - possible missing edges or undocumented components. These have ≤1 connection - possible missing edges or undocumented components.
- **170 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes. - **154 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes.
## Suggested Questions ## Suggested Questions
_Questions this graph is uniquely positioned to answer:_ _Questions this graph is uniquely positioned to answer:_
- **Why does `New()` connect `Series Acquisition Tests` to `Cover Fetch Test Helpers`, `Bookmarks API Tests`, `Cover & Acquire Internals`, `Session Middleware`, `Store Tests`, `Web UI Handlers`?** - **Why does `New()` connect `Series Acquisition Tests` to `Bookmarks API Tests`, `Cover & Acquire Internals`, `Session Middleware`, `Web UI Handlers`, `Open`?**
_High betweenness centrality (0.047) - this node is a cross-community bridge._ _High betweenness centrality (0.051) - this node is a cross-community bridge._
- **Why does `Open()` connect `Store Tests` to `Cover Fetch Test Helpers`, `Web UI Handlers`, `Series Acquisition Tests`, `Bookmarks API Tests`?** - **Why does `Open()` connect `Open` to `Repo Hard Constraints`, `Cover Fetch Test Helpers`, `Series Acquisition Tests`, `Bookmarks API Tests`, `Store Tests`, `Store`?**
_High betweenness centrality (0.032) - this node is a cross-community bridge._ _High betweenness centrality (0.038) - this node is a cross-community bridge._
- **Why does `newRouter()` connect `Bookmarks API Tests` to `Cover Fetch Test Helpers`, `Bookmarks API Handler`, `Series Acquisition Tests`?** - **Why does `newRouter()` connect `Bookmarks API Tests` to `Cover Fetch Test Helpers`, `Bookmarks API Handler`, `Series Acquisition Tests`?**
_High betweenness centrality (0.025) - this node is a cross-community bridge._ _High betweenness centrality (0.030) - this node is a cross-community bridge._
- **Are the 47 inferred relationships involving `testConfig()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverNeverEchoesNonImage()`) actually correct?** - **Are the 47 inferred relationships involving `testConfig()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverNeverEchoesNonImage()`) actually correct?**
_`testConfig()` has 47 INFERRED edges - model-reasoned connections that need verification._ _`testConfig()` has 47 INFERRED edges - model-reasoned connections that need verification._
- **Are the 8 inferred relationships involving `newWebTestServer()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverRejectsUnknownAddress()`) actually correct?** - **Are the 8 inferred relationships involving `newWebTestServer()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverRejectsUnknownAddress()`) actually correct?**
_`newWebTestServer()` has 8 INFERRED edges - model-reasoned connections that need verification._ _`newWebTestServer()` has 8 INFERRED edges - model-reasoned connections that need verification._
- **Are the 6 inferred relationships involving `newTestStore()` (e.g. with `TestCreateAndGetSession()` and `TestDeleteSessionIsPerReader()`) actually correct?** - **Are the 12 inferred relationships involving `newTestStore()` (e.g. with `TestAcquireDoesNotBlockTheWrite()` and `TestAcquireFailureLeavesTheBookmarkIntact()`) actually correct?**
_`newTestStore()` has 6 INFERRED edges - model-reasoned connections that need verification._ _`newTestStore()` has 12 INFERRED edges - model-reasoned connections that need verification._
- **What connects `bookmarkmanager/backend`, `ctxKey`, `loginView` to the rest of the system?** - **What connects `bookmarkmanager/backend`, `ctxKey`, `loginView` to the rest of the system?**
_573 weakly-connected nodes found - possible documentation gaps or missing edges._ _553 weakly-connected nodes found - possible documentation gaps or missing edges._
File diff suppressed because one or more lines are too long
+803 -829
View File
File diff suppressed because it is too large Load Diff
+33 -28
View File
@@ -140,8 +140,8 @@
"semantic_hash": "3a08979e4603aae5c32a58d5b6c39765" "semantic_hash": "3a08979e4603aae5c32a58d5b6c39765"
}, },
"CLAUDE.md": { "CLAUDE.md": {
"mtime": 1786501942.7367291, "mtime": 1786856633.445473,
"ast_hash": "16b34d73e066d56d8f04523e6b3f6bd9", "ast_hash": "fab288c23be960d9c6afbfe3f21ff41c",
"semantic_hash": "" "semantic_hash": ""
}, },
"DEPLOY.md": { "DEPLOY.md": {
@@ -200,8 +200,8 @@
"semantic_hash": "e69a8340a371579ca3ea689660f7d7bd" "semantic_hash": "e69a8340a371579ca3ea689660f7d7bd"
}, },
"AGENTS.md": { "AGENTS.md": {
"mtime": 1786501942.7367291, "mtime": 1786856633.445473,
"ast_hash": "16b34d73e066d56d8f04523e6b3f6bd9", "ast_hash": "fab288c23be960d9c6afbfe3f21ff41c",
"semantic_hash": "" "semantic_hash": ""
}, },
"userscript/test/logic.test.js": { "userscript/test/logic.test.js": {
@@ -215,9 +215,9 @@
"semantic_hash": "8f3c0132eb4787a2c8736eb99f7689af" "semantic_hash": "8f3c0132eb4787a2c8736eb99f7689af"
}, },
"REDEPLOY.md": { "REDEPLOY.md": {
"mtime": 1786363889.552731, "mtime": 1786856643.0770833,
"ast_hash": "d0baf08b95e7b5986234a9f36759c12e", "ast_hash": "e5e910f0244a040e2ccdc669b17eb4db",
"semantic_hash": "d0baf08b95e7b5986234a9f36759c12e" "semantic_hash": ""
}, },
"docs/design-system.md": { "docs/design-system.md": {
"mtime": 1786022513.9623306, "mtime": 1786022513.9623306,
@@ -245,14 +245,14 @@
"semantic_hash": "385b36f58488b7e6d93eb6d6034e9ee3" "semantic_hash": "385b36f58488b7e6d93eb6d6034e9ee3"
}, },
"backend/internal/latest/browser.go": { "backend/internal/latest/browser.go": {
"mtime": 1786499529.7688599, "mtime": 1786856879.3397639,
"ast_hash": "ed129a7f00601ea90c877ff29fa21220", "ast_hash": "fbdba4bb56d8c804087bcb6e141e52ad",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/browser_test.go": { "backend/internal/latest/browser_test.go": {
"mtime": 1786262323.6964688, "mtime": 1786856254.36163,
"ast_hash": "e900f92971486f47d7ef76e9a95217fe", "ast_hash": "800fa6aa471ada054a3c48943ad17ab7",
"semantic_hash": "e900f92971486f47d7ef76e9a95217fe" "semantic_hash": ""
}, },
"backend/internal/latest/fetch.go": { "backend/internal/latest/fetch.go": {
"mtime": 1786499529.7688599, "mtime": 1786499529.7688599,
@@ -260,23 +260,23 @@
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/poller.go": { "backend/internal/latest/poller.go": {
"mtime": 1786499529.7688599, "mtime": 1786856670.129715,
"ast_hash": "44fef6074ac2eaffc8233f46aad5236b", "ast_hash": "6b7060ff52994729832d9fcc3cc97dad",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/poller_test.go": { "backend/internal/latest/poller_test.go": {
"mtime": 1786499529.7688599, "mtime": 1786856311.8535168,
"ast_hash": "64bc838c822f1bf33bbf9e291215454b", "ast_hash": "b6837fab377c94fa1d2f20a931975a4a",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/sites.go": { "backend/internal/latest/sites.go": {
"mtime": 1786499529.7688599, "mtime": 1786856865.4529688,
"ast_hash": "b744cc685363317a526cc3bebceea39e", "ast_hash": "2f00456ac9d1d8148ba1bc6cbceb24f6",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/sites_test.go": { "backend/internal/latest/sites_test.go": {
"mtime": 1786499529.7725692, "mtime": 1786856890.1231265,
"ast_hash": "eabca9014a306e3c71d238b0ae499f61", "ast_hash": "0db2028a6073f342fee61ad15c2e5f0f",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/smoke_image_test.go": { "backend/internal/latest/smoke_image_test.go": {
@@ -415,7 +415,7 @@
"semantic_hash": "e44a2f6f624db044e19508bc5ab05592" "semantic_hash": "e44a2f6f624db044e19508bc5ab05592"
}, },
"CONTEXT.md": { "CONTEXT.md": {
"mtime": 1786850406.6525955, "mtime": 1786855457.9327722,
"ast_hash": "24548f60414b4c5ff58538acaada5345", "ast_hash": "24548f60414b4c5ff58538acaada5345",
"semantic_hash": "" "semantic_hash": ""
}, },
@@ -425,13 +425,13 @@
"semantic_hash": "6c6f3e4c4c2f57867894280bce728c50" "semantic_hash": "6c6f3e4c4c2f57867894280bce728c50"
}, },
"backend/AGENTS.md": { "backend/AGENTS.md": {
"mtime": 1786501942.7367291, "mtime": 1786856656.6826186,
"ast_hash": "23d8dbbcb9796c679d25a74bb3847f88", "ast_hash": "d5610ba24076b57e17b9d76241acaa65",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/CLAUDE.md": { "backend/CLAUDE.md": {
"mtime": 1786501942.7367291, "mtime": 1786856656.6826186,
"ast_hash": "23d8dbbcb9796c679d25a74bb3847f88", "ast_hash": "d5610ba24076b57e17b9d76241acaa65",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/web/templates/app.html": { "backend/internal/web/templates/app.html": {
@@ -560,9 +560,9 @@
"semantic_hash": "46cf7822d4f667e3cab36b547abe5e97" "semantic_hash": "46cf7822d4f667e3cab36b547abe5e97"
}, },
"backend/internal/latest/cover.go": { "backend/internal/latest/cover.go": {
"mtime": 1786363889.5565126, "mtime": 1786856664.2957466,
"ast_hash": "e6749cfe3cd7c2e71d4392dde84f55f9", "ast_hash": "d5f2248c3d11de74bf5a3977651b17c2",
"semantic_hash": "e6749cfe3cd7c2e71d4392dde84f55f9" "semantic_hash": ""
}, },
"backend/internal/latest/cover_fetch_test.go": { "backend/internal/latest/cover_fetch_test.go": {
"mtime": 1786363889.5565126, "mtime": 1786363889.5565126,
@@ -623,5 +623,10 @@
"mtime": 1786501942.7367291, "mtime": 1786501942.7367291,
"ast_hash": "1aa17575ab20f2f36583602999a6a60f", "ast_hash": "1aa17575ab20f2f36583602999a6a60f",
"semantic_hash": "" "semantic_hash": ""
},
"backend/internal/latest/smoke_comix_test.go": {
"mtime": 1786856356.3432186,
"ast_hash": "111fdbb75fc68ac2ab1013bc916063cf",
"semantic_hash": ""
} }
} }