180ee78b1f
Tracks read progress on comix.to and kagane.to alongside asura and demonic, in both the userscript and the backend. Implements `docs/superpowers/plans/2026-08-03-comix-kagane-support.md`. ## Userscript - `comix` adapter — `/title/<id>-<slug>`; only the id prefix is identity (the slug follows the title). No `og:image`, so the cover is matched by `alt`. - `kagane` adapter — reader URLs are uuids with no chapter number, so it comes out of `og:title`; anchor scanning is structurally impossible, replaced by `latestChapterFromApi` against kagane's same-origin JSON API. - `seriesId` threaded through `latestChapterFromAnchors` so comix can scope its scan to its own series and a recommendation strip cannot win the maximum. - `@match` for both hosts, panel chips, v1.6.0. ## Backend - `latestChapterFrom` cases: comix parses the SSR JSON state blob (`latestChapterUrl`, scoped to the series id); kagane parses API JSON (`chapter_no`). - Poller allowlist extended; `Poller.BrowserFetch` with `fetcherFor(site)` routes kagane to a browser fetcher. Nil means kagane is not polled at all — never a fallback to the TLS fetcher, which would only ever retrieve a challenge page. - `BrowserFetcher`: chromedp against a `headless-shell` sidecar. kagane sits behind a Cloudflare JS challenge that no TLS fingerprint clears, and the request is made inside the page rather than by replaying `cf_clearance`. - `BROWSER_WS_URL` wiring, sidecar in both compose files (no `ports:`, dedicated non-external network), Dockerfile on `golang:1.26-alpine` — chromedp requires go 1.26. - Web UI `--comix` / `--kagane` tokens in both colour branches. ## Notes for review - `series_url` is client-supplied and a headless browser is a strong SSRF primitive, so kagane's host is pinned twice: in `fetchableSeriesURL` and again in `kaganeAPIURL`. - Three chained defects found during verification made the browser path dead under Compose (sidecar flag collision, Chrome's Host-header DNS-rebinding check, the wrong chromedp option). Fixed; the compose comments record the wrong configurations too, so they don't get "simplified" back. - `ALLOWED_ORIGINS` now includes both new origins. Without it every write from comix/kagane silently fails CORS preflight, parks in the retry queue, and drops at the cap. ## Verification 221 backend tests, 32 userscript tests, static `CGO_ENABLED=0` build, both compose configs. Two gaps, both real: 1. The userscript on live pages via Violentmonkey needs a human browser profile — not run. Check: comix series page (title/cover, no chapter), comix chapter page (records the number; an *older* chapter must not regress it), comix SPA navigation without reload, kagane series page (og:image cover), kagane reader (number from `og:title`), both chips opening the right sites. 2. The kagane browser path has not completed end-to-end anywhere. Dial/navigate/fetch is confirmed, but Cloudflare 403'd headless-shell's Chrome on every attempt from the dev sandbox, and comix's poll-through-Docker was blocked by that environment's TLS interception. Both environment-dependent rather than branch defects — the first real deploy is the actual verification. Reviewed-on: #13 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
169 lines
6.8 KiB
Go
169 lines
6.8 KiB
Go
package latest
|
|
|
|
import "testing"
|
|
|
|
// Trimmed from https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af
|
|
// fetched 2026-07-26. The first anchor is the "First Chapter" shortcut: it is a
|
|
// real chapter link with no "Chapter N" text, and it must not be mistaken for
|
|
// the latest just because it parses.
|
|
const asuraSeriesFixture = `
|
|
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/1" class="py-3 rounded-md bg-[#E8E8E8]"><svg class="w-4 h-4"></svg>First Chapter</a>
|
|
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/179" data-astro-prefetch="hover" class="group flex"><span class="font-medium">Chapter 179</span></a>
|
|
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/181" data-astro-prefetch="hover" class="group flex"><span class="font-medium">Chapter 181</span></a>
|
|
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/180" data-astro-prefetch="hover" class="group flex"><span class="font-medium">Chapter 180</span></a>
|
|
`
|
|
|
|
// A chapter link belonging to a different series, of the kind a "you might also
|
|
// like" strip would introduce. Slug scoping must exclude it.
|
|
const asuraCrossSeriesFixture = asuraSeriesFixture + `
|
|
<a href="/comics/some-other-series-aabbccdd/chapter/999" class="group flex"><span>Chapter 999</span></a>
|
|
`
|
|
|
|
// Trimmed from https://demonicscans.org/manga/Catastrophic-Necromancer fetched
|
|
// 2026-07-26. Note the raw "&", the doubled space after <a, and the decimal
|
|
// chapters, all as they appear live.
|
|
const demonicSeriesFixture = `
|
|
<a href="/chaptered.php?manga=11799&chapter=0.5" class="chplinks" title="Catastrophic Necromancer 0.5">Chapter 0.5</a>
|
|
<a href="/chaptered.php?manga=11799&chapter=294" class="chplinks" title="Catastrophic Necromancer 294">Chapter 294</a>
|
|
<a href="/chaptered.php?manga=11799&chapter=296" class="chplinks" title="Catastrophic Necromancer 296">Chapter 296</a>
|
|
<a href="/chaptered.php?manga=11799&chapter=295" class="chplinks" title="Catastrophic Necromancer 295">Chapter 295</a>
|
|
`
|
|
|
|
// What Cloudflare serves instead of the page when an IP's bot score flips.
|
|
const challengeFixture = `<!DOCTYPE html><html><head><title>Just a moment...</title>
|
|
<script src="/cdn-cgi/challenge-platform/h/b/orchestrate/chl_page/v1"></script></head>
|
|
<body><div id="challenge-running">Checking your browser</div></body></html>`
|
|
|
|
// Trimmed from the server-rendered HTML of
|
|
// https://comix.to/title/n8we-dungeons-and-crayons fetched 2026-08-03. comix is
|
|
// an SPA: the page ships a JSON state blob rather than a list of chapter
|
|
// anchors, and latestChapterUrl is where the newest chapter actually lives.
|
|
const comixSeriesFixture = `
|
|
{"firstChapterUrl":"/title/n8we-dungeons-and-crayons/5038739-chapter-1","latestChapterUrl":"/title/n8we-dungeons-and-crayons/11139891-chapter-80"},
|
|
{""manga","recommended","n8we",1]":{"items":[{"latestChapterUrl":"/title/qqwrm-full-time-awakening/99999999-chapter-999"}]}
|
|
`
|
|
|
|
// The kagane branch is fed by the browser fetcher, so the body is API JSON, not
|
|
// HTML. Trimmed from GET /api/v2/series/<uuid> on 2026-08-03.
|
|
const kaganeAPIFixture = `
|
|
{"series_id":"019f84bc-9ba0-7ed9-86f5-8b905ec7c28b","title":"Infinite Decryption",
|
|
"series_books":[{"book_id":"a","title":"Episode 1","chapter_no":"1","sort_no":1},
|
|
{"book_id":"b","title":"Episode 41","chapter_no":"41","sort_no":41},
|
|
{"book_id":"c","title":"Episode 40.5","chapter_no":"40.5","sort_no":40}]}
|
|
`
|
|
|
|
func TestLatestChapterFrom(t *testing.T) {
|
|
const asuraURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
|
|
const demonicURL = "https://demonicscans.org/manga/Catastrophic-Necromancer"
|
|
|
|
tests := []struct {
|
|
name string
|
|
site string
|
|
seriesURL string
|
|
body string
|
|
wantOK bool
|
|
wantNum float64
|
|
wantLabel string
|
|
}{
|
|
{
|
|
name: "asura takes the max, not the last listed",
|
|
site: "asura", seriesURL: asuraURL, body: asuraSeriesFixture,
|
|
wantOK: true, wantNum: 181, wantLabel: "Chapter 181",
|
|
},
|
|
{
|
|
name: "asura ignores another series' chapter links",
|
|
site: "asura", seriesURL: asuraURL, body: asuraCrossSeriesFixture,
|
|
wantOK: true, wantNum: 181, wantLabel: "Chapter 181",
|
|
},
|
|
{
|
|
name: "asura scoping survives a build-hash rotation",
|
|
site: "asura",
|
|
seriesURL: "https://asurascans.com/comics/chronicles-of-the-demon-faction-059befe1",
|
|
body: asuraCrossSeriesFixture,
|
|
wantOK: true, wantNum: 181, wantLabel: "Chapter 181",
|
|
},
|
|
{
|
|
name: "asura with an unparseable series url",
|
|
site: "asura", seriesURL: "https://asurascans.com/", body: asuraSeriesFixture,
|
|
wantOK: false,
|
|
},
|
|
{
|
|
name: "demonic takes the max across raw and escaped ampersands",
|
|
site: "demonic", seriesURL: demonicURL, body: demonicSeriesFixture,
|
|
wantOK: true, wantNum: 296, wantLabel: "Chapter 296",
|
|
},
|
|
{
|
|
name: "demonic keeps decimal chapters parseable",
|
|
site: "demonic", seriesURL: demonicURL,
|
|
body: `<a href="/chaptered.php?manga=11799&chapter=0.5">Chapter 0.5</a>`,
|
|
wantOK: true, wantNum: 0.5, wantLabel: "Chapter 0.5",
|
|
},
|
|
{
|
|
name: "empty body",
|
|
site: "asura", seriesURL: asuraURL, body: "",
|
|
wantOK: false,
|
|
},
|
|
{
|
|
name: "cloudflare challenge page",
|
|
site: "asura", seriesURL: asuraURL, body: challengeFixture,
|
|
wantOK: false,
|
|
},
|
|
{
|
|
name: "demonic markup handed to the asura rule",
|
|
site: "asura", seriesURL: asuraURL, body: demonicSeriesFixture,
|
|
wantOK: false,
|
|
},
|
|
{
|
|
name: "unknown site",
|
|
site: "mangadex", seriesURL: "https://example.com/x", body: asuraSeriesFixture,
|
|
wantOK: false,
|
|
},
|
|
{
|
|
name: "comix reads latestChapterUrl, scoped to this series",
|
|
site: "comix",
|
|
seriesURL: "https://comix.to/title/n8we-dungeons-and-crayons",
|
|
body: comixSeriesFixture,
|
|
wantOK: true, wantNum: 80, wantLabel: "Chapter 80",
|
|
},
|
|
{
|
|
name: "comix yields nothing on a challenge page",
|
|
site: "comix",
|
|
seriesURL: "https://comix.to/title/n8we-dungeons-and-crayons",
|
|
body: challengeFixture,
|
|
wantOK: false,
|
|
},
|
|
{
|
|
name: "kagane takes the max chapter_no from API json",
|
|
site: "kagane",
|
|
seriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b",
|
|
body: kaganeAPIFixture,
|
|
wantOK: true, wantNum: 41, wantLabel: "Chapter 41",
|
|
},
|
|
{
|
|
name: "kagane yields nothing on a challenge page",
|
|
site: "kagane",
|
|
seriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b",
|
|
body: challengeFixture,
|
|
wantOK: false,
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
got, ok := latestChapterFrom(tt.site, tt.seriesURL, tt.body)
|
|
if ok != tt.wantOK {
|
|
t.Fatalf("ok = %v, want %v (got %+v)", ok, tt.wantOK, got)
|
|
}
|
|
if !tt.wantOK {
|
|
return
|
|
}
|
|
if got.Num != tt.wantNum {
|
|
t.Errorf("Num = %v, want %v", got.Num, tt.wantNum)
|
|
}
|
|
if got.Label != tt.wantLabel {
|
|
t.Errorf("Label = %q, want %q", got.Label, tt.wantLabel)
|
|
}
|
|
})
|
|
}
|
|
}
|