sulthan
ddbd57070d
Closes #98.
comix.to began answering plain-TLS fetches with a Cloudflare JavaScript
challenge on 2026-08-12, so every poll got a 403 interstitial. Its cover host
`static.comix.to` is gated the same way. comix therefore joins kagane and
novelfull as a browser-backed Site.
## What changed
- **Registry** (`internal/latest/sites.go`): comix gains a `Browser` entry —
`comixRead`, `Done: body != "" && !isInterstitial(body)`, `Fallback: false`.
Skip-when-no-browser falls out of the existing routing; no site-string compare
was added anywhere.
- **Read shape** (`internal/latest/browser.go`): an in-tab `fetch()` of the
Series URL, not a DOM render. comix is an SPA — rendering it costs ~65
requests for the same server-rendered HTML one fetch returns (24.5 KB,
~480 ms measured). `comixSeriesPageURL` pins scheme + host + `/title/<slug>`
and rebuilds the address, so a client-supplied `series_url` cannot aim the
browser anywhere else.
- **Cover bytes**: `comixImageURLRe` pins `https://static.comix.to/<path>.<ext>`;
`BrowserFetcher.Image` now gates on `browserOnlyCoverURL` rather than a
kagane-only regex, so both Sites' image URLs route through the one path.
Bytes come from direct navigation, not a page-context fetch — comix's Series
page sets `cross-origin-embedder-policy: require-corp`, which fails one.
- **Parsers and stored Series identity: untouched.** The in-tab body is the same
server-rendered HTML the existing fixtures were cut from.
## Verification
- `go test ./...` green (needs Docker).
- New seam tests: comix routes to the browser when one is configured, and is
not fetched at all when none is (`TestComixUsesBrowserFetcher`,
`TestComixSkippedWhenNoBrowserFetcher`); URL-pin and cover-gate table tests.
- Live proof against the real browser unit, `TestSmokeComix` (env-gated):
page 24793 bytes in one in-tab fetch, chapter 53, cover accepted by the pin,
26862 bytes of `image/jpg` retrieved.
- Two-axis review run; findings were stale comments on `BrowserFetcher`, `Get`
and the `Fallback` field, fixed in f000cc7.
Docs updated: root `AGENTS.md` (constraint + smoke command, including the note
that this dev machine's ISP DNS-hijacks `comix.to`), `backend/AGENTS.md`
(poller, cover pipeline, `BROWSER_WS_URL`), `REDEPLOY.md` §8 degrade note.
Reviewed-on: #105
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
73 lines
2.2 KiB
Go
73 lines
2.2 KiB
Go
package latest
|
|
|
|
import (
|
|
"context"
|
|
"os"
|
|
"testing"
|
|
"time"
|
|
|
|
"bookmarkmanager/backend/internal/store"
|
|
)
|
|
|
|
// TestSmokeComix answers "is comix's challenge clearing from this browser right
|
|
// now" — a live, time-varying fact, so a red run is something to re-check
|
|
// before it is a defect. Needs the real browser unit with outbound network:
|
|
//
|
|
// cd chrome && BROWSER_BIND_ADDR=127.0.0.1 docker compose up -d --build
|
|
// SMOKE_BROWSER_WS_URL=ws://127.0.0.1:9222 go test -run TestSmokeComix ./internal/latest
|
|
//
|
|
// It walks the whole read: the in-tab page fetch, both parses, and the Cover
|
|
// bytes by direct navigation to static.comix.to. The Cover address comes out of
|
|
// the page rather than being pinned in the test, because a stored one rots.
|
|
func TestSmokeComix(t *testing.T) {
|
|
ws := os.Getenv("SMOKE_BROWSER_WS_URL")
|
|
if ws == "" {
|
|
t.Skip("SMOKE_BROWSER_WS_URL unset")
|
|
}
|
|
const seriesURL = "https://comix.to/title/m12d-classmate"
|
|
|
|
f, err := NewBrowserFetcher(ws)
|
|
if err != nil {
|
|
t.Fatalf("NewBrowserFetcher: %v", err)
|
|
}
|
|
defer f.Close()
|
|
|
|
ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
|
|
defer cancel()
|
|
|
|
body, status, err := f.Get(ctx, seriesURL)
|
|
if err != nil {
|
|
t.Fatalf("Get: %v", err)
|
|
}
|
|
t.Logf("status=%d bytes=%d", status, len(body))
|
|
if status != 200 {
|
|
t.Fatalf("status = %d, want 200 — the sidecar is not clearing the challenge", status)
|
|
}
|
|
chapter, ok := latestChapterFrom("comix", seriesURL, body)
|
|
if !ok {
|
|
t.Fatalf("no latest chapter in %d bytes — page shape changed", len(body))
|
|
}
|
|
t.Logf("latest chapter: %v %q", chapter.Num, chapter.Label)
|
|
|
|
cover, ok := coverFrom("comix", seriesURL, body)
|
|
if !ok {
|
|
t.Fatalf("no cover address in %d bytes — page shape changed", len(body))
|
|
}
|
|
t.Logf("cover: %s", cover)
|
|
if !browserOnlyCoverURL(cover) {
|
|
t.Fatalf("cover %q is not claimed by the browser gate: the pin and the live URL shape disagree", cover)
|
|
}
|
|
|
|
bytes, contentType, err := f.Image(ctx, cover)
|
|
if err != nil {
|
|
t.Fatalf("Image: %v", err)
|
|
}
|
|
if len(bytes) < 1000 {
|
|
t.Fatalf("cover is %d bytes, want a real image", len(bytes))
|
|
}
|
|
t.Logf("fetched %d bytes of %s", len(bytes), contentType)
|
|
if _, ok := store.CoverContentType(contentType); !ok {
|
|
t.Fatalf("content type %q is not storable", contentType)
|
|
}
|
|
}
|