Each Site runs its own poll goroutine, paced by Rest and Gap from the
registry (sites.go) instead of the shared cooldown/interval/stagger/batch
config. Rest is enforced by the due query's WHERE clause; the Lane sleeps
its effective gap between fetches (hour/eligible, floored at 1s).
Lane-local failures: two refusals stop that Site for 15m, a lost browser
stops only the round's remaining browser Lanes, and cover heals moved to
background goroutines so a slow CDN cannot consume a Lane's gap. The five
LATEST_CHAPTER_POLL_* pace env vars are gone; only the kill switch
remains.
Closes#98.
comix.to began answering plain-TLS fetches with a Cloudflare JavaScript
challenge on 2026-08-12, so every poll got a 403 interstitial. Its cover host
`static.comix.to` is gated the same way. comix therefore joins kagane and
novelfull as a browser-backed Site.
## What changed
- **Registry** (`internal/latest/sites.go`): comix gains a `Browser` entry —
`comixRead`, `Done: body != "" && !isInterstitial(body)`, `Fallback: false`.
Skip-when-no-browser falls out of the existing routing; no site-string compare
was added anywhere.
- **Read shape** (`internal/latest/browser.go`): an in-tab `fetch()` of the
Series URL, not a DOM render. comix is an SPA — rendering it costs ~65
requests for the same server-rendered HTML one fetch returns (24.5 KB,
~480 ms measured). `comixSeriesPageURL` pins scheme + host + `/title/<slug>`
and rebuilds the address, so a client-supplied `series_url` cannot aim the
browser anywhere else.
- **Cover bytes**: `comixImageURLRe` pins `https://static.comix.to/<path>.<ext>`;
`BrowserFetcher.Image` now gates on `browserOnlyCoverURL` rather than a
kagane-only regex, so both Sites' image URLs route through the one path.
Bytes come from direct navigation, not a page-context fetch — comix's Series
page sets `cross-origin-embedder-policy: require-corp`, which fails one.
- **Parsers and stored Series identity: untouched.** The in-tab body is the same
server-rendered HTML the existing fixtures were cut from.
## Verification
- `go test ./...` green (needs Docker).
- New seam tests: comix routes to the browser when one is configured, and is
not fetched at all when none is (`TestComixUsesBrowserFetcher`,
`TestComixSkippedWhenNoBrowserFetcher`); URL-pin and cover-gate table tests.
- Live proof against the real browser unit, `TestSmokeComix` (env-gated):
page 24793 bytes in one in-tab fetch, chapter 53, cover accepted by the pin,
26862 bytes of `image/jpg` retrieved.
- Two-axis review run; findings were stale comments on `BrowserFetcher`, `Get`
and the `Fallback` field, fixed in f000cc7.
Docs updated: root `AGENTS.md` (constraint + smoke command, including the note
that this dev machine's ISP DNS-hijacks `comix.to`), `backend/AGENTS.md`
(poller, cover pipeline, `BROWSER_WS_URL`), `REDEPLOY.md` §8 degrade note.
Reviewed-on: #105
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
Two new glossary terms in `CONTEXT.md`, settled in a design session, plus the graphify refresh.
- **Poll Lane** — one Site's own stream of Polls, carrying the pace at which that Site is willing to be asked. No Lane can slow, block or borrow from another's; a Reader never has one.
- **Sighting** — what a Reader's browser happened to see of a Series's Latest Chapter. Reports the same fact as a Poll, carries none of its authority.
- **Latest Chapter** amended: it no longer claims to be discovered without the reader present, since a Sighting establishes it between Polls.
No code. The work these terms describe is specified in #98 (comix), #101 (Poll Lanes), #102 (admin page) and #103 (Sightings).
Reviewed-on: #104
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
Closes#96.
## What
Removes every reference that still invites a Reader onto `asuracomic.net`.
The domain's deep links 301 to the `asurascans.com` **root**, discarding the
path (re-checked 2026-07-25), so a page on it never yields a series document
client-side and a stored address on it never yields a series page server-side.
#95 already pinned each Site to one hostname, so the backend rejects such an
address cleanly; this is the cleanup around that.
| File | Change |
|---|---|
| `userscript/manga-bookmark.user.js` | drops the `@match`, narrows the asura adapter to `/(^\|\.)asurascans\.com$/` |
| `userscript/test/logic.test.js` | new test pinning the narrowed host match |
| `.env.example`, `docker-compose.yml` | origin dropped from the `ALLOWED_ORIGINS` default |
| `DEPLOY.md` | same, and the sample list gains the two novel origins it was missing |
| `backend/api_test.go` | CORS fixtures and round-trip seed move to `asurascans.com` |
| `README.md`, `AGENTS.md` | notes say the host is dropped, not "stays matched" |
## Behaviour
- A Reader landing on `asuracomic.net` gets no userscript UI. Previously the
script loaded and could do nothing useful — the redirect had already
discarded the path.
- A request whose `Origin` is `https://asuracomic.net` is no longer reflected
by a deployment using the shipped defaults.
- No backend logic changed: the CORS rule, the address gate and the poller are
untouched. `AllowedOrigins` is data, not code.
## Security invariant preserved
CORS still reflects `Origin` only when it appears in `ALLOWED_ORIGINS`, with
`GET,PUT,DELETE,OPTIONS` and a `204` preflight — `TestCORSPreflight` and
`TestCORSDisallowedOrigin` still pin both halves, now against a live origin.
This change only removes a value from the allowlist, which is a narrowing.
## Verification
- `go test ./...` — full backend suite green (real Postgres per package).
- `node --test test/*.test.js` — 66/66 green, up one from the new match test.
## Deploy note (does not happen on merge)
The live allowlist comes from the VPS `.env`, not from these defaults, so the
origin must be dropped there in the same deploy. The one-off row repair for any
stored `asuracomic.net` address is in #96.
Reviewed-on: #97
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
Closes#94.
## What
Two phases per the spec, in three feature commits plus two review-fix commits:
**Phase one — one registry entry per Site** (`2d134fb`)
The six per-site comparison points that used to live across three files collapse
into one `sites` map in `backend/internal/latest/sites.go`: Latest Chapter parse,
Cover parse, browser-backed list, fetcher route, host pins, and the browser
payload read all become lookups into it. `browserBackedSites()` is derived from
the registry (sorted, deterministic); `fetcherFor` and `fetchableSeriesURL` keep
their signatures and become lookups; `BrowserFetcher.Get` dispatches through the
entries' `Read`/`Done` while the tab lifecycle stays in `BrowserFetcher.run`.
**Phase two — one shared Series-page read** (`f215130`)
`readSeriesPage` (new `read.go`) performs the read the Poll and the Acquisition
have in common: gate, route, fetch, parse Latest Chapter, parse Cover address.
It returns facts only — polling and persistence policies (stamp order, cooldowns,
cover policy) stay with the callers; `acquire.go` gained the comment naming the
deliberate post-fetch stamp order. The poll's legacy cover heal and the
no-chapter byte-count diagnostic were restored after review (`d998f87`) so the
claims "the Poll keeps its own Cover policy" and "pinning is the only
behavioural change" both hold.
## Behaviour
- All six Sites now pin their host exactly; asura/demonic/comix previously
accepted any https host. For asura this is a strict improvement: its dead old
domain redirects deep links to the site root and would parse the wrong
document.
- Everything else is unchanged: existing parse tables, the challenge-body table
and the gate table pass unmodified except the one deliberate exception — the
gate table gains the three new pin cases.
## Security invariants preserved
- The address gate is recognisably the same rule, now a single registry lookup:
`https` + exact hostname match, all callers route through it. No fetch path
was widened; asura/demonic/comix were narrowed.
- The second host pin inside each browser entry's Read is retained deliberately
(browser = strong SSRF primitive, `series_url` is client-supplied) and is not
deduplicated against the shared gate.
- Review hardening: `fetcherFor` now fails closed for unknown site strings
(previously fell through to the TLS fetcher on an unreachable path), and the
browser dispatch iterates a sorted list so outcomes cannot depend on map order.
- The security review's log-injection finding was checked against Go's
`url.Parse` and does not hold: control characters are rejected anywhere in a
URL, so a client-supplied value in a log line cannot carry a newline.
## Review
Reviewed on three axes (spec, standards, security) by read-only subagents over
`672c16f..f1b26f4`. No blocking findings; all minor/nit findings addressed in
`d998f87` and `700de20`. Verified end to end with `go test ./...` (Docker
Postgres per test package) on every commit.
## Out of scope (tracked separately)
- Dropping asuracomic.net (CORS allowlist, userscript match, API fixtures,
live env) — separate issue, per spec.
Reviewed-on: #95
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
Follows the spec published on #91: remove the client-side latest-chapter scan for lightnovelworld rather than porting #87's truncation into a second codebase.
- lightnovelworld.latestChapterFromAnchors deleted, not stubbed: absence is what the background-fetch guard keys off.
- computeLatestChapter tolerates an adapter with no scanner (yields null) and is exported as the test seam.
- backgroundRefreshLatest skips a scanner-less Site before the due filter: no Series page fetched, no freshness timestamp recorded, no batch slot consumed. The on-page path (maybeCaptureLatestOnSeriesPage) routes through the same null-tolerant computation.
- novelfull's scanner, the shared max-chapter helper and all four manga Sites untouched.
- userscript/AGENTS.md records the Poll-only contract for this Site and why.
All seven acceptance criteria from the spec met. node --check clean; novel suite 30/30 (the regression pin fails if a lnw scan is reintroduced, scoped or not); manga suite 35/35, manga userscript byte-for-byte unchanged.
Two-axis code review: no hard standard violations, spec-clean; one follow-up commit matching the sibling adapter guard from the manga script.
Reviewed-on: #93
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
Implements spec #80 / ADR-0008 — Gitea issues #86, #87, #88, #89, #90, all closed.
A Reader bookmarks a novel on lightnovelworld and it never shows a New Chapter, because the Series identity was derived from the chapter address instead of read from the page. One Series can publish under several Chapter Slugs, so the derived key points at a slug that 404s.
- **#89** — the userscript's lnw adapter stops deriving `seriesUrl`/`seriesId` from the path. It reads the page's own pointer (`a[aria-label='All Chapter']`), falling back to the microdata breadcrumb's second crumb, and carries `chapterSlug` on the page object, stored nowhere.
- **#87** — the Poll's lnw chapter scan is unscoped (no stored-slug pattern can cover a Series' whole list) and truncated at the `wpd-threads` comment thread, the one region a visitor can write to. Marker absent means skip and log with the body length, never scan whole. Corrects the `maxBodyBytes` headroom comment to the measured 3.5x.
- **#86** — the scan fixture is now text trimmed from a real, wholly-fetched Series page instead of a hand-written cross-series anchor that no live page carries.
- **#90** — stale stored rows repair themselves on the next chapter visit: a pure transform over cache, queue and last-checked map, silent to the Reader, with progress, favourite and lifecycle bucket preserved when two rows merge.
- **#88** — an env-gated live canary (`SMOKE_LNW_SERIES_URL`) proving the marker still occurs exactly once and still follows the last chapter anchor, asserted against the production symbols themselves.
Verified on the merged branch: `go test ./...` green, `gofmt -l internal/latest/` silent, both userscripts `node --check` clean, 35/35 + 29/29 logic tests. Live canary green (marker once at byte 612,182 of 651,795). #90 verified on device with Playwright.
Open follow-up: **#91** — the userscript's client-side latest-chapter scan is still scoped to the derived slug.
Reviewed-on: #92
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
sulthan