lnw scan fixture: real page text, better novel #86

New Issue

2026-08-11T11:28:08+07:00

sulthan commented

2026-08-11 11:28:08 +07:00

Parent

#80 (spec) — implements part of #77.

What to build

The backend's lightnovelworld chapter-scan test stops passing because of a fiction. Today's fixture is trimmed from a-will-eternal and carries a hand-written cross-series anchor annotated "the last anchor is another series" — the live survey found no Series page on this Site that carries a foreign chapter anchor, so the test that proves the scan "ignores another series" proves nothing about a page the Site actually serves.

Replace it with text trimmed from a real fetched Series page, and pick a different novel: a-will-eternal's Chapter Slug and series slug coincide — which is exactly why the defect in the parent spec slipped through — and its chapter numbering is the anomaly filed as #79.

This is a prefactor. Scan behaviour does not change here; the existing scoped scan must still be green against the new fixture.

Note when fetching: these Series pages run up to ~1.18 MB and the interesting part sits far below the fold. A probe that reads the first few hundred KB will mislead you — two earlier investigations reached opposite conclusions on exactly this. Fetch the whole document. Only the .net domain is live; .com is shut down and serves a notice page.

Acceptance criteria

The lightnovelworld fixture is text trimmed from a real page fetched from the live Site, with the fetch date recorded the way the neighbouring fixtures record theirs
The invented cross-series anchor is gone, and no test asserts behaviour against a page shape the Site does not produce
The fixture's novel is neither a-will-eternal nor any other novel whose Chapter Slug equals its series slug
cd backend && go test ./... passes with the scan logic unchanged

Blocked by

None — can start immediately.

## Parent #80 (spec) — implements part of #77. ## What to build The backend's lightnovelworld chapter-scan test stops passing because of a fiction. Today's fixture is trimmed from `a-will-eternal` and carries a hand-written cross-series anchor annotated "the last anchor is another series" — the live survey found no Series page on this Site that carries a foreign chapter anchor, so the test that proves the scan "ignores another series" proves nothing about a page the Site actually serves. Replace it with text trimmed from a real fetched Series page, and pick a different novel: `a-will-eternal`'s Chapter Slug and series slug coincide — which is exactly why the defect in the parent spec slipped through — and its chapter numbering is the anomaly filed as #79. This is a prefactor. Scan behaviour does not change here; the existing scoped scan must still be green against the new fixture. Note when fetching: these Series pages run up to ~1.18 MB and the interesting part sits far below the fold. A probe that reads the first few hundred KB will mislead you — two earlier investigations reached opposite conclusions on exactly this. Fetch the whole document. Only the `.net` domain is live; `.com` is shut down and serves a notice page. ## Acceptance criteria - [ ] The lightnovelworld fixture is text trimmed from a real page fetched from the live Site, with the fetch date recorded the way the neighbouring fixtures record theirs - [ ] The invented cross-series anchor is gone, and no test asserts behaviour against a page shape the Site does not produce - [ ] The fixture's novel is neither `a-will-eternal` nor any other novel whose Chapter Slug equals its series slug - [ ] `cd backend && go test ./...` passes with the scan logic unchanged ## Blocked by - None — can start immediately.

sulthan added the ready-for-agent label 2026-08-11 11:28:08 +07:00

sulthan referenced this issue

2026-08-11 11:28:08 +07:00

Poll's lnw chapter scan: unscoped and comment-truncated #87

sulthan self-assigned this 2026-08-11 11:34:30 +07:00

sulthan commented

2026-08-11 11:53:57 +07:00

Landed on main as 01aed58 (merge of ticket/86-lnw-scan-fixture; commits 8d310bb, 5bee351).

lnwSeriesFixture is now text trimmed from a real, wholly-fetched /novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/ page: 8 verbatim

chapter-anchor blocks spanning both Chapter Slugs (1, 50, 99 under …-not; 100, 200, 300, 404, 423 under …-not-them-all), the wpd-threads marker exactly once, and a real wpdiscuz comment block after it. The invented overgeared-chapter-9999 anchor and the 'ignores another series' assertion are gone. Scan logic untouched: sites.go and fetch.go are byte-for-byte identical.

Completeness proof, worth recording: the page's wire body is ~32 KB because it is zstd-encoded, decoding to ~307 KB — curl's size_download misleads here. The document was verified whole (proper </html>, 425 absolute chapter anchors, full wpdiscuz region) across two independent fetches.

Two deviations, both accepted:

The pinned page serves zero comments today (data-comments-count=0), so the real comment block was taken verbatim from a sibling page on the same Site and theme (the-sword-illuminates-the-great-wilderness, comment #wpd-comm-358_0), with one composed element — a bare to a real chapter of a real different novel. Provenance is disclosed in the fixture doc comment.

Anchor order mirrors the real page (newest-first: …-not-them-all before …-not), the reverse of the ticket's literal listing. Reordering would fabricate a page shape the Site never serves; scan outcomes and #87's truncation are unaffected.

a-will-eternal remains in two URL-shape-only tests, where it is real and carries no fiction. go test ./... green.

Landed on main as 01aed58 (merge of ticket/86-lnw-scan-fixture; commits 8d310bb, 5bee351). lnwSeriesFixture is now text trimmed from a real, wholly-fetched /novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/ page: 8 verbatim <li> chapter-anchor blocks spanning both Chapter Slugs (1, 50, 99 under …-not; 100, 200, 300, 404, 423 under …-not-them-all), the wpd-threads marker exactly once, and a real wpdiscuz comment block after it. The invented overgeared-chapter-9999 anchor and the 'ignores another series' assertion are gone. Scan logic untouched: sites.go and fetch.go are byte-for-byte identical. Completeness proof, worth recording: the page's wire body is ~32 KB because it is zstd-encoded, decoding to ~307 KB — curl's size_download misleads here. The document was verified whole (proper </html>, 425 absolute chapter anchors, full wpdiscuz region) across two independent fetches. Two deviations, both accepted: 1. The pinned page serves zero comments today (data-comments-count=0), so the real comment block was taken verbatim from a sibling page on the same Site and theme (the-sword-illuminates-the-great-wilderness, comment #wpd-comm-358_0), with one composed element — a bare <a href> to a real chapter of a real different novel. Provenance is disclosed in the fixture doc comment. 2. Anchor order mirrors the real page (newest-first: …-not-them-all before …-not), the reverse of the ticket's literal listing. Reordering would fabricate a page shape the Site never serves; scan outcomes and #87's truncation are unaffected. a-will-eternal remains in two URL-shape-only tests, where it is real and carries no fiction. go test ./... green.

sulthan closed this issue

2026-08-11 11:53:57 +07:00

sulthan referenced this issue

2026-08-11 13:38:12 +07:00

lightnovelworld Series identity is read from the chapter page (#80) #92

sulthan referenced this issue from a commit

2026-08-11 18:21:52 +07:00

lightnovelworld Series identity is read from the chapter page (#80) (#92)

Sign in to join this conversation.