sulthan/mangaBookmark
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

lightnovelworld Series identity is read from the chapter page (#80) (#92)

Browse Source 
Implements spec #80 / ADR-0008 — Gitea issues #86, #87, #88, #89, #90, all closed.

A Reader bookmarks a novel on lightnovelworld and it never shows a New Chapter, because the Series identity was derived from the chapter address instead of read from the page. One Series can publish under several Chapter Slugs, so the derived key points at a slug that 404s.

- **#89** — the userscript's lnw adapter stops deriving `seriesUrl`/`seriesId` from the path. It reads the page's own pointer (`a[aria-label='All Chapter']`), falling back to the microdata breadcrumb's second crumb, and carries `chapterSlug` on the page object, stored nowhere.
- **#87** — the Poll's lnw chapter scan is unscoped (no stored-slug pattern can cover a Series' whole list) and truncated at the `wpd-threads` comment thread, the one region a visitor can write to. Marker absent means skip and log with the body length, never scan whole. Corrects the `maxBodyBytes` headroom comment to the measured 3.5x.
- **#86** — the scan fixture is now text trimmed from a real, wholly-fetched Series page instead of a hand-written cross-series anchor that no live page carries.
- **#90** — stale stored rows repair themselves on the next chapter visit: a pure transform over cache, queue and last-checked map, silent to the Reader, with progress, favourite and lifecycle bucket preserved when two rows merge.
- **#88** — an env-gated live canary (`SMOKE_LNW_SERIES_URL`) proving the marker still occurs exactly once and still follows the last chapter anchor, asserted against the production symbols themselves.

Verified on the merged branch: `go test ./...` green, `gofmt -l internal/latest/` silent, both userscripts `node --check` clean, 35/35 + 29/29 logic tests. Live canary green (marker once at byte 612,182 of 651,795). #90 verified on device with Playwright.

Open follow-up: **#91** — the userscript's client-side latest-chapter scan is still scoped to the derived slug.

Reviewed-on: #92
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #92.
This commit is contained in:
Sulthan Zaki
2026-08-11 18:21:50 +07:00
committed by sulthan
parent 90d8ab72ad
commit e7e22a12a5
13 changed files with 1731 additions and 551 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/internal/latest/sites_test.go
+137 -19
View File
@@ -1,6 +1,9 @@
package latest
import "testing"
import (
"strings"
"testing"
)
// Trimmed from https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af
// fetched 2026-07-26. The first anchor is the "First Chapter" shortcut: it is a
@@ -69,14 +72,113 @@ const novelfullSeriesFixture = `
<a href="/release-that-witch/chapter-9999.html">Chapter 9999</a>
`
// Trimmed from https://lightnovelworld.net/novel/a-will-eternal/ fetched
// 2026-08-05. Its chapter anchors are absolute and flat — /<slug>-chapter-<n>/
// at the site root, not under /novel/. The last anchor is another series'.
// Trimmed from
// https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/
// fetched 2026-08-11, whole document (~307 KB decoded; the wire body is ~32 KB
// zstd-compressed). This
// novel publishes its chapters under two Chapter Slugs: 1–99 at
// …-not-chapter-<n>/ and 100–423 at …-not-them-all-chapter-<n>/, and the page
// lists them newest-first, so the …-not-them-all anchors precede the …-not
// anchors. Every anchor through the comment-thread marker is verbatim page
// text (the site renders this novel's chapter titles as "[ ... words ]"). The
// comment block after the marker is the real wpdiscuz comment #wpd-comm-358_0
// from https://lightnovelworld.net/novel/the-sword-illuminates-the-great-wilderness/
// — the pinned page serves zero comments — with its share/link/vote/reply
// boilerplate trimmed. The comment's body carried no link, so the bare <a
// href> to https://lightnovelworld.net/overgeared-chapter-2059/ inside
// wpd-comment-text is the one composed element; that URL is a real chapter of
// a real different novel (overgeared; fetched, HTTP 200).
const lnwSeriesFixture = `
<a href="https://lightnovelworld.net/a-will-eternal-chapter-1/">Chapter 1</a>
<a href="https://lightnovelworld.net/a-will-eternal-chapter-1317/">Chapter 1317</a>
<a href="https://lightnovelworld.net/a-will-eternal-chapter-1298/">Chapter 1298</a>
<a href="https://lightnovelworld.net/overgeared-chapter-9999/">Chapter 9999</a>
<li data-ID="102741">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-404/">
<div class="epl-num">Vol. 1 Ch. 404</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">April 12, 2026</div>
</a>
</li>
<li data-ID="102780">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-423/">
<div class="epl-num">Vol. 1 Ch. 423</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">April 7, 2026</div>
</a>
</li>
<li data-ID="102527">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-300/">
<div class="epl-num">Vol. 1 Ch. 300</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">April 4, 2026</div>
</a>
</li>
<li data-ID="102325">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-200/">
<div class="epl-num">Vol. 1 Ch. 200</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">March 29, 2026</div>
</a>
</li>
<li data-ID="102121">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-100/">
<div class="epl-num">Vol. 1 Ch. 100</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">March 22, 2026</div>
</a>
</li>
<li data-ID="26014">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-chapter-99/">
<div class="epl-num">Vol. 1 Ch. 99</div>
<div class="epl-title">Chapter 99</div>
<div class="epl-date">November 5, 2025</div>
</a>
</li>
<li data-ID="25916">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-chapter-50/">
<div class="epl-num">Vol. 1 Ch. 50</div>
<div class="epl-title">Chapter 50</div>
<div class="epl-date">October 29, 2025</div>
</a>
</li>
<li class='tseplsfrst' data-ID="25818">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-chapter-1/">
<div class="epl-num">Vol. 1 Ch. 1</div>
<div class="epl-title">Chapter 01</div>
<div class="epl-date">October 11, 2025</div>
</a>
</li>
<div id="wpd-threads" class="wpd-thread-wrapper">
<div class="wpd-thread-list">
<div id='wpd-comm-358_0' class='comment byuser comment-author-jimbear even thread-even depth-1 wpd-comment wpd_comment_level-1'><div class="wpd-comment-wrap wpd-blog-user wpd-blog-subscriber">
<div class="wpd-comment-left ">
<div class="wpd-avatar ">
<img alt='hasbi asy' src='https://secure.gravatar.com/avatar/3c1792cb31cab842f90e7c463f0948e98536cf938aebbd2f678f937ca60fb799?s=64&#038;d=mm&#038;r=g' srcset='https://secure.gravatar.com/avatar/3c1792cb31cab842f90e7c463f0948e98536cf938aebbd2f678f937ca60fb799?s=128&#038;d=mm&#038;r=g 2x' class='avatar avatar-64 photo' height='64' width='64' decoding='async'/>
</div>
<div class="wpd-comment-label" wpd-tooltip="Member" wpd-tooltip-position="right">
<span>Member</span>
</div>
</div>
<div id="comment-358" class="wpd-comment-right">
<div class="wpd-comment-header">
<div class="wpd-comment-author ">
hasbi asy
</div>
<div class="wpd-comment-date" title="July 9, 2026 1:44 am">
<i class='far fa-clock' aria-hidden='true'></i>
1 month ago
</div>
</div>
<div class="wpd-comment-text">
<p>where&#8217;s everyone</p>
<a href="https://lightnovelworld.net/overgeared-chapter-2059/">https://lightnovelworld.net/overgeared-chapter-2059/</a>
</div>
</div>
</div>
<div id='wpdiscuz_form_anchor-358_0'></div>
</div>
</div>
`
// Trimmed from https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af
@@ -98,8 +200,10 @@ const kaganeCoverFixture = `{"series_covers":[{"cover_id":"019fe11a-84d1-714b-9c
// Trimmed from https://novelfull.com/reverend-insanity.html on 2026-08-10.
const novelfullCoverFixture = `<meta name="image" content="https://novelfull.com/uploads/webp/novel/reverend-insanity-82661d911a.webp">`
// Trimmed from https://lightnovelworld.net/novel/a-will-eternal/ on 2026-08-10.
const lnwCoverFixture = `<meta content='https://lightnovelworld.net/wp-content/uploads/2026/03/a-will-eternal-1.webp' property='og:image'>`
// Trimmed from
// https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/
// on 2026-08-11.
const lnwCoverFixture = `<meta property="og:image" content="https://i1.wp.com/lightnovelworld.net/wp-content/uploads/2025/10/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all.jpg" />`
func TestCoverFrom(t *testing.T) {
const comixURL = "https://comix.to/title/n8we-dungeons-and-crayons"
@@ -139,7 +243,7 @@ func TestCoverFrom(t *testing.T) {
{
name: "lightnovelworld reads og image",
site: "lightnovelworld", body: lnwCoverFixture, wantOK: true,
wantCover: "https://lightnovelworld.net/wp-content/uploads/2026/03/a-will-eternal-1.webp",
wantCover: "https://i1.wp.com/lightnovelworld.net/wp-content/uploads/2025/10/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all.jpg",
},
{
name: "later metadata cover survives empty match",
@@ -307,24 +411,38 @@ func TestLatestChapterFrom(t *testing.T) {
body: novelfullSeriesFixture,
wantOK: false,
},
// Stored before the slug split, so the address carries the ...-not
// Chapter Slug; the 100-423 block under the other slug must still win.
// The body is the chapter-list portion of lnwSeriesFixture with the
// comment block omitted; the marker is kept, because a body without it
// is skipped, not scanned.
{
name: "lightnovelworld takes the max and ignores another series",
name: "lightnovelworld max spans both chapter slugs",
site: "lightnovelworld",
seriesURL: "https://lightnovelworld.net/novel/a-will-eternal/",
body: lnwSeriesFixture,
wantOK: true, wantNum: 1317, wantLabel: "Chapter 1317",
seriesURL: "https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not/",
body: strings.SplitN(lnwSeriesFixture, lnwCommentMarker, 2)[0] + lnwCommentMarker,
wantOK: true, wantNum: 423, wantLabel: "Chapter 423",
},
{
name: "lightnovelworld tolerates a series url with no trailing slash",
name: "lightnovelworld comment anchor cannot set the latest chapter",
site: "lightnovelworld",
seriesURL: "https://lightnovelworld.net/novel/a-will-eternal",
seriesURL: "https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/",
body: lnwSeriesFixture,
wantOK: true, wantNum: 1317, wantLabel: "Chapter 1317",
wantOK: true, wantNum: 423, wantLabel: "Chapter 423",
},
// Marker removed from the fixture, comment block still present: a
// redesign must degrade into a skip, never into the comment's number.
{
name: "lightnovelworld body without the comment marker is skipped",
site: "lightnovelworld",
seriesURL: "https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/",
body: strings.ReplaceAll(lnwSeriesFixture, lnwCommentMarker, ""),
wantOK: false,
},
{
name: "lightnovelworld yields nothing on a challenge page",
site: "lightnovelworld",
seriesURL: "https://lightnovelworld.net/novel/a-will-eternal/",
seriesURL: "https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/",
body: challengeFixture,
wantOK: false,
},