Unscope lnw chapter scan, truncate at comment thread (#87)

The Poll scoped the lightnovelworld scan to the slug derived from the
stored address, so a Series publishing chapters under more than one
Chapter Slug froze its Latest Chapter. The branch now matches any
chapter-shaped address on the host (lnwChapterRe) and scans only the
body before the wpdiscuz thread marker (wpd-threads); a body without
the marker is skipped and logged with its length, never scanned whole.
The size-cap comment now states the measured ~3.5x headroom.
This commit is contained in:
2026-08-11 12:59:21 +07:00
parent e864770eb8
commit ecce453d13
3 changed files with 63 additions and 20 deletions
+3 -2
View File
@@ -11,8 +11,9 @@ import (
)
// maxBodyBytes caps what a single series page can cost in memory. Real pages
// measured 100-400 KB on 2026-07-26, so this is roughly 10x headroom and mostly
// guards against a proxy handing back something enormous.
// measured 100-400 KB on 2026-07-26; lightnovelworld runs larger — 685 KB and
// 1.18 MB measured 2026-08-11 — so the headroom there is roughly 3.5x, and the
// cap mostly guards against a proxy handing back something enormous.
const maxBodyBytes = 4 << 20
// chromeUA matches the client profile below. A Chrome fingerprint paired with a