Unscope lnw chapter scan, truncate at comment thread (#87)
The Poll scoped the lightnovelworld scan to the slug derived from the stored address, so a Series publishing chapters under more than one Chapter Slug froze its Latest Chapter. The branch now matches any chapter-shaped address on the host (lnwChapterRe) and scans only the body before the wpdiscuz thread marker (wpd-threads); a body without the marker is skipped and logged with its length, never scanned whole. The size-cap comment now states the measured ~3.5x headroom.
This commit is contained in:
@@ -11,8 +11,9 @@ import (
|
||||
)
|
||||
|
||||
// maxBodyBytes caps what a single series page can cost in memory. Real pages
|
||||
// measured 100-400 KB on 2026-07-26, so this is roughly 10x headroom and mostly
|
||||
// guards against a proxy handing back something enormous.
|
||||
// measured 100-400 KB on 2026-07-26; lightnovelworld runs larger — 685 KB and
|
||||
// 1.18 MB measured 2026-08-11 — so the headroom there is roughly 3.5x, and the
|
||||
// cap mostly guards against a proxy handing back something enormous.
|
||||
const maxBodyBytes = 4 << 20
|
||||
|
||||
// chromeUA matches the client profile below. A Chrome fingerprint paired with a
|
||||
|
||||
Reference in New Issue
Block a user