feat: one registry entry per Site, one shared Series-page read #95
@@ -70,6 +70,15 @@ Reader present. Performed once per Series no matter how many Readers bookmarked
|
||||
a Poll is work done on behalf of the Series, never on behalf of a Reader.
|
||||
_Avoid_: scrape, refresh, check, sync
|
||||
|
||||
**Acquisition**:
|
||||
The single read of a Series page made the moment the Series first exists, giving it
|
||||
both its Latest Chapter and its Cover without waiting out the Poll queue. Distinct
|
||||
from a Poll in the two ways that matter: a Reader is present — it is triggered by
|
||||
their first Bookmark of that Series — and it is the only read that establishes a
|
||||
Cover rather than refreshing facts. It happens once in a Series's life; every later
|
||||
read of the same page is a Poll.
|
||||
_Avoid_: initial poll, first fetch, prefetch, warm-up
|
||||
|
||||
**New Chapter**:
|
||||
The state where Latest Chapter is ahead of Progress. The single condition the ember
|
||||
accent is permitted to signal.
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
# ADR-0009: A Site answers fixed questions; an unusual Site owns its own fetch
|
||||
|
||||
Date: 2026-08-11
|
||||
Status: accepted
|
||||
|
||||
## Decision
|
||||
|
||||
Per-Site knowledge lives in one registry in `backend/internal/latest/sites.go`,
|
||||
keyed by the stored site string. An entry answers a fixed set of questions: the
|
||||
hostname a `series_url` must carry, how to find the Latest Chapter in a body,
|
||||
how to find the Cover address in a body, and — for a Site behind a JavaScript
|
||||
challenge — how to read its payload from a cleared tab and how to tell that the
|
||||
payload arrived.
|
||||
|
||||
The question set does not grow to accommodate one Site. When a Site needs
|
||||
something the set cannot express, that Site gets an optional override and
|
||||
performs its own fetch, leaving the other entries untouched. The override is a
|
||||
per-Site escape hatch, not a stage every Site passes through, and it is added to
|
||||
the registry type when the first Site needs it rather than in advance.
|
||||
|
||||
Two things stay outside the registry. Cover *bytes* are routed by address shape
|
||||
in `fetchCoverBytes`, never by Site name, so the Poll and the Acquisition cannot
|
||||
drift apart. And the host pin inside a browser entry's read is kept even though
|
||||
`fetchableSeriesURL` has already pinned the same host: a headless browser is a
|
||||
strong SSRF primitive and `series_url` arrives in a client-supplied PUT body, so
|
||||
the second check is deliberate and must not be deduplicated.
|
||||
|
||||
## Why
|
||||
|
||||
Before the registry, the site string was compared in six places across three
|
||||
files: the Latest Chapter switch (`sites.go:102`), the Cover switch
|
||||
(`sites.go:263`), the browser-backed list and the fetcher choice
|
||||
(`poller.go:60`, `poller.go:132`), the host pins (`poller.go:314-325`), and the
|
||||
payload read (`browser.go:96-119`). Nothing tied them together, so adding a
|
||||
seventh Site meant finding all six unaided, and a Site added to five of them
|
||||
failed at the sixth in production rather than at compile time.
|
||||
|
||||
The Sites are not alike and the registry does not ask them to be. asura strips a
|
||||
rotating build hash from its slug before scoping a regex; comix reads a JSON
|
||||
blob embedded in server-rendered HTML; kagane's chapter list exists only in its
|
||||
JSON API, which must be called from inside the page so the request carries the
|
||||
clearance cookie; lightnovelworld must truncate the body at the comment thread
|
||||
first. What they have in common is not behaviour, it is the questions they
|
||||
answer. Arbitrary behaviour behind one entry is the point.
|
||||
|
||||
Making a browser Site contribute a read and a completion test, rather than
|
||||
letting it drive the browser, was chosen because the tab lifecycle in
|
||||
`BrowserFetcher.run` is load-bearing and shared. It holds one tab open across
|
||||
re-reads, because a Cloudflare interstitial needs several seconds of live page
|
||||
to solve itself and write clearance into the shared cookie jar; reading once and
|
||||
closing the tab, which is what this did before 2026-08-08, never clears
|
||||
anything. It also serialises the browser, binds the caller's deadline to the
|
||||
tab, distinguishes a lost browser from a retryable read, and paces re-reads.
|
||||
Spreading that across per-Site adapters would put one subtle, measured loop
|
||||
behind six doors.
|
||||
|
||||
This costs the adapters little, because `chromedp.Run` takes an Action and
|
||||
`chromedp.Tasks` is an Action. A Site that must click, wait on a selector, and
|
||||
then evaluate expresses all of it as its read. Only a Site needing something
|
||||
outside the per-tab loop — its own cadence, two tabs, a tab held between calls,
|
||||
cookies set before navigation — falls outside, and that Site takes the override.
|
||||
|
||||
## Considered options
|
||||
|
||||
**Widen the shared interface whenever a Site needs something new.** Rejected:
|
||||
one Site's requirement becomes a field on all seven entries, and the entries
|
||||
that ignore it still have to be read and understood by anyone adding the eighth.
|
||||
|
||||
**Give every Site the whole fetch.** Rejected: it makes the browser lifecycle
|
||||
above a per-Site concern, and pulls `chromedp` into adapters for five Sites that
|
||||
never open a browser.
|
||||
|
||||
**A Go `interface` with a method set instead of a registry of records.**
|
||||
Rejected: most Sites differ in one or two answers, and three share a single
|
||||
Cover implementation, so a method set produces near-empty types. A missing
|
||||
answer is a nil value caught at dispatch, which is where an unknown Site is
|
||||
already handled.
|
||||
|
||||
## Consequences
|
||||
|
||||
Adding a Site is one registry entry. The existing dispatch functions —
|
||||
`latestChapterFrom`, `coverFrom`, `fetchableSeriesURL` — become registry
|
||||
lookups, so the table tests that drive them by site string are unchanged.
|
||||
|
||||
A future architecture review will see an override that only one Site uses and
|
||||
read it as an inconsistency to collapse. It is not. Collapsing it means either
|
||||
widening the question set for every Site or moving the shared tab lifecycle into
|
||||
the adapters, and both were rejected here on the evidence above.
|
||||
|
||||
An unknown site string resolves to the zero entry and fails the existing
|
||||
not-fetchable and no-fetcher paths, which log and skip. That is unchanged.
|
||||
Reference in New Issue
Block a user