docs: record ADR-0009 Site registry and define Acquisition (#94)

2026-08-12 00:09:22 +07:00
parent 672c16ffbf
commit 4a92e956cf
2 changed files with 100 additions and 0 deletions
@@ -0,0 +1,91 @@
+# ADR-0009: A Site answers fixed questions; an unusual Site owns its own fetch
+
+Date: 2026-08-11
+Status: accepted
+
+## Decision
+
+Per-Site knowledge lives in one registry in `backend/internal/latest/sites.go`,
+keyed by the stored site string. An entry answers a fixed set of questions: the
+hostname a `series_url` must carry, how to find the Latest Chapter in a body,
+how to find the Cover address in a body, and — for a Site behind a JavaScript
+challenge — how to read its payload from a cleared tab and how to tell that the
+payload arrived.
+
+The question set does not grow to accommodate one Site. When a Site needs
+something the set cannot express, that Site gets an optional override and
+performs its own fetch, leaving the other entries untouched. The override is a
+per-Site escape hatch, not a stage every Site passes through, and it is added to
+the registry type when the first Site needs it rather than in advance.
+
+Two things stay outside the registry. Cover *bytes* are routed by address shape
+in `fetchCoverBytes`, never by Site name, so the Poll and the Acquisition cannot
+drift apart. And the host pin inside a browser entry's read is kept even though
+`fetchableSeriesURL` has already pinned the same host: a headless browser is a
+strong SSRF primitive and `series_url` arrives in a client-supplied PUT body, so
+the second check is deliberate and must not be deduplicated.
+
+## Why
+
+Before the registry, the site string was compared in six places across three
+files: the Latest Chapter switch (`sites.go:102`), the Cover switch
+(`sites.go:263`), the browser-backed list and the fetcher choice
+(`poller.go:60`, `poller.go:132`), the host pins (`poller.go:314-325`), and the
+payload read (`browser.go:96-119`). Nothing tied them together, so adding a
+seventh Site meant finding all six unaided, and a Site added to five of them
+failed at the sixth in production rather than at compile time.
+
+The Sites are not alike and the registry does not ask them to be. asura strips a
+rotating build hash from its slug before scoping a regex; comix reads a JSON
+blob embedded in server-rendered HTML; kagane's chapter list exists only in its
+JSON API, which must be called from inside the page so the request carries the
+clearance cookie; lightnovelworld must truncate the body at the comment thread
+first. What they have in common is not behaviour, it is the questions they
+answer. Arbitrary behaviour behind one entry is the point.
+
+Making a browser Site contribute a read and a completion test, rather than
+letting it drive the browser, was chosen because the tab lifecycle in
+`BrowserFetcher.run` is load-bearing and shared. It holds one tab open across
+re-reads, because a Cloudflare interstitial needs several seconds of live page
+to solve itself and write clearance into the shared cookie jar; reading once and
+closing the tab, which is what this did before 2026-08-08, never clears
+anything. It also serialises the browser, binds the caller's deadline to the
+tab, distinguishes a lost browser from a retryable read, and paces re-reads.
+Spreading that across per-Site adapters would put one subtle, measured loop
+behind six doors.
+
+This costs the adapters little, because `chromedp.Run` takes an Action and
+`chromedp.Tasks` is an Action. A Site that must click, wait on a selector, and
+then evaluate expresses all of it as its read. Only a Site needing something
+outside the per-tab loop — its own cadence, two tabs, a tab held between calls,
+cookies set before navigation — falls outside, and that Site takes the override.
+
+## Considered options
+
+**Widen the shared interface whenever a Site needs something new.** Rejected:
+one Site's requirement becomes a field on all seven entries, and the entries
+that ignore it still have to be read and understood by anyone adding the eighth.
+
+**Give every Site the whole fetch.** Rejected: it makes the browser lifecycle
+above a per-Site concern, and pulls `chromedp` into adapters for five Sites that
+never open a browser.
+
+**A Go `interface` with a method set instead of a registry of records.**
+Rejected: most Sites differ in one or two answers, and three share a single
+Cover implementation, so a method set produces near-empty types. A missing
+answer is a nil value caught at dispatch, which is where an unknown Site is
+already handled.
+
+## Consequences
+
+Adding a Site is one registry entry. The existing dispatch functions —
+`latestChapterFrom`, `coverFrom`, `fetchableSeriesURL` — become registry
+lookups, so the table tests that drive them by site string are unchanged.
+
+A future architecture review will see an override that only one Site uses and
+read it as an inconsistency to collapse. It is not. Collapsing it means either
+widening the question set for every Site or moving the shared tab lifecycle into
+the adapters, and both were rejected here on the evidence above.
+
+An unknown site string resolves to the zero entry and fails the existing
+not-fetchable and no-fetcher paths, which log and skip. That is unchanged.