From 4a92e956cfb3ae7c17b3f31bce86d81fac82a9aa Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Wed, 12 Aug 2026 00:09:22 +0700 Subject: [PATCH] docs: record ADR-0009 Site registry and define Acquisition (#94) --- CONTEXT.md | 9 ++ ...09-a-site-answers-questions-its-own-way.md | 91 +++++++++++++++++++ 2 files changed, 100 insertions(+) create mode 100644 docs/adr/0009-a-site-answers-questions-its-own-way.md diff --git a/CONTEXT.md b/CONTEXT.md index 2501a21..19163e3 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -70,6 +70,15 @@ Reader present. Performed once per Series no matter how many Readers bookmarked a Poll is work done on behalf of the Series, never on behalf of a Reader. _Avoid_: scrape, refresh, check, sync +**Acquisition**: +The single read of a Series page made the moment the Series first exists, giving it +both its Latest Chapter and its Cover without waiting out the Poll queue. Distinct +from a Poll in the two ways that matter: a Reader is present — it is triggered by +their first Bookmark of that Series — and it is the only read that establishes a +Cover rather than refreshing facts. It happens once in a Series's life; every later +read of the same page is a Poll. +_Avoid_: initial poll, first fetch, prefetch, warm-up + **New Chapter**: The state where Latest Chapter is ahead of Progress. The single condition the ember accent is permitted to signal. diff --git a/docs/adr/0009-a-site-answers-questions-its-own-way.md b/docs/adr/0009-a-site-answers-questions-its-own-way.md new file mode 100644 index 0000000..972f45b --- /dev/null +++ b/docs/adr/0009-a-site-answers-questions-its-own-way.md @@ -0,0 +1,91 @@ +# ADR-0009: A Site answers fixed questions; an unusual Site owns its own fetch + +Date: 2026-08-11 +Status: accepted + +## Decision + +Per-Site knowledge lives in one registry in `backend/internal/latest/sites.go`, +keyed by the stored site string. An entry answers a fixed set of questions: the +hostname a `series_url` must carry, how to find the Latest Chapter in a body, +how to find the Cover address in a body, and — for a Site behind a JavaScript +challenge — how to read its payload from a cleared tab and how to tell that the +payload arrived. + +The question set does not grow to accommodate one Site. When a Site needs +something the set cannot express, that Site gets an optional override and +performs its own fetch, leaving the other entries untouched. The override is a +per-Site escape hatch, not a stage every Site passes through, and it is added to +the registry type when the first Site needs it rather than in advance. + +Two things stay outside the registry. Cover *bytes* are routed by address shape +in `fetchCoverBytes`, never by Site name, so the Poll and the Acquisition cannot +drift apart. And the host pin inside a browser entry's read is kept even though +`fetchableSeriesURL` has already pinned the same host: a headless browser is a +strong SSRF primitive and `series_url` arrives in a client-supplied PUT body, so +the second check is deliberate and must not be deduplicated. + +## Why + +Before the registry, the site string was compared in six places across three +files: the Latest Chapter switch (`sites.go:102`), the Cover switch +(`sites.go:263`), the browser-backed list and the fetcher choice +(`poller.go:60`, `poller.go:132`), the host pins (`poller.go:314-325`), and the +payload read (`browser.go:96-119`). Nothing tied them together, so adding a +seventh Site meant finding all six unaided, and a Site added to five of them +failed at the sixth in production rather than at compile time. + +The Sites are not alike and the registry does not ask them to be. asura strips a +rotating build hash from its slug before scoping a regex; comix reads a JSON +blob embedded in server-rendered HTML; kagane's chapter list exists only in its +JSON API, which must be called from inside the page so the request carries the +clearance cookie; lightnovelworld must truncate the body at the comment thread +first. What they have in common is not behaviour, it is the questions they +answer. Arbitrary behaviour behind one entry is the point. + +Making a browser Site contribute a read and a completion test, rather than +letting it drive the browser, was chosen because the tab lifecycle in +`BrowserFetcher.run` is load-bearing and shared. It holds one tab open across +re-reads, because a Cloudflare interstitial needs several seconds of live page +to solve itself and write clearance into the shared cookie jar; reading once and +closing the tab, which is what this did before 2026-08-08, never clears +anything. It also serialises the browser, binds the caller's deadline to the +tab, distinguishes a lost browser from a retryable read, and paces re-reads. +Spreading that across per-Site adapters would put one subtle, measured loop +behind six doors. + +This costs the adapters little, because `chromedp.Run` takes an Action and +`chromedp.Tasks` is an Action. A Site that must click, wait on a selector, and +then evaluate expresses all of it as its read. Only a Site needing something +outside the per-tab loop — its own cadence, two tabs, a tab held between calls, +cookies set before navigation — falls outside, and that Site takes the override. + +## Considered options + +**Widen the shared interface whenever a Site needs something new.** Rejected: +one Site's requirement becomes a field on all seven entries, and the entries +that ignore it still have to be read and understood by anyone adding the eighth. + +**Give every Site the whole fetch.** Rejected: it makes the browser lifecycle +above a per-Site concern, and pulls `chromedp` into adapters for five Sites that +never open a browser. + +**A Go `interface` with a method set instead of a registry of records.** +Rejected: most Sites differ in one or two answers, and three share a single +Cover implementation, so a method set produces near-empty types. A missing +answer is a nil value caught at dispatch, which is where an unknown Site is +already handled. + +## Consequences + +Adding a Site is one registry entry. The existing dispatch functions — +`latestChapterFrom`, `coverFrom`, `fetchableSeriesURL` — become registry +lookups, so the table tests that drive them by site string are unchanged. + +A future architecture review will see an override that only one Site uses and +read it as an inconsistency to collapse. It is not. Collapsing it means either +widening the question set for every Site or moving the shared tab lifecycle into +the adapters, and both were rejected here on the evidence above. + +An unknown site string resolves to the zero entry and fails the existing +not-fetchable and no-fetcher paths, which log and skip. That is unchanged.