417f809d7511046f8809126ef82f00c10d9ab33d
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 sulthan
|
8e4fa6448e |
Spec #136: Finished belongs to the Series — owner-owned poll gate, Lifecycle bucket dropped (#163)
Closes #136. Spec #136 end to end: `finished` becomes a fact about the Series, written only by the owner, and the reader-facing Lifecycle bucket is gone. ## What landed - **#157** — `series.finished_at bigint NOT NULL DEFAULT 0` plus the migration whose statement order is load-bearing (seed from the buckets, then flip them); both Lane queries lose the `HAVING COUNT(*) FILTER (WHERE b.status <> 'finished')` clause and gate on `finished_at = 0` instead, with the due-query/eligible-count force asymmetry kept deliberate and commented; `StatusFinished`, its API special-case 400, the web tab and the templates' Finished bucket deleted. - **#158** — owner Finish control on the Series detail page: confirm-gated finish, instant un-finish, admin accent (never ember, nothing is destroyed), `Store.SetSeriesFinished`, the two routes behind the owner gate, and the state displayed on the list row without offering the control there. - **#160** — reader side: derived `finished` bool on the flat Bookmark (`s.finished_at > 0`), rendered as a text-only label in both userscripts and on the web card; read-only inbound by omission from `Upsert`'s explicit `series` column list, same mechanism that already protects `cover`. - **#161** — glossary and the stale Reader-count divergence note catch up. - **#159** — `finished` joins the admin filter vocabulary (predicate `finished_at > 0`, label `Finished`, own aggregate count, figure last in the stats block as informational); the four clock-driven hygiene predicates (stale, never-checked, no-cover, no-chapter) exclude finished Series while unpollable, orphan and sighting-raised deliberately do not. ## Verification `go vet ./...` and `go test ./...` green on the merged branch (Docker-backed, throwaway `postgres:17-alpine` per package). Each ticket also passed a two-axis review (spec + standards) on its own branch before merge. Reviewed-on: #163 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
0a245a0dde |
docs: name the Stall and the Correction in CONTEXT.md (#133)
Two domain terms the admin surfaces need and CONTEXT.md did not carry: - **Stall** — a Poll Lane that owed Polls, made none, and has nothing to say for it; distinct from a refusing Site and a Paused Lane. - **Correction** — an owner-set Latest Chapter for a Series no Poll can read; lower authority than a Sighting. Docs only. Branch cut fresh off `origin/main`, so it carries nothing from the research branch. Reviewed-on: #133 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
249f11e1fe |
docs: name the admin dashboard's domain terms in CONTEXT.md (#128)
Glossary-only change; no code touched. Charting the admin dashboard map (#114) settled four terms the glossary did not carry: - **Orphan Series** (#125) — a Series no Reader bookmarks; a state of the Series, never a Lifecycle bucket. - **Lane Pass** (#117) — one sweep of a Poll Lane, including a pass that declined to work and why. - **Forced Poll** (#119, #120) — a Poll the owner asks for by marking the Series, which jumps the waiting rules but never the Site's refusal, and which may replace a Cover. - **Paused Lane** (#119) — a bounded, restart-surviving stop on one Site, distinct from the deploy-time kill switch. **Acquisition** is amended in the same pass: establishing a Cover is no longer unique to it, since a Forced Poll can replace one. Reviewed-on: #128 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
3303a55b20 |
feat: one Poll Lane per Site, replacing the shared pace (#100) (#106)
Closes #100. Each Site runs its own Poll Lane: an independent goroutine with its own rest and pace from the registry (`backend/internal/latest/sites.go`), replacing the shared cooldown/interval/stagger/batch configuration. Rest (1h, all six Sites including the browser trio) is enforced by the due query's WHERE clause; the Lane sleeps its effective gap between fetches — the registry 10s, or rest/eligible when a Site holds enough Series, floored at 1s with a Site-naming warning when the floor engages. Lane-local failure handling: - Two challenge-held results stop that Site's Lane for 15m; the probes keep their stamp, untried Series stay due. - A lost browser sets a shared Poller flag: the other browser Lanes skip their passes for the same 15m (no stamp-per-pass-per-Lane on a dead tab), then decay and probe again. - Browser wake gate preserved (5 due, or one waiting 15m, ADR-0005); one tab shared by the three browser Sites; "browser lane behind by X" logged every pass. - Cover work (healing a stored source URL and filling a blank from the series page) runs in the background so a slow CDN cannot consume a Lane's gap. Removed: `LATEST_CHAPTER_POLL_{COOLDOWN,BROWSER_COOLDOWN,INTERVAL,BATCH,STAGGER}` and the 6h browser rest. Only `LATEST_CHAPTER_POLL_ENABLED` remains; DEPLOY.md documents the exact `.env` edit. ADR-0010 records the decisions. Reviewed-on: #106 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
3f53c79cf4 |
docs: add Poll Lane and Sighting to the shared vocabulary (#104)
Two new glossary terms in `CONTEXT.md`, settled in a design session, plus the graphify refresh. - **Poll Lane** — one Site's own stream of Polls, carrying the pace at which that Site is willing to be asked. No Lane can slow, block or borrow from another's; a Reader never has one. - **Sighting** — what a Reader's browser happened to see of a Series's Latest Chapter. Reports the same fact as a Poll, carries none of its authority. - **Latest Chapter** amended: it no longer claims to be discovered without the reader present, since a Sighting establishes it between Polls. No code. The work these terms describe is specified in #98 (comix), #101 (Poll Lanes), #102 (admin page) and #103 (Sightings). Reviewed-on: #104 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
21615be2bd |
feat: one registry entry per Site, one shared Series-page read (#95)
Closes #94. ## What Two phases per the spec, in three feature commits plus two review-fix commits: **Phase one — one registry entry per Site** (`2d134fb`) The six per-site comparison points that used to live across three files collapse into one `sites` map in `backend/internal/latest/sites.go`: Latest Chapter parse, Cover parse, browser-backed list, fetcher route, host pins, and the browser payload read all become lookups into it. `browserBackedSites()` is derived from the registry (sorted, deterministic); `fetcherFor` and `fetchableSeriesURL` keep their signatures and become lookups; `BrowserFetcher.Get` dispatches through the entries' `Read`/`Done` while the tab lifecycle stays in `BrowserFetcher.run`. **Phase two — one shared Series-page read** (`f215130`) `readSeriesPage` (new `read.go`) performs the read the Poll and the Acquisition have in common: gate, route, fetch, parse Latest Chapter, parse Cover address. It returns facts only — polling and persistence policies (stamp order, cooldowns, cover policy) stay with the callers; `acquire.go` gained the comment naming the deliberate post-fetch stamp order. The poll's legacy cover heal and the no-chapter byte-count diagnostic were restored after review (`d998f87`) so the claims "the Poll keeps its own Cover policy" and "pinning is the only behavioural change" both hold. ## Behaviour - All six Sites now pin their host exactly; asura/demonic/comix previously accepted any https host. For asura this is a strict improvement: its dead old domain redirects deep links to the site root and would parse the wrong document. - Everything else is unchanged: existing parse tables, the challenge-body table and the gate table pass unmodified except the one deliberate exception — the gate table gains the three new pin cases. ## Security invariants preserved - The address gate is recognisably the same rule, now a single registry lookup: `https` + exact hostname match, all callers route through it. No fetch path was widened; asura/demonic/comix were narrowed. - The second host pin inside each browser entry's Read is retained deliberately (browser = strong SSRF primitive, `series_url` is client-supplied) and is not deduplicated against the shared gate. - Review hardening: `fetcherFor` now fails closed for unknown site strings (previously fell through to the TLS fetcher on an unreachable path), and the browser dispatch iterates a sorted list so outcomes cannot depend on map order. - The security review's log-injection finding was checked against Go's `url.Parse` and does not hold: control characters are rejected anywhere in a URL, so a client-supplied value in a log line cannot carry a newline. ## Review Reviewed on three axes (spec, standards, security) by read-only subagents over `672c16f..f1b26f4`. No blocking findings; all minor/nit findings addressed in `d998f87` and `700de20`. Verified end to end with `go test ./...` (Docker Postgres per test package) on every commit. ## Out of scope (tracked separately) - Dropping asuracomic.net (CORS allowlist, userscript match, API fixtures, live env) — separate issue, per spec. Reviewed-on: #95 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
f1eb7d514c |
Record the lightnovelworld series-identity decision (#77) (#81)
Docs only. No code, no tests, nothing to run. Implementation is specified in #80. Outcome of a grilling session on 2026-08-11 against #77, backed by live measurement of lightnovelworld over 2026-08-10/11. ## What changed **`docs/adr/0008-series-identity-is-discovered-not-derived.md`** (new) A Series identity is discovered from the Site's own links, never derived from an address. On lightnovelworld the userscript reads the chapter page's `All Chapter` anchor instead of building a `/novel/<slug>/` address by string manipulation. A Chapter Slug is not an identity and is not stored. The backend's chapter scan drops its per-Series scoping and runs against the body truncated before the visitor comment thread. Evidence in the ADR: 3 of 41 sampled novels serve chapters under a slug that differs from their series slug, divergence runs in both directions, one novel serves chapters under two slugs, and neither slug is computable from the other. The pointer was checked on 8 chapter pages and agreed every time. Three narrower selectors are recorded as rejected, each with the measurement that killed it. Three rejected options are recorded with reasons: correcting the stored address only, which keeps an identity the Site does not guarantee; scoping the scan to a container, which the probe refuted; and a SQL migration, which is impossible because the database holds no source for the correct slug. **`CONTEXT.md`** - **Series** - identity is the canonical slug the Site publishes, never the title and never a Chapter Slug. - **Chapter Slug** - new term. A slug a Site builds its chapter addresses from. Not an identity: one Series may have several, and none is computable from another. - **Latest Chapter** - now the highest-numbered chapter, explicitly not a date and not the Site's own newest-chapter banner. Settles #79. **`docs/research/lightnovelworld-chapter-vs-series-slug.md`** (new, committed with its corrections) The 41-novel survey behind the ADR. Two claims are struck through and corrected in place, with the date and sample size of the probe that refuted each: the `ul.clstyle` container it named is the hidden, empty "Latest Reading" template rather than the chapter list, and its caveat about the comment region understated the risk, because that region is writable by any visitor while the scan takes an unbounded maximum into a Series row shared by every Reader (ADR-0003). ## Review notes Nothing here constrains code that exists today - the ADR describes work not yet written. The part worth disagreeing with, if any of it is wrong, is the fail-closed rule: a missing truncation marker means skip the Series and log, never scan the whole page. Related: #77 (the defect), #80 (the spec), #79 (the numbering anomaly, closed by decision), #71 (the same size cap seen from the cover side). Reviewed-on: #81 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
30c57bd39c |
Define Cover and record hosting its bytes (#47) (#64)
Defines **Cover** in the glossary and records ADR-0007, the decision behind #47's fix. ## Why these two files, and why now `CONTEXT.md` named Cover inside the **Series** entry — "facts true regardless of who is reading — title, cover, Latest Chapter" — but never said *what* one is. That gap is the bug. Nothing in the model distinguished "an address on a Site" from "an image a Reader's browser can display", so both clients were left to work it out independently, and one of them got it wrong. kagane serves covers with `cross-origin-resource-policy: same-origin`, the web UI rewrote them to a proxy in its templates, the JSON API did not, and the panel rendered a broken-image glyph. The new entry closes the ambiguity: *an address no client can load is not a Cover, it is a missing one.* ADR-0007 records what follows from that — the backend fetches, stores and serves every Site's cover bytes — plus the alternatives that were rejected and, more importantly, the two places this deliberately departs from existing precedent: - **Destination-class control instead of a host allowlist.** `fetchableSeriesURL` sets the allowlist precedent for `series_url`, and covers do not follow it. Cover hosts are CDNs that move independently of their Site — demonicscans serves its covers from `readermc.org` — so an allowlist would stop producing Covers the day a Site switched CDN, and that failure would look exactly like #47. The resolve-then-classify step is what actually stops the SSRF. - **A public cover route where the kagane proxy is session-gated.** An `<img>` cannot send a bearer token, and it cannot be given one either: the panel's shadow root is `mode: "open"`, so the host page's JavaScript can read any `src` the script sets. Both are security-adjacent departures, which is precisely why they are written down rather than left in a commit message. ## Scope Documentation only — no code, no schema, no behaviour. The implementation is #56–#63. ## Why this should merge promptly rather than sit All eight implementation tickets cite `docs/adr/0007-backend-hosts-cover-bytes.md` as the authority for decisions they must not relitigate, and they are written in the vocabulary this glossary entry defines. An agent picking up #56 reads both from `main`. Until this lands they get a 404 and either invent a rationale or stall — so this PR gates the tickets, not the other way round. Related: #47 (bug), #55 (spec), #54 (deferred admin refetch). Reviewed-on: #64 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
08749df050 |
feat(backend)!: run on Postgres with a migration-owned schema (#28)
Swap modernc.org/sqlite for jackc/pgx/v5 with no observable change: same endpoints, same wire format, same updated_at ordering rule. The schema now comes from numbered SQL embedded in the binary and applied on startup, one transaction each, recorded in schema_migrations. That replaces two pieces of SQLite-era machinery, both deleted rather than ported: the column probing (Postgres has ADD COLUMN IF NOT EXISTS, and there is no legacy database left to probe) and the Asura key rewrite, which has run clean on every start for months now that the userscripts strip build hashes before writing. Its regexp survives as latest.asuraBuildHash, where the poller still needs it to scope chapter links to a series whose slug carries a rotating hash. Types get real: favorite is a boolean, chapter numbers double precision, timestamps stay unix-ms bigint. SQLite's null-safe IS NOT becomes IS DISTINCT FROM, which is what implements the rule that only reading progress reorders a list. Inside COALESCE/NULLIF the status and kind parameters need an explicit ::text -- there is no target column to infer from and Postgres refuses to guess. Tests lose their free t.TempDir() database, so Docker is now a hard prerequisite for `go test ./...`: internal/pgtest starts one postgres:17-alpine per test binary and hands each test a database of its own. Also lands CONTEXT.md and the four ADRs written while scoping #18. BREAKING CHANGE: DB_PATH is retired for DATABASE_URL, which is required and has no default. Compose gains a postgres service on an internal network with its own volume; POSTGRES_PASSWORD joins .env. The old bookmarks-data volume is deliberately left undeclared so `docker compose down -v` cannot take the pre-migration database with it. main is not deployable until #25 and #26 land. Closes #20 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |