sulthan
08749df050
Swap modernc.org/sqlite for jackc/pgx/v5 with no observable change: same endpoints, same wire format, same updated_at ordering rule. The schema now comes from numbered SQL embedded in the binary and applied on startup, one transaction each, recorded in schema_migrations. That replaces two pieces of SQLite-era machinery, both deleted rather than ported: the column probing (Postgres has ADD COLUMN IF NOT EXISTS, and there is no legacy database left to probe) and the Asura key rewrite, which has run clean on every start for months now that the userscripts strip build hashes before writing. Its regexp survives as latest.asuraBuildHash, where the poller still needs it to scope chapter links to a series whose slug carries a rotating hash. Types get real: favorite is a boolean, chapter numbers double precision, timestamps stay unix-ms bigint. SQLite's null-safe IS NOT becomes IS DISTINCT FROM, which is what implements the rule that only reading progress reorders a list. Inside COALESCE/NULLIF the status and kind parameters need an explicit ::text -- there is no target column to infer from and Postgres refuses to guess. Tests lose their free t.TempDir() database, so Docker is now a hard prerequisite for `go test ./...`: internal/pgtest starts one postgres:17-alpine per test binary and hands each test a database of its own. Also lands CONTEXT.md and the four ADRs written while scoping #18. BREAKING CHANGE: DB_PATH is retired for DATABASE_URL, which is required and has no default. Compose gains a postgres service on an internal network with its own volume; POSTGRES_PASSWORD joins .env. The old bookmarks-data volume is deliberately left undeclared so `docker compose down -v` cannot take the pre-migration database with it. main is not deployable until #25 and #26 land. Closes #20 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
45 lines
2.4 KiB
Markdown
45 lines
2.4 KiB
Markdown
# Series is a shared entity, and only the Poll may update it
|
|
|
|
Status: accepted
|
|
|
|
Facts about a Series that are true regardless of who is reading — title, cover, canonical
|
|
URL, Latest Chapter — moved off the Bookmark onto a shared `series` row keyed
|
|
`(site, series_id)`. A Bookmark now holds only what differs between Readers: Progress,
|
|
Favourite, Lifecycle bucket. Fifty Readers tracking one Series produce fifty Bookmarks
|
|
and one Series, so the Series is polled once rather than fifty times.
|
|
|
|
## Why
|
|
|
|
The poller checks at most 84 series/hour (batch 14 per 10-minute tick). With ~50 Readers
|
|
holding ~30 Series each, polling per Bookmark means a 1,500-item sweep — roughly 18 hours
|
|
against a configured 1-hour cooldown, quietly breaking the New Chapter signal that is the
|
|
product's reason to exist. Deduplicating to distinct Series cuts the sweep several-fold,
|
|
and because the Series row now knows how many Readers hold it, the poll queue is ordered
|
|
`reader_count DESC, latest_checked_at ASC` — popular Series stay fresh and the long tail
|
|
absorbs the shortfall. That ordering is only expressible because the split happened.
|
|
|
|
Raising throughput instead was rejected: sweeping 400 Series hourly needs the stagger
|
|
cut from 20s to ~9s, doubling request rate against sites that already bot-score the
|
|
single VPS IP.
|
|
|
|
## Only the Poll writes Series fields
|
|
|
|
A client may supply `title`, `cover` and `series_url` only when creating a Series nobody
|
|
has bookmarked yet. After that, client-supplied values are ignored; only the backend's
|
|
own fetch updates them.
|
|
|
|
This is a security boundary, not tidiness. Those values are scraped from third-party
|
|
pages, which `AGENTS.md` requires be treated as attacker-controlled. Before the split, a
|
|
hostile or compromised site could corrupt exactly one Reader's row. After it, the same
|
|
write lands on a row every Reader sees — one Reader's browser becomes a write path into
|
|
everyone else's UI, and a cover URL can point anywhere. The backend's own fetch is the
|
|
higher-trust source: its network, its parser, no third-party JavaScript in the path.
|
|
|
|
## Consequences
|
|
|
|
- `Store.Upsert` decomposes one incoming flat body across two tables and enforces the
|
|
ownership rule at that seam.
|
|
- The `updated_at` ordering rule stays on the Bookmark, where Progress lives. Unchanged.
|
|
- Per-Reader title overrides are deliberately not supported; they would reintroduce the
|
|
duplication this removes.
|