sulthan
08749df050
Swap modernc.org/sqlite for jackc/pgx/v5 with no observable change: same endpoints, same wire format, same updated_at ordering rule. The schema now comes from numbered SQL embedded in the binary and applied on startup, one transaction each, recorded in schema_migrations. That replaces two pieces of SQLite-era machinery, both deleted rather than ported: the column probing (Postgres has ADD COLUMN IF NOT EXISTS, and there is no legacy database left to probe) and the Asura key rewrite, which has run clean on every start for months now that the userscripts strip build hashes before writing. Its regexp survives as latest.asuraBuildHash, where the poller still needs it to scope chapter links to a series whose slug carries a rotating hash. Types get real: favorite is a boolean, chapter numbers double precision, timestamps stay unix-ms bigint. SQLite's null-safe IS NOT becomes IS DISTINCT FROM, which is what implements the rule that only reading progress reorders a list. Inside COALESCE/NULLIF the status and kind parameters need an explicit ::text -- there is no target column to infer from and Postgres refuses to guess. Tests lose their free t.TempDir() database, so Docker is now a hard prerequisite for `go test ./...`: internal/pgtest starts one postgres:17-alpine per test binary and hands each test a database of its own. Also lands CONTEXT.md and the four ADRs written while scoping #18. BREAKING CHANGE: DB_PATH is retired for DATABASE_URL, which is required and has no default. Compose gains a postgres service on an internal network with its own volume; POSTGRES_PASSWORD joins .env. The old bookmarks-data volume is deliberately left undeclared so `docker compose down -v` cannot take the pre-migration database with it. main is not deployable until #25 and #26 land. Closes #20 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
96 lines
7.1 KiB
Markdown
96 lines
7.1 KiB
Markdown
Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system.
|
|
|
|
- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + Postgres over `jackc/pgx/v5` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`.
|
|
Single binary, split into packages under `backend/internal/`: `store`
|
|
(Bookmark type, Postgres persistence, migration runner), `latest` (background
|
|
poller, site parsers, TLS fetcher), `session` (cookie signing, login
|
|
rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON
|
|
bookmark handlers), `userscript` (userscript-serving handler), `web`
|
|
(browser UI handler + `templates/` + `static/`, `go:embed`-ed).
|
|
`backend/main.go` is the composition root — the only place that wires
|
|
packages together into `newRouter`. Root-level `*_test.go` hold
|
|
integration tests that exercise the full router; unit tests for a
|
|
package live beside it under `internal/`.
|
|
- **Schema is migration-owned.** `internal/store/migrations/*.sql` is
|
|
`go:embed`-ed and applied on every start by `store.migrate`: one numbered
|
|
file per change, one transaction each, versions recorded in
|
|
`schema_migrations`. Files are **append-only** — editing an applied one
|
|
changes nothing on a database that already ran it. No column probing, no
|
|
data-fixup migrations: both were SQLite-era machinery and are gone.
|
|
- **Tests need Docker.** `internal/pgtest` starts one `postgres:17-alpine`
|
|
container per test binary (`TestMain` -> `pgtest.Main`) and hands each test
|
|
its own database (`pgtest.URL(t)`). A package whose tests touch the store
|
|
must have that `TestMain`.
|
|
- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`), with a `kind` column (`manga`|`novel`) splitting the two libraries. Sync **last-write-wins**. Schema and endpoint list in plan.
|
|
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
|
|
- **Web UI:** same binary serve password-gated browser UI on second
|
|
hostname — `GET /` (list, or login page when no session),
|
|
`POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints
|
|
under `/ui/*`. Templates + assets `go:embed`-ed under
|
|
`backend/internal/web/`, so `backend/Dockerfile` must copy the whole
|
|
`internal/` tree, not just `*.go`. Sessions stateless
|
|
HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, and when empty,
|
|
web routes not registered at all. UI mutations read-modify-write
|
|
through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one
|
|
place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`.
|
|
**Design-tool caveat:** templates link `/static/style.css` root-absolutely
|
|
(correct — served from `/`), but impeccable detector resolves
|
|
stylesheet href with `path.resolve(fileDir, href)`, drops directory
|
|
on leading `/` and silently skip file. Relative href don't help
|
|
either: template's directory isn't its served path. So
|
|
`detect.mjs backend/internal/web/templates` reports **false clean** —
|
|
always pass `backend/internal/web/static` too. One finding there,
|
|
`overused-font` on "Instrument Serif", deliberate identity choice, not debt.
|
|
- **Every action that moves series out of list is confirm-gated.**
|
|
Archive, finish, remove each open own `.confirm-row` disclosure
|
|
(`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈
|
|
`archive|finish|remove`); restore fire instantly since it's the reversal.
|
|
Remove's row wear ember wash, two reversible ones wear `.calm` grey.
|
|
`--ember` stay reserved for new-chapter signal: busy bar and inline
|
|
error use `--mute`.
|
|
- **Latest-chapter poller:** ticker goroutine in same binary re-check
|
|
each bookmarked series' newest published chapter from backend's own
|
|
network access, so `latest_chapter` stay fresh when user not
|
|
browsing. Second, parallel signal — userscript keep own
|
|
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
|
|
Two independent clocks: per-bookmark cooldown (`latest_checked_at` column,
|
|
enforced by `Store.DueForLatestCheck`'s WHERE clause) and wake interval.
|
|
Row stamped *before* fetch so broken series wait out full
|
|
cooldown instead of retrying every tick, and writes go through
|
|
`Store.Get` + `Store.Upsert` so new chapter never reorders list.
|
|
Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth
|
|
against fingerprint-based blocking; any failure log and skip. kagane and
|
|
novelfull sit behind Cloudflare JavaScript challenges the TLS client can't
|
|
clear, so they are browser-only: fetched over CDP via `BROWSER_WS_URL`, and
|
|
simply not polled when that's unset. See
|
|
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
|
|
Poller's `Store.Get` + `Store.Upsert` not wrapped in transaction, so
|
|
userscript `PUT` that commits between the two can get overwritten by
|
|
poller's stale re-read — reverting that read progress and, since stored
|
|
value now differs, moving `updated_at` and reordering list. Known,
|
|
accepted limitation for single-user deployment, not bug to fix.
|
|
- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
|
|
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
|
|
`finished`, orthogonal to `favorite`. Archived and finished appear only in
|
|
own tab — not in All, Updated, Favourites, or recent strip. Poller keeps
|
|
checking archived series and skip finished ones. `finished` settable
|
|
only from web UI; `PUT /bookmarks/{key}` reject it with 400.
|
|
**Empty incoming status means "keep stored one"** — resolved on the
|
|
`VALUES` side of `Store.Upsert`, not conflict clause, since
|
|
`excluded.*` is post-evaluation row and default applied there would
|
|
wipe bucket on every PUT from client that predates column. See
|
|
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
|
|
- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list),
|
|
`DATABASE_URL` (Postgres connection URL, required — no default),
|
|
`PORT` (default `8080`), `WEB_PASSWORD`
|
|
(gates browser UI; unset disable it),
|
|
`LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER`
|
|
(background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`).
|
|
`USERSCRIPT_PATH` and `NOVEL_USERSCRIPT_PATH` (files served at
|
|
`/u/{token}/manga-bookmark.user.js` and `/u/{token}/novel-bookmark.user.js`,
|
|
defaults `/userscript/manga-bookmark.user.js` and
|
|
`/userscript/novel-bookmark.user.js`, both supplied by bindmount).
|
|
`BROWSER_WS_URL` (headless-shell CDP endpoint for kagane and novelfull;
|
|
unset disables browser polling and leaves those sites to the userscript
|
|
alone).
|