Compare commits

...

2 Commits

Author SHA1 Message Date
sulthan 3f7664ef9b feat(backend): give every Bookmark an owner (Reader table) (#22)
A readers table appears, keyed by Discord user ID and carrying the SHA-256
of the owner's userscript token (the global API token today). Startup seeds
exactly one Reader from OWNER_DISCORD_ID, idempotently, and a run-once
migration (0004, version-table-gated) attaches existing bookmarks to it
before reshaping: the surrogate key column is dropped and bookmarks are
keyed (reader_id, site, series_id) with an FK to readers ON DELETE CASCADE,
so a duplicate bookmark for one Reader and Series is impossible at the
database level.

Every store read and write is now scoped to the reader it names; handlers
act as the seeded owner while the global token remains the only credential.
Authentication and the wire format are untouched: the flat JSON still
carries key/site/series_id, with key derived on read.

OWNER_DISCORD_ID is a new required env var (compose + docs updated).
2026-08-08 07:59:40 +07:00
sulthan 984965ed9f Split Series from Bookmark, keeping the wire format flat (#21) (#29)
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 07:19:54 +07:00
18 changed files with 1255 additions and 268 deletions
+4
View File
@@ -4,6 +4,10 @@
# openssl rand -hex 32
API_TOKEN=changeme-generate-a-long-random-token
# The owner's Discord user ID — the one Reader every bookmark belongs to
# (seeded at startup). Discord snowflake, e.g. 1046923170000000000.
OWNER_DISCORD_ID=changeme-your-discord-user-id
# Comma-separated origins allowed to call the API (CORS). Both Asura domains
# plus Demonic, Comix, Kagane, and the two novel sites. Add/remove as the
# sites' hostnames change.
+6 -1
View File
@@ -35,6 +35,11 @@ Edit `.env`:
# Required — long random secret, also goes in the userscript.
API_TOKEN=<paste output of: openssl rand -hex 32>
# Required — the owner's Discord user ID. Seeds the one Reader every bookmark
# belongs to; the value is the snowflake in your Discord profile (Settings →
# Advanced → Developer Mode → right-click your name → Copy User ID).
OWNER_DISCORD_ID=<discord user id>
# CORS allowlist — leave as-is unless a site changes hostname.
ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to
@@ -250,7 +255,7 @@ it; see `REDEPLOY.md` §1 for when to remove it.)
| `fetch` fails in the userscript, `curl` works | Origin missing from `ALLOWED_ORIGINS`, or mixed content (backend not HTTPS). |
| 401 with the right token | Trailing space/newline in `API_TOKEN`; regenerate and restart. |
| Panel button absent | URL didn't match an adapter, or user scripts disabled in Bromite. |
| `compose ... config` errors about `API_TOKEN` or `POSTGRES_PASSWORD` | Run compose from the dir with `.env`, or export the vars. Both are required and neither has a fallback. |
| `compose ... config` errors about `API_TOKEN`, `OWNER_DISCORD_ID` or `POSTGRES_PASSWORD` | Run compose from the dir with `.env`, or export the vars. All three are required and none has a fallback. |
| `bookmark-api` restarts in a loop, `password authentication failed for user "bookmarks"` | `POSTGRES_PASSWORD` was changed after first boot; Postgres only applies it to an empty `postgres-data`. Restore the old value, or reset the role (`REDEPLOY.md` troubleshooting). |
| `bookmark-api` never logs `listening on :8080` | It is blocked on `postgres` passing `pg_isready`, or a migration failed. `docker compose -f docker-compose.yml -f docker-compose.prod.yml logs postgres`. |
+1
View File
@@ -26,6 +26,7 @@ Bromite userscript (isolated world, Shadow DOM UI, localStorage cache)
| Var | Default | Notes |
|-----|---------|-------|
| `API_TOKEN` | *(required)* | Bearer token shared with the userscript. |
| `OWNER_DISCORD_ID` | *(required)* | Discord user ID of the owner; seeds the one Reader all bookmarks belong to. |
| `ALLOWED_ORIGINS` | Asura + Demonic + Comix + Kagane origins | Comma-separated CORS allowlist. |
| `DATABASE_URL` | *(required)* | Postgres connection URL, e.g. `postgres://bookmarks:…@postgres:5432/bookmarks?sslmode=disable`. Compose builds it from `POSTGRES_PASSWORD`. |
| `PORT` | `8080` | Plain HTTP; TLS terminated by the proxy. |
+2 -1
View File
@@ -59,7 +59,7 @@ network can reach it — so every command below goes in through the container:
```bash
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -c '\dt'
# -> bookmarks, schema_migrations
# -> bookmarks, schema_migrations, series
```
Inside the container that connects over the local socket as the `bookmarks`
@@ -100,6 +100,7 @@ docker run --rm -v "$BACKUP_DIR":/backup postgres:17-alpine \
pg_restore --list "/backup/bookmarks-$STAMP.dump" | grep 'TABLE DATA'
# -> 1234; 0 0 TABLE DATA public bookmarks bookmarks
# -> 1235; 0 0 TABLE DATA public schema_migrations bookmarks
# -> 1236; 0 0 TABLE DATA public series series
# 2. Sanity-check the live row count you just captured.
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks \
+30 -11
View File
@@ -21,7 +21,22 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
container per test binary (`TestMain` -> `pgtest.Main`) and hands each test
its own database (`pgtest.URL(t)`). A package whose tests touch the store
must have that `TestMain`.
- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`), with a `kind` column (`manga`|`novel`) splitting the two libraries. Sync **last-write-wins**. Schema and endpoint list in plan.
- **Single-owner store, three tables.** `readers` is keyed by Discord user ID
and carries the SHA-256 of the owner's userscript token (the global
`API_TOKEN` today; issue #22). The seed creates exactly one row at startup.
`series` keyed `(site, series_id)`
(`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the
shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`),
Latest Chapter, `latest_checked_at` — and `bookmarks` holds only what
differs between readers: progress, favourite, lifecycle bucket,
`updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no
surrogate id; the wire `key` is derived as `site:series_id` on read — and
every store read/write is scoped to the reader it names. `Store.OwnerID()`
is the seeded owner, which every handler passes while the global token is
still the only credential. Sync **last-write-wins**; the wire format stays
flat (ADR-0004). `Store.Upsert` decomposes one flat body across two tables
and enforces the ownership rule: client `title`/`series_url`/`cover` are
written only when the series row is new (ADR-0003).
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- **Web UI:** same binary serve password-gated browser UI on second
hostname — `GET /` (list, or login page when no session),
@@ -53,22 +68,25 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
network access, so `latest_chapter` stay fresh when user not
browsing. Second, parallel signal — userscript keep own
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
Two independent clocks: per-bookmark cooldown (`latest_checked_at` column,
Two independent clocks: per-series cooldown (`series.latest_checked_at`,
enforced by `Store.DueForLatestCheck`'s WHERE clause) and wake interval.
Row stamped *before* fetch so broken series wait out full
cooldown instead of retrying every tick, and writes go through
`Store.Get` + `Store.Upsert` so new chapter never reorders list.
The poller walks **Series, not Bookmarks** — a series referenced by several
bookmarks is fetched once per cycle, and the due queue orders
`reader_count DESC, latest_checked_at ASC` (ADR-0003). Series row stamped
*before* fetch so broken series wait out full cooldown instead of retrying
every tick; found chapter written straight to the series row via
`Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
order — is never touched.
Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth
against fingerprint-based blocking; any failure log and skip. kagane and
novelfull sit behind Cloudflare JavaScript challenges the TLS client can't
clear, so they are browser-only: fetched over CDP via `BROWSER_WS_URL`, and
simply not polled when that's unset. See
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
Poller's `Store.Get` + `Store.Upsert` not wrapped in transaction, so
userscript `PUT` that commits between the two can get overwritten by
poller's stale re-read — reverting that read progress and, since stored
value now differs, moving `updated_at` and reordering list. Known,
accepted limitation for single-user deployment, not bug to fix.
The poller's series write is a single-column UPDATE
(`Store.SetLatestChapter`), not a read-modify-write of the whole bookmark:
it cannot revert read progress or move `updated_at`, so the old
stale-re-read race is gone with the Get+Upsert flow.
- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
`finished`, orthogonal to `favorite`. Archived and finished appear only in
@@ -80,7 +98,8 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
`excluded.*` is post-evaluation row and default applied there would
wipe bucket on every PUT from client that predates column. See
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list),
- **Config via env:** `API_TOKEN`, `OWNER_DISCORD_ID` (seeds the owner Reader;
required), `ALLOWED_ORIGINS` (comma list),
`DATABASE_URL` (Postgres connection URL, required — no default),
`PORT` (default `8080`), `WEB_PASSWORD`
(gates browser UI; unset disable it),
+138 -7
View File
@@ -2,6 +2,7 @@ package main
import (
"bytes"
"crypto/sha256"
"encoding/json"
"fmt"
"net/http"
@@ -35,7 +36,9 @@ func newTestServer(t *testing.T) http.Handler {
func newTestStore(t *testing.T) *store.Store {
t.Helper()
s, err := store.Open(pgtest.URL(t))
s, err := store.Open(pgtest.URL(t), store.Owner{
DiscordID: "test-owner", TokenHash: sha256.Sum256([]byte("owner-token-hash")),
})
if err != nil {
t.Fatalf("store.Open: %v", err)
}
@@ -50,26 +53,35 @@ func auth(req *http.Request) *http.Request {
func floatPtr(f float64) *float64 { return &f }
// seedForCheck inserts a bookmark and forces its latest_checked_at.
// seedForCheck inserts a bookmark (and with it its series) and forces the
// series' latest_checked_at.
func seedForCheck(t *testing.T, s *store.Store, key, seriesURL string, checkedAt int64) {
t.Helper()
if _, err := s.Upsert(store.Bookmark{
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key,
Site: "asura",
SeriesID: key,
Site: site,
SeriesID: seriesID,
SeriesURL: seriesURL,
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed %q: %v", key, err)
}
if err := s.MarkLatestChecked(key, checkedAt); err != nil {
if err := s.MarkLatestChecked(site, seriesID, checkedAt); err != nil {
t.Fatalf("seed mark %q: %v", key, err)
}
}
func readLatestCheckedAt(t *testing.T, s *store.Store, key string) int64 {
t.Helper()
ts, err := s.LatestCheckedAt(key)
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
ts, err := s.LatestCheckedAt(site, seriesID)
if err != nil {
t.Fatalf("LatestCheckedAt %q: %v", key, err)
}
@@ -229,6 +241,125 @@ func TestBookmarkRoundTrip(t *testing.T) {
}
}
// The wire contract (ADR-0004): GET and PUT speak exactly the flat field set
// they always did, with the series-owned fields as siblings of the bookmark
// fields, not nested. Asserted as a key set, not by inspection.
func TestFlatWireFieldSet(t *testing.T) {
srv := newTestServer(t)
key := "comix:some-title"
in := store.Bookmark{
Key: key,
Site: "comix",
SeriesID: "some-title",
Title: "Some Title",
SeriesURL: "https://comix.to/title/some-title",
Cover: "https://comix.to/covers/some-title.jpg",
LastChapter: "Chapter 7",
LastChapterNum: 7,
LastChapterURL: "https://comix.to/title/some-title/ch/7",
Favorite: true,
LatestChapter: "Chapter 8",
LatestChapterNum: floatPtr(8),
Status: store.StatusArchived,
Kind: store.KindManga,
}
body, _ := json.Marshal(in)
wantKeys := map[string]bool{
"key": true, "site": true, "series_id": true, "title": true,
"series_url": true, "cover": true, "last_chapter": true,
"last_chapter_num": true, "last_chapter_url": true, "favorite": true,
"latest_chapter": true, "latest_chapter_num": true, "updated_at": true,
"status": true, "kind": true,
}
checkFlat := func(t *testing.T, payload []byte) map[string]json.RawMessage {
t.Helper()
var obj map[string]json.RawMessage
if err := json.Unmarshal(payload, &obj); err != nil {
t.Fatalf("decode: %v", err)
}
if len(obj) != len(wantKeys) {
t.Fatalf("field count = %d, want %d (%s)", len(obj), len(wantKeys), payload)
}
for k := range obj {
if !wantKeys[k] {
t.Fatalf("unexpected field %q", k)
}
}
return obj
}
// PUT
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body))))
if rr.Code != http.StatusOK {
t.Fatalf("PUT status = %d, want 200", rr.Code)
}
checkFlat(t, rr.Body.Bytes())
// Every field round-trips with its value, and updated_at is server-stamped.
var stored store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &stored); err != nil {
t.Fatalf("decode PUT response: %v", err)
}
latestNum := floatPtr(8)
want := store.Bookmark{
Key: key, Site: "comix", SeriesID: "some-title",
Title: in.Title, SeriesURL: in.SeriesURL, Cover: in.Cover,
LastChapter: in.LastChapter, LastChapterNum: in.LastChapterNum,
LastChapterURL: in.LastChapterURL, Favorite: true,
LatestChapter: in.LatestChapter, LatestChapterNum: latestNum,
Status: store.StatusArchived, Kind: store.KindManga,
}
if stored.Title != want.Title || stored.SeriesURL != want.SeriesURL || stored.Cover != want.Cover ||
stored.LastChapter != want.LastChapter || stored.LastChapterNum != want.LastChapterNum ||
stored.LastChapterURL != want.LastChapterURL || stored.Favorite != want.Favorite ||
stored.LatestChapter != want.LatestChapter ||
stored.LatestChapterNum == nil || *stored.LatestChapterNum != *want.LatestChapterNum ||
stored.Status != want.Status || stored.Kind != want.Kind {
t.Fatalf("PUT response = %+v, want %+v", stored, want)
}
if stored.UpdatedAt == 0 {
t.Fatal("updated_at not server-stamped")
}
// GET reports the same flat shape.
list := getBookmarks(t, srv)
if len(list) != 1 {
t.Fatalf("list = %d items, want 1", len(list))
}
body2, _ := json.Marshal(list[0])
checkFlat(t, body2)
}
// A PUT naming an existing series must ignore client-supplied title, cover and
// URL — the security boundary from ADR-0003, where a hostile site's scraped
// values could otherwise land on a shared row — while progress still lands.
func TestPutExistingSeriesIgnoresClientTitleCoverURL(t *testing.T) {
srv := newTestServer(t)
key := "asura:solo"
first := putBookmark(t, srv, key, store.Bookmark{
Title: "Solo Leveling",
SeriesURL: "https://asurascans.com/comics/solo",
Cover: "https://asurascans.com/covers/solo.jpg",
LastChapterNum: 10,
})
second := putBookmark(t, srv, key, store.Bookmark{
Title: "Scraped Rename",
SeriesURL: "https://evil.example/solo",
Cover: "https://evil.example/solo.jpg",
LastChapterNum: 11,
})
if second.Title != first.Title || second.SeriesURL != first.SeriesURL || second.Cover != first.Cover {
t.Fatalf("stored = %+v, want original title/url/cover kept", second)
}
if second.LastChapterNum != 11 {
t.Fatalf("LastChapterNum = %v, want 11 — progress must still land", second.LastChapterNum)
}
}
// putBookmark PUTs b at key and returns the bookmark the server echoes back,
// which is the row as actually stored (not the request payload).
func putBookmark(t *testing.T, srv http.Handler, key string, b store.Bookmark) store.Bookmark {
+7 -4
View File
@@ -13,6 +13,9 @@ import (
// Handler serves the userscript-facing JSON bookmark API.
type Handler struct {
Store *store.Store
// ReaderID is the Reader this request acts as. Authentication is still the
// single global token, so that is always the seeded owner (issue #22).
ReaderID int64
}
func writeJSON(w http.ResponseWriter, status int, v any) {
@@ -25,9 +28,9 @@ func writeJSON(w http.ResponseWriter, status int, v any) {
}
}
// List returns all bookmarks. GET /bookmarks
// List returns all bookmarks of the acting Reader. GET /bookmarks
func (h *Handler) List(w http.ResponseWriter, r *http.Request) {
items, err := h.Store.List()
items, err := h.Store.List(h.ReaderID)
if err != nil {
log.Printf("list: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
@@ -91,7 +94,7 @@ func (h *Handler) Put(w http.ResponseWriter, r *http.Request) {
// reading progress actually moved. Any client value is ignored.
b.UpdatedAt = time.Now().UnixMilli()
stored, err := h.Store.Upsert(b)
stored, err := h.Store.Upsert(h.ReaderID, b)
if err != nil {
log.Printf("upsert: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
@@ -109,7 +112,7 @@ func (h *Handler) Delete(w http.ResponseWriter, r *http.Request) {
http.Error(w, "missing key", http.StatusBadRequest)
return
}
if err := h.Store.Delete(key); err != nil {
if err := h.Store.Delete(h.ReaderID, key); err != nil {
log.Printf("delete: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
+29 -46
View File
@@ -23,9 +23,9 @@ type Fetcher interface {
// Two clocks, deliberately independent:
//
// - Interval is how often this goroutine wakes up and looks.
// - Cooldown is how long one bookmark rests since its own last check.
// - Cooldown is how long one series rests since its own last check.
//
// Only the cooldown is per bookmark, and it is enforced by the WHERE clause in
// Only the cooldown is per series, and it is enforced by the WHERE clause in
// DueForLatestCheck rather than by any timer. Shortening Interval therefore
// cannot shorten anyone's cooldown; it only makes the poller wake up and find
// nothing due more often.
@@ -78,7 +78,7 @@ func (p *Poller) Run(ctx context.Context) {
}
}
// runOnce processes one batch of due bookmarks.
// runOnce processes one batch of due series.
func (p *Poller) runOnce(ctx context.Context) {
cutoff := p.Now().Add(-p.Cooldown).UnixMilli()
due, err := p.Store.DueForLatestCheck(cutoff, p.Batch)
@@ -88,7 +88,7 @@ func (p *Poller) runOnce(ctx context.Context) {
}
checked := 0
for i, b := range due {
for i, sr := range due {
if ctx.Err() != nil {
break
}
@@ -107,7 +107,7 @@ func (p *Poller) runOnce(ctx context.Context) {
if stopped {
break
}
p.checkOne(ctx, b)
p.checkOne(ctx, sr)
checked++
}
// due vs checked is how you tell which constraint is binding: ticks that
@@ -119,10 +119,10 @@ func (p *Poller) runOnce(ctx context.Context) {
// checkOne re-checks one series. Every failure path here is "log and move on":
// the poller is a best-effort enhancement, and no single bad series may stall a
// batch or take down the process.
func (p *Poller) checkOne(ctx context.Context, b store.Bookmark) {
func (p *Poller) checkOne(ctx context.Context, sr store.Series) {
defer func() {
if r := recover(); r != nil {
log.Printf("latest poll %q: recovered from panic: %v", b.Key, r)
log.Printf("latest poll %q: recovered from panic: %v", sr.Key(), r)
}
}()
@@ -130,8 +130,8 @@ func (p *Poller) checkOne(ctx context.Context, b store.Bookmark) {
// mid-request still consumes the cooldown. Otherwise a renamed or deleted
// series would be retried on every single tick forever. The userscript
// stamps in the same order and for the same reason (L471-473).
if err := p.Store.MarkLatestChecked(b.Key, p.Now().UnixMilli()); err != nil {
log.Printf("latest poll %q: mark checked: %v", b.Key, err)
if err := p.Store.MarkLatestChecked(sr.Site, sr.SeriesID, p.Now().UnixMilli()); err != nil {
log.Printf("latest poll %q: mark checked: %v", sr.Key(), err)
return
}
@@ -142,69 +142,52 @@ func (p *Poller) checkOne(ctx context.Context, b store.Bookmark) {
// link-local/internal addresses or non-https schemes. The cooldown above
// is already consumed, so a row that never passes this check is retried at
// cooldown pace rather than hot-looping.
if !fetchableSeriesURL(b.Site, b.SeriesURL) {
log.Printf("latest poll %q: not fetchable: site=%q url=%q", b.Key, b.Site, b.SeriesURL)
if !fetchableSeriesURL(sr.Site, sr.SeriesURL) {
log.Printf("latest poll %q: not fetchable: site=%q url=%q", sr.Key(), sr.Site, sr.SeriesURL)
return
}
f := p.fetcherFor(b.Site)
f := p.fetcherFor(sr.Site)
if f == nil {
log.Printf("latest poll %q: no fetcher for site %q", b.Key, b.Site)
log.Printf("latest poll %q: no fetcher for site %q", sr.Key(), sr.Site)
return
}
body, status, err := f.Get(ctx, b.SeriesURL)
body, status, err := f.Get(ctx, sr.SeriesURL)
if err != nil {
log.Printf("latest poll %q: fetch %s: %v", b.Key, b.SeriesURL, err)
log.Printf("latest poll %q: fetch %s: %v", sr.Key(), sr.SeriesURL, err)
return
}
if status != 200 {
log.Printf("latest poll %q: fetch %s: status %d", b.Key, b.SeriesURL, status)
log.Printf("latest poll %q: fetch %s: status %d", sr.Key(), sr.SeriesURL, status)
return
}
latest, ok := latestChapterFrom(b.Site, b.SeriesURL, body)
latest, ok := latestChapterFrom(sr.Site, sr.SeriesURL, body)
if !ok {
// Most likely a challenge page or a layout change. Either way the row is
// already stamped, so this waits out a cooldown instead of hot-looping.
log.Printf("latest poll %q: no chapter links in %d bytes", b.Key, len(body))
log.Printf("latest poll %q: no chapter links in %d bytes", sr.Key(), len(body))
return
}
// Re-read: the row may have been updated or deleted while the fetch was in
// flight, and writing b back wholesale would undo that.
//
// ponytail: non-transactional read-modify-write, wrap Get+Upsert in a tx if
// this ever runs for more than one user. A client PUT that commits between
// these two statements is lost to the stale re-read — reverting read
// progress or a status change, and moving updated_at because the stored
// value now differs. Accepted for a single-user deployment: the window is
// milliseconds and the loser is one poll cycle.
cur, found, err := p.Store.Get(b.Key)
if err != nil {
log.Printf("latest poll %q: reread: %v", b.Key, err)
return
}
if !found {
return
}
// Equality, not >, mirroring the userscript (L427): a site that retracts a
// chapter should correct the stored number downward.
if cur.LatestChapterNum != nil && *cur.LatestChapterNum == latest.Num {
// chapter should correct the stored number downward. The comparison is
// against the due-query snapshot; a concurrent write in between only costs
// one redundant UPDATE of the same absolute value, never a wrong one.
if sr.LatestChapterNum != nil && *sr.LatestChapterNum == latest.Num {
return
}
num := latest.Num
cur.LatestChapter = latest.Label
cur.LatestChapterNum = &num
// A candidate only. last_chapter_num is untouched, so the CASE in Upsert
// keeps the stored updated_at and the bookmark list does not reorder.
cur.UpdatedAt = p.Now().UnixMilli()
if _, err := p.Store.Upsert(cur); err != nil {
log.Printf("latest poll %q: upsert: %v", b.Key, err)
// Series-level write: the row is shared, so one update refreshes every
// bookmark joining to it, and the bookmark's updated_at is never touched —
// a newly published chapter is not reading progress and must not reorder
// the list.
if err := p.Store.SetLatestChapter(sr.Site, sr.SeriesID, latest.Label, latest.Num); err != nil {
log.Printf("latest poll %q: set latest chapter: %v", sr.Key(), err)
return
}
log.Printf("latest poll %q: latest is now %s", b.Key, latest.Label)
log.Printf("latest poll %q: latest is now %s", sr.Key(), latest.Label)
}
// fetchableSeriesURL reports whether site is a site latestChapterFrom knows how
+95 -35
View File
@@ -2,8 +2,10 @@ package latest
import (
"context"
"crypto/sha256"
"errors"
"os"
"strings"
"sync"
"testing"
"time"
@@ -14,37 +16,53 @@ import (
func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
// newTestStore opens a store on a Postgres database of this test's own.
func newTestStore(t *testing.T) *store.Store {
// testOwner is the owner every test store seeds. A second reader, where a
// test needs one, is created by opening the same database as a second owner.
var testOwner = store.Owner{DiscordID: "test-owner", TokenHash: sha256.Sum256([]byte("owner-token-hash"))}
// newTestStore opens a store on a Postgres database of this test's own and
// returns the URL, for helpers that need a second connection to the same
// database (see TestRunOnceFetchesSharedSeriesOnce).
func newTestStore(t *testing.T) (*store.Store, string) {
t.Helper()
s, err := store.Open(pgtest.URL(t))
url := pgtest.URL(t)
s, err := store.Open(url, testOwner)
if err != nil {
t.Fatalf("Open: %v", err)
}
t.Cleanup(func() { s.Close() })
return s
return s, url
}
// seedForCheck inserts a bookmark and forces its latest_checked_at.
// seedForCheck inserts a bookmark (and with it its series) and forces the
// series' latest_checked_at.
func seedForCheck(t *testing.T, s *store.Store, key, seriesURL string, checkedAt int64) {
t.Helper()
if _, err := s.Upsert(store.Bookmark{
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key,
Site: "asura",
SeriesID: key,
Site: site,
SeriesID: seriesID,
SeriesURL: seriesURL,
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed %q: %v", key, err)
}
if err := s.MarkLatestChecked(key, checkedAt); err != nil {
if err := s.MarkLatestChecked(site, seriesID, checkedAt); err != nil {
t.Fatalf("seed mark %q: %v", key, err)
}
}
func readLatestCheckedAt(t *testing.T, s *store.Store, key string) int64 {
t.Helper()
ts, err := s.LatestCheckedAt(key)
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
ts, err := s.LatestCheckedAt(site, seriesID)
if err != nil {
t.Fatalf("LatestCheckedAt %q: %v", key, err)
}
@@ -101,7 +119,7 @@ func newTestPoller(t *testing.T, s *store.Store, f Fetcher, at time.Time) *Polle
}
func TestRunOnceRecordsLatestChapter(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
const url = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
seedForCheck(t, s, "asura:chronicles-of-the-demon-faction-f886a8af", url, 0)
@@ -109,7 +127,7 @@ func TestRunOnceRecordsLatestChapter(t *testing.T) {
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).runOnce(context.Background())
b, ok, err := s.Get("asura:chronicles-of-the-demon-faction-f886a8af")
b, ok, err := s.Get(s.OwnerID(), "asura:chronicles-of-the-demon-faction-f886a8af")
if err != nil || !ok {
t.Fatalf("Get: %v ok=%v", err, ok)
}
@@ -127,19 +145,19 @@ func TestRunOnceRecordsLatestChapter(t *testing.T) {
// The whole point of the updated_at CASE in Upsert: a newly published chapter is
// not reading progress and must not move the series up the list.
func TestRunOnceDoesNotReorderList(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
const url = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
const key = "asura:chronicles-of-the-demon-faction-f886a8af"
// "other" is the most recently read, so it must stay at the top of List().
if _, err := s.Upsert(store.Bookmark{
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: "asura:other", Site: "asura", SeriesID: "other",
SeriesURL: "https://asurascans.com/comics/other", UpdatedAt: 9_000_000,
}); err != nil {
t.Fatalf("seed other: %v", err)
}
seedForCheck(t, s, key, url, 0)
before, _, err := s.Get(key)
before, _, err := s.Get(s.OwnerID(), key)
if err != nil {
t.Fatalf("Get before: %v", err)
}
@@ -147,7 +165,7 @@ func TestRunOnceDoesNotReorderList(t *testing.T) {
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, time.UnixMilli(9_999_999)).runOnce(context.Background())
after, _, err := s.Get(key)
after, _, err := s.Get(s.OwnerID(), key)
if err != nil {
t.Fatalf("Get after: %v", err)
}
@@ -155,7 +173,7 @@ func TestRunOnceDoesNotReorderList(t *testing.T) {
t.Fatalf("updated_at moved from %d to %d on a latest-chapter bump",
before.UpdatedAt, after.UpdatedAt)
}
list, err := s.List()
list, err := s.List(s.OwnerID())
if err != nil {
t.Fatalf("List: %v", err)
}
@@ -178,7 +196,7 @@ func TestRunOnceMarksCheckedOnFailure(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
const url = "https://asurascans.com/comics/x"
seedForCheck(t, s, "asura:x", url, 0)
@@ -189,7 +207,7 @@ func TestRunOnceMarksCheckedOnFailure(t *testing.T) {
if got := readLatestCheckedAt(t, s, "asura:x"); got != now.UnixMilli() {
t.Fatalf("latest_checked_at = %d, want %d", got, now.UnixMilli())
}
b, _, err := s.Get("asura:x")
b, _, err := s.Get(s.OwnerID(), "asura:x")
if err != nil {
t.Fatalf("Get: %v", err)
}
@@ -201,7 +219,7 @@ func TestRunOnceMarksCheckedOnFailure(t *testing.T) {
}
func TestRunOnceRespectsBatchLimit(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
for i := 0; i < 20; i++ {
key := "asura:s" + string(rune('a'+i))
seedForCheck(t, s, key, "https://asurascans.com/comics/"+key, 0)
@@ -217,9 +235,51 @@ func TestRunOnceRespectsBatchLimit(t *testing.T) {
}
}
// The point of the split (ADR-0003): a series referenced by several bookmarks
// is fetched once per due cycle, not once per bookmark. Two bookmarks share a
// series when two readers track it (issue #22).
func TestRunOnceFetchesSharedSeriesOnce(t *testing.T) {
s, url := newTestStore(t)
// The slug must match the fixture's own anchors: asura's parser scopes
// chapter links to the stored slug.
const slug = "chronicles-of-the-demon-faction-f886a8af"
const seriesURL = "https://asurascans.com/comics/" + slug
seedForCheck(t, s, "asura:"+slug, seriesURL, 0)
// A second reader tracks the same series. The seed is the only
// reader-creation path, so a second Open as a different owner is how a
// test gets a second reader on the same database.
other, err := store.Open(url, store.Owner{DiscordID: "second-reader", TokenHash: sha256.Sum256([]byte("second-token-hash"))})
if err != nil {
t.Fatalf("Open second reader: %v", err)
}
t.Cleanup(func() { other.Close() })
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
Key: "asura:" + slug, Site: "asura", SeriesID: slug, UpdatedAt: 2000,
}); err != nil {
t.Fatalf("seed second reader: %v", err)
}
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, time.UnixMilli(5_000_000)).runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched shared series %d times, want 1", got)
}
// Both bookmarks join to the same updated series row.
for _, st := range []*store.Store{s, other} {
b, ok, err := st.Get(st.OwnerID(), "asura:"+slug)
if err != nil || !ok {
t.Fatalf("Get: %v ok=%v", err, ok)
}
if b.LatestChapterNum == nil || *b.LatestChapterNum != 181 {
t.Fatalf("LatestChapterNum = %v, want 181", b.LatestChapterNum)
}
}
}
// One unreachable series must not abandon the rest of the batch.
func TestRunOnceOneBadSeriesDoesNotStallBatch(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
keys := []string{"asura:a", "asura:b", "asura:c", "asura:d", "asura:e"}
for _, k := range keys {
seedForCheck(t, s, k, "https://asurascans.com/comics/"+k, 0)
@@ -247,7 +307,7 @@ func TestRunOnceOneBadSeriesDoesNotStallBatch(t *testing.T) {
// The cooldown is enforced by the due query, so a second immediate pass must do
// nothing at all — this is what makes the tick interval independent of it.
func TestRunOnceHonoursCooldownAcrossPasses(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
const url = "https://asurascans.com/comics/x"
seedForCheck(t, s, "asura:x", url, 0)
@@ -277,12 +337,12 @@ func TestRunOnceHonoursCooldownAcrossPasses(t *testing.T) {
// A site that retracts a chapter should correct the stored number downward,
// mirroring the userscript's equality check (L427) rather than a >.
func TestRunOnceCorrectsDownward(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
const url = "https://demonicscans.org/manga/Catastrophic-Necromancer"
const key = "demonic:Catastrophic-Necromancer"
high := 400.0
if _, err := s.Upsert(store.Bookmark{
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key, Site: "demonic", SeriesID: "Catastrophic-Necromancer",
SeriesURL: url, LatestChapter: "Chapter 400", LatestChapterNum: &high,
UpdatedAt: 1000,
@@ -293,7 +353,7 @@ func TestRunOnceCorrectsDownward(t *testing.T) {
f := &fakeFetcher{body: demonicSeriesFixture, status: 200}
newTestPoller(t, s, f, time.UnixMilli(5_000_000)).runOnce(context.Background())
b, _, err := s.Get(key)
b, _, err := s.Get(s.OwnerID(), key)
if err != nil {
t.Fatalf("Get: %v", err)
}
@@ -319,9 +379,9 @@ func TestCheckOneValidatesSeriesURLBeforeFetching(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
key := tt.site + ":x"
if _, err := s.Upsert(store.Bookmark{
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key, Site: tt.site, SeriesID: "x", SeriesURL: tt.seriesURL,
UpdatedAt: 1000,
}); err != nil {
@@ -330,8 +390,8 @@ func TestCheckOneValidatesSeriesURLBeforeFetching(t *testing.T) {
now := time.UnixMilli(4_000_000)
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).checkOne(context.Background(), store.Bookmark{
Key: key, Site: tt.site, SeriesURL: tt.seriesURL,
newTestPoller(t, s, f, now).checkOne(context.Background(), store.Series{
Site: tt.site, SeriesID: "x", SeriesURL: tt.seriesURL,
})
if got := f.callCount(); got != tt.wantCalls {
@@ -346,7 +406,7 @@ func TestCheckOneValidatesSeriesURLBeforeFetching(t *testing.T) {
// A cancelled context must abandon the batch rather than run it to completion.
func TestRunOnceStopsOnCancelledContext(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
for _, k := range []string{"asura:a", "asura:b", "asura:c"} {
seedForCheck(t, s, k, "https://asurascans.com/comics/"+k, 0)
}
@@ -395,8 +455,8 @@ func TestFetchableSeriesURL(t *testing.T) {
// receive a challenge page, and the browser fetcher is the whole reason kagane
// is pollable at all.
func TestKaganeSkippedWhenNoBrowserFetcher(t *testing.T) {
s := newTestStore(t)
if _, err := s.Upsert(store.Bookmark{
s, _ := newTestStore(t)
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: "kagane:019f84bc-9ba0-7ed9-86f5-8b905ec7c28b",
Site: "kagane",
SeriesID: "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b",
@@ -421,9 +481,9 @@ func TestKaganeSkippedWhenNoBrowserFetcher(t *testing.T) {
// With a browser fetcher wired up, kagane goes to it and not to the TLS one.
func TestKaganeUsesBrowserFetcher(t *testing.T) {
s := newTestStore(t)
s, _ := newTestStore(t)
key := "kagane:019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"
if _, err := s.Upsert(store.Bookmark{
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key,
Site: "kagane",
SeriesID: "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b",
@@ -448,7 +508,7 @@ func TestKaganeUsesBrowserFetcher(t *testing.T) {
if len(browserF.calls) != 1 {
t.Fatalf("browser fetcher calls = %v, want 1", browserF.calls)
}
got, found, err := s.Get(key)
got, found, err := s.Get(s.OwnerID(), key)
if err != nil || !found {
t.Fatalf("Get: %v found=%v", err, found)
}
@@ -0,0 +1,45 @@
-- One row per distinct work, shared by every bookmark that tracks it
-- (ADR-0003). Keyed (site, series_id), the pair a bookmark key decomposes
-- into. title/series_url/cover are written once, at creation, and never
-- again: client-supplied values are ignored once the row exists and the
-- poller is the only party that may change them. kind and the latest-chapter
-- fields are last-write-wins like the bookmark's own fields.
CREATE TABLE series (
site text NOT NULL,
series_id text NOT NULL,
title text NOT NULL DEFAULT '',
series_url text NOT NULL DEFAULT '',
cover text NOT NULL DEFAULT '',
kind text NOT NULL DEFAULT 'manga',
latest_chapter text NOT NULL DEFAULT '',
latest_chapter_num double precision,
-- When the server last polled this series, unix ms; 0 means never, and sorts
-- first so a new bookmark is picked up on the next tick with no special case.
latest_checked_at bigint NOT NULL DEFAULT 0,
PRIMARY KEY (site, series_id)
);
-- Backfill from today's rows. The bookmark key's uniqueness makes
-- (site, series_id) unique in practice; DISTINCT is belt and braces.
INSERT INTO series (site, series_id, title, series_url, cover, kind,
latest_chapter, latest_chapter_num, latest_checked_at)
SELECT DISTINCT site, series_id, title, series_url, cover, kind,
latest_chapter, latest_chapter_num, latest_checked_at
FROM bookmarks;
-- The bookmark keeps only what differs between readers (ADR-0003): progress,
-- favourite, lifecycle bucket. The dropped columns now live on series.
ALTER TABLE bookmarks
DROP COLUMN title,
DROP COLUMN series_url,
DROP COLUMN cover,
DROP COLUMN kind,
DROP COLUMN latest_chapter,
DROP COLUMN latest_chapter_num,
DROP COLUMN latest_checked_at;
-- A bookmark may not point at a series that does not exist. No cascade: a
-- series outlives its last bookmark, and deleting one is not a store operation.
ALTER TABLE bookmarks
ADD CONSTRAINT bookmarks_series_fk
FOREIGN KEY (site, series_id) REFERENCES series (site, series_id);
@@ -0,0 +1,17 @@
-- One row per person. Keyed by their Discord user ID; carries the SHA-256 of
-- their userscript token and when they were created. Hashed because a token
-- in the database is a token anyone with the database can replay; SHA-256 is
-- enough because the tokens are high-entropy random values with nothing to
-- brute-force. No one can register yet, so this table holds exactly the one
-- owner row the seed creates at startup (see Store.Open).
CREATE TABLE readers (
id bigserial PRIMARY KEY,
discord_id text NOT NULL UNIQUE,
token_sha256 bytea NOT NULL UNIQUE,
created_at timestamptz NOT NULL DEFAULT now()
);
-- Every bookmark now belongs to a reader. Added nullable: rows created before
-- this migration have no owner yet — 0004 attaches them to the seeded owner
-- before NOT NULL and the composite key land.
ALTER TABLE bookmarks ADD COLUMN reader_id bigint;
@@ -0,0 +1,19 @@
-- Attach every pre-existing bookmark to the owner reader, seeded between the
-- two migrate passes (Store.Open). The oldest reader is the owner by
-- construction: only the seed creates readers, and it runs once per database.
-- Run-once via the version table, like every migration.
UPDATE bookmarks SET reader_id = (SELECT id FROM readers ORDER BY id LIMIT 1);
-- Ownership lands structurally: reader_id becomes part of the key, so a
-- bookmark is one Reader's progress on one Series and a duplicate for the
-- same pair is impossible at the database level. Deleting a Reader takes
-- their bookmarks with them. The old text key is gone — the wire "key" is
-- derived as site:series_id on read, and nothing references the column.
-- Dropping it drops the primary key it carried; the composite key replaces
-- it, and the FK index the series constraint needs is created automatically.
ALTER TABLE bookmarks
ALTER COLUMN reader_id SET NOT NULL,
DROP COLUMN key,
ADD PRIMARY KEY (reader_id, site, series_id),
ADD CONSTRAINT bookmarks_reader_fk
FOREIGN KEY (reader_id) REFERENCES readers (id) ON DELETE CASCADE;
+287 -96
View File
@@ -19,6 +19,11 @@ import (
//
// LastChapter* is the user's read progress; LatestChapter* is the newest
// chapter the site has published, captured opportunistically by the userscript.
//
// Title, SeriesURL, Cover, Kind and LatestChapter* live on the shared Series
// row (ADR-0003) and are joined in on read; Bookmark carries only what differs
// between readers: progress, favourite, lifecycle bucket, updated_at. The wire
// format stays flat regardless — see ADR-0004.
type Bookmark struct {
Key string `json:"key"`
Site string `json:"site"`
@@ -42,6 +47,35 @@ type Bookmark struct {
Kind string `json:"kind"`
}
// Series is one distinct work, shared by every bookmark that tracks it. It is
// keyed (site, series_id) — the pair a bookmark key decomposes into — and
// exists once no matter how many bookmarks point at it (ADR-0003).
//
// Title, SeriesURL and Cover are written once, at creation: a PUT naming an
// existing Series has them ignored, and only the backend's own Poll may change
// them. Kind and the latest-chapter fields are last-write-wins like the
// bookmark's own fields. Never serialized: the wire format is the flat
// Bookmark (ADR-0004).
type Series struct {
Site string
SeriesID string
Title string
SeriesURL string
Cover string
Kind string
LatestChapter string
LatestChapterNum *float64 // nil until first captured
LatestCheckedAt int64 // unix ms; see MarkLatestChecked
// readerCount is the number of bookmarks referencing this series, filled
// only by the due-queue query that orders on it.
readerCount int
}
// Key returns the canonical identity in bookmark-key form ("<site>:<series_id>"),
// used by the poller's logs and by tests asserting on the due queue.
func (s Series) Key() string { return s.Site + ":" + s.SeriesID }
// HasNewChapter reports whether the site has published past the read point.
// A nil LatestChapterNum means nothing has been captured yet, which is not the
// same as "nothing new".
@@ -122,37 +156,104 @@ const (
var migrations embed.FS
// bookmarkColumns is the only value ever concatenated into query text. It is a
// compile-time constant; every request value is bound as a parameter.
const bookmarkColumns = `key, site, series_id, title, series_url, cover,
last_chapter, last_chapter_num, last_chapter_url,
favorite, latest_chapter, latest_chapter_num, updated_at, status, kind`
// compile-time constant; every request value is bound as a parameter. The
// series-owned fields are joined in from the series table, in scanBookmark
// order, so the flat Bookmark reads back whole despite the split (ADR-0004).
const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover,
b.last_chapter, b.last_chapter_num, b.last_chapter_url,
b.favorite, s.latest_chapter, s.latest_chapter_num, b.updated_at, b.status, s.kind`
// seriesColumns is the series row in scanSeries order, used by the poller's
// due query. latest_checked_at lives only on series — see MarkLatestChecked
// for why it stays off every client-visible write.
const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at`
// Owner is the person running the service: the first Reader, and the only one
// until registration exists. The seed makes sure exactly one readers row
// matches their Discord ID, carrying the SHA-256 of their userscript token —
// which today is the global API token.
type Owner struct {
DiscordID string
// TokenHash is the SHA-256 of the userscript token; the array shape makes
// it a compile error to store anything that is not a hash.
TokenHash [32]byte
}
// Store is the Postgres-backed bookmark store.
type Store struct {
db *sql.DB
// ownerID is the seeded owner Reader (issue #22). Authentication is still
// the single global token, so every request acts as this Reader; the store
// methods take the id explicitly so the scoping survives per-Reader auth.
ownerID int64
}
// OwnerID returns the seeded owner Reader's id — the Reader every request
// acts as while the global token is still the only credential.
func (s *Store) OwnerID() int64 { return s.ownerID }
// readersMigration is the version that creates the readers table. The owner
// seed runs between two migrate passes, so that the run-once migration which
// attaches existing bookmarks (0004) finds the owner row.
const readersMigration = 3
// allMigrations is the migrate() cap that applies every pending version.
const allMigrations = 0
// Open connects to Postgres at url — a libpq connection URL such as
// "postgres://user:pass@host:5432/bookmarks?sslmode=disable" — and brings its
// schema up to date.
func Open(url string) (*Store, error) {
// "postgres://user:pass@host:5432/bookmarks?sslmode=disable" — brings its
// schema up to date, and seeds the owner Reader.
func Open(url string, owner Owner) (*Store, error) {
db, err := sql.Open("pgx", url)
if err != nil {
return nil, fmt.Errorf("open postgres: %w", err)
}
if err := migrate(db); err != nil {
// Schema runs in two passes with the seed between: 0003 creates the
// readers table, the owner row must exist before 0004 attaches the
// existing bookmarks to it. Anything past 0004 is applied by the second
// pass.
if err := migrate(db, readersMigration); err != nil {
db.Close()
return nil, fmt.Errorf("migrate schema: %w", err)
}
if err := seedOwner(db, owner); err != nil {
db.Close()
return nil, fmt.Errorf("seed owner: %w", err)
}
if err := migrate(db, allMigrations); err != nil {
db.Close()
return nil, fmt.Errorf("migrate: %w", err)
}
return &Store{db: db}, nil
var ownerID int64
if err := db.QueryRow(
`SELECT id FROM readers WHERE discord_id = $1`, owner.DiscordID).Scan(&ownerID); err != nil {
db.Close()
return nil, fmt.Errorf("resolve owner: %w", err)
}
return &Store{db: db, ownerID: ownerID}, nil
}
// seedOwner makes sure the configured owner exists as exactly one readers row,
// and keeps its token hash current on every start: rotating the userscript
// token must refresh the hash, or the stored credential goes stale.
func seedOwner(db *sql.DB, o Owner) error {
if _, err := db.Exec(`
INSERT INTO readers (discord_id, token_sha256) VALUES ($1, $2)
ON CONFLICT (discord_id) DO UPDATE SET token_sha256 = EXCLUDED.token_sha256`,
o.DiscordID, o.TokenHash[:]); err != nil {
return fmt.Errorf("seed owner: %w", err)
}
return nil
}
// migrate applies every embedded migration this database has not recorded, in
// filename order, each in its own transaction. Files are named
// "<version>_<name>.sql" and are append-only: editing an applied file changes
// nothing, because schema_migrations is how a database remembers what it ran.
// Runs on every start and is a no-op once current.
func migrate(db *sql.DB) error {
// filename order, each in its own transaction. upto caps the highest version
// applied; 0 means all. Files are named "<version>_<name>.sql" and are
// append-only: editing an applied file changes nothing, because
// schema_migrations is how a database remembers what it ran. Runs on every
// start and is a no-op once current.
func migrate(db *sql.DB, upto int64) error {
if _, err := db.Exec(`CREATE TABLE IF NOT EXISTS schema_migrations (
version bigint PRIMARY KEY,
applied_at timestamptz NOT NULL DEFAULT now())`); err != nil {
@@ -170,6 +271,9 @@ func migrate(db *sql.DB) error {
if err != nil {
return fmt.Errorf("migration %q: filename must start with a version number", name)
}
if upto > 0 && version > upto {
continue
}
body, err := migrations.ReadFile(name)
if err != nil {
return err
@@ -219,7 +323,7 @@ func scanBookmark(scan func(...any) error) (Bookmark, error) {
latestChapterNum sql.NullFloat64
)
if err := scan(
&b.Key, &b.Site, &b.SeriesID, &b.Title, &b.SeriesURL, &b.Cover,
&b.Site, &b.SeriesID, &b.Title, &b.SeriesURL, &b.Cover,
&b.LastChapter, &b.LastChapterNum, &b.LastChapterURL,
&b.Favorite, &b.LatestChapter, &latestChapterNum, &b.UpdatedAt, &b.Status, &b.Kind,
); err != nil {
@@ -228,6 +332,9 @@ func scanBookmark(scan func(...any) error) (Bookmark, error) {
if latestChapterNum.Valid {
b.LatestChapterNum = &latestChapterNum.Float64
}
// The wire identity is derived: there is no stored key column, the
// bookmark is keyed (reader_id, site, series_id) (issue #22).
b.Key = b.Site + ":" + b.SeriesID
// An unrecognised bucket (a hand-edited row) would leave the row in no list
// at all, so anything outside the three known buckets reads as the default
// rather than being passed through.
@@ -237,14 +344,39 @@ func scanBookmark(scan func(...any) error) (Bookmark, error) {
return b, nil
}
// scanSeries reads one row in seriesColumns order, plus the due query's
// reader_count column. latest_chapter_num is NULL until the first capture,
// same as on the bookmark read path.
func scanSeries(scan func(...any) error) (Series, error) {
var (
sr Series
latestChapterNum sql.NullFloat64
)
if err := scan(
&sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover,
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt,
&sr.readerCount,
); err != nil {
return Series{}, err
}
if latestChapterNum.Valid {
sr.LatestChapterNum = &latestChapterNum.Float64
}
return sr, nil
}
// Close releases the underlying database handle.
func (s *Store) Close() error { return s.db.Close() }
// List returns every bookmark, newest activity first.
func (s *Store) List() ([]Bookmark, error) {
rows, err := s.db.Query(`SELECT ` + bookmarkColumns + `
FROM bookmarks
ORDER BY updated_at DESC`)
// List returns every bookmark of one reader, newest activity first.
// Series-owned fields are joined in, so each Bookmark reads back whole and
// flat (ADR-0004).
func (s *Store) List(readerID int64) ([]Bookmark, error) {
rows, err := s.db.Query(`SELECT `+bookmarkColumns+`
FROM bookmarks b
JOIN series s ON s.site = b.site AND s.series_id = b.series_id
WHERE b.reader_id = $1
ORDER BY b.updated_at DESC`, readerID)
if err != nil {
return nil, fmt.Errorf("query bookmarks: %w", err)
}
@@ -261,12 +393,19 @@ func (s *Store) List() ([]Bookmark, error) {
return out, rows.Err()
}
// Get returns one bookmark by key. A missing key is not an error: ok is false
// and err is nil. UI mutations read-modify-write through this so they preserve
// the fields they do not touch.
func (s *Store) Get(key string) (Bookmark, bool, error) {
// Get returns one bookmark of one reader by key. A missing key is not an
// error: ok is false and err is nil. UI mutations read-modify-write through
// this so they preserve the fields they do not touch.
func (s *Store) Get(readerID int64, key string) (Bookmark, bool, error) {
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
return Bookmark{}, false, nil
}
b, err := scanBookmark(s.db.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks WHERE key = $1`, key).Scan)
`SELECT `+bookmarkColumns+` FROM bookmarks b
JOIN series s ON s.site = b.site AND s.series_id = b.series_id
WHERE b.reader_id = $1 AND b.site = $2 AND b.series_id = $3`,
readerID, site, seriesID).Scan)
if errors.Is(err, sql.ErrNoRows) {
return Bookmark{}, false, nil
}
@@ -276,15 +415,25 @@ func (s *Store) Get(key string) (Bookmark, bool, error) {
return b, true, nil
}
// Upsert inserts or replaces a bookmark by key (last-write-wins) and returns
// the row as actually stored.
// Upsert inserts or replaces one reader's bookmark by key (last-write-wins)
// and returns the row as actually stored — one flat object with the
// series-owned fields joined in, exactly as GET reports it (ADR-0004). A
// bookmark is keyed (reader_id, site, series_id), so the same key upserts two
// independent rows for two readers.
//
// The flat body is decomposed across two tables in one transaction. The series
// row is written first (the bookmarks FK requires it to exist), then the
// bookmark row. On the series side, title/series_url/cover are applied only
// when the row is brand new: once a series exists, client-supplied values are
// ignored, because the row is shared and the values are scraped page content —
// see ADR-0003. Kind and the latest-chapter fields are last-write-wins.
//
// b.UpdatedAt is only a candidate: it is applied when the row is new or when
// last_chapter_num changes, and otherwise the stored value is kept. Clients
// order their list by updated_at, so favoriting a series or recording a newly
// published chapter must not disturb that order — only real reading progress
// does. Callers must therefore use the returned bookmark, not the argument.
func (s *Store) Upsert(b Bookmark) (Bookmark, error) {
func (s *Store) Upsert(readerID int64, b Bookmark) (Bookmark, error) {
tx, err := s.db.Begin()
if err != nil {
return Bookmark{}, fmt.Errorf("begin %q: %w", b.Key, err)
@@ -296,53 +445,65 @@ func (s *Store) Upsert(b Bookmark) (Bookmark, error) {
latestNum = *b.LatestChapterNum
}
// IS DISTINCT FROM is Postgres's null-safe comparison, and it is what
// implements the ordering rule. Within DO UPDATE, a bare column is the
// stored row and excluded.* is the incoming one; a brand-new key never
// reaches this clause, so it keeps the fresh timestamp from VALUES.
//
// The status and kind columns resolve on the VALUES side, not in the
// conflict clause: excluded.* is the row *after* these expressions are
// evaluated, so a default applied there would look identical to a real
// 'reading' / 'manga' and would overwrite an archived or novel row on
// every PUT from a client that knows nothing about the column. Resolved
// once here, an empty incoming status or kind means "keep what is
// stored", and only a brand-new row falls through to the literal
// default. The subquery runs inside this transaction, so it sees the
// row this statement is about to conflict with.
// The kind column resolves on the VALUES side, not in the conflict clause:
// excluded.* is the row *after* these expressions are evaluated, so a
// default applied there would look identical to a real 'manga' and would
// overwrite a novel series on every PUT from a client that knows nothing
// about the column. Resolved once here, an empty incoming kind means "keep
// what is stored", and only a brand-new row falls through to the literal
// default. The subquery runs inside this transaction, so it sees the row
// this statement is about to conflict with. Same pattern as the status
// COALESCE on the bookmark insert below.
//
// The ::text casts are load-bearing: inside COALESCE/NULLIF there is no
// target column to infer the parameter type from, and Postgres rejects the
// statement rather than guessing.
if _, err := tx.Exec(`
INSERT INTO bookmarks (`+bookmarkColumns+`)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13,
COALESCE(NULLIF($14::text, ''), (SELECT status FROM bookmarks WHERE key = $1), 'reading'),
COALESCE(NULLIF($15::text, ''), (SELECT kind FROM bookmarks WHERE key = $1), 'manga'))
ON CONFLICT (key) DO UPDATE SET
site=excluded.site, series_id=excluded.series_id, title=excluded.title,
series_url=excluded.series_url, cover=excluded.cover,
INSERT INTO series (site, series_id, title, series_url, cover, kind,
latest_chapter, latest_chapter_num)
VALUES ($1, $2, $3, $4, $5,
COALESCE(NULLIF($6::text, ''), (SELECT kind FROM series WHERE site = $1 AND series_id = $2), 'manga'),
$7, $8)
ON CONFLICT (site, series_id) DO UPDATE SET
kind=excluded.kind,
latest_chapter=excluded.latest_chapter,
latest_chapter_num=excluded.latest_chapter_num`,
b.Site, b.SeriesID, b.Title, b.SeriesURL, b.Cover, b.Kind,
b.LatestChapter, latestNum); err != nil {
return Bookmark{}, fmt.Errorf("upsert series for %q: %w", b.Key, err)
}
// IS DISTINCT FROM is Postgres's null-safe comparison, and it is what
// implements the ordering rule. Within DO UPDATE, a bare column is the
// stored row and excluded.* is the incoming one; a brand-new key never
// reaches this clause, so it keeps the fresh timestamp from VALUES.
if _, err := tx.Exec(`
INSERT INTO bookmarks (reader_id, site, series_id, last_chapter, last_chapter_num,
last_chapter_url, favorite, status, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7,
COALESCE(NULLIF($8::text, ''), (SELECT status FROM bookmarks WHERE reader_id = $1 AND site = $2 AND series_id = $3), 'reading'),
$9)
ON CONFLICT (reader_id, site, series_id) DO UPDATE SET
last_chapter=excluded.last_chapter, last_chapter_num=excluded.last_chapter_num,
last_chapter_url=excluded.last_chapter_url,
favorite=excluded.favorite,
latest_chapter=excluded.latest_chapter,
latest_chapter_num=excluded.latest_chapter_num,
status=excluded.status,
kind=excluded.kind,
updated_at=CASE
WHEN bookmarks.last_chapter_num IS DISTINCT FROM excluded.last_chapter_num
THEN excluded.updated_at
ELSE bookmarks.updated_at
END`,
b.Key, b.Site, b.SeriesID, b.Title, b.SeriesURL, b.Cover,
readerID, b.Site, b.SeriesID,
b.LastChapter, b.LastChapterNum, b.LastChapterURL,
b.Favorite, b.LatestChapter, latestNum, b.UpdatedAt,
b.Status, b.Kind); err != nil {
b.Favorite, b.Status, b.UpdatedAt); err != nil {
return Bookmark{}, fmt.Errorf("upsert %q: %w", b.Key, err)
}
stored, err := scanBookmark(tx.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks WHERE key = $1`, b.Key).Scan)
`SELECT `+bookmarkColumns+` FROM bookmarks b
JOIN series s ON s.site = b.site AND s.series_id = b.series_id
WHERE b.reader_id = $1 AND b.site = $2 AND b.series_id = $3`,
readerID, b.Site, b.SeriesID).Scan)
if err != nil {
return Bookmark{}, fmt.Errorf("read back %q: %w", b.Key, err)
}
@@ -352,79 +513,109 @@ func (s *Store) Upsert(b Bookmark) (Bookmark, error) {
return stored, nil
}
// Delete removes a bookmark by key. Deleting a missing key is not an error.
func (s *Store) Delete(key string) error {
if _, err := s.db.Exec(`DELETE FROM bookmarks WHERE key = $1`, key); err != nil {
// Delete removes one reader's bookmark by key. Deleting a missing key is not
// an error.
func (s *Store) Delete(readerID int64, key string) error {
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
return nil
}
if _, err := s.db.Exec(
`DELETE FROM bookmarks WHERE reader_id = $1 AND site = $2 AND series_id = $3`,
readerID, site, seriesID); err != nil {
return fmt.Errorf("delete %q: %w", key, err)
}
return nil
}
// DueForLatestCheck returns bookmarks whose server-side latest-chapter check has
// aged past cutoffMs, least-recently-checked first, at most limit of them.
// DueForLatestCheck returns series whose server-side latest-chapter check has
// aged past cutoffMs, ordered by how many bookmarks reference them (descending)
// then least-recently-checked first, at most limit of them.
//
// Oldest-first is what keeps the poller fair when the backlog outgrows its
// The reader_count ordering is the point of the split (ADR-0003): a series
// shared by several readers is fetched once per due cycle, and the popular
// ones stay freshest while the long tail absorbs any shortfall. Within one
// reader count, oldest-first keeps the poll fair when the backlog outgrows its
// throughput: the most neglected series is always next, so a large collection
// refreshes uniformly slower rather than leaving a tail that never refreshes at
// all. The userscript sorts its own queue the same way (L453).
//
// Bookmarks with no series_url are skipped — there is nothing to fetch, which
// is the same filter the userscript applies at L452.
//
// Finished series are excluded: nothing more is coming, so fetching them only
// burns requests. Archived ones are deliberately still polled — knowing what a
// shelved series is up to is the whole reason for archiving instead of deleting.
func (s *Store) DueForLatestCheck(cutoffMs int64, limit int) ([]Bookmark, error) {
rows, err := s.db.Query(`SELECT `+bookmarkColumns+`
FROM bookmarks
WHERE series_url <> ''
AND status IS DISTINCT FROM 'finished'
AND latest_checked_at <= $1
ORDER BY latest_checked_at ASC
// Series with no series_url are skipped — there is nothing to fetch, which is
// the same filter the userscript applies at L452. Series whose only bookmarks
// are finished are skipped too: nothing more is coming, so fetching them only
// burns requests. Archived bookmarks still count — knowing what a shelved
// series is up to is the whole reason for archiving instead of deleting.
// A series with no bookmarks at all never appears: the join excludes it.
func (s *Store) DueForLatestCheck(cutoffMs int64, limit int) ([]Series, error) {
rows, err := s.db.Query(`SELECT `+seriesColumns+`, COUNT(*) AS reader_count
FROM series s
JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id
WHERE s.series_url <> ''
AND s.latest_checked_at <= $1
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at
HAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
ORDER BY reader_count DESC, s.latest_checked_at ASC
LIMIT $2`, cutoffMs, limit)
if err != nil {
return nil, fmt.Errorf("query due bookmarks: %w", err)
return nil, fmt.Errorf("query due series: %w", err)
}
defer rows.Close()
out := []Bookmark{}
out := []Series{}
for rows.Next() {
b, err := scanBookmark(rows.Scan)
sr, err := scanSeries(rows.Scan)
if err != nil {
return nil, fmt.Errorf("scan due bookmark: %w", err)
return nil, fmt.Errorf("scan due series: %w", err)
}
out = append(out, b)
out = append(out, sr)
}
return out, rows.Err()
}
// MarkLatestChecked records that the server looked at key at ts, whatever the
// look turned up. Marking a missing key is not an error: the row may have been
// deleted while a fetch was in flight.
// MarkLatestChecked records that the server looked at a series at ts, whatever
// the look turned up. Marking a missing series is not an error: the row may
// have been orphaned while a fetch was in flight.
//
// This is the one write that does not go through Upsert, and the column is kept
// out of bookmarkColumns on purpose. PUT /bookmarks/{key} decodes a whole
// Bookmark from the client and Upsert writes every column it knows about, so a
// userscript PUT — which has no idea this field exists — would write a zero and
// reset the cooldown, making the poller re-fetch that series every tick for as
// long as the user kept reading it.
func (s *Store) MarkLatestChecked(key string, ts int64) error {
// out of the client-visible read path on purpose. PUT /bookmarks/{key} decodes
// a whole Bookmark from the client and Upsert writes every series column it
// knows about, so a userscript PUT — which has no idea this field exists —
// would write a zero and reset the cooldown, making the poller re-fetch that
// series every tick for as long as the user kept reading it.
func (s *Store) MarkLatestChecked(site, seriesID string, ts int64) error {
if _, err := s.db.Exec(
`UPDATE bookmarks SET latest_checked_at = $1 WHERE key = $2`, ts, key); err != nil {
return fmt.Errorf("mark checked %q: %w", key, err)
`UPDATE series SET latest_checked_at = $1 WHERE site = $2 AND series_id = $3`,
ts, site, seriesID); err != nil {
return fmt.Errorf("mark checked %s:%s: %w", site, seriesID, err)
}
return nil
}
// LatestCheckedAt reads the column MarkLatestChecked writes. It exists for
// tests outside this package (the poller's own tests assert on cooldown
// bookkeeping) — see MarkLatestChecked for why the field itself stays off
// Bookmark.
func (s *Store) LatestCheckedAt(key string) (int64, error) {
// bookkeeping) — see MarkLatestChecked for why the field stays off the
// client-visible row.
func (s *Store) LatestCheckedAt(site, seriesID string) (int64, error) {
var ts int64
if err := s.db.QueryRow(
`SELECT latest_checked_at FROM bookmarks WHERE key = $1`, key).Scan(&ts); err != nil {
return 0, fmt.Errorf("latest checked at %q: %w", key, err)
`SELECT latest_checked_at FROM series WHERE site = $1 AND series_id = $2`,
site, seriesID).Scan(&ts); err != nil {
return 0, fmt.Errorf("latest checked at %s:%s: %w", site, seriesID, err)
}
return ts, nil
}
// SetLatestChapter records the newest chapter the poll found on a series page.
// The poller walks Series rather than Bookmarks, so this is a series-level
// write: the row is shared, and updating it once refreshes every bookmark that
// joins to it. Touching a missing series is not an error.
func (s *Store) SetLatestChapter(site, seriesID, label string, num float64) error {
if _, err := s.db.Exec(
`UPDATE series SET latest_chapter = $3, latest_chapter_num = $4
WHERE site = $1 AND series_id = $2`,
site, seriesID, label, num); err != nil {
return fmt.Errorf("set latest chapter %s:%s: %w", site, seriesID, err)
}
return nil
}
+532 -42
View File
@@ -1,7 +1,12 @@
package store
import (
"bytes"
"crypto/sha256"
"database/sql"
"os"
"strconv"
"strings"
"testing"
"time"
@@ -10,9 +15,13 @@ import (
func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
// testOwner is the owner every test store seeds. Tests that need a second
// reader insert one directly (see secondReader).
var testOwner = Owner{DiscordID: "test-owner", TokenHash: sha256.Sum256([]byte("owner-token-hash"))}
func newTestStore(t *testing.T) *Store {
t.Helper()
store, err := Open(pgtest.URL(t))
store, err := Open(pgtest.URL(t), testOwner)
if err != nil {
t.Fatalf("Open: %v", err)
}
@@ -20,29 +29,44 @@ func newTestStore(t *testing.T) *Store {
return store
}
// secondReader inserts an extra reader row and returns its id. The store API
// has no reader-creation path yet — the seed is the only one — so tests that
// need reader isolation insert directly.
func secondReader(t *testing.T, s *Store) int64 {
t.Helper()
hash := sha256.Sum256([]byte("second-token-hash"))
var id int64
if err := s.db.QueryRow(
`INSERT INTO readers (discord_id, token_sha256) VALUES ($1, $2) RETURNING id`,
"second-"+strconv.FormatInt(time.Now().UnixNano(), 10), hash[:]).Scan(&id); err != nil {
t.Fatalf("seed second reader: %v", err)
}
return id
}
// The migration runner runs on every start, so a second Open against a
// database it already built must be a no-op rather than a duplicate-table
// error, and must leave the rows alone.
func TestOpenIsIdempotent(t *testing.T) {
url := pgtest.URL(t)
first, err := Open(url)
first, err := Open(url, testOwner)
if err != nil {
t.Fatalf("Open: %v", err)
}
if _, err := first.Upsert(Bookmark{
if _, err := first.Upsert(first.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
first.Close()
second, err := Open(url)
second, err := Open(url, testOwner)
if err != nil {
t.Fatalf("reopen: %v", err)
}
t.Cleanup(func() { second.Close() })
list, err := second.List()
list, err := second.List(second.OwnerID())
if err != nil {
t.Fatalf("List: %v", err)
}
@@ -53,14 +77,14 @@ func TestOpenIsIdempotent(t *testing.T) {
func TestStoreGet(t *testing.T) {
store := newTestStore(t)
if _, err := store.Upsert(Bookmark{
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", LastChapterNum: 45, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("Upsert: %v", err)
}
got, ok, err := store.Get("asura:solo")
got, ok, err := store.Get(store.OwnerID(), "asura:solo")
if err != nil {
t.Fatalf("Get: %v", err)
}
@@ -74,7 +98,7 @@ func TestStoreGet(t *testing.T) {
func TestStoreGetMissing(t *testing.T) {
store := newTestStore(t)
_, ok, err := store.Get("asura:nope")
_, ok, err := store.Get(store.OwnerID(), "asura:nope")
if err != nil {
t.Fatalf("Get missing returned error %v, want nil", err)
}
@@ -124,30 +148,41 @@ func TestBookmarkContinueURL(t *testing.T) {
}
// readLatestCheckedAt reads the column directly. It is deliberately absent from
// Bookmark (see Store.Upsert), so tests cannot assert on it any other way.
// Series (see Store.MarkLatestChecked), so tests cannot assert on it any other
// way. The key splits the same way the API handler derives site/series_id.
func readLatestCheckedAt(t *testing.T, s *Store, key string) int64 {
t.Helper()
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
var ts int64
if err := s.db.QueryRow(
`SELECT latest_checked_at FROM bookmarks WHERE key = $1`, key).Scan(&ts); err != nil {
`SELECT latest_checked_at FROM series WHERE site = $1 AND series_id = $2`,
site, seriesID).Scan(&ts); err != nil {
t.Fatalf("read latest_checked_at %q: %v", key, err)
}
return ts
}
// seedForCheck inserts a bookmark and forces its latest_checked_at.
// seedForCheck inserts a bookmark (and with it its series) and forces the
// series' latest_checked_at.
func seedForCheck(t *testing.T, s *Store, key, seriesURL string, checkedAt int64) {
t.Helper()
if _, err := s.Upsert(Bookmark{
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
if _, err := s.Upsert(s.OwnerID(), Bookmark{
Key: key,
Site: "asura",
SeriesID: key,
Site: site,
SeriesID: seriesID,
SeriesURL: seriesURL,
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed %q: %v", key, err)
}
if err := s.MarkLatestChecked(key, checkedAt); err != nil {
if err := s.MarkLatestChecked(site, seriesID, checkedAt); err != nil {
t.Fatalf("seed mark %q: %v", key, err)
}
}
@@ -198,8 +233,8 @@ func TestDueForLatestCheckOldestFirstAndLimited(t *testing.T) {
if len(due) != 2 {
t.Fatalf("got %d rows, want 2 (limit)", len(due))
}
if due[0].Key != "asura:a" || due[1].Key != "asura:b" {
t.Fatalf("got %q,%q; want asura:a,asura:b (oldest first)", due[0].Key, due[1].Key)
if due[0].Key() != "asura:a" || due[1].Key() != "asura:b" {
t.Fatalf("got %q,%q; want asura:a,asura:b (oldest first)", due[0].Key(), due[1].Key())
}
}
@@ -207,30 +242,31 @@ func TestMarkLatestChecked(t *testing.T) {
s := newTestStore(t)
seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", 0)
if err := s.MarkLatestChecked("asura:x", 4242); err != nil {
if err := s.MarkLatestChecked("asura", "x", 4242); err != nil {
t.Fatalf("MarkLatestChecked: %v", err)
}
if got := readLatestCheckedAt(t, s, "asura:x"); got != 4242 {
t.Fatalf("latest_checked_at = %d, want 4242", got)
}
// A missing key is not an error: the row may have been deleted mid-fetch.
if err := s.MarkLatestChecked("asura:gone", 1); err != nil {
t.Fatalf("MarkLatestChecked on missing key: %v", err)
// A missing series is not an error: its bookmarks may have been deleted
// mid-fetch.
if err := s.MarkLatestChecked("asura", "gone", 1); err != nil {
t.Fatalf("MarkLatestChecked on missing series: %v", err)
}
}
// Upsert must not touch latest_checked_at. If the column ever migrates into
// bookmarkColumns, this fails and the cooldown is silently dead.
// the client-visible write path, this fails and the cooldown is silently dead.
func TestUpsertPreservesLatestCheckedAt(t *testing.T) {
s := newTestStore(t)
seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", 999)
b, ok, err := s.Get("asura:x")
b, ok, err := s.Get(s.OwnerID(), "asura:x")
if err != nil || !ok {
t.Fatalf("Get: %v ok=%v", err, ok)
}
b.Title = "changed"
if _, err := s.Upsert(b); err != nil {
if _, err := s.Upsert(s.OwnerID(), b); err != nil {
t.Fatalf("Upsert: %v", err)
}
if got := readLatestCheckedAt(t, s, "asura:x"); got != 999 {
@@ -240,7 +276,7 @@ func TestUpsertPreservesLatestCheckedAt(t *testing.T) {
func TestUpsertDefaultsStatusToReading(t *testing.T) {
store := newTestStore(t)
stored, err := store.Upsert(Bookmark{
stored, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
UpdatedAt: time.Now().UnixMilli(),
})
@@ -260,13 +296,13 @@ func TestUpsertEmptyStatusPreservesStored(t *testing.T) {
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Status: StatusArchived, UpdatedAt: time.Now().UnixMilli(),
}
if _, err := store.Upsert(base); err != nil {
if _, err := store.Upsert(store.OwnerID(), base); err != nil {
t.Fatalf("seed: %v", err)
}
base.Status = ""
base.LastChapterNum = 12
stored, err := store.Upsert(base)
stored, err := store.Upsert(store.OwnerID(), base)
if err != nil {
t.Fatalf("Upsert: %v", err)
}
@@ -285,11 +321,11 @@ func TestLatestPollRoundTripPreservesArchived(t *testing.T) {
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Status: StatusArchived, UpdatedAt: time.Now().UnixMilli(),
}
if _, err := store.Upsert(base); err != nil {
if _, err := store.Upsert(store.OwnerID(), base); err != nil {
t.Fatalf("seed: %v", err)
}
cur, found, err := store.Get(base.Key)
cur, found, err := store.Get(store.OwnerID(), base.Key)
if err != nil || !found {
t.Fatalf("Get: found=%v err=%v", found, err)
}
@@ -299,7 +335,7 @@ func TestLatestPollRoundTripPreservesArchived(t *testing.T) {
cur.LatestChapterNum = &num
cur.UpdatedAt = time.Now().UnixMilli()
stored, err := store.Upsert(cur)
stored, err := store.Upsert(store.OwnerID(), cur)
if err != nil {
t.Fatalf("Upsert: %v", err)
}
@@ -314,12 +350,12 @@ func TestUpsertReplacesStatusWhenGiven(t *testing.T) {
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Status: StatusArchived, UpdatedAt: time.Now().UnixMilli(),
}
if _, err := store.Upsert(base); err != nil {
if _, err := store.Upsert(store.OwnerID(), base); err != nil {
t.Fatalf("seed: %v", err)
}
base.Status = StatusReading
stored, err := store.Upsert(base)
stored, err := store.Upsert(store.OwnerID(), base)
if err != nil {
t.Fatalf("Upsert: %v", err)
}
@@ -336,14 +372,14 @@ func TestUpsertStatusChangeKeepsUpdatedAt(t *testing.T) {
LastChapter: "45", LastChapterNum: 45,
UpdatedAt: time.Now().UnixMilli(),
}
first, err := store.Upsert(base)
first, err := store.Upsert(store.OwnerID(), base)
if err != nil {
t.Fatalf("seed: %v", err)
}
base.Status = StatusArchived
base.UpdatedAt = first.UpdatedAt + 60_000
stored, err := store.Upsert(base)
stored, err := store.Upsert(store.OwnerID(), base)
if err != nil {
t.Fatalf("Upsert: %v", err)
}
@@ -361,8 +397,8 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
{"asura:archived", StatusArchived},
{"asura:finished", StatusFinished},
} {
if _, err := store.Upsert(Bookmark{
Key: tc.key, Site: "asura", SeriesID: tc.key,
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: tc.key, Site: "asura", SeriesID: strings.TrimPrefix(tc.key, "asura:"),
SeriesURL: "https://asurascans.com/comics/" + tc.key,
Status: tc.status, UpdatedAt: time.Now().UnixMilli(),
}); err != nil {
@@ -375,8 +411,8 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
t.Fatalf("DueForLatestCheck: %v", err)
}
got := map[string]bool{}
for _, b := range due {
got[b.Key] = true
for _, sr := range due {
got[sr.Key()] = true
}
if !got["asura:reading"] || !got["asura:archived"] {
t.Fatalf("due = %v, want reading and archived present", got)
@@ -426,7 +462,7 @@ func TestDisplayChapter(t *testing.T) {
func TestUpsertKindDefaultsToManga(t *testing.T) {
store := newTestStore(t)
got, err := store.Upsert(Bookmark{
got, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", UpdatedAt: 1000,
})
if err != nil {
@@ -439,7 +475,7 @@ func TestUpsertKindDefaultsToManga(t *testing.T) {
func TestUpsertKindRoundTrips(t *testing.T) {
store := newTestStore(t)
got, err := store.Upsert(Bookmark{
got, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "lightnovelworld:a-will-eternal", Site: "lightnovelworld",
SeriesID: "a-will-eternal", Kind: KindNovel, UpdatedAt: 1000,
})
@@ -455,14 +491,14 @@ func TestUpsertKindRoundTrips(t *testing.T) {
// must keep the stored library, not silently demote a novel to manga.
func TestUpsertEmptyKindKeepsStoredValue(t *testing.T) {
store := newTestStore(t)
if _, err := store.Upsert(Bookmark{
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "lightnovelworld:a-will-eternal", Site: "lightnovelworld",
SeriesID: "a-will-eternal", Kind: KindNovel, LastChapterNum: 10, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
got, err := store.Upsert(Bookmark{
got, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "lightnovelworld:a-will-eternal", Site: "lightnovelworld",
SeriesID: "a-will-eternal", Kind: "", LastChapterNum: 11, UpdatedAt: 2000,
})
@@ -476,3 +512,457 @@ func TestUpsertEmptyKindKeepsStoredValue(t *testing.T) {
t.Fatalf("LastChapterNum = %v, want 11 — progress in the same request must still land", got.LastChapterNum)
}
}
// The 0002 backfill must survive a database that already ran 0001 with real
// rows: one series row per distinct (site, series_id) carrying the moved
// columns, and the bookmark keeping the rest. That is the upgrade path for
// every deployed database, so it is exercised rather than trusted.
func TestMigration0002BackfillsExistingBookmarks(t *testing.T) {
url := pgtest.URL(t)
db, err := sql.Open("pgx", url)
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { db.Close() })
// Run only 0001, as a database created before this change would have.
// migrate() normally creates the version table first; do the same here.
if _, err := db.Exec(`CREATE TABLE IF NOT EXISTS schema_migrations (
version bigint PRIMARY KEY,
applied_at timestamptz NOT NULL DEFAULT now())`); err != nil {
t.Fatalf("create version table: %v", err)
}
body, err := migrations.ReadFile("migrations/0001_bookmarks.sql")
if err != nil {
t.Fatalf("read 0001: %v", err)
}
if err := applyMigration(db, 1, string(body)); err != nil {
t.Fatalf("apply 0001: %v", err)
}
if _, err := db.Exec(`
INSERT INTO bookmarks (key, site, series_id, title, series_url, cover,
last_chapter, last_chapter_num, last_chapter_url, favorite, latest_chapter,
latest_chapter_num, latest_checked_at, status, kind, updated_at)
VALUES ('asura:solo', 'asura', 'solo', 'Solo Leveling',
'https://asurascans.com/comics/solo', 'https://asurascans.com/covers/solo.jpg',
'Chapter 10', 10, 'https://asurascans.com/comics/solo/ch/10', true,
'Chapter 11', 11, 123456, 'reading', 'manga', 1000)`); err != nil {
t.Fatalf("seed legacy row: %v", err)
}
// Bring it current through the production path: Open runs the schema to
// 0003, seeds the owner, then applies 0004 which attaches this row. 0002
// must have backfilled the series row, not lost data.
st, err := Open(url, testOwner)
if err != nil {
t.Fatalf("Open after migrate: %v", err)
}
defer st.Close()
var (
title string
checked int64
fav bool
lastNum float64
)
if err := db.QueryRow(`SELECT title, latest_checked_at FROM series
WHERE site = 'asura' AND series_id = 'solo'`).Scan(&title, &checked); err != nil {
t.Fatalf("series row missing after migration: %v", err)
}
if title != "Solo Leveling" || checked != 123456 {
t.Fatalf("series = (%q, %d), want backfilled title and latest_checked_at", title, checked)
}
// The key column is gone; the bookmark is read by its composite key.
if err := db.QueryRow(`SELECT favorite, last_chapter_num FROM bookmarks
WHERE reader_id = $1 AND site = 'asura' AND series_id = 'solo'`,
st.OwnerID()).Scan(&fav, &lastNum); err != nil {
t.Fatalf("bookmark row missing after migration: %v", err)
}
if !fav || lastNum != 10 {
t.Fatalf("bookmark = (%v, %v), want favorite and progress kept", fav, lastNum)
}
}
// readSeries reads the series row directly, for asserting on what Upsert
// actually stored rather than what the joined Bookmark reports.
func readSeries(t *testing.T, s *Store, site, seriesID string) Series {
t.Helper()
sr, err := scanSeries(s.db.QueryRow(
`SELECT `+seriesColumns+`, 0 AS reader_count FROM series s
WHERE s.site = $1 AND s.series_id = $2`, site, seriesID).Scan)
if err != nil {
t.Fatalf("read series %s:%s: %v", site, seriesID, err)
}
return sr
}
// The first PUT for a series creates its row from the client's title, cover
// and URL — there is no other source for them (ADR-0003).
func TestUpsertCreatesSeriesFromClient(t *testing.T) {
store := newTestStore(t)
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", SeriesURL: "https://asurascans.com/comics/solo",
Cover: "https://asurascans.com/covers/solo.jpg", Kind: KindManga,
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("Upsert: %v", err)
}
sr := readSeries(t, store, "asura", "solo")
if sr.Title != "Solo Leveling" || sr.SeriesURL != "https://asurascans.com/comics/solo" ||
sr.Cover != "https://asurascans.com/covers/solo.jpg" {
t.Fatalf("series = %+v, want client title/url/cover stored", sr)
}
}
// A PUT naming an existing series must not overwrite its title, cover or URL:
// the row is shared, and those values are scraped page content (ADR-0003).
func TestUpsertExistingSeriesIgnoresClientTitleCoverURL(t *testing.T) {
store := newTestStore(t)
base := Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", SeriesURL: "https://asurascans.com/comics/solo",
Cover: "https://asurascans.com/covers/solo.jpg", LastChapterNum: 10,
UpdatedAt: 1000,
}
if _, err := store.Upsert(store.OwnerID(), base); err != nil {
t.Fatalf("seed: %v", err)
}
// Same series, hostile/compromised values, real progress advance.
base.Title = "Scraped Rename"
base.SeriesURL = "https://evil.example/solo"
base.Cover = "https://evil.example/solo.jpg"
base.LastChapterNum = 11
got, err := store.Upsert(store.OwnerID(), base)
if err != nil {
t.Fatalf("Upsert: %v", err)
}
if got.Title != "Solo Leveling" || got.SeriesURL != "https://asurascans.com/comics/solo" ||
got.Cover != "https://asurascans.com/covers/solo.jpg" {
t.Fatalf("stored = %+v, want original title/url/cover kept", got)
}
if got.LastChapterNum != 11 {
t.Fatalf("LastChapterNum = %v, want 11 — progress in the same request must still land", got.LastChapterNum)
}
}
// Kind and latest-chapter are last-write-wins even on an existing series: the
// poller and the userscript both report the latest chapter, and kind is only
// known to whichever client created the row.
func TestUpsertExistingSeriesAcceptsKindAndLatest(t *testing.T) {
store := newTestStore(t)
base := Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", Kind: KindManga,
UpdatedAt: 1000,
}
if _, err := store.Upsert(store.OwnerID(), base); err != nil {
t.Fatalf("seed: %v", err)
}
num := 12.0
base.Kind = KindNovel
base.LatestChapter = "Chapter 12"
base.LatestChapterNum = &num
got, err := store.Upsert(store.OwnerID(), base)
if err != nil {
t.Fatalf("Upsert: %v", err)
}
if got.Kind != KindNovel || got.LatestChapter != "Chapter 12" ||
got.LatestChapterNum == nil || *got.LatestChapterNum != 12 {
t.Fatalf("stored = %+v, want kind and latest chapter updated", got)
}
}
// Deleting the last bookmark must leave the series row behind, so a later
// re-bookmark shows title and cover immediately instead of waiting for a poll.
func TestDeleteKeepsSeriesRow(t *testing.T) {
store := newTestStore(t)
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", Cover: "https://asurascans.com/covers/solo.jpg",
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if err := store.Delete(store.OwnerID(), "asura:solo"); err != nil {
t.Fatalf("Delete: %v", err)
}
sr := readSeries(t, store, "asura", "solo")
if sr.Title != "Solo Leveling" {
t.Fatalf("series = %+v, want it kept after the last bookmark is deleted", sr)
}
// Re-bookmark with nothing but progress: the stored title/cover come back.
stored, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
LastChapterNum: 5, UpdatedAt: 2000,
})
if err != nil {
t.Fatalf("re-upsert: %v", err)
}
if stored.Title != "Solo Leveling" || stored.Cover != "https://asurascans.com/covers/solo.jpg" {
t.Fatalf("re-bookmark = %+v, want title/cover from the surviving series row", stored)
}
}
// seedSecondReader inserts an extra bookmark on an existing series, owned by a
// second reader. Two bookmarks can share a series only across readers now
// (issue #22); the due queue's reader-count ordering counts them all.
func seedSecondReader(t *testing.T, s *Store, key, site, seriesID string, updatedAt int64) {
t.Helper()
if _, err := s.Upsert(secondReader(t, s), Bookmark{
Key: key, Site: site, SeriesID: seriesID, UpdatedAt: updatedAt,
}); err != nil {
t.Fatalf("seed second reader %q: %v", key, err)
}
}
// The whole point of the split: a shared series is due once, ordered ahead of
// single-reader series by how many bookmarks reference it.
func TestDueForLatestCheckOrdersByReaderCountThenAge(t *testing.T) {
s := newTestStore(t)
// "pop" has two bookmarks but was checked most recently; "solo" has one and
// was checked long ago. Reader count must win over age.
seedForCheck(t, s, "asura:pop", "https://asurascans.com/comics/pop", 900)
seedSecondReader(t, s, "asura:pop:2", "asura", "pop", 1001)
seedForCheck(t, s, "asura:solo", "https://asurascans.com/comics/solo", 100)
due, err := s.DueForLatestCheck(1000, 10)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
if len(due) != 2 {
t.Fatalf("due = %d rows, want 2", len(due))
}
if due[0].Key() != "asura:pop" || due[1].Key() != "asura:solo" {
t.Fatalf("due order = %q, %q; want asura:pop (2 readers), asura:solo (1)",
due[0].Key(), due[1].Key())
}
}
// A series with no bookmarks must never appear in the due queue, and nothing
// in the store ever deletes it (see TestDeleteKeepsSeriesRow).
func TestDueForLatestCheckExcludesOrphanSeries(t *testing.T) {
s := newTestStore(t)
seedForCheck(t, s, "asura:kept", "https://asurascans.com/comics/kept", 0)
if _, err := s.db.Exec(`
INSERT INTO series (site, series_id, title, series_url, cover, kind,
latest_chapter, latest_chapter_num, latest_checked_at)
VALUES ('asura', 'orphan', 'Orphan', 'https://asurascans.com/comics/orphan',
'', 'manga', '', NULL, 0)`); err != nil {
t.Fatalf("seed orphan series: %v", err)
}
due, err := s.DueForLatestCheck(1000, 10)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
if len(due) != 1 || due[0].Key() != "asura:kept" {
t.Fatalf("due = %+v, want only the bookmarked series", due)
}
var n int
if err := s.db.QueryRow(
`SELECT count(*) FROM series WHERE site = 'asura' AND series_id = 'orphan'`).Scan(&n); err != nil {
t.Fatalf("count orphan series: %v", err)
}
if n != 1 {
t.Fatalf("orphan series count = %d, want 1 (never deleted)", n)
}
}
// The seed must never multiply the owner row: reopening the same database with
// a different token hash refreshes the stored hash, not the row. That is what
// keeps the readers table at exactly one row across restarts and token
// rotations.
func TestSeedOwnerIdempotentAndRefreshesTokenHash(t *testing.T) {
url := pgtest.URL(t)
first, err := Open(url, Owner{DiscordID: "owner", TokenHash: sha256.Sum256([]byte("hash-v1"))})
if err != nil {
t.Fatalf("Open: %v", err)
}
ownerID := first.OwnerID()
first.Close()
second, err := Open(url, Owner{DiscordID: "owner", TokenHash: sha256.Sum256([]byte("hash-v2"))})
if err != nil {
t.Fatalf("reopen: %v", err)
}
defer second.Close()
if second.OwnerID() != ownerID {
t.Fatalf("owner id = %d after reopen, want %d (same row)", second.OwnerID(), ownerID)
}
var (
n int
hash []byte
)
if err := second.db.QueryRow(`SELECT count(*), (SELECT token_sha256 FROM readers LIMIT 1) FROM readers`).Scan(&n, &hash); err != nil {
t.Fatalf("read readers: %v", err)
}
if n != 1 {
t.Fatalf("readers count = %d, want 1", n)
}
want := sha256.Sum256([]byte("hash-v2"))
if !bytes.Equal(hash, want[:]) {
t.Fatalf("token hash = %x, want the refreshed sha256", hash)
}
}
// The upgrade path for a deployed database: bookmarks created before readers
// existed must all land on the seeded owner, the key column must be gone, and
// the same database must be able to hold two readers' bookmarks for one series.
func TestMigration0004AttachesBookmarksToOwner(t *testing.T) {
url := pgtest.URL(t)
db, err := sql.Open("pgx", url)
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { db.Close() })
// A database at the state before #21 shipped: 0001 applied, bookmarks
// keyed by <site>:<series_id>, no series table. Rows land before 0002, the
// way a real deployment's data did.
if err := migrate(db, 1); err != nil {
t.Fatalf("migrate to 0001: %v", err)
}
for _, key := range []string{"asura:solo", "demonic:catastrophic-necromancer"} {
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
if _, err := db.Exec(`
INSERT INTO bookmarks (key, site, series_id, updated_at)
VALUES ($1, $2, $3, 1000)`, key, site, seriesID); err != nil {
t.Fatalf("seed legacy row %q: %v", key, err)
}
}
// 0002 backfills the series rows, as it did in the real upgrade.
if err := migrate(db, 2); err != nil {
t.Fatalf("migrate to 0002: %v", err)
}
st, err := Open(url, testOwner)
if err != nil {
t.Fatalf("Open: %v", err)
}
defer st.Close()
var (
attached int
readers int
)
if err := st.db.QueryRow(
`SELECT count(*) FROM bookmarks WHERE reader_id = $1`, st.OwnerID()).Scan(&attached); err != nil {
t.Fatalf("count attached bookmarks: %v", err)
}
if attached != 2 {
t.Fatalf("bookmarks attached to owner = %d, want all 2", attached)
}
if err := st.db.QueryRow(`SELECT count(*) FROM readers`).Scan(&readers); err != nil {
t.Fatalf("count readers: %v", err)
}
if readers != 1 {
t.Fatalf("readers = %d, want 1", readers)
}
// The surrogate key column is gone; only the composite key remains.
if _, err := st.db.Query(`SELECT key FROM bookmarks`); err == nil {
t.Fatal("bookmarks.key still exists after the migration")
}
}
// One Reader and Series pair must admit at most one bookmark, enforced by the
// primary key itself — a raw INSERT that skips the upsert must fail.
func TestBookmarkDuplicateImpossibleAtDatabaseLevel(t *testing.T) {
st := newTestStore(t)
// A series row on its own, no bookmark: the raw inserts below must only
// ever collide on the bookmark primary key.
if _, err := st.db.Exec(
`INSERT INTO series (site, series_id) VALUES ('asura', 'solo')`); err != nil {
t.Fatalf("seed series: %v", err)
}
insert := func() error {
_, err := st.db.Exec(`
INSERT INTO bookmarks (reader_id, site, series_id, updated_at)
VALUES ($1, 'asura', 'solo', 1000)`, st.OwnerID())
return err
}
if err := insert(); err != nil {
t.Fatalf("first insert: %v", err)
}
if err := insert(); err == nil {
t.Fatal("duplicate bookmark for the same reader and series was accepted")
}
}
// Every read and write is scoped to the reader it names: a second reader sees
// an empty list, cannot read or delete the owner's row, and a delete by the
// wrong reader leaves the row alone.
func TestStoreScopesBookmarksToReader(t *testing.T) {
st := newTestStore(t)
other := secondReader(t, st)
if _, err := st.Upsert(st.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed owner bookmark: %v", err)
}
otherList, err := st.List(other)
if err != nil {
t.Fatalf("List(other): %v", err)
}
if len(otherList) != 0 {
t.Fatalf("other reader's list = %+v, want empty", otherList)
}
if _, ok, err := st.Get(other, "asura:solo"); err != nil || ok {
t.Fatalf("Get(other, asura:solo) = ok:%v err:%v, want not found", ok, err)
}
if err := st.Delete(other, "asura:solo"); err != nil {
t.Fatalf("Delete(other): %v", err)
}
ownerList, err := st.List(st.OwnerID())
if err != nil {
t.Fatalf("List(owner): %v", err)
}
if len(ownerList) != 1 || ownerList[0].Key != "asura:solo" {
t.Fatalf("owner's list after other's delete = %+v, want the row intact", ownerList)
}
// The same key under a second reader is an independent bookmark.
if _, err := st.Upsert(other, Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", UpdatedAt: 2000,
}); err != nil {
t.Fatalf("upsert other's bookmark: %v", err)
}
if got, err := st.List(other); err != nil || len(got) != 1 {
t.Fatalf("other's list after own upsert = %+v err:%v, want 1 row", got, err)
}
}
// Deleting a reader must take their bookmarks with them (ON DELETE CASCADE)
// while leaving the shared series row behind.
func TestDeleteReaderCascadesToBookmarks(t *testing.T) {
st := newTestStore(t)
other := secondReader(t, st)
if _, err := st.Upsert(other, Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed other's bookmark: %v", err)
}
if _, err := st.db.Exec(`DELETE FROM readers WHERE id = $1`, other); err != nil {
t.Fatalf("delete reader: %v", err)
}
var n int
if err := st.db.QueryRow(`SELECT count(*) FROM bookmarks`).Scan(&n); err != nil {
t.Fatalf("count bookmarks: %v", err)
}
if n != 0 {
t.Fatalf("bookmarks after reader delete = %d, want 0 (cascade)", n)
}
sr := readSeries(t, st, "asura", "solo")
if sr.Title != "Solo Leveling" {
t.Fatalf("series = %+v, want it kept after its only reader was deleted", sr)
}
}
+9 -5
View File
@@ -32,6 +32,9 @@ const RecentCount = 5
// representations (HTML versus JSON) to different clients under different auth.
type Handler struct {
store *store.Store
// readerID is the Reader this UI acts as — the seeded owner, while the web
// password is still the only credential (issue #22).
readerID int64
tmpl *template.Template
key []byte
password string
@@ -81,13 +84,14 @@ type loginView struct {
// New parses every template up front so a broken one kills the process at
// startup rather than the first request that touches it.
func New(s *store.Store, apiToken, webPassword string) (*Handler, error) {
func New(s *store.Store, readerID int64, apiToken, webPassword string) (*Handler, error) {
tmpl, err := template.ParseFS(templateFS, "templates/*.html")
if err != nil {
return nil, err
}
return &Handler{
store: s,
readerID: readerID,
tmpl: tmpl,
key: session.Key(apiToken, webPassword),
password: webPassword,
@@ -216,7 +220,7 @@ func libOf(q string) string {
// archived favourite therefore shows only under Archived: Favourites means
// "favourites I am currently reading".
func (h *Handler) buildListView(lib, tab string) (listView, error) {
all, err := h.store.List() // already ordered updated_at DESC
all, err := h.store.List(h.readerID) // already ordered updated_at DESC
if err != nil {
return listView{}, err
}
@@ -374,7 +378,7 @@ func (h *Handler) loadForMutation(w http.ResponseWriter, r *http.Request) (store
http.Error(w, "missing key", http.StatusBadRequest)
return store.Bookmark{}, false
}
b, ok, err := h.store.Get(key)
b, ok, err := h.store.Get(h.readerID, key)
if err != nil {
log.Printf("ui get %q: %v", key, err)
http.Error(w, "internal error", http.StatusInternalServerError)
@@ -397,7 +401,7 @@ func (h *Handler) loadForMutation(w http.ResponseWriter, r *http.Request) (store
// describe the whole library, so they are rebuilt out of band on every
// mutation, at the cost of one extra list read per toggle.
func (h *Handler) saveAndRenderCard(w http.ResponseWriter, r *http.Request, b store.Bookmark) {
stored, err := h.store.Upsert(b)
stored, err := h.store.Upsert(h.readerID, b)
if err != nil {
log.Printf("ui upsert %q: %v", b.Key, err)
http.Error(w, "internal error", http.StatusInternalServerError)
@@ -489,7 +493,7 @@ func (h *Handler) uiDelete(w http.ResponseWriter, r *http.Request) {
http.Error(w, "missing key", http.StatusBadRequest)
return
}
if err := h.store.Delete(key); err != nil {
if err := h.store.Delete(h.readerID, key); err != nil {
log.Printf("ui delete %q: %v", key, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
+15 -3
View File
@@ -2,6 +2,7 @@ package main
import (
"context"
"crypto/sha256"
"errors"
"log"
"net/http"
@@ -30,6 +31,9 @@ type Config struct {
Port string
// WebPassword gates the browser UI. Empty disables the web routes entirely.
WebPassword string
// OwnerDiscordID identifies the seeded owner Reader (issue #22). Required:
// bookmarks are scoped to a Reader, and without an owner there is none.
OwnerDiscordID string
// UserscriptPath is the file served at /u/{token}/manga-bookmark.user.js.
// Supplied by a bindmount so the script can be edited without a rebuild.
UserscriptPath string
@@ -148,6 +152,7 @@ func loadConfig() Config {
DatabaseURL: os.Getenv("DATABASE_URL"),
Port: envOr("PORT", "8080"),
WebPassword: os.Getenv("WEB_PASSWORD"),
OwnerDiscordID: os.Getenv("OWNER_DISCORD_ID"),
UserscriptPath: envOr("USERSCRIPT_PATH", "/userscript/manga-bookmark.user.js"),
NovelUserscriptPath: envOr("NOVEL_USERSCRIPT_PATH", "/userscript/novel-bookmark.user.js"),
LatestPoll: loadLatestPoll(),
@@ -173,7 +178,7 @@ func newRouter(s *store.Store, cfg Config) http.Handler {
mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", userscript.Handler(cfg.Token, cfg.UserscriptPath))
mux.HandleFunc("GET /u/{token}/novel-bookmark.user.js", userscript.Handler(cfg.Token, cfg.NovelUserscriptPath))
h := &api.Handler{Store: s}
h := &api.Handler{Store: s, ReaderID: s.OwnerID()}
protected := http.NewServeMux()
protected.HandleFunc("GET /bookmarks", h.List)
protected.HandleFunc("PUT /bookmarks/{key}", h.Put)
@@ -187,7 +192,7 @@ func newRouter(s *store.Store, cfg Config) http.Handler {
// deployment that forgets WEB_PASSWORD exposes nothing rather than
// exposing an unprotected list.
if cfg.WebPassword != "" {
wh, err := web.New(s, cfg.Token, cfg.WebPassword)
wh, err := web.New(s, s.OwnerID(), cfg.Token, cfg.WebPassword)
if err != nil {
log.Fatalf("web handler: %v", err)
}
@@ -217,11 +222,18 @@ func main() {
if cfg.Token == "" {
log.Fatal("API_TOKEN is required")
}
if cfg.OwnerDiscordID == "" {
log.Fatal("OWNER_DISCORD_ID is required")
}
if cfg.DatabaseURL == "" {
log.Fatal("DATABASE_URL is required")
}
s, err := store.Open(cfg.DatabaseURL)
// The owner's userscript token is the global API token today (issue #22);
// the readers row carries its SHA-256, not the token itself.
owner := store.Owner{DiscordID: cfg.OwnerDiscordID, TokenHash: sha256.Sum256([]byte(cfg.Token))}
s, err := store.Open(cfg.DatabaseURL, owner)
if err != nil {
log.Fatalf("open store: %v", err)
}
+17 -17
View File
@@ -56,7 +56,7 @@ func TestIndexWithoutSessionShowsLogin(t *testing.T) {
func TestIndexWithSessionShowsList(t *testing.T) {
cfg := webConfig()
srv, st := newWebTestServer(t, cfg)
if _, err := st.Upsert(store.Bookmark{
if _, err := st.Upsert(st.OwnerID(), store.Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", LastChapter: "45", LastChapterNum: 45,
UpdatedAt: time.Now().UnixMilli(),
@@ -203,7 +203,7 @@ func TestStaticAssetsServed(t *testing.T) {
// seed inserts one bookmark and returns it as stored.
func seed(t *testing.T, st *store.Store, b store.Bookmark) store.Bookmark {
t.Helper()
stored, err := st.Upsert(b)
stored, err := st.Upsert(st.OwnerID(), b)
if err != nil {
t.Fatalf("Upsert: %v", err)
}
@@ -257,7 +257,7 @@ func TestFavoriteTogglesWithoutReordering(t *testing.T) {
t.Fatalf("favorite status = %d, want 200", rr.Code)
}
after, ok, err := st.Get("asura:solo")
after, ok, err := st.Get(st.OwnerID(), "asura:solo")
if err != nil || !ok {
t.Fatalf("Get after favorite: %v ok=%v", err, ok)
}
@@ -275,7 +275,7 @@ func TestFavoriteTogglesWithoutReordering(t *testing.T) {
// Toggling again turns it back off.
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, uiRequest(t, cfg, http.MethodPost, "/ui/bookmarks/asura:solo/favorite", nil))
back, _, _ := st.Get("asura:solo")
back, _, _ := st.Get(st.OwnerID(), "asura:solo")
if back.Favorite {
t.Fatal("Favorite = true after a second toggle, want false")
}
@@ -334,7 +334,7 @@ func TestChapterOverrideMovesUpdatedAt(t *testing.T) {
t.Fatalf("chapter override status = %d, want 200", rr.Code)
}
after, ok, err := st.Get("asura:solo")
after, ok, err := st.Get(st.OwnerID(), "asura:solo")
if err != nil || !ok {
t.Fatalf("Get after override: %v ok=%v", err, ok)
}
@@ -374,7 +374,7 @@ func TestChapterOverrideNoOpPreservesURLAndUpdatedAt(t *testing.T) {
t.Fatalf("chapter no-op status = %d, want 200", rr.Code)
}
after, ok, err := st.Get("asura:solo")
after, ok, err := st.Get(st.OwnerID(), "asura:solo")
if err != nil || !ok {
t.Fatalf("Get after no-op override: %v ok=%v", err, ok)
}
@@ -408,7 +408,7 @@ func TestChapterOverrideRejectsBadInput(t *testing.T) {
if rr.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400", rr.Code)
}
after, _, _ := st.Get("asura:solo")
after, _, _ := st.Get(st.OwnerID(), "asura:solo")
if after.LastChapterNum != 45 {
t.Fatalf("chapter changed to %v on invalid input", after.LastChapterNum)
}
@@ -459,7 +459,7 @@ func TestUIDeleteRemovesRow(t *testing.T) {
if !strings.Contains(body, `id="new-count" hx-swap-oob="true"`) {
t.Fatalf("delete body = %q, want the out-of-band badge", body)
}
if _, ok, _ := st.Get("asura:solo"); ok {
if _, ok, _ := st.Get(st.OwnerID(), "asura:solo"); ok {
t.Fatal("row still present after delete")
}
}
@@ -538,7 +538,7 @@ func seedStatusRows(t *testing.T, st *store.Store) {
}
for _, b := range rows {
b.UpdatedAt = time.Now().UnixMilli()
if _, err := st.Upsert(b); err != nil {
if _, err := st.Upsert(st.OwnerID(), b); err != nil {
t.Fatalf("seed %s: %v", b.Key, err)
}
}
@@ -612,7 +612,7 @@ func TestRecentStripCarriesUnreadOnlyAndOnlyOnAll(t *testing.T) {
Status: store.StatusReading, LastChapterNum: 40, LatestChapter: "40",
LatestChapterNum: floatPtr(40), UpdatedAt: time.Now().UnixMilli(),
}
if _, err := st.Upsert(caught); err != nil {
if _, err := st.Upsert(st.OwnerID(), caught); err != nil {
t.Fatalf("seed %s: %v", caught.Key, err)
}
@@ -632,12 +632,12 @@ func TestRecentStripCarriesUnreadOnlyAndOnlyOnAll(t *testing.T) {
}
// Nothing new anywhere: the strip has nothing to say and does not render.
reading, _, err := st.Get("asura:reading")
reading, _, err := st.Get(st.OwnerID(), "asura:reading")
if err != nil {
t.Fatalf("Get: %v", err)
}
reading.LatestChapterNum = floatPtr(reading.LastChapterNum)
if _, err := st.Upsert(reading); err != nil {
if _, err := st.Upsert(st.OwnerID(), reading); err != nil {
t.Fatalf("Upsert: %v", err)
}
// The section still ships (an out-of-band swap needs the id to exist) but
@@ -662,7 +662,7 @@ func TestRecentStripCapped(t *testing.T) {
Status: store.StatusReading, LastChapterNum: 1, LatestChapter: "2",
LatestChapterNum: floatPtr(2), UpdatedAt: time.Now().UnixMilli() + int64(i),
}
if _, err := st.Upsert(b); err != nil {
if _, err := st.Upsert(st.OwnerID(), b); err != nil {
t.Fatalf("seed %s: %v", b.Key, err)
}
}
@@ -692,7 +692,7 @@ func TestUIStatusSetsBucket(t *testing.T) {
if rr := postStatus(t, srv, cfg, "asura:reading", want); rr.Code != http.StatusOK {
t.Fatalf("set %s: status = %d, body %s", want, rr.Code, rr.Body.String())
}
b, ok, err := st.Get("asura:reading")
b, ok, err := st.Get(st.OwnerID(), "asura:reading")
if err != nil || !ok {
t.Fatalf("Get: ok=%v err=%v", ok, err)
}
@@ -710,7 +710,7 @@ func TestUIStatusRejectsUnknownValue(t *testing.T) {
if rr := postStatus(t, srv, cfg, "asura:reading", "dropped"); rr.Code != http.StatusBadRequest {
t.Fatalf("status = %d, want 400", rr.Code)
}
b, _, _ := st.Get("asura:reading")
b, _, _ := st.Get(st.OwnerID(), "asura:reading")
if b.Status != store.StatusReading {
t.Fatalf("stored status = %q, want it untouched", b.Status)
}
@@ -736,12 +736,12 @@ func TestUIStatusDoesNotReorderList(t *testing.T) {
srv, st := newWebTestServer(t, cfg)
seedStatusRows(t, st)
before, _, _ := st.Get("asura:reading")
before, _, _ := st.Get(st.OwnerID(), "asura:reading")
time.Sleep(2 * time.Millisecond)
if rr := postStatus(t, srv, cfg, "asura:reading", store.StatusArchived); rr.Code != http.StatusOK {
t.Fatalf("status = %d", rr.Code)
}
after, _, _ := st.Get("asura:reading")
after, _, _ := st.Get(st.OwnerID(), "asura:reading")
if after.UpdatedAt != before.UpdatedAt {
t.Fatalf("UpdatedAt moved %d -> %d", before.UpdatedAt, after.UpdatedAt)
}
+2
View File
@@ -15,6 +15,8 @@ services:
environment:
# API_TOKEN is required — compose refuses to start without it.
API_TOKEN: ${API_TOKEN:?set API_TOKEN in .env}
# Owner's Discord user ID — required, seeds the one Reader row.
OWNER_DISCORD_ID: ${OWNER_DISCORD_ID:?set OWNER_DISCORD_ID in .env}
ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to,https://novelfull.com,https://lightnovelworld.net}
# The bookmarks database. Host is the compose service name; the password
# comes from .env so it is never committed.