feat(backend)!: run on Postgres with a migration-owned schema (#28)

Swap modernc.org/sqlite for jackc/pgx/v5 with no observable change:
same endpoints, same wire format, same updated_at ordering rule.

The schema now comes from numbered SQL embedded in the binary and
applied on startup, one transaction each, recorded in
schema_migrations. That replaces two pieces of SQLite-era machinery,
both deleted rather than ported: the column probing (Postgres has ADD
COLUMN IF NOT EXISTS, and there is no legacy database left to probe)
and the Asura key rewrite, which has run clean on every start for
months now that the userscripts strip build hashes before writing. Its
regexp survives as latest.asuraBuildHash, where the poller still needs
it to scope chapter links to a series whose slug carries a rotating
hash.

Types get real: favorite is a boolean, chapter numbers double
precision, timestamps stay unix-ms bigint. SQLite's null-safe IS NOT
becomes IS DISTINCT FROM, which is what implements the rule that only
reading progress reorders a list. Inside COALESCE/NULLIF the status
and kind parameters need an explicit ::text -- there is no target
column to infer from and Postgres refuses to guess.

Tests lose their free t.TempDir() database, so Docker is now a hard
prerequisite for `go test ./...`: internal/pgtest starts one
postgres:17-alpine per test binary and hands each test a database of
its own.

Also lands CONTEXT.md and the four ADRs written while scoping #18.

BREAKING CHANGE: DB_PATH is retired for DATABASE_URL, which is
required and has no default. Compose gains a postgres service on an
internal network with its own volume; POSTGRES_PASSWORD joins .env.
The old bookmarks-data volume is deliberately left undeclared so
`docker compose down -v` cannot take the pre-migration database with
it. main is not deployable until #25 and #26 land.

Closes #20

Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #28.
This commit is contained in:
2026-08-08 06:52:20 +07:00
committed by sulthan
parent b9f9aea82c
commit 08749df050
31 changed files with 961 additions and 966 deletions
+15 -4
View File
@@ -1,8 +1,8 @@
Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system.
- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`.
- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + Postgres over `jackc/pgx/v5` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`.
Single binary, split into packages under `backend/internal/`: `store`
(Bookmark type, SQLite persistence, migrations), `latest` (background
(Bookmark type, Postgres persistence, migration runner), `latest` (background
poller, site parsers, TLS fetcher), `session` (cookie signing, login
rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON
bookmark handlers), `userscript` (userscript-serving handler), `web`
@@ -11,6 +11,16 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
packages together into `newRouter`. Root-level `*_test.go` hold
integration tests that exercise the full router; unit tests for a
package live beside it under `internal/`.
- **Schema is migration-owned.** `internal/store/migrations/*.sql` is
`go:embed`-ed and applied on every start by `store.migrate`: one numbered
file per change, one transaction each, versions recorded in
`schema_migrations`. Files are **append-only** — editing an applied one
changes nothing on a database that already ran it. No column probing, no
data-fixup migrations: both were SQLite-era machinery and are gone.
- **Tests need Docker.** `internal/pgtest` starts one `postgres:17-alpine`
container per test binary (`TestMain` -> `pgtest.Main`) and hands each test
its own database (`pgtest.URL(t)`). A package whose tests touch the store
must have that `TestMain`.
- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`), with a `kind` column (`manga`|`novel`) splitting the two libraries. Sync **last-write-wins**. Schema and endpoint list in plan.
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- **Web UI:** same binary serve password-gated browser UI on second
@@ -70,8 +80,9 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
`excluded.*` is post-evaluation row and default applied there would
wipe bucket on every PUT from client that predates column. See
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list), `DB_PATH`
(default `/data/bookmarks.db`), `PORT` (default `8080`), `WEB_PASSWORD`
- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list),
`DATABASE_URL` (Postgres connection URL, required — no default),
`PORT` (default `8080`), `WEB_PASSWORD`
(gates browser UI; unset disable it),
`LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER`
(background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`).
-81
View File
@@ -1,81 +0,0 @@
Guidance for Claude Code working under `backend/`. See root `CLAUDE.md` for the project-wide architecture diagram, hard constraints, and design system.
- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`.
Single binary, split into packages under `backend/internal/`: `store`
(Bookmark type, SQLite persistence, migrations), `latest` (background
poller, site parsers, TLS fetcher), `session` (cookie signing, login
rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON
bookmark handlers), `userscript` (userscript-serving handler), `web`
(browser UI handler + `templates/` + `static/`, `go:embed`-ed).
`backend/main.go` is the composition root — the only place that wires
packages together into `newRouter`. Root-level `*_test.go` hold
integration tests that exercise the full router; unit tests for a
package live beside it under `internal/`.
- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`|`comix`|`kagane`). Sync **last-write-wins**. Schema and endpoint list in plan.
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- **Web UI:** same binary serve password-gated browser UI on second
hostname — `GET /` (list, or login page when no session),
`POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints
under `/ui/*`. Templates + assets `go:embed`-ed under
`backend/internal/web/`, so `backend/Dockerfile` must copy the whole
`internal/` tree, not just `*.go`. Sessions stateless
HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, and when empty,
web routes not registered at all. UI mutations read-modify-write
through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one
place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`.
**Design-tool caveat:** templates link `/static/style.css` root-absolutely
(correct — served from `/`), but impeccable detector resolves
stylesheet href with `path.resolve(fileDir, href)`, drops directory
on leading `/` and silently skip file. Relative href don't help
either: template's directory isn't its served path. So
`detect.mjs backend/internal/web/templates` reports **false clean** —
always pass `backend/internal/web/static` too. One finding there,
`overused-font` on "Instrument Serif", deliberate identity choice, not debt.
- **Every action that moves series out of list is confirm-gated.**
Archive, finish, remove each open own `.confirm-row` disclosure
(`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈
`archive|finish|remove`); restore fire instantly since it's the reversal.
Remove's row wear ember wash, two reversible ones wear `.calm` grey.
`--ember` stay reserved for new-chapter signal: busy bar and inline
error use `--mute`.
- **Latest-chapter poller:** ticker goroutine in same binary re-check
each bookmarked series' newest published chapter from backend's own
network access, so `latest_chapter` stay fresh when user not
browsing. Second, parallel signal — userscript keep own
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
Two independent clocks: per-bookmark cooldown (`latest_checked_at` column,
enforced by `Store.DueForLatestCheck`'s WHERE clause) and wake interval.
Row stamped *before* fetch so broken series wait out full
cooldown instead of retrying every tick, and writes go through
`Store.Get` + `Store.Upsert` so new chapter never reorders list.
Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth
against fingerprint-based blocking; any failure log and skip. kagane sits
behind a Cloudflare JavaScript challenge the TLS client can't clear, so it is
browser-only: fetched over CDP via `BROWSER_WS_URL`, and simply not polled
when that's unset. See
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
Poller's `Store.Get` + `Store.Upsert` not wrapped in transaction, so
userscript `PUT` that commits between the two can get overwritten by
poller's stale re-read — reverting that read progress and, since stored
value now differs, moving `updated_at` and reordering list. Known,
accepted limitation for single-user deployment, not bug to fix.
- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
`finished`, orthogonal to `favorite`. Archived and finished appear only in
own tab — not in All, Updated, Favourites, or recent strip. Poller keeps
checking archived series and skip finished ones. `finished` settable
only from web UI; `PUT /bookmarks/{key}` reject it with 400.
**Empty incoming status means "keep stored one"** — resolved on the
`VALUES` side of `Store.Upsert`, not conflict clause, since
`excluded.*` is post-evaluation row and default applied there would
wipe bucket on every PUT from client that predates column. See
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list), `DB_PATH`
(default `/data/bookmarks.db`), `PORT` (default `8080`), `WEB_PASSWORD`
(gates browser UI; unset disable it),
`LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER`
(background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`).
`USERSCRIPT_PATH` (file served at `/u/{token}/manga-bookmark.user.js`,
default `/userscript/manga-bookmark.user.js`, supplied by bindmount).
`BROWSER_WS_URL` (headless-shell CDP endpoint for kagane; unset disables
browser polling and leaves that site to the userscript alone).
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+3 -7
View File
@@ -14,21 +14,17 @@ RUN go mod download
COPY *.go ./
COPY internal/ ./internal/
# Static binary: pure-Go sqlite means CGO_ENABLED=0 -> no libc dependency.
# Static binary: the Postgres driver (jackc/pgx) is pure Go, so CGO_ENABLED=0
# leaves no libc dependency.
# -trimpath + -ldflags strip paths and debug info for a smaller image.
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/server .
# Data dir with the runtime user's ownership so the mounted volume inherits it.
RUN mkdir -p /out/data
# --- runtime stage: distroless static, non-root ---
FROM gcr.io/distroless/static:nonroot
WORKDIR /
COPY --from=build /out/server /server
COPY --from=build --chown=65532:65532 /out/data /data
VOLUME ["/data"]
EXPOSE 8080
USER nonroot:nonroot
ENV DB_PATH=/data/bookmarks.db PORT=8080
ENV PORT=8080
ENTRYPOINT ["/server"]
+13 -20
View File
@@ -12,6 +12,7 @@ import (
"testing"
"time"
"bookmarkmanager/backend/internal/pgtest"
"bookmarkmanager/backend/internal/store"
)
@@ -25,15 +26,21 @@ func testConfig() Config {
}
}
func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
func newTestServer(t *testing.T) http.Handler {
t.Helper()
dbPath := filepath.Join(t.TempDir(), "test.db")
s, err := store.Open(dbPath)
return newRouter(newTestStore(t), testConfig())
}
func newTestStore(t *testing.T) *store.Store {
t.Helper()
s, err := store.Open(pgtest.URL(t))
if err != nil {
t.Fatalf("store.Open: %v", err)
}
t.Cleanup(func() { s.Close() })
return newRouter(s, testConfig())
return s
}
func auth(req *http.Request) *http.Request {
@@ -417,12 +424,7 @@ func TestLoadConfigWebPassword(t *testing.T) {
// moved into bookmarkColumns, this test catches it: the PUT would reset the
// cooldown and the poller would re-fetch that series on every single tick.
func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "test.db")
s, err := store.Open(dbPath)
if err != nil {
t.Fatalf("store.Open: %v", err)
}
t.Cleanup(func() { s.Close() })
s := newTestStore(t)
srv := newRouter(s, testConfig())
seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", 777)
@@ -454,12 +456,7 @@ func TestUserscriptServedWithWebUIDisabled(t *testing.T) {
t.Fatalf("write script: %v", err)
}
dbPath := filepath.Join(t.TempDir(), "nopass.db")
s, err := store.Open(dbPath)
if err != nil {
t.Fatalf("store.Open: %v", err)
}
t.Cleanup(func() { s.Close() })
s := newTestStore(t)
cfg := testConfig() // WebPassword empty
cfg.UserscriptPath = path
@@ -480,11 +477,7 @@ func TestNovelUserscriptServed(t *testing.T) {
t.Fatalf("write script: %v", err)
}
s, err := store.Open(filepath.Join(dir, "test.db"))
if err != nil {
t.Fatalf("store.Open: %v", err)
}
t.Cleanup(func() { s.Close() })
s := newTestStore(t)
cfg := testConfig()
cfg.NovelUserscriptPath = novelPath
+5 -13
View File
@@ -7,7 +7,7 @@ require (
github.com/bogdanfinn/tls-client v1.15.1
github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f
github.com/chromedp/chromedp v0.16.0
modernc.org/sqlite v1.34.4
github.com/jackc/pgx/v5 v5.10.0
)
require (
@@ -18,27 +18,19 @@ require (
github.com/bogdanfinn/utls v1.7.7-barnius // indirect
github.com/bogdanfinn/websocket v1.5.5-barnius // indirect
github.com/chromedp/sysutil v1.1.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68 // indirect
github.com/gobwas/httphead v0.1.0 // indirect
github.com/gobwas/pool v0.2.1 // indirect
github.com/gobwas/ws v1.4.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/klauspost/compress v1.18.2 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/tam7t/hpkp v0.0.0-20160821193359-2b70b4024ed5 // indirect
golang.org/x/crypto v0.46.0 // indirect
golang.org/x/net v0.48.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.32.0 // indirect
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
modernc.org/libc v1.55.3 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/strutil v1.2.0 // indirect
modernc.org/token v1.1.0 // indirect
)
+14 -44
View File
@@ -20,10 +20,9 @@ github.com/chromedp/chromedp v0.16.0 h1:rOO4deOm4CbZgBCa8mD9g2rDyIoNs0BkgvNrlbp5
github.com/chromedp/chromedp v0.16.0/go.mod h1:rbuGKFT1vMcFcFqKfPIO1GpX/N+2s8onm2qMxZLbU5U=
github.com/chromedp/sysutil v1.1.0 h1:PUFNv5EcprjqXZD9nJb9b/c9ibAbxiYo4exNWZyipwM=
github.com/chromedp/sysutil v1.1.0/go.mod h1:WiThHUdltqCNKGc4gaU50XgYjwjYIhKWoHGPTUfWTJ8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68 h1:KZaTBSyshWX3MP5jukJcNSuXDQTO+rNpt0J564dX/eg=
github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68/go.mod h1:tphK2c80bpPhMOI4v6bIc2xWywPfbqi1Z06+RcrMkDg=
github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU=
@@ -32,28 +31,27 @@ github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og=
github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw=
github.com/gobwas/ws v1.4.0 h1:CTaoG1tojrh4ucGPcoJFiAQUAsEWekEWvLy7GsVNqGs=
github.com/gobwas/ws v1.4.0/go.mod h1:G3gNqMNtPppf5XUz7O4shetPpcZ1VJ7zt18dlUeakrc=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80 h1:6Yzfa6GP0rIo/kULo2bwGEkFvCePZ3qHDDTC3/J9Swo=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/tam7t/hpkp v0.0.0-20160821193359-2b70b4024ed5 h1:YqAladjX7xpA6BM04leXMWAEjS0mTZ5kUU9KRBriQJc=
@@ -64,8 +62,6 @@ go.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=
go.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=
golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU=
golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0=
golang.org/x/mod v0.30.0 h1:fDEXFVZ/fmCKProc/yAXXUijritrDzahmwwefnjoPFk=
golang.org/x/mod v0.30.0/go.mod h1:lAsf5O2EvJeSFMiBxXDki7sCgAxEUcZHXoXMKT4GJKc=
golang.org/x/net v0.0.0-20211104170005-ce137452f963/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU=
golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY=
@@ -81,33 +77,7 @@ golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU=
golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.39.0 h1:ik4ho21kwuQln40uelmciQPp9SipgNDdrafrYA4TmQQ=
golang.org/x/tools v0.39.0/go.mod h1:JnefbkDPyD8UU2kI5fuf8ZX4/yUeh9W877ZeBONxUqQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ=
modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ=
modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y=
modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s=
modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE=
modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ=
modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw=
modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4=
modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U=
modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w=
modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E=
modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc=
modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss=
modernc.org/sqlite v1.34.4 h1:sjdARozcL5KJBvYQvLlZEmctRgW9xqIZc2ncN7PU0P8=
modernc.org/sqlite v1.34.4/go.mod h1:3QQFCG2SEMtc2nv+Wq4cQCH7Hjcg+p/RMlS1XK+zwbk=
modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA=
modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+6 -3
View File
@@ -3,18 +3,21 @@ package latest
import (
"context"
"errors"
"path/filepath"
"os"
"sync"
"testing"
"time"
"bookmarkmanager/backend/internal/pgtest"
"bookmarkmanager/backend/internal/store"
)
// newTestStore opens a fresh SQLite store in a temp dir.
func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
// newTestStore opens a store on a Postgres database of this test's own.
func newTestStore(t *testing.T) *store.Store {
t.Helper()
s, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
s, err := store.Open(pgtest.URL(t))
if err != nil {
t.Fatalf("Open: %v", err)
}
+9 -5
View File
@@ -5,8 +5,6 @@ import (
"regexp"
"strconv"
"strings"
"bookmarkmanager/backend/internal/store"
)
// latestChapter is the newest chapter a series page advertises.
@@ -22,6 +20,12 @@ type latestChapter struct {
// before using the slug to scope anything.
var asuraSlugRe = regexp.MustCompile(`/comics/([^/?#]+)`)
// asuraBuildHash matches the trailing "-xxxxxxxx" site-wide build ID Asura
// appends to every series slug. It rotates on each site redeploy, so it is
// never part of a stable series_id. Must stay in sync with stripBuildHash in
// userscript/manga-bookmark.user.js.
var asuraBuildHash = regexp.MustCompile(`-[0-9a-f]{8}$`)
// demonicChapterRe matches the pre-redirect anchors demonic series pages link
// through. Both the raw "&" and the HTML-escaped "&amp;" forms occur.
var demonicChapterRe = regexp.MustCompile(`chaptered\.php\?manga=\d+&(?:amp;)?chapter=([0-9.]+)`)
@@ -74,9 +78,9 @@ func latestChapterFrom(site, seriesURL, body string) (latestChapter, bool) {
}
// Stored URLs predating a redeploy may carry a stale build hash;
// chapter hrefs in the fetched body carry the current one. Strip to
// the stable ID (same rule as migrateAsuraKeys) and make the hash
// optional in the pattern, so scoping survives rotations.
slug := store.AsuraBuildHash.ReplaceAllString(m[1], "")
// the stable ID and make the hash optional in the pattern, so scoping
// survives rotations.
slug := asuraBuildHash.ReplaceAllString(m[1], "")
// Compiled per call rather than cached: this runs once per fetch, which
// is at most a few times a minute, and the slug varies per series.
re = regexp.MustCompile(`/comics/` + regexp.QuoteMeta(slug) + `(?:-[0-9a-f]{8})?/chapter/([0-9.]+)`)
+120
View File
@@ -0,0 +1,120 @@
// Package pgtest runs the Postgres the test suite needs: one throwaway
// container per test binary, one fresh database per test. Docker is therefore
// a hard prerequisite for `go test ./...`.
//
// Rolled by hand rather than pulled in as a dependency — it is one `docker
// run`, one `docker port` and a ping loop, against a module list that is
// otherwise stdlib plus what the poller genuinely needs.
package pgtest
import (
"database/sql"
"fmt"
"os/exec"
"strconv"
"strings"
"sync/atomic"
"testing"
"time"
_ "github.com/jackc/pgx/v5/stdlib"
)
const (
image = "postgres:17-alpine"
readyLimit = 60 * time.Second
)
var (
adminURL string
dbSeq atomic.Int64
)
// Main starts the container, runs the package's tests and tears the container
// down. Every test package that touches the store calls it from TestMain:
//
// func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
func Main(m *testing.M) int {
id, url, err := start()
if err != nil {
fmt.Println("pgtest:", err)
return 1
}
defer exec.Command("docker", "rm", "-f", id).Run()
adminURL = url
return m.Run()
}
// URL creates a database of its own for t and returns a connection URL for it.
// Nothing drops it again: the container goes away wholesale when Main returns.
func URL(t testing.TB) string {
t.Helper()
if adminURL == "" {
t.Fatal("pgtest: no container; this package needs TestMain to call pgtest.Main")
}
// Generated, never derived from the test name, so it needs no quoting and
// cannot collide when tests run in parallel.
name := "test_" + strconv.FormatInt(dbSeq.Add(1), 10)
admin, err := sql.Open("pgx", adminURL)
if err != nil {
t.Fatalf("pgtest: open admin connection: %v", err)
}
defer admin.Close()
if _, err := admin.Exec(`CREATE DATABASE ` + name); err != nil {
t.Fatalf("pgtest: create database %s: %v", name, err)
}
return strings.Replace(adminURL, "/postgres?", "/"+name+"?", 1)
}
// start launches the container and waits for it to accept queries, returning
// its id and a connection URL for the default database.
func start() (id, url string, err error) {
out, err := exec.Command("docker", "run", "-d", "--rm",
"-e", "POSTGRES_PASSWORD=pgtest",
"-P", image,
// Durability buys nothing for a database that dies with the test
// binary, and turning it off is most of the container's start-up cost.
"-c", "fsync=off", "-c", "full_page_writes=off",
).Output()
if err != nil {
return "", "", fmt.Errorf("docker run %s: %w", image, err)
}
id = strings.TrimSpace(string(out))
port, err := exec.Command("docker", "port", id, "5432/tcp").Output()
if err != nil {
exec.Command("docker", "rm", "-f", id).Run()
return "", "", fmt.Errorf("docker port: %w", err)
}
// "0.0.0.0:32768" (and possibly a second, IPv6 line); the port is all we want.
first, _, _ := strings.Cut(strings.TrimSpace(string(port)), "\n")
url = fmt.Sprintf("postgres://postgres:pgtest@127.0.0.1:%s/postgres?sslmode=disable",
first[strings.LastIndex(first, ":")+1:])
if err := waitReady(url); err != nil {
exec.Command("docker", "rm", "-f", id).Run()
return "", "", err
}
return id, url, nil
}
func waitReady(url string) error {
db, err := sql.Open("pgx", url)
if err != nil {
return err
}
defer db.Close()
deadline := time.Now().Add(readyLimit)
for {
if err = db.Ping(); err == nil {
return nil
}
if time.Now().After(deadline) {
return fmt.Errorf("postgres not ready after %s: %w", readyLimit, err)
}
time.Sleep(200 * time.Millisecond)
}
}
@@ -0,0 +1,28 @@
-- One row per tracked series, keyed "<site>:<series_id>".
--
-- Everything is NOT NULL with a default except latest_chapter_num, where NULL
-- is a distinct state: nothing has been captured yet, which is not the same as
-- chapter zero.
--
-- Timestamps are unix milliseconds as bigint, not timestamptz: the userscripts
-- send Date.now() over the wire and the ordering rule compares them directly.
CREATE TABLE bookmarks (
key text PRIMARY KEY,
site text NOT NULL,
series_id text NOT NULL,
title text NOT NULL DEFAULT '',
series_url text NOT NULL DEFAULT '',
cover text NOT NULL DEFAULT '',
last_chapter text NOT NULL DEFAULT '',
last_chapter_num double precision NOT NULL DEFAULT 0,
last_chapter_url text NOT NULL DEFAULT '',
favorite boolean NOT NULL DEFAULT false,
latest_chapter text NOT NULL DEFAULT '',
latest_chapter_num double precision,
-- When the server last polled this series, unix ms; 0 means never, and sorts
-- first so a new bookmark is picked up on the next tick with no special case.
latest_checked_at bigint NOT NULL DEFAULT 0,
status text NOT NULL DEFAULT 'reading',
kind text NOT NULL DEFAULT 'manga',
updated_at bigint NOT NULL
);
+98 -193
View File
@@ -2,13 +2,17 @@ package store
import (
"database/sql"
"embed"
"errors"
"fmt"
"io/fs"
"path"
"regexp"
"slices"
"strconv"
"strings"
_ "modernc.org/sqlite"
_ "github.com/jackc/pgx/v5/stdlib"
)
// Bookmark is one tracked series, keyed "<site>:<series_id>" across both sites.
@@ -114,222 +118,119 @@ const (
StatusFinished = "finished"
)
const schema = `
CREATE TABLE IF NOT EXISTS bookmarks (
key TEXT PRIMARY KEY,
site TEXT NOT NULL,
series_id TEXT NOT NULL,
title TEXT,
series_url TEXT,
cover TEXT,
last_chapter TEXT,
last_chapter_num REAL,
last_chapter_url TEXT,
favorite INTEGER NOT NULL DEFAULT 0,
latest_chapter TEXT NOT NULL DEFAULT '',
latest_chapter_num REAL,
latest_checked_at INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'reading',
kind TEXT NOT NULL DEFAULT 'manga',
updated_at INTEGER NOT NULL
);`
// The columns above that databases created before them will be missing.
// SQLite has no ADD COLUMN IF NOT EXISTS, so each is added only when absent.
var addedColumns = []struct{ name, ddl string }{
{"favorite", `ALTER TABLE bookmarks ADD COLUMN favorite INTEGER NOT NULL DEFAULT 0`},
{"latest_chapter", `ALTER TABLE bookmarks ADD COLUMN latest_chapter TEXT NOT NULL DEFAULT ''`},
{"latest_chapter_num", `ALTER TABLE bookmarks ADD COLUMN latest_chapter_num REAL`},
// When the server last looked at this series, unix ms; 0 means never, and
// sorts first so a new bookmark is picked up on the next tick with no
// special case. Deliberately NOT in bookmarkColumns — see MarkLatestChecked.
{"latest_checked_at", `ALTER TABLE bookmarks ADD COLUMN latest_checked_at INTEGER NOT NULL DEFAULT 0`},
// Lifecycle bucket. The DEFAULT backfills every pre-existing row as
// 'reading', so there is no separate migration step.
{"status", `ALTER TABLE bookmarks ADD COLUMN status TEXT NOT NULL DEFAULT 'reading'`},
// Library bucket. The DEFAULT backfills every pre-existing row as 'manga',
// which is what every row written before novels existed actually is.
{"kind", `ALTER TABLE bookmarks ADD COLUMN kind TEXT NOT NULL DEFAULT 'manga'`},
}
//go:embed migrations/*.sql
var migrations embed.FS
// bookmarkColumns is the only value ever concatenated into query text. It is a
// compile-time constant; every request value is bound as a parameter.
const bookmarkColumns = `key, site, series_id, title, series_url, cover,
last_chapter, last_chapter_num, last_chapter_url,
favorite, latest_chapter, latest_chapter_num, updated_at, status, kind`
// Store is the SQLite-backed bookmark store.
// Store is the Postgres-backed bookmark store.
type Store struct {
db *sql.DB
}
// OpenStore opens (or creates) the SQLite database at path and applies the schema.
func Open(path string) (*Store, error) {
// busy_timeout guards against SQLITE_BUSY under the reverse proxy's
// concurrent requests; a single writer connection keeps writes serialized.
dsn := path
if !strings.Contains(dsn, "?") {
dsn += "?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)"
}
db, err := sql.Open("sqlite", dsn)
// Open connects to Postgres at url — a libpq connection URL such as
// "postgres://user:pass@host:5432/bookmarks?sslmode=disable" — and brings its
// schema up to date.
func Open(url string) (*Store, error) {
db, err := sql.Open("pgx", url)
if err != nil {
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
return nil, fmt.Errorf("open postgres: %w", err)
}
db.SetMaxOpenConns(1)
if _, err := db.Exec(schema); err != nil {
if err := migrate(db); err != nil {
db.Close()
return nil, fmt.Errorf("apply schema: %w", err)
}
if err := migrateColumns(db); err != nil {
db.Close()
return nil, fmt.Errorf("migrate schema: %w", err)
}
if err := migrateAsuraKeys(db); err != nil {
db.Close()
return nil, fmt.Errorf("migrate asura keys: %w", err)
return nil, fmt.Errorf("migrate: %w", err)
}
return &Store{db: db}, nil
}
// migrateColumns brings a pre-existing bookmarks table up to the current
// schema. Safe to run on every start: columns already present are skipped.
func migrateColumns(db *sql.DB) error {
have, err := existingColumns(db, "bookmarks")
// migrate applies every embedded migration this database has not recorded, in
// filename order, each in its own transaction. Files are named
// "<version>_<name>.sql" and are append-only: editing an applied file changes
// nothing, because schema_migrations is how a database remembers what it ran.
// Runs on every start and is a no-op once current.
func migrate(db *sql.DB) error {
if _, err := db.Exec(`CREATE TABLE IF NOT EXISTS schema_migrations (
version bigint PRIMARY KEY,
applied_at timestamptz NOT NULL DEFAULT now())`); err != nil {
return fmt.Errorf("create version table: %w", err)
}
names, err := fs.Glob(migrations, "migrations/*.sql")
if err != nil {
return err
}
for _, c := range addedColumns {
if _, ok := have[c.name]; ok {
continue
slices.Sort(names)
for _, name := range names {
version, err := strconv.ParseInt(strings.SplitN(path.Base(name), "_", 2)[0], 10, 64)
if err != nil {
return fmt.Errorf("migration %q: filename must start with a version number", name)
}
if _, err := db.Exec(c.ddl); err != nil {
return fmt.Errorf("add column %q: %w", c.name, err)
body, err := migrations.ReadFile(name)
if err != nil {
return err
}
if err := applyMigration(db, version, string(body)); err != nil {
return fmt.Errorf("migration %q: %w", name, err)
}
}
return nil
}
// AsuraBuildHash matches the trailing "-xxxxxxxx" site-wide build ID Asura
// appends to every series slug. It rotates on each site redeploy, so it
// must not be part of series_id. Must stay in sync with stripBuildHash in
// userscript/manga-bookmark.user.js.
var AsuraBuildHash = regexp.MustCompile(`-[0-9a-f]{8}$`)
// migrateAsuraKeys rewrites asura bookmarks whose series_id still carries
// the build hash to the stable, hashless ID. Rows keyed with a hash are
// orphaned on every Asura redeploy (old-hash URLs 302 to new-hash ones, so
// detection yields a key that never matches). When two hash-generations of
// one series collide, the row with the newest updated_at wins and the rest
// are deleted. Idempotent: hashless IDs never match the regex.
func migrateAsuraKeys(db *sql.DB) error {
rows, err := db.Query(`SELECT key, series_id, updated_at FROM bookmarks WHERE site = 'asura'`)
// applyMigration runs one migration and records its version in the same
// transaction, so an interrupted start leaves neither half behind.
func applyMigration(db *sql.DB, version int64, body string) error {
tx, err := db.Begin()
if err != nil {
return fmt.Errorf("list asura rows: %w", err)
}
type row struct {
key, id string
updated int64
}
var all []row
for rows.Next() {
var r row
if err := rows.Scan(&r.key, &r.id, &r.updated); err != nil {
rows.Close()
return fmt.Errorf("scan asura row: %w", err)
}
all = append(all, r)
}
if err := rows.Close(); err != nil {
return err
}
defer tx.Rollback()
groups := map[string][]row{}
for _, r := range all {
stripped := AsuraBuildHash.ReplaceAllString(r.id, "")
groups[stripped] = append(groups[stripped], r)
var applied bool
if err := tx.QueryRow(
`SELECT EXISTS (SELECT 1 FROM schema_migrations WHERE version = $1)`,
version).Scan(&applied); err != nil {
return err
}
for stripped, g := range groups {
winner := 0
for i := range g {
if g[i].updated > g[winner].updated {
winner = i
}
}
// Losers go first: rewriting the winner to the stripped key while a
// pre-existing hashless row still holds it is a primary-key collision.
for i, r := range g {
if i == winner {
continue
}
if _, err := db.Exec(`DELETE FROM bookmarks WHERE key = ?`, r.key); err != nil {
return fmt.Errorf("drop duplicate %q: %w", r.key, err)
}
}
if r := g[winner]; r.id != stripped {
if _, err := db.Exec(
`UPDATE bookmarks SET key = ?, series_id = ? WHERE key = ?`,
"asura:"+stripped, stripped, r.key); err != nil {
return fmt.Errorf("rewrite key %q: %w", r.key, err)
}
}
if applied {
return nil
}
return nil
// No parameters, so this goes over the simple protocol and a migration may
// hold more than one statement.
if _, err := tx.Exec(body); err != nil {
return err
}
if _, err := tx.Exec(`INSERT INTO schema_migrations (version) VALUES ($1)`, version); err != nil {
return err
}
return tx.Commit()
}
func existingColumns(db *sql.DB, table string) (map[string]struct{}, error) {
rows, err := db.Query(`SELECT name FROM pragma_table_info(?)`, table)
if err != nil {
return nil, fmt.Errorf("read %s columns: %w", table, err)
}
defer rows.Close()
out := map[string]struct{}{}
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
return nil, fmt.Errorf("scan column name: %w", err)
}
out[name] = struct{}{}
}
return out, rows.Err()
}
// scanBookmark reads one row in bookmarkColumns order, translating SQLite's
// integer bool and nullable latest_chapter_num into Go types.
//
// The optional columns are read through Null* types because rows predating
// this code (or written by hand) may hold NULL where the app only ever writes
// zero values. Only latest_chapter_num distinguishes the two: everywhere else
// NULL and the zero value mean the same thing to clients.
// scanBookmark reads one row in bookmarkColumns order. Every column is NOT
// NULL except latest_chapter_num, where NULL means "never captured" — a
// distinct state from chapter zero, and the reason for the pointer.
func scanBookmark(scan func(...any) error) (Bookmark, error) {
var (
b Bookmark
title, seriesURL, cover sql.NullString
lastChapter, lastChapterURL, latestChapter sql.NullString
status sql.NullString
lastChapterNum, latestChapterNum sql.NullFloat64
favorite sql.NullInt64
b Bookmark
latestChapterNum sql.NullFloat64
)
if err := scan(
&b.Key, &b.Site, &b.SeriesID, &title, &seriesURL, &cover,
&lastChapter, &lastChapterNum, &lastChapterURL,
&favorite, &latestChapter, &latestChapterNum, &b.UpdatedAt, &status, &b.Kind,
&b.Key, &b.Site, &b.SeriesID, &b.Title, &b.SeriesURL, &b.Cover,
&b.LastChapter, &b.LastChapterNum, &b.LastChapterURL,
&b.Favorite, &b.LatestChapter, &latestChapterNum, &b.UpdatedAt, &b.Status, &b.Kind,
); err != nil {
return Bookmark{}, err
}
b.Title = title.String
b.SeriesURL = seriesURL.String
b.Cover = cover.String
b.LastChapter = lastChapter.String
b.LastChapterNum = lastChapterNum.Float64
b.LastChapterURL = lastChapterURL.String
b.Favorite = favorite.Int64 != 0
b.LatestChapter = latestChapter.String
if latestChapterNum.Valid {
b.LatestChapterNum = &latestChapterNum.Float64
}
// A NULL, empty, or unrecognised bucket (e.g. a hand-edited row) would
// leave the row in no list at all, so anything outside the three known
// buckets reads as the default rather than being passed through.
b.Status = status.String
// An unrecognised bucket (a hand-edited row) would leave the row in no list
// at all, so anything outside the three known buckets reads as the default
// rather than being passed through.
if b.Status != StatusReading && b.Status != StatusArchived && b.Status != StatusFinished {
b.Status = StatusReading
}
@@ -365,7 +266,7 @@ func (s *Store) List() ([]Bookmark, error) {
// the fields they do not touch.
func (s *Store) Get(key string) (Bookmark, bool, error) {
b, err := scanBookmark(s.db.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks WHERE key = ?`, key).Scan)
`SELECT `+bookmarkColumns+` FROM bookmarks WHERE key = $1`, key).Scan)
if errors.Is(err, sql.ErrNoRows) {
return Bookmark{}, false, nil
}
@@ -395,9 +296,10 @@ func (s *Store) Upsert(b Bookmark) (Bookmark, error) {
latestNum = *b.LatestChapterNum
}
// IS NOT is SQLite's null-safe comparison. Within DO UPDATE, a bare column
// is the stored row and excluded.* is the incoming one; a brand-new key
// never reaches this clause, so it keeps the fresh timestamp from VALUES.
// IS DISTINCT FROM is Postgres's null-safe comparison, and it is what
// implements the ordering rule. Within DO UPDATE, a bare column is the
// stored row and excluded.* is the incoming one; a brand-new key never
// reaches this clause, so it keeps the fresh timestamp from VALUES.
//
// The status and kind columns resolve on the VALUES side, not in the
// conflict clause: excluded.* is the row *after* these expressions are
@@ -408,12 +310,16 @@ func (s *Store) Upsert(b Bookmark) (Bookmark, error) {
// stored", and only a brand-new row falls through to the literal
// default. The subquery runs inside this transaction, so it sees the
// row this statement is about to conflict with.
//
// The ::text casts are load-bearing: inside COALESCE/NULLIF there is no
// target column to infer the parameter type from, and Postgres rejects the
// statement rather than guessing.
if _, err := tx.Exec(`
INSERT INTO bookmarks (`+bookmarkColumns+`)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
COALESCE(NULLIF(?, ''), (SELECT status FROM bookmarks WHERE key = ?), 'reading'),
COALESCE(NULLIF(?, ''), (SELECT kind FROM bookmarks WHERE key = ?), 'manga'))
ON CONFLICT(key) DO UPDATE SET
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13,
COALESCE(NULLIF($14::text, ''), (SELECT status FROM bookmarks WHERE key = $1), 'reading'),
COALESCE(NULLIF($15::text, ''), (SELECT kind FROM bookmarks WHERE key = $1), 'manga'))
ON CONFLICT (key) DO UPDATE SET
site=excluded.site, series_id=excluded.series_id, title=excluded.title,
series_url=excluded.series_url, cover=excluded.cover,
last_chapter=excluded.last_chapter, last_chapter_num=excluded.last_chapter_num,
@@ -424,20 +330,19 @@ func (s *Store) Upsert(b Bookmark) (Bookmark, error) {
status=excluded.status,
kind=excluded.kind,
updated_at=CASE
WHEN bookmarks.last_chapter_num IS NOT excluded.last_chapter_num
WHEN bookmarks.last_chapter_num IS DISTINCT FROM excluded.last_chapter_num
THEN excluded.updated_at
ELSE bookmarks.updated_at
END`,
b.Key, b.Site, b.SeriesID, b.Title, b.SeriesURL, b.Cover,
b.LastChapter, b.LastChapterNum, b.LastChapterURL,
b.Favorite, b.LatestChapter, latestNum, b.UpdatedAt,
b.Status, b.Key,
b.Kind, b.Key); err != nil {
b.Status, b.Kind); err != nil {
return Bookmark{}, fmt.Errorf("upsert %q: %w", b.Key, err)
}
stored, err := scanBookmark(tx.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks WHERE key = ?`, b.Key).Scan)
`SELECT `+bookmarkColumns+` FROM bookmarks WHERE key = $1`, b.Key).Scan)
if err != nil {
return Bookmark{}, fmt.Errorf("read back %q: %w", b.Key, err)
}
@@ -449,7 +354,7 @@ func (s *Store) Upsert(b Bookmark) (Bookmark, error) {
// Delete removes a bookmark by key. Deleting a missing key is not an error.
func (s *Store) Delete(key string) error {
if _, err := s.db.Exec(`DELETE FROM bookmarks WHERE key = ?`, key); err != nil {
if _, err := s.db.Exec(`DELETE FROM bookmarks WHERE key = $1`, key); err != nil {
return fmt.Errorf("delete %q: %w", key, err)
}
return nil
@@ -472,11 +377,11 @@ func (s *Store) Delete(key string) error {
func (s *Store) DueForLatestCheck(cutoffMs int64, limit int) ([]Bookmark, error) {
rows, err := s.db.Query(`SELECT `+bookmarkColumns+`
FROM bookmarks
WHERE series_url IS NOT NULL AND series_url <> ''
AND status IS NOT 'finished'
AND latest_checked_at <= ?
WHERE series_url <> ''
AND status IS DISTINCT FROM 'finished'
AND latest_checked_at <= $1
ORDER BY latest_checked_at ASC
LIMIT ?`, cutoffMs, limit)
LIMIT $2`, cutoffMs, limit)
if err != nil {
return nil, fmt.Errorf("query due bookmarks: %w", err)
}
@@ -505,7 +410,7 @@ func (s *Store) DueForLatestCheck(cutoffMs int64, limit int) ([]Bookmark, error)
// long as the user kept reading it.
func (s *Store) MarkLatestChecked(key string, ts int64) error {
if _, err := s.db.Exec(
`UPDATE bookmarks SET latest_checked_at = ? WHERE key = ?`, ts, key); err != nil {
`UPDATE bookmarks SET latest_checked_at = $1 WHERE key = $2`, ts, key); err != nil {
return fmt.Errorf("mark checked %q: %w", key, err)
}
return nil
@@ -518,7 +423,7 @@ func (s *Store) MarkLatestChecked(key string, ts int64) error {
func (s *Store) LatestCheckedAt(key string) (int64, error) {
var ts int64
if err := s.db.QueryRow(
`SELECT latest_checked_at FROM bookmarks WHERE key = ?`, key).Scan(&ts); err != nil {
`SELECT latest_checked_at FROM bookmarks WHERE key = $1`, key).Scan(&ts); err != nil {
return 0, fmt.Errorf("latest checked at %q: %w", key, err)
}
return ts, nil
+26 -311
View File
@@ -1,80 +1,54 @@
package store
import (
"database/sql"
"path/filepath"
"os"
"testing"
"time"
"bookmarkmanager/backend/internal/pgtest"
)
func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
func newTestStore(t *testing.T) *Store {
t.Helper()
store, err := Open(filepath.Join(t.TempDir(), "test.db"))
store, err := Open(pgtest.URL(t))
if err != nil {
t.Fatalf("OpenStore: %v", err)
t.Fatalf("Open: %v", err)
}
t.Cleanup(func() { store.Close() })
return store
}
func TestOpenStoreMigratesLegacySchema(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "legacy.db")
legacy, err := sql.Open("sqlite", dbPath)
// The migration runner runs on every start, so a second Open against a
// database it already built must be a no-op rather than a duplicate-table
// error, and must leave the rows alone.
func TestOpenIsIdempotent(t *testing.T) {
url := pgtest.URL(t)
first, err := Open(url)
if err != nil {
t.Fatalf("open legacy db: %v", err)
t.Fatalf("Open: %v", err)
}
if _, err := legacy.Exec(`
CREATE TABLE bookmarks (
key TEXT PRIMARY KEY,
site TEXT NOT NULL,
series_id TEXT NOT NULL,
title TEXT,
series_url TEXT,
cover TEXT,
last_chapter TEXT,
last_chapter_num REAL,
last_chapter_url TEXT,
updated_at INTEGER NOT NULL
)`); err != nil {
t.Fatalf("create legacy schema: %v", err)
}
if _, err := legacy.Exec(`
INSERT INTO bookmarks (key, site, series_id, title, last_chapter, last_chapter_num, updated_at)
VALUES ('asura:legacy', 'asura', 'legacy', 'Legacy Series', 'Chapter 7', 7, 123)`); err != nil {
t.Fatalf("seed legacy row: %v", err)
}
if err := legacy.Close(); err != nil {
t.Fatalf("close legacy db: %v", err)
if _, err := first.Upsert(Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
first.Close()
store, err := Open(dbPath)
second, err := Open(url)
if err != nil {
t.Fatalf("OpenStore on legacy db: %v", err)
t.Fatalf("reopen: %v", err)
}
t.Cleanup(func() { store.Close() })
t.Cleanup(func() { second.Close() })
list, err := store.List()
list, err := second.List()
if err != nil {
t.Fatalf("List: %v", err)
}
if len(list) != 1 || list[0].Key != "asura:legacy" {
t.Fatalf("legacy row lost: %+v", list)
if len(list) != 1 || list[0].Key != "asura:solo" {
t.Fatalf("rows after reopen = %+v, want only the seeded one", list)
}
got := list[0]
if got.Title != "Legacy Series" || got.LastChapterNum != 7 || got.UpdatedAt != 123 {
t.Fatalf("legacy data mangled: %+v", got)
}
if got.Favorite || got.LatestChapter != "" || got.LatestChapterNum != nil {
t.Fatalf("new columns should default empty, got %+v", got)
}
// Reopening an already-migrated database must be a no-op, not an error.
store2, err := Open(dbPath)
if err != nil {
t.Fatalf("OpenStore is not idempotent: %v", err)
}
store2.Close()
}
func TestStoreGet(t *testing.T) {
@@ -155,7 +129,7 @@ func readLatestCheckedAt(t *testing.T, s *Store, key string) int64 {
t.Helper()
var ts int64
if err := s.db.QueryRow(
`SELECT latest_checked_at FROM bookmarks WHERE key = ?`, key).Scan(&ts); err != nil {
`SELECT latest_checked_at FROM bookmarks WHERE key = $1`, key).Scan(&ts); err != nil {
t.Fatalf("read latest_checked_at %q: %v", key, err)
}
return ts
@@ -264,53 +238,6 @@ func TestUpsertPreservesLatestCheckedAt(t *testing.T) {
}
}
// migrateColumns must be able to bring a database created before this column up
// to date, not just create it fresh.
func TestMigrateAddsLatestCheckedAt(t *testing.T) {
path := filepath.Join(t.TempDir(), "old.db")
old, err := sql.Open("sqlite", path)
if err != nil {
t.Fatalf("open: %v", err)
}
// A pre-latest_checked_at table, matching the schema as it shipped before.
if _, err := old.Exec(`CREATE TABLE bookmarks (
key TEXT PRIMARY KEY, site TEXT NOT NULL, series_id TEXT NOT NULL,
title TEXT, series_url TEXT, cover TEXT,
last_chapter TEXT, last_chapter_num REAL, last_chapter_url TEXT,
favorite INTEGER NOT NULL DEFAULT 0,
latest_chapter TEXT NOT NULL DEFAULT '', latest_chapter_num REAL,
updated_at INTEGER NOT NULL)`); err != nil {
t.Fatalf("create old table: %v", err)
}
if _, err := old.Exec(
`INSERT INTO bookmarks (key, site, series_id, series_url, updated_at)
VALUES ('asura:x', 'asura', 'x', 'https://asurascans.com/comics/x', 5)`); err != nil {
t.Fatalf("seed old row: %v", err)
}
if err := old.Close(); err != nil {
t.Fatalf("close: %v", err)
}
s, err := Open(path)
if err != nil {
t.Fatalf("OpenStore on pre-existing db: %v", err)
}
t.Cleanup(func() { s.Close() })
// The migrated row must default to 0 (never checked) and so be due.
if got := readLatestCheckedAt(t, s, "asura:x"); got != 0 {
t.Fatalf("migrated latest_checked_at = %d, want 0", got)
}
due, err := s.DueForLatestCheck(1000, 10)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
if len(due) != 1 {
t.Fatalf("got %d due rows after migration, want 1", len(due))
}
}
func TestUpsertDefaultsStatusToReading(t *testing.T) {
store := newTestStore(t)
stored, err := store.Upsert(Bookmark{
@@ -425,41 +352,6 @@ func TestUpsertStatusChangeKeepsUpdatedAt(t *testing.T) {
}
}
// A database written before the column existed must gain it, with every
// pre-existing row landing in the reading bucket.
func TestMigrationAddsStatusToLegacyDatabase(t *testing.T) {
path := filepath.Join(t.TempDir(), "legacy.db")
db, err := sql.Open("sqlite", path)
if err != nil {
t.Fatalf("open: %v", err)
}
if _, err := db.Exec(`
CREATE TABLE bookmarks (
key TEXT PRIMARY KEY, site TEXT NOT NULL, series_id TEXT NOT NULL,
title TEXT, series_url TEXT, cover TEXT,
last_chapter TEXT, last_chapter_num REAL, last_chapter_url TEXT,
updated_at INTEGER NOT NULL);
INSERT INTO bookmarks (key, site, series_id, updated_at)
VALUES ('asura:old', 'asura', 'old', 1)`); err != nil {
t.Fatalf("seed legacy: %v", err)
}
db.Close()
store, err := Open(path)
if err != nil {
t.Fatalf("OpenStore: %v", err)
}
defer store.Close()
b, ok, err := store.Get("asura:old")
if err != nil || !ok {
t.Fatalf("Get: ok=%v err=%v", ok, err)
}
if b.Status != StatusReading {
t.Fatalf("Status = %q, want %q", b.Status, StatusReading)
}
}
// Archiving is the reason to keep polling — the point is to come back to a
// series that has moved on. A finished series has nothing left to publish.
func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
@@ -494,135 +386,6 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
}
}
// Asura slugs used to include the site build hash; rows keyed with it must
// be rewritten to the stable ID on open, merging hash-generations of the
// same series into the newest row.
func TestOpenStoreMigratesAsuraBuildHashKeys(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "hash.db")
store, err := Open(dbPath)
if err != nil {
t.Fatalf("open: %v", err)
}
seed := []Bookmark{
{Key: "asura:swordmasters-youngest-son-f886a8af", Site: "asura",
SeriesID: "swordmasters-youngest-son-f886a8af", Title: "Old gen",
LastChapterNum: 50, UpdatedAt: 100},
{Key: "asura:swordmasters-youngest-son-059befe1", Site: "asura",
SeriesID: "swordmasters-youngest-son-059befe1", Title: "Re-bookmarked",
LastChapterNum: 60, UpdatedAt: 200},
{Key: "asura:overgeared-059befe1", Site: "asura",
SeriesID: "overgeared-059befe1", Title: "Single gen", UpdatedAt: 150},
// Hash-like suffix on another site must be left alone.
{Key: "demonic:x-deadbeef", Site: "demonic",
SeriesID: "x-deadbeef", Title: "Not asura", UpdatedAt: 300},
}
for _, b := range seed {
if _, err := store.Upsert(b); err != nil {
t.Fatalf("seed %s: %v", b.Key, err)
}
}
if err := store.Close(); err != nil {
t.Fatalf("close: %v", err)
}
reopened, err := Open(dbPath)
if err != nil {
t.Fatalf("reopen: %v", err)
}
t.Cleanup(func() { reopened.Close() })
list, err := reopened.List()
if err != nil {
t.Fatalf("List: %v", err)
}
byKey := map[string]Bookmark{}
for _, b := range list {
byKey[b.Key] = b
}
if len(list) != 3 {
t.Fatalf("want 3 rows after merge, got %d: %+v", len(list), list)
}
merged, ok := byKey["asura:swordmasters-youngest-son"]
if !ok {
t.Fatalf("merged key missing: %+v", byKey)
}
// Newest row wins the merge.
if merged.Title != "Re-bookmarked" || merged.LastChapterNum != 60 || merged.UpdatedAt != 200 {
t.Fatalf("merge kept wrong row: %+v", merged)
}
if merged.SeriesID != "swordmasters-youngest-son" {
t.Fatalf("series_id not stripped: %q", merged.SeriesID)
}
if _, ok := byKey["asura:overgeared-059befe1"]; ok {
t.Fatal("single-generation hashed key not rewritten")
}
if _, ok := byKey["asura:overgeared"]; !ok {
t.Fatal("single-generation row missing under stripped key")
}
if _, ok := byKey["demonic:x-deadbeef"]; !ok {
t.Fatal("non-asura row touched")
}
// Idempotent: a third open changes nothing.
third, err := Open(dbPath)
if err != nil {
t.Fatalf("third open: %v", err)
}
third.Close()
}
// A hashed row and a pre-existing hashless row of the same series collide on
// the stripped key. The winner rewrite must happen only after the loser is
// gone, or the UPDATE hits a primary-key collision and OpenStore fails.
func TestOpenStoreMigratesAsuraHashlessCollision(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "collision.db")
store, err := Open(dbPath)
if err != nil {
t.Fatalf("open: %v", err)
}
seed := []Bookmark{
{Key: "asura:overgeared", Site: "asura", SeriesID: "overgeared",
Title: "Hashless", LastChapterNum: 10, UpdatedAt: 100},
{Key: "asura:overgeared-059befe1", Site: "asura",
SeriesID: "overgeared-059befe1", Title: "Hashed newer",
LastChapterNum: 20, UpdatedAt: 200},
}
for _, b := range seed {
if _, err := store.Upsert(b); err != nil {
t.Fatalf("seed %s: %v", b.Key, err)
}
}
if err := store.Close(); err != nil {
t.Fatalf("close: %v", err)
}
reopened, err := Open(dbPath)
if err != nil {
t.Fatalf("reopen: %v", err)
}
t.Cleanup(func() { reopened.Close() })
list, err := reopened.List()
if err != nil {
t.Fatalf("List: %v", err)
}
if len(list) != 1 {
t.Fatalf("want 1 row after merge, got %d: %+v", len(list), list)
}
merged := list[0]
if merged.Key != "asura:overgeared" {
t.Fatalf("merged key = %q, want asura:overgeared", merged.Key)
}
if merged.SeriesID != "overgeared" {
t.Fatalf("series_id = %q, want overgeared", merged.SeriesID)
}
if merged.Title != "Hashed newer" || merged.LastChapterNum != 20 || merged.UpdatedAt != 200 {
t.Fatalf("merge kept wrong row: %+v", merged)
}
}
func TestDisplayChapter(t *testing.T) {
cases := []struct {
name string
@@ -713,51 +476,3 @@ func TestUpsertEmptyKindKeepsStoredValue(t *testing.T) {
t.Fatalf("LastChapterNum = %v, want 11 — progress in the same request must still land", got.LastChapterNum)
}
}
// A database created before this column exists must gain it, backfilled as
// manga, without losing anything.
func TestLegacyDatabaseGainsKindAsManga(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "legacy.db")
legacy, err := sql.Open("sqlite", dbPath)
if err != nil {
t.Fatalf("open legacy db: %v", err)
}
if _, err := legacy.Exec(`
CREATE TABLE bookmarks (
key TEXT PRIMARY KEY,
site TEXT NOT NULL,
series_id TEXT NOT NULL,
title TEXT,
series_url TEXT,
cover TEXT,
last_chapter TEXT,
last_chapter_num REAL,
last_chapter_url TEXT,
updated_at INTEGER NOT NULL
)`); err != nil {
t.Fatalf("create legacy schema: %v", err)
}
if _, err := legacy.Exec(`
INSERT INTO bookmarks (key, site, series_id, title, updated_at)
VALUES ('asura:legacy', 'asura', 'legacy', 'Legacy Series', 123)`); err != nil {
t.Fatalf("seed legacy row: %v", err)
}
if err := legacy.Close(); err != nil {
t.Fatalf("close legacy db: %v", err)
}
store, err := Open(dbPath)
if err != nil {
t.Fatalf("Open on legacy db: %v", err)
}
t.Cleanup(func() { store.Close() })
list, err := store.List()
if err != nil {
t.Fatalf("List: %v", err)
}
if len(list) != 1 || list[0].Kind != KindManga {
t.Fatalf("legacy row should backfill as manga, got %+v", list)
}
}
+11 -5
View File
@@ -24,8 +24,10 @@ import (
type Config struct {
Token string
AllowedOrigins []string
DBPath string
Port string
// DatabaseURL is the Postgres connection URL; required, no default,
// because a wrong guess would silently start on an empty database.
DatabaseURL string
Port string
// WebPassword gates the browser UI. Empty disables the web routes entirely.
WebPassword string
// UserscriptPath is the file served at /u/{token}/manga-bookmark.user.js.
@@ -143,7 +145,7 @@ func loadLatestPoll() LatestPoll {
func loadConfig() Config {
c := Config{
Token: os.Getenv("API_TOKEN"),
DBPath: envOr("DB_PATH", "/data/bookmarks.db"),
DatabaseURL: os.Getenv("DATABASE_URL"),
Port: envOr("PORT", "8080"),
WebPassword: os.Getenv("WEB_PASSWORD"),
UserscriptPath: envOr("USERSCRIPT_PATH", "/userscript/manga-bookmark.user.js"),
@@ -215,8 +217,11 @@ func main() {
if cfg.Token == "" {
log.Fatal("API_TOKEN is required")
}
if cfg.DatabaseURL == "" {
log.Fatal("DATABASE_URL is required")
}
s, err := store.Open(cfg.DBPath)
s, err := store.Open(cfg.DatabaseURL)
if err != nil {
log.Fatalf("open store: %v", err)
}
@@ -236,7 +241,8 @@ func main() {
}
go func() {
log.Printf("listening on :%s (db=%s, origins=%v)", cfg.Port, cfg.DBPath, cfg.AllowedOrigins)
// The connection URL carries a password, so it stays out of the log.
log.Printf("listening on :%s (origins=%v)", cfg.Port, cfg.AllowedOrigins)
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Fatalf("serve: %v", err)
}
+1 -6
View File
@@ -5,7 +5,6 @@ import (
"net/http"
"net/http/httptest"
"net/url"
"path/filepath"
"strconv"
"strings"
"testing"
@@ -28,11 +27,7 @@ func webConfig() Config {
// can seed rows and assert on what the handlers wrote back.
func newWebTestServer(t *testing.T, cfg Config) (http.Handler, *store.Store) {
t.Helper()
st, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatalf("store.Open: %v", err)
}
t.Cleanup(func() { st.Close() })
st := newTestStore(t)
return newRouter(st, cfg), st
}