A newly bookmarked Series acquires its Cover at creation (#59) (#68)

Closes #59.

Part of spec #55, and the ticket that fixes the reported bug #47. Architecture: `docs/adr/0007-backend-hosts-cover-bytes.md`. Does not close #47 or #55.

## What changed

A Reader bookmarks a Series nobody holds yet — the exact case in #47 — and within seconds the list shows its artwork instead of a broken image. The first Bookmark to create a Series fires `Store.OnSeriesCreated` after commit, and the new `latest.Acquirer` turns that into **one** series-page fetch that yields both the Latest Chapter and the cover URL. The bytes go through the gated cover fetcher from #57 and are stored content-addressed through #56, so the wire carries an absolute URL on this deployment's own origin — never a third-party address, and never one that 404s.

### Store

- Migration `0009_series_cover_address.sql` adds `series.cover_address`. The two facts are now split: `series.cover` is the third-party source address the bytes came from (the acquisition path's dedupe key), `series.cover_address` is the SHA-256 they are stored under. An empty `cover_address` is precisely what "no Cover yet" means, which is the distinction both the API and the UI depend on.
- `SetSeriesCover` writes the address only after the bytes are on disk, so the wire can never name an object that is not there.
- `CoverWireURL` builds `PUBLIC_BASE_URL + /covers/<sha256>` for every scanned row, and returns `""` for a blank address.
- The cover columns are gone from `Upsert`'s `INSERT` and its `DO UPDATE`. A client-supplied cover cannot reach the shared Series row on any path, not just the creation path.
- `Open` now rejects a base URL that is not an absolute `http(s)` origin: `PUBLIC_BASE_URL=bookmarks.example.com` would otherwise start cleanly and emit addresses no browser can load.

### Acquisition

- `internal/latest/acquire.go`: one fetch, gated by the poller's own `fetchableSeriesURL` (a `series_url` arrives in a client-supplied PUT body, so without the gate a token-holder chooses what the server fetches from its own network position).
- Asynchronous and log-and-drop. The Bookmark, its progress and its Latest Chapter are already committed; a Site that is down or a cover that cannot be produced disturbs none of them.
- Bounded by a two-slot semaphore. A bulk sync creating N Series would otherwise fire N simultaneous requests from one IP — the traffic shape the poller's stagger exists to avoid.
- Cancelled at shutdown (shares the poller's context) and stamps `latest_checked_at`, so the poller does not refetch the same page a tick later.
- Browser-backed Sites (kagane, novelfull) are deliberately skipped: their pages only yield a Cloudflare challenge to the TLS client, so the request would be spent for nothing. They arrive in #62.

### Wire and route

- `GET /covers/{address}` serves the bytes publicly and uncredentialed with `Cache-Control: public, max-age=604800, immutable`. The address is gated by a `^[0-9a-f]{64}$` pattern and cross-checked against a pure function of itself before any filesystem read, so no request shaped like a traversal reaches disk.
- `PUT /bookmarks/{key}` still accepts a `cover` field and discards it, permanently. Rejecting it would break every installed userscript the moment this deploys, and ADR-0004's compatibility argument depends on those scripts continuing to work. The decode site says so in place of a TODO nobody intends to keep.
- `store.CoverContentType` canonicalises comix's non-standard `image/jpg` to `image/jpeg`, so one image cannot land under two spellings. This one was found by the live smoke test, not by reading.

### Config

`PUBLIC_BASE_URL` is new and required (cover URLs must go out absolute — the userscript renders them on third-party origins, where a relative path resolves against the Site). Documented in `.env.example`, `docker-compose.yml` (`:?` so compose fails too), `DEPLOY.md` and `backend/AGENTS.md`.

## Acceptance criteria

All twelve of #59's criteria are met; the checklist on the issue is ticked with the evidence.

## Verification

- `go test ./...` green (Docker-backed Postgres suite).
- Live smoke against a real backend + Postgres: bookmarking `comix:n8we-dungeons-and-crayons` produced `"cover": "http://127.0.0.1:8099/covers/8ce74d80…"` and `"latest_chapter": "Chapter 81"` within seconds of the PUT; `curl` on that address returned `200`, `Content-Type: image/jpeg`, `Cache-Control: public, max-age=604800, immutable`, and a 280x420 JPEG. That run is what surfaced the `image/jpg` content type.
- Mutation-checked the asynchrony test: removing the `go` from `Acquire` turns `TestAcquireDoesNotBlockTheWrite` red.

## Reviewed

Both axes of `/code-review` were run against this diff before commit. Their findings that were actionable here are folded in: the concurrency bound, the shutdown tie, the `PUBLIC_BASE_URL` validation, the missing `latest_checked_at` stamp, and a test that could not fail.

## Known sequencing

A kagane/novelfull Series created between this deploy and #62 has no cover source at all: the acquisition skips those Sites and `Upsert` no longer persists the userscript-scraped address. This is #59's stated boundary rather than a defect, but it is a user-visible gap on two Sites and should order #62 accordingly.

Reviewed-on: #68
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #68.
This commit is contained in:
2026-08-10 04:07:53 +07:00
committed by sulthan
parent b6b88bde8a
commit 92eba07da7
18 changed files with 971 additions and 115 deletions
@@ -0,0 +1,8 @@
-- The Cover splits into two facts. `cover` keeps the third-party address the
-- bytes come from, which is what the acquisition path refetches and dedupes
-- on; `cover_address` is the content address of the bytes once they are
-- actually stored, and is what the wire's absolute URL is built from.
--
-- Empty `cover_address` therefore means "no Cover yet" rather than "a Cover
-- that 404s", which is the distinction the API and the UI both depend on.
ALTER TABLE series ADD COLUMN cover_address text NOT NULL DEFAULT '';
+152 -40
View File
@@ -30,12 +30,20 @@ import (
// between readers: progress, favourite, lifecycle bucket, updated_at. The wire
// format stays flat regardless — see ADR-0004.
type Bookmark struct {
Key string `json:"key"`
Site string `json:"site"`
SeriesID string `json:"series_id"`
Title string `json:"title"`
SeriesURL string `json:"series_url"`
Cover string `json:"cover"`
Key string `json:"key"`
Site string `json:"site"`
SeriesID string `json:"series_id"`
Title string `json:"title"`
SeriesURL string `json:"series_url"`
// Cover is the wire value: an absolute URL on this deployment's own
// origin once the bytes exist, and "" until they do — never a third-party
// address and never an address that 404s (ADR-0007). A client may still
// send this field and it is discarded on the way in; see Upsert.
Cover string `json:"cover"`
// CoverSource is the third-party address the bytes were fetched from. It
// stays off the wire: it is the acquisition path's dedupe key, and no
// client is ever asked to render one.
CoverSource string `json:"-"`
LastChapter string `json:"last_chapter"`
LastChapterNum float64 `json:"last_chapter_num"`
LastChapterURL string `json:"last_chapter_url"`
@@ -62,11 +70,16 @@ type Bookmark struct {
// bookmark's own fields. Never serialized: the wire format is the flat
// Bookmark (ADR-0004).
type Series struct {
Site string
SeriesID string
Title string
SeriesURL string
Site string
SeriesID string
Title string
SeriesURL string
// Cover is the third-party source address the bytes come from, and
// CoverAddress the content address they are stored under. A blank
// CoverAddress is what "no Cover yet" means: the poll fills it and never
// replaces a filled one (ADR-0007).
Cover string
CoverAddress string
Kind string
LatestChapter string
LatestChapterNum *float64 // nil until first captured
@@ -156,23 +169,27 @@ func KaganeImageID(cover string) (string, bool) {
return m[1], true
}
// IsCoverContentType reports whether a fetched response is safe to store and serve.
func IsCoverContentType(contentType string) bool {
// CoverContentType canonicalises a fetched response's media type and reports
// whether the bytes are safe to store and serve. comix answers "image/jpg",
// which no standard lists but browsers accept; it is stored as the real name
// rather than passed through, so one image never lands under two spellings.
func CoverContentType(contentType string) (string, bool) {
switch contentType {
case "image/jpg":
return "image/jpeg", true
case "image/webp", "image/jpeg", "image/png", "image/avif", "image/gif":
return true
return contentType, true
default:
return false
return "", false
}
}
// CoverURL is the src the web UI puts in an <img>. For every site but kagane
// that is Cover as stored. kagane serves its images behind a Cloudflare
// challenge *and* with `cross-origin-resource-policy: same-origin`, so no page
// on another origin can load one however it asks (verified 2026-08-08); those
// go through the backend's own proxy instead.
// CoverURL is the src the web UI puts in an <img>. Cover already is an address
// on this origin, so for every site but kagane it is used as-is. kagane's
// bytes still arrive through the browser-backed proxy, which is keyed by image
// id rather than by content address until #62 moves it onto the same path.
func (b Bookmark) CoverURL() string {
if imageID, ok := KaganeImageID(b.Cover); ok {
if imageID, ok := KaganeImageID(b.CoverSource); ok {
return "/img/kagane/" + imageID
}
return b.Cover
@@ -200,14 +217,14 @@ var migrations embed.FS
// compile-time constant; every request value is bound as a parameter. The
// series-owned fields are joined in from the series table, in scanBookmark
// order, so the flat Bookmark reads back whole despite the split (ADR-0004).
const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover,
const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover, s.cover_address,
b.last_chapter, b.last_chapter_num, b.last_chapter_url,
b.favorite, s.latest_chapter, s.latest_chapter_num, b.updated_at, b.status, s.kind`
// seriesColumns is the series row in scanSeries order, used by the poller's
// due query. latest_checked_at lives only on series — see MarkLatestChecked
// for why it stays off every client-visible write.
const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover,
const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover, s.cover_address,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at`
// Owner is the person running the service: the first Reader, seeded at startup
@@ -230,6 +247,16 @@ type Store struct {
// method takes a reader id explicitly, so ownership is never implicit.
ownerID int64
coverDir string
// coverBaseURL is this deployment's public origin. Cover addresses are
// absolute because the userscript renders them on third-party origins,
// where a relative path would resolve against the Site (ADR-0007).
coverBaseURL string
// OnSeriesCreated fires once, after commit, for a Series no Reader had
// bookmarked before. It is how creation-time Cover and Latest Chapter
// acquisition is triggered without the write waiting on a third-party
// Site; nil disables it, which is what every test that does not care
// about acquisition leaves it as.
OnSeriesCreated func(Series)
}
// OwnerID returns the seeded owner Reader's id: the administrator, and the
@@ -365,10 +392,19 @@ const allMigrations = 0
// Open connects to Postgres at url — a libpq connection URL such as
// "postgres://user:pass@host:5432/bookmarks?sslmode=disable" — brings its
// schema up to date, seeds the owner Reader, and prepares cover storage.
func Open(url string, owner Owner, coverDir string) (*Store, error) {
func Open(url string, owner Owner, coverDir, coverBaseURL string) (*Store, error) {
if strings.TrimSpace(coverDir) == "" {
return nil, errors.New("cover directory is required")
}
// Every wire Cover is this string with a path glued on, rendered by a
// userscript on a Site's own origin: anything but an absolute origin
// produces addresses no client can load, silently (ADR-0007).
base := strings.TrimRight(coverBaseURL, "/")
if host, ok := strings.CutPrefix(base, "https://"); !ok || host == "" {
if host, ok := strings.CutPrefix(base, "http://"); !ok || host == "" {
return nil, fmt.Errorf("cover base URL %q is not an absolute http(s) origin", coverBaseURL)
}
}
if err := os.MkdirAll(coverDir, 0o755); err != nil {
return nil, fmt.Errorf("create cover directory: %w", err)
}
@@ -414,7 +450,9 @@ func Open(url string, owner Owner, coverDir string) (*Store, error) {
db.Close()
return nil, fmt.Errorf("resolve owner: %w", err)
}
return &Store{db: db, ownerID: ownerID, coverDir: coverDir}, nil
return &Store{
db: db, ownerID: ownerID, coverDir: coverDir, coverBaseURL: base,
}, nil
}
// seedOwner makes sure the configured owner exists as exactly one readers row.
@@ -517,18 +555,20 @@ func applyMigration(db *sql.DB, version int64, body string) error {
// scanBookmark reads one row in bookmarkColumns order. Every column is NOT
// NULL except latest_chapter_num, where NULL means "never captured" — a
// distinct state from chapter zero, and the reason for the pointer.
func scanBookmark(scan func(...any) error) (Bookmark, error) {
func (s *Store) scanBookmark(scan func(...any) error) (Bookmark, error) {
var (
b Bookmark
coverAddress string
latestChapterNum sql.NullFloat64
)
if err := scan(
&b.Site, &b.SeriesID, &b.Title, &b.SeriesURL, &b.Cover,
&b.Site, &b.SeriesID, &b.Title, &b.SeriesURL, &b.CoverSource, &coverAddress,
&b.LastChapter, &b.LastChapterNum, &b.LastChapterURL,
&b.Favorite, &b.LatestChapter, &latestChapterNum, &b.UpdatedAt, &b.Status, &b.Kind,
); err != nil {
return Bookmark{}, err
}
b.Cover = s.CoverWireURL(coverAddress)
if latestChapterNum.Valid {
b.LatestChapterNum = &latestChapterNum.Float64
}
@@ -553,7 +593,7 @@ func scanSeries(scan func(...any) error) (Series, error) {
latestChapterNum sql.NullFloat64
)
if err := scan(
&sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover,
&sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover, &sr.CoverAddress,
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt,
&sr.readerCount,
); err != nil {
@@ -582,7 +622,10 @@ func kaganeCoverSourceURL(imageID string) string {
}
func (s *Store) getCover(sourceURL string) ([]byte, string, bool, error) {
address := coverSourceAddress(sourceURL)
return s.getCoverByAddress(coverSourceAddress(sourceURL))
}
func (s *Store) getCoverByAddress(address string) ([]byte, string, bool, error) {
var relativePath, contentType string
err := s.db.QueryRow(
`SELECT path, content_type FROM covers WHERE address = $1`, address,
@@ -608,9 +651,11 @@ func (s *Store) getCover(sourceURL string) ([]byte, string, bool, error) {
}
func (s *Store) putCover(sourceURL string, body []byte, contentType string) error {
if !IsCoverContentType(contentType) {
stored, ok := CoverContentType(contentType)
if !ok {
return fmt.Errorf("put cover %q: unsupported content type %q", sourceURL, contentType)
}
contentType = stored
address := coverSourceAddress(sourceURL)
relativePath := coverRelativePath(address)
coverPath := filepath.Join(s.coverDir, filepath.FromSlash(relativePath))
@@ -670,6 +715,55 @@ func (s *Store) PutKaganeCover(imageID string, body []byte, contentType string)
return s.putCover(kaganeCoverSourceURL(imageID), body, contentType)
}
// CoverAddress is the content address bytes fetched from sourceURL are stored
// under. It is a pure function of the URL, so the acquisition path can name a
// Cover before it has the bytes.
func CoverAddress(sourceURL string) string { return coverSourceAddress(sourceURL) }
// coverAddressRe is the shape of a stored address: the hex SHA-256 of a source
// URL. Request paths reach CoverByAddress, so the shape is checked before the
// value is ever turned into a filesystem path.
var coverAddressRe = regexp.MustCompile(`^[0-9a-f]{64}$`)
// CoverByAddress returns the immutable object at one content address. An
// address that is not a stored one - malformed, unknown, or recorded but with
// its file gone - is reported with ok=false rather than as an error.
func (s *Store) CoverByAddress(address string) ([]byte, string, bool, error) {
if !coverAddressRe.MatchString(address) {
return nil, "", false, nil
}
return s.getCoverByAddress(address)
}
// CoverWireURL is the absolute URL a client renders for a stored Cover, and ""
// for a Series that has none yet. A blank is a real state, not a placeholder
// address: it is what tells both clients to draw their own fallback instead of
// requesting bytes that do not exist (ADR-0007).
func (s *Store) CoverWireURL(address string) string {
if address == "" {
return ""
}
return s.coverBaseURL + "/covers/" + address
}
// SetSeriesCover stores the bytes and points the Series at them, but only
// while the Series has no Cover: acquisition at creation and the poll both
// call this, and whichever arrives second must not overwrite the first. The
// bytes themselves are content-addressed and immutable, so storing them twice
// is free.
func (s *Store) SetSeriesCover(site, seriesID, sourceURL string, body []byte, contentType string) error {
if err := s.putCover(sourceURL, body, contentType); err != nil {
return err
}
if _, err := s.db.Exec(`
UPDATE series SET cover = $3, cover_address = $4
WHERE site = $1 AND series_id = $2 AND cover_address = ''`,
site, seriesID, sourceURL, coverSourceAddress(sourceURL)); err != nil {
return fmt.Errorf("set cover for %q: %w", site+":"+seriesID, err)
}
return nil
}
// List returns every bookmark of one reader, newest activity first.
// Series-owned fields are joined in, so each Bookmark reads back whole and
// flat (ADR-0004).
@@ -686,7 +780,7 @@ func (s *Store) List(readerID int64) ([]Bookmark, error) {
out := []Bookmark{}
for rows.Next() {
b, err := scanBookmark(rows.Scan)
b, err := s.scanBookmark(rows.Scan)
if err != nil {
return nil, fmt.Errorf("scan bookmark: %w", err)
}
@@ -703,7 +797,7 @@ func (s *Store) Get(readerID int64, key string) (Bookmark, bool, error) {
if !ok {
return Bookmark{}, false, nil
}
b, err := scanBookmark(s.db.QueryRow(
b, err := s.scanBookmark(s.db.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks b
JOIN series s ON s.site = b.site AND s.series_id = b.series_id
WHERE b.reader_id = $1 AND b.site = $2 AND b.series_id = $3`,
@@ -760,18 +854,28 @@ func (s *Store) Upsert(readerID int64, b Bookmark) (Bookmark, error) {
// The ::text casts are load-bearing: inside COALESCE/NULLIF there is no
// target column to infer the parameter type from, and Postgres rejects the
// statement rather than guessing.
if _, err := tx.Exec(`
INSERT INTO series (site, series_id, title, series_url, cover, kind,
//
// The cover columns are absent on purpose: the Cover is acquired
// server-side (ADR-0007), so a client-supplied one is not written even
// when the row is brand new.
//
// xmax is zero only on a row this statement inserted, which is how a
// Series nobody had bookmarked before is told apart from one that already
// existed — DO UPDATE returns a row either way.
var created bool
if err := tx.QueryRow(`
INSERT INTO series (site, series_id, title, series_url, kind,
latest_chapter, latest_chapter_num)
VALUES ($1, $2, $3, $4, $5,
COALESCE(NULLIF($6::text, ''), (SELECT kind FROM series WHERE site = $1 AND series_id = $2), 'manga'),
$7, $8)
VALUES ($1, $2, $3, $4,
COALESCE(NULLIF($5::text, ''), (SELECT kind FROM series WHERE site = $1 AND series_id = $2), 'manga'),
$6, $7)
ON CONFLICT (site, series_id) DO UPDATE SET
kind=excluded.kind,
latest_chapter=excluded.latest_chapter,
latest_chapter_num=excluded.latest_chapter_num`,
b.Site, b.SeriesID, b.Title, b.SeriesURL, b.Cover, b.Kind,
b.LatestChapter, latestNum); err != nil {
latest_chapter_num=excluded.latest_chapter_num
RETURNING xmax = 0`,
b.Site, b.SeriesID, b.Title, b.SeriesURL, b.Kind,
b.LatestChapter, latestNum).Scan(&created); err != nil {
return Bookmark{}, fmt.Errorf("upsert series for %q: %w", b.Key, err)
}
@@ -801,7 +905,7 @@ func (s *Store) Upsert(readerID int64, b Bookmark) (Bookmark, error) {
return Bookmark{}, fmt.Errorf("upsert %q: %w", b.Key, err)
}
stored, err := scanBookmark(tx.QueryRow(
stored, err := s.scanBookmark(tx.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks b
JOIN series s ON s.site = b.site AND s.series_id = b.series_id
WHERE b.reader_id = $1 AND b.site = $2 AND b.series_id = $3`,
@@ -812,6 +916,14 @@ func (s *Store) Upsert(readerID int64, b Bookmark) (Bookmark, error) {
if err := tx.Commit(); err != nil {
return Bookmark{}, fmt.Errorf("commit %q: %w", b.Key, err)
}
// After commit, never inside the transaction: the hook reaches a
// third-party Site, and the Reader's write must not wait on it.
if created && s.OnSeriesCreated != nil {
s.OnSeriesCreated(Series{
Site: b.Site, SeriesID: b.SeriesID, Title: stored.Title,
SeriesURL: stored.SeriesURL, Kind: stored.Kind,
})
}
return stored, nil
}
+167 -37
View File
@@ -21,9 +21,12 @@ func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
// reader register one (see secondReader).
var testOwner = Owner{DiscordID: "test-owner", TokenHash: sha256.Sum256([]byte("owner-token-hash"))}
// testCoverBaseURL is the public origin every stored cover URL is built from.
const testCoverBaseURL = "https://bookmarks.test"
func newTestStore(t *testing.T) *Store {
t.Helper()
store, err := Open(pgtest.URL(t), testOwner, t.TempDir())
store, err := Open(pgtest.URL(t), testOwner, t.TempDir(), testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}
@@ -49,7 +52,7 @@ func secondReader(t *testing.T, s *Store) int64 {
func TestOpenIsIdempotent(t *testing.T) {
url := pgtest.URL(t)
coverDir := t.TempDir()
first, err := Open(url, testOwner, coverDir)
first, err := Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}
@@ -60,7 +63,7 @@ func TestOpenIsIdempotent(t *testing.T) {
}
first.Close()
second, err := Open(url, testOwner, coverDir)
second, err := Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("reopen: %v", err)
}
@@ -113,7 +116,7 @@ func TestReaderTokenInfo(t *testing.T) {
func TestRotateTokenInvalidatesOldAndSurvivesRestart(t *testing.T) {
url := pgtest.URL(t)
coverDir := t.TempDir()
store, err := Open(url, testOwner, coverDir)
store, err := Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}
@@ -145,7 +148,7 @@ func TestRotateTokenInvalidatesOldAndSurvivesRestart(t *testing.T) {
}
store.Close()
reopened, err := Open(url, testOwner, coverDir)
reopened, err := Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("reopen: %v", err)
}
@@ -547,32 +550,39 @@ func TestDisplayChapter(t *testing.T) {
}
}
// CoverURL reads the source address for the kagane branch and the wire value
// otherwise, so both are set the way scanBookmark sets them.
func TestCoverURL(t *testing.T) {
cases := []struct {
name string
cover string
want string
name string
coverSource string
cover string
want string
}{
{
"kagane routes through the proxy",
"https://kagane.to/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed",
"https://bookmarks.test/covers/" + CoverAddress("kagane"),
"/img/kagane/019fe11a-84c3-7fc3-a84b-88787374b617",
},
{
"another site is served as stored",
"https://gg.asuracomic.net/storage/media/1/conversions/cover.webp",
"another site is served from our own origin",
"https://gg.asuracomic.net/storage/media/1/conversions/cover.webp",
"https://bookmarks.test/covers/" + CoverAddress("asura"),
"https://bookmarks.test/covers/" + CoverAddress("asura"),
},
{
"a lookalike host is not rewritten",
"https://evil.example/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed",
"https://evil.example/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed",
"https://bookmarks.test/covers/" + CoverAddress("evil"),
"https://bookmarks.test/covers/" + CoverAddress("evil"),
},
{"no cover stays empty", "", ""},
{"no cover stays empty", "", "", ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := (Bookmark{Cover: tc.cover}).CoverURL(); got != tc.want {
b := Bookmark{CoverSource: tc.coverSource, Cover: tc.cover}
if got := b.CoverURL(); got != tc.want {
t.Errorf("CoverURL() = %q, want %q", got, tc.want)
}
})
@@ -672,7 +682,7 @@ func TestMigration0002BackfillsExistingBookmarks(t *testing.T) {
// Bring it current through the production path: Open runs the schema to
// 0003, seeds the owner, then applies 0004 which attaches this row. 0002
// must have backfilled the series row, not lost data.
st, err := Open(url, testOwner, t.TempDir())
st, err := Open(url, testOwner, t.TempDir(), testCoverBaseURL)
if err != nil {
t.Fatalf("Open after migrate: %v", err)
}
@@ -756,39 +766,143 @@ func readSeries(t *testing.T, s *Store, site, seriesID string) Series {
return sr
}
// The first PUT for a series creates its row from the client's title, cover
// and URL — there is no other source for them (ADR-0003).
// The first PUT for a series creates its row from the client's title and URL —
// there is no other source for them (ADR-0003). The Cover is not among them:
// it is acquired server-side, so a client-supplied one is dropped even on a
// brand-new row (ADR-0007).
func TestUpsertCreatesSeriesFromClient(t *testing.T) {
store := newTestStore(t)
if _, err := store.Upsert(store.OwnerID(), Bookmark{
stored, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", SeriesURL: "https://asurascans.com/comics/solo",
Cover: "https://asurascans.com/covers/solo.jpg", Kind: KindManga,
UpdatedAt: 1000,
}); err != nil {
})
if err != nil {
t.Fatalf("Upsert: %v", err)
}
if stored.Cover != "" {
t.Fatalf("Cover = %q, want empty — a client cover is never stored", stored.Cover)
}
sr := readSeries(t, store, "asura", "solo")
if sr.Title != "Solo Leveling" || sr.SeriesURL != "https://asurascans.com/comics/solo" ||
sr.Cover != "https://asurascans.com/covers/solo.jpg" {
t.Fatalf("series = %+v, want client title/url/cover stored", sr)
if sr.Title != "Solo Leveling" || sr.SeriesURL != "https://asurascans.com/comics/solo" {
t.Fatalf("series = %+v, want client title/url stored", sr)
}
if sr.Cover != "" {
t.Fatalf("series cover = %q, want empty", sr.Cover)
}
}
// A PUT naming an existing series must not overwrite its title, cover or URL:
// the row is shared, and those values are scraped page content (ADR-0003).
// The hook is what starts creation-time acquisition, so it must fire exactly
// once per Series — on the PUT that created it, and on no later one, whichever
// Reader sends it.
func TestOnSeriesCreatedFiresOnceForANewSeries(t *testing.T) {
store := newTestStore(t)
var created []Series
store.OnSeriesCreated = func(sr Series) { created = append(created, sr) }
b := Bookmark{
Key: "comix:solo", Site: "comix", SeriesID: "solo", Title: "Solo Leveling",
SeriesURL: "https://comix.to/series/solo", Kind: KindManga, UpdatedAt: 1000,
}
if _, err := store.Upsert(store.OwnerID(), b); err != nil {
t.Fatalf("Upsert: %v", err)
}
b.LastChapterNum = 12
b.UpdatedAt = 2000
if _, err := store.Upsert(store.OwnerID(), b); err != nil {
t.Fatalf("second Upsert: %v", err)
}
if _, err := store.Upsert(secondReader(t, store), b); err != nil {
t.Fatalf("second reader Upsert: %v", err)
}
if len(created) != 1 {
t.Fatalf("hook fired %d times, want 1: %+v", len(created), created)
}
if created[0].Site != "comix" || created[0].SeriesID != "solo" ||
created[0].SeriesURL != "https://comix.to/series/solo" {
t.Fatalf("hook got %+v, want the created series' identity and URL", created[0])
}
}
// Acquisition at creation and the poll both write covers, and whichever
// arrives second must leave the first one alone: a Cover is replaced by
// nothing short of the series row being rebuilt.
func TestSetSeriesCoverDoesNotOverwrite(t *testing.T) {
store := newTestStore(t)
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
first := "https://asurascans.com/covers/first.jpg"
if err := store.SetSeriesCover("asura", "solo", first, []byte("first"), "image/jpeg"); err != nil {
t.Fatalf("SetSeriesCover: %v", err)
}
if err := store.SetSeriesCover("asura", "solo", "https://asurascans.com/covers/second.jpg",
[]byte("second"), "image/jpeg"); err != nil {
t.Fatalf("second SetSeriesCover: %v", err)
}
got, ok, err := store.Get(store.OwnerID(), "asura:solo")
if err != nil || !ok {
t.Fatalf("Get = %v, %v", ok, err)
}
if want := "https://bookmarks.test/covers/" + CoverAddress(first); got.Cover != want {
t.Fatalf("Cover = %q, want the first one %q", got.Cover, want)
}
}
// The address comes straight off a public request path, so anything that is
// not a stored address must be a miss rather than a filesystem lookup.
func TestCoverByAddress(t *testing.T) {
store := newTestStore(t)
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
source := "https://asurascans.com/covers/solo.jpg"
if err := store.SetSeriesCover("asura", "solo", source, []byte("bytes"), "image/jpeg"); err != nil {
t.Fatalf("SetSeriesCover: %v", err)
}
body, contentType, ok, err := store.CoverByAddress(CoverAddress(source))
if err != nil || !ok {
t.Fatalf("CoverByAddress = %v, %v", ok, err)
}
if string(body) != "bytes" || contentType != "image/jpeg" {
t.Fatalf("CoverByAddress = %q, %q, want the stored bytes", body, contentType)
}
for _, address := range []string{"", "../../etc/passwd", "ZZ" + CoverAddress(source)[2:],
CoverAddress("never stored")} {
_, _, ok, err := store.CoverByAddress(address)
if err != nil || ok {
t.Fatalf("CoverByAddress(%q) = %v, %v, want a clean miss", address, ok, err)
}
}
}
// A PUT naming an existing series must not overwrite its title or URL: the row
// is shared, and those values are scraped page content (ADR-0003). An acquired
// Cover is likewise untouched by any client.
func TestUpsertExistingSeriesIgnoresClientTitleCoverURL(t *testing.T) {
store := newTestStore(t)
base := Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", SeriesURL: "https://asurascans.com/comics/solo",
Cover: "https://asurascans.com/covers/solo.jpg", LastChapterNum: 10,
UpdatedAt: 1000,
LastChapterNum: 10, UpdatedAt: 1000,
}
if _, err := store.Upsert(store.OwnerID(), base); err != nil {
t.Fatalf("seed: %v", err)
}
acquired := "https://asurascans.com/covers/solo.jpg"
if err := store.SetSeriesCover("asura", "solo", acquired, []byte("bytes"), "image/jpeg"); err != nil {
t.Fatalf("SetSeriesCover: %v", err)
}
// Same series, hostile/compromised values, real progress advance.
base.Title = "Scraped Rename"
@@ -799,8 +913,9 @@ func TestUpsertExistingSeriesIgnoresClientTitleCoverURL(t *testing.T) {
if err != nil {
t.Fatalf("Upsert: %v", err)
}
wantCover := "https://bookmarks.test/covers/" + CoverAddress(acquired)
if got.Title != "Solo Leveling" || got.SeriesURL != "https://asurascans.com/comics/solo" ||
got.Cover != "https://asurascans.com/covers/solo.jpg" {
got.Cover != wantCover {
t.Fatalf("stored = %+v, want original title/url/cover kept", got)
}
if got.LastChapterNum != 11 {
@@ -836,16 +951,19 @@ func TestUpsertExistingSeriesAcceptsKindAndLatest(t *testing.T) {
}
// Deleting the last bookmark must leave the series row behind, so a later
// re-bookmark shows title and cover immediately instead of waiting for a poll.
// re-bookmark shows title and cover immediately instead of re-acquiring them.
func TestDeleteKeepsSeriesRow(t *testing.T) {
store := newTestStore(t)
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", Cover: "https://asurascans.com/covers/solo.jpg",
UpdatedAt: 1000,
Title: "Solo Leveling", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
acquired := "https://asurascans.com/covers/solo.jpg"
if err := store.SetSeriesCover("asura", "solo", acquired, []byte("bytes"), "image/jpeg"); err != nil {
t.Fatalf("SetSeriesCover: %v", err)
}
if err := store.Delete(store.OwnerID(), "asura:solo"); err != nil {
t.Fatalf("Delete: %v", err)
}
@@ -863,7 +981,8 @@ func TestDeleteKeepsSeriesRow(t *testing.T) {
if err != nil {
t.Fatalf("re-upsert: %v", err)
}
if stored.Title != "Solo Leveling" || stored.Cover != "https://asurascans.com/covers/solo.jpg" {
wantCover := "https://bookmarks.test/covers/" + CoverAddress(acquired)
if stored.Title != "Solo Leveling" || stored.Cover != wantCover {
t.Fatalf("re-bookmark = %+v, want title/cover from the surviving series row", stored)
}
}
@@ -940,14 +1059,14 @@ func TestDueForLatestCheckExcludesOrphanSeries(t *testing.T) {
func TestSeedOwnerIdempotentAndRefreshesTokenHash(t *testing.T) {
url := pgtest.URL(t)
coverDir := t.TempDir()
first, err := Open(url, Owner{DiscordID: "owner", TokenHash: sha256.Sum256([]byte("hash-v1"))}, coverDir)
first, err := Open(url, Owner{DiscordID: "owner", TokenHash: sha256.Sum256([]byte("hash-v1"))}, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}
ownerID := first.OwnerID()
first.Close()
second, err := Open(url, Owner{DiscordID: "owner", TokenHash: sha256.Sum256([]byte("hash-v2"))}, coverDir)
second, err := Open(url, Owner{DiscordID: "owner", TokenHash: sha256.Sum256([]byte("hash-v2"))}, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("reopen: %v", err)
}
@@ -1004,7 +1123,7 @@ func TestMigration0004AttachesBookmarksToOwner(t *testing.T) {
t.Fatalf("migrate to 0002: %v", err)
}
st, err := Open(url, testOwner, t.TempDir())
st, err := Open(url, testOwner, t.TempDir(), testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}
@@ -1281,7 +1400,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
func TestKaganeCoverPersistsAcrossReopen(t *testing.T) {
url := pgtest.URL(t)
coverDir := t.TempDir()
first, err := Open(url, testOwner, coverDir)
first, err := Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}
@@ -1293,7 +1412,7 @@ func TestKaganeCoverPersistsAcrossReopen(t *testing.T) {
t.Fatalf("close first store: %v", err)
}
second, err := Open(url, testOwner, coverDir)
second, err := Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("reopen: %v", err)
}
@@ -1308,15 +1427,26 @@ func TestKaganeCoverPersistsAcrossReopen(t *testing.T) {
}
func TestOpenRequiresCoverDirectory(t *testing.T) {
if _, err := Open(pgtest.URL(t), testOwner, ""); err == nil || !strings.Contains(err.Error(), "cover directory is required") {
if _, err := Open(pgtest.URL(t), testOwner, "", testCoverBaseURL); err == nil || !strings.Contains(err.Error(), "cover directory is required") {
t.Fatalf("Open without cover directory = %v, want required-directory error", err)
}
}
// A base URL without a scheme reads like a hostname and starts cleanly, but
// every Cover it puts on the wire is an address no browser can resolve.
func TestOpenRequiresAbsoluteCoverBaseURL(t *testing.T) {
for _, base := range []string{"", "bookmarks.test", "https://", "ftp://bookmarks.test"} {
if _, err := Open(pgtest.URL(t), testOwner, t.TempDir(), base); err == nil ||
!strings.Contains(err.Error(), "absolute http(s) origin") {
t.Fatalf("Open with base %q = %v, want absolute-origin error", base, err)
}
}
}
func TestKaganeCoverIsContentAddressedOnFilesystem(t *testing.T) {
url := pgtest.URL(t)
coverDir := t.TempDir()
first, err := Open(url, testOwner, coverDir)
first, err := Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}