Compare commits

...

15 Commits

Author SHA1 Message Date
sulthan f532e50516 Orphan removal: the database is the guard (#155) 2026-08-22 09:58:11 +07:00
sulthan bc64a1d894 backend: remove orphan series from the admin (ticket #155)
(*Store).RemoveSeries deletes one series row by (site, series_id) via a
plain parameterized DELETE; a bookmarks_series_fk violation outside
23503 is translated into the ErrSeriesHasBookmarks sentinel so no
driver type escapes the store. The caller reads the row's cover
address before the delete and reclaims it after: ReclaimCover's guard
cannot pass while a series row still points at the address.

POST /admin/series/{key}/remove answers the list row with the removed
row's fragment plus the heading re-rendered out of band at the fresh
count (HX-Reswap: delete removes the row through the same button that
swaps the refusal back in), and navigates from the detail page to the
No-Readers list (HX-Redirect for htmx, a 303 for plain clients). A
removal that races a fresh Bookmark is a refusal, not a 500: the row
re-renders at its new count with the fact spelled out. The control
renders only at zero Reader count on both surfaces, gated by hx-confirm
with the brief's copy.
2026-08-22 09:55:01 +07:00
sulthan ba2b6eebbb Latest Chapter provenance: one derived line naming the actor class (#152) 2026-08-22 09:37:08 +07:00
sulthan f40894b108 Cover byte reclamation: one guarded helper, file first (#154) 2026-08-22 09:36:45 +07:00
sulthan a4ea80dcc2 Cover byte reclamation: one guarded helper, file first, covers row last (#154) 2026-08-22 09:34:02 +07:00
sulthan a4491babed Latest Chapter provenance line naming the actor class (#152) 2026-08-22 09:32:25 +07:00
sulthan e1d61534bb A Forced Poll replaces the Cover; an ordinary pass still only fills a blank one (#153) 2026-08-22 09:23:59 +07:00
sulthan ad09569f2e Series URL repair, owner-typed and gated by the poller's own fetch gate (#151) 2026-08-22 09:23:59 +07:00
sulthan 35a86f5eb9 Correct the three-outcome comment; keep doc comment attached to storeCover (#153) 2026-08-22 09:23:18 +07:00
sulthan 424d2c6600 Series URL repair: owner-typed, gated by the poller's own fetch gate (#151) 2026-08-22 09:21:00 +07:00
sulthan 7e1cbdde9e Forced Poll replaces the Cover; ordinary pass still fills only a blank one (#153) 2026-08-22 09:19:15 +07:00
sulthan 448631c78e Cover addresses derived from the bytes; ReplaceSeriesCover (#150)
# Conflicts:
#	backend/internal/store/store_test.go
2026-08-22 09:08:37 +07:00
sulthan f3b2722568 Latest Chapter correction: one numeric input, owner-gated (#149) 2026-08-22 09:08:06 +07:00
sulthan b9fc83217d #150: address covers by bytes; ReplaceSeriesCover 2026-08-22 09:05:53 +07:00
sulthan c9b1f2a334 Latest Chapter correction: one numeric input, owner-gated (#149) 2026-08-22 09:04:24 +07:00
25 changed files with 2327 additions and 130 deletions
+1 -1
View File
@@ -99,7 +99,7 @@ Go backend:
- SQL always parameterized (`$N`). Only compile-time constants (`bookmarkColumns`) may be concatenated into query text — never a request value, not even a validated one.
- `html/template` only for anything a browser parses, never `text/template`. Never wrap stored or fetched strings in `template.HTML`/`JS`/`URL`; that switches off the escaping every template depends on.
- Any outbound fetch of a client-supplied URL passes `fetchableSeriesURL` (site + `https` + host check) first. `series_url` arrives in a PUT body, so without the gate the poller will probe arbitrary hosts from the server's own network position. New fetch path reuses the gate rather than re-deriving one.
- Any outbound fetch of a client-supplied URL passes `FetchableSeriesURL` (site + `https` + host check) first. `series_url` arrives in a PUT body, so without the gate the poller will probe arbitrary hosts from the server's own network position. New fetch path reuses the gate rather than re-deriving one.
- Cap every remote body with `io.LimitReader` (`maxBodyBytes`). An unbounded read is an OOM handed to whatever is on the other end.
- Compare secrets with `hmac.Equal` / `subtle.ConstantTimeCompare`, never `==`. A credential is matched by the SHA-256 the `readers` table holds, which is already a fixed-width equality — a new secret comparison must not regress to `==`.
- Errors: generic text to the client (`http.Error(w, "internal error", 500)`), detail to `log.Printf`. Never log `TOKEN_KEY`, a Reader's credential, `DISCORD_CLIENT_SECRET`, a session id, or a whole `Authorization` header.
+4 -4
View File
@@ -32,7 +32,7 @@ func TestPublicCoverServesStoredBytesUnauthenticated(t *testing.T) {
// The wire URL is what a client actually requests, so the path under test
// is taken from it rather than rebuilt by hand.
wire := st.CoverWireURL(store.CoverAddress(sourceURL))
wire := st.CoverWireURL(store.CoverAddressForBytes([]byte("\x00webp-bytes")))
path, ok := strings.CutPrefix(wire, testCoverBaseURL)
if !ok {
t.Fatalf("wire URL %q is not on the public origin %q", wire, testCoverBaseURL)
@@ -57,7 +57,7 @@ func TestPublicCoverServesStoredBytesUnauthenticated(t *testing.T) {
func TestPublicCoverRejectsUnknownAddress(t *testing.T) {
srv, _ := newWebTestServer(t, testConfig())
cases := map[string]string{
"unknown": "/covers/" + store.CoverAddress("https://cdn.example/never-stored.jpg"),
"unknown": "/covers/" + store.CoverAddressForBytes([]byte("never-stored")),
"malformed": "/covers/not-an-address",
"traversal": "/covers/../../etc/passwd",
"empty": "/covers/",
@@ -86,7 +86,7 @@ func TestPublicCoverNeverEchoesNonImage(t *testing.T) {
}
// A legitimate row, then the content type flipped behind the store's back:
// the bytes exist at the address, so only the type is hostile.
address := store.CoverAddress(sourceURL)
address := store.CoverAddressForBytes([]byte("<script>"))
if err := st.SetSeriesCover("asura", "solo", sourceURL, []byte("<script>"), "image/png"); err != nil {
t.Fatalf("seed row: %v", err)
}
@@ -127,7 +127,7 @@ func TestListRendersAcquiredCover(t *testing.T) {
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rr.Code)
}
want := `src="` + testCoverBaseURL + "/covers/" + store.CoverAddress(sourceURL) + `"`
want := `src="` + testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("\xff\xd8jpeg")) + `"`
if !strings.Contains(rr.Body.String(), want) {
t.Fatalf("rendered list does not contain %s", want)
}
+7 -7
View File
@@ -123,10 +123,10 @@ func TestAcquireFillsChapterAndCoverFromOneFetch(t *testing.T) {
if got.LatestChapterNum == nil || *got.LatestChapterNum != 181 {
t.Fatalf("LatestChapterNum = %v, want 181", got.LatestChapterNum)
}
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(acquireCoverURL); got.Cover != want {
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("cover-bytes")); got.Cover != want {
t.Fatalf("Cover = %q, want the absolute address %q", got.Cover, want)
}
body, contentType, ok, err := s.CoverByAddress(store.CoverAddress(acquireCoverURL))
body, contentType, ok, err := s.CoverByAddress(store.CoverAddressForBytes([]byte("cover-bytes")))
if err != nil || !ok {
t.Fatalf("CoverByAddress = %v, %v", ok, err)
}
@@ -182,7 +182,7 @@ func TestAcquireSkipsAnExistingSeries(t *testing.T) {
t.Fatalf("cover fetches = %d, want 1", got)
}
got := readBookmark(t, s, acquireKey)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(acquireCoverURL); got.Cover != want {
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("cover-bytes")); got.Cover != want {
t.Fatalf("Cover = %q, want the acquired one %q", got.Cover, want)
}
}
@@ -343,10 +343,10 @@ func TestAcquireKaganeCoverThroughBrowser(t *testing.T) {
t.Fatalf("browser cover fetched URL %q, want %q", got, kaganeCoverSrc)
}
got := readBookmark(t, s, kaganeKey)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(kaganeCoverSrc); got.Cover != want {
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("cover-bytes")); got.Cover != want {
t.Fatalf("Cover = %q, want the content-addressed URL %q", got.Cover, want)
}
body, contentType, ok, err := s.CoverByAddress(store.CoverAddress(kaganeCoverSrc))
body, contentType, ok, err := s.CoverByAddress(store.CoverAddressForBytes([]byte("cover-bytes")))
if err != nil || !ok {
t.Fatalf("CoverByAddress = %v, %v", ok, err)
}
@@ -381,7 +381,7 @@ func TestAcquireNovelfullCoverOverPlainTLS(t *testing.T) {
t.Fatalf("cover fetched from %q, want %q", got, novelfullCoverURL)
}
got := readBookmark(t, s, novelfullKey)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(novelfullCoverURL); got.Cover != want {
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("cover-bytes")); got.Cover != want {
t.Fatalf("Cover = %q, want %q", got.Cover, want)
}
}
@@ -451,7 +451,7 @@ func TestAcquireNovelfullCoverWithoutBrowser(t *testing.T) {
t.Fatalf("cover fetches = %d, want 1", got)
}
got := readBookmark(t, s, novelfullKey)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(novelfullCoverURL); got.Cover != want {
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("cover-bytes")); got.Cover != want {
t.Fatalf("Cover = %q, want %q", got.Cover, want)
}
}
+2 -2
View File
@@ -124,7 +124,7 @@ func (f *BrowserFetcher) Get(ctx context.Context, seriesURL string) (string, int
// request must be made from inside the page so it carries the clearance
// cookie, and the API is the only place the list exists. Refusing any other
// address is the per-Site half of the SSRF gate, kept deliberately behind
// fetchableSeriesURL (see browserRead.Read).
// FetchableSeriesURL (see browserRead.Read).
func kaganeRead(seriesURL string, out *string) (chromedp.Action, bool) {
apiURL, ok := kaganeAPIURL(seriesURL)
if !ok {
@@ -326,7 +326,7 @@ func (f *BrowserFetcher) run(ctx context.Context, target string, read chromedp.A
// kaganeAPIURL maps a stored series_url to the JSON endpoint carrying its
// chapter list. Returning false for anything else is a second line of defence
// behind fetchableSeriesURL: a headless browser is a strong SSRF primitive and
// behind FetchableSeriesURL: a headless browser is a strong SSRF primitive and
// series_url is client-supplied, so the host is pinned here too.
func kaganeAPIURL(seriesURL string) (string, bool) {
u, err := url.Parse(seriesURL)
+1 -1
View File
@@ -174,7 +174,7 @@ func (f *TLSCoverFetcher) Fetch(ctx context.Context, sourceURL string) ([]byte,
return body, contentType, nil
}
// This gate deliberately differs from fetchableSeriesURL: cover hosts are
// This gate deliberately differs from FetchableSeriesURL: cover hosts are
// site-independent CDNs, so a Site host allowlist would reject valid covers.
func (f *TLSCoverFetcher) validateURL(ctx context.Context, u *url.URL) error {
if u == nil || u.Scheme != "https" || u.Host == "" || u.User != nil {
+56 -5
View File
@@ -98,6 +98,23 @@ func (p *Poller) fillBlankCover(ctx context.Context, sr store.Series, cover stri
}()
}
// replaceCover is the Forced Poll's Cover path: the owner asked to accept the
// page as it now stands, so where fillBlankCover leaves a non-blank Cover
// alone (ADR-0007) this writes through whatever the page's Cover URL answers
// with, whether one exists or not. The accepted consequence (issue #135):
// refreshing the Cover and re-reading the chapters are one act — there is no
// Cover-only refetch.
func (p *Poller) replaceCover(ctx context.Context, sr store.Series, cover string) {
if cover == "" {
return
}
p.coverWG.Add(1)
go func() {
defer p.coverWG.Done()
p.storeCover(ctx, sr, cover)
}()
}
// prefetchCover heals Series that already carry a third-party source URL but
// no stored address — the state left by client-supplied covers before
// acquisition moved server-side. Every Site takes the same path; fetchCoverBytes
@@ -122,18 +139,43 @@ func (p *Poller) prefetchCover(ctx context.Context, sr store.Series) {
p.storeCover(ctx, sr, sr.Cover)
}
// storeCover fetches bytes for sourceURL and points the Series at them. Every
// failure is logged against the Series and swallowed so the chapter poll
// cannot see it.
// storeCover fetches bytes for sourceURL and points the Series at them: a
// fill-only write for an ordinary pass, a write-through for a forced one
// (issue #135). Every failure is logged against the Series and swallowed so
// the chapter poll cannot see it.
func (p *Poller) storeCover(ctx context.Context, sr store.Series, sourceURL string) {
bytes, contentType, err := fetchCoverBytes(ctx, sourceURL, p.CoverFetch, p.CoverBytesFetch)
if err != nil {
log.Printf("latest poll %q: fetch cover %s: %v", sr.Key(), sourceURL, err)
return
}
if !sr.Forced {
if err := p.Store.SetSeriesCover(sr.Site, sr.SeriesID, sourceURL, bytes, contentType); err != nil {
log.Printf("latest poll %q: persist cover: %v", sr.Key(), err)
}
return
}
// The forced write replaces whether or not a Cover exists, and the row
// then tells the three outcomes apart: a blank filled, identical artwork
// re-served — an honest no-op — or a replacement whose previous address
// is stranded and reclaimed below. A failed reclaim is logged and the
// stranded bytes stay served until a later call reclaims them.
previous, current, err := p.Store.ReplaceSeriesCover(sr.Site, sr.SeriesID, sourceURL, bytes, contentType)
if err != nil {
log.Printf("latest poll %q: persist cover: %v", sr.Key(), err)
return
}
switch {
case previous == "":
log.Printf("latest poll %q: cover filled at %s", sr.Key(), current)
case previous == current:
log.Printf("latest poll %q: cover unchanged, the site re-serves the same bytes", sr.Key())
default:
if err := p.Store.ReclaimCover(previous); err != nil {
log.Printf("latest poll %q: reclaim cover %s: %v", sr.Key(), previous, err)
}
log.Printf("latest poll %q: cover replaced %s -> %s", sr.Key(), previous, current)
}
}
// fetcherFor returns the fetcher a site's page needs, or nil when the site
@@ -621,7 +663,12 @@ func (p *Poller) checkOne(ctx context.Context, sr store.Series) (outcome readOut
p.healCover(ctx, sr)
// Cover fill is independent of the chapter signal: a page that lost its
// chapter list may keep its og:image, and a blank Series heals either way.
// A forced pass writes the Cover through the replace path instead.
if sr.Forced {
p.replaceCover(ctx, sr, facts.Cover)
} else {
p.fillBlankCover(ctx, sr, facts.Cover)
}
if !facts.HasLatest {
// Most likely a challenge page or a layout change. Either way the row is
// already stamped, so this waits out a rest instead of hot-looping.
@@ -710,7 +757,7 @@ func (p *Poller) waitCovers() {
p.coverWG.Wait()
}
// fetchableSeriesURL reports whether site is a Site the registry knows and
// FetchableSeriesURL reports whether site is a Site the registry knows and
// seriesURL is safe to hand to a fetcher: an https URL whose host matches the
// Site's pinned hostname exactly. series_url comes from client-supplied PUT
// bodies, so this is a defence against the poller being used to probe
@@ -719,7 +766,11 @@ func (p *Poller) waitCovers() {
// browser Site guards a control that executes JavaScript and carries cookies,
// a parser Site guards a wasted request — but the rule is one rule, from the
// registry.
func fetchableSeriesURL(site, seriesURL string) bool {
//
// The owner's series URL repair (issue #151) is a second caller: the web
// layer validates with this same gate before storing a repair, so there is
// never a second copy of it.
func FetchableSeriesURL(site, seriesURL string) bool {
s, known := sites[site]
if !known {
return false
+376 -18
View File
@@ -6,8 +6,10 @@ import (
"database/sql"
"errors"
"fmt"
"io/fs"
"log"
"os"
"path/filepath"
"strings"
"sync"
"testing"
@@ -214,9 +216,9 @@ func TestRunOncePrefetchesPublicCover(t *testing.T) {
if got := covers.callCount(); got != 1 {
t.Fatalf("cover fetch calls = %d, want 1", got)
}
body, contentType, found, err := s.GetCover(coverURL)
body, contentType, found, err := s.CoverByAddress(store.CoverAddressForBytes([]byte("cover-bytes")))
if err != nil || !found {
t.Fatalf("GetCover: %v found=%v", err, found)
t.Fatalf("CoverByAddress: %v found=%v", err, found)
}
if string(body) != "cover-bytes" || contentType != "image/jpeg" {
t.Fatalf("stored cover = (%q, %q), want (cover-bytes, image/jpeg)", body, contentType)
@@ -588,8 +590,8 @@ func TestFetchableSeriesURL(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := fetchableSeriesURL(tt.site, tt.seriesURL); got != tt.want {
t.Errorf("fetchableSeriesURL(%q, %q) = %v, want %v",
if got := FetchableSeriesURL(tt.site, tt.seriesURL); got != tt.want {
t.Errorf("FetchableSeriesURL(%q, %q) = %v, want %v",
tt.site, tt.seriesURL, got, tt.want)
}
})
@@ -663,7 +665,7 @@ func TestNovelfullUsesTLSWhenNoBrowserFetcher(t *testing.T) {
if err != nil || !found {
t.Fatalf("Get: %v found=%v", err, found)
}
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(novelfullCoverURL); got.Cover != want {
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("cover-bytes")); got.Cover != want {
t.Fatalf("Cover = %q, want %q", got.Cover, want)
}
if got.LatestChapterNum == nil || *got.LatestChapterNum != 2334 {
@@ -811,7 +813,7 @@ func TestRunOncePrefetchesKaganeCover(t *testing.T) {
p.runOnce(context.Background())
p.waitCovers()
body, contentType, ok, err := s.CoverByAddress(store.CoverAddress(coverURL))
body, contentType, ok, err := s.CoverByAddress(store.CoverAddressForBytes([]byte("cover-bytes")))
if err != nil || !ok {
t.Fatalf("CoverByAddress: %v found=%v", err, ok)
}
@@ -821,7 +823,7 @@ func TestRunOncePrefetchesKaganeCover(t *testing.T) {
if got := covers.callCount(); got != 1 {
t.Fatalf("cover fetch calls = %d, want 1", got)
}
if got := readBookmark(t, s, key); got.Cover != testCoverBaseURL+"/covers/"+store.CoverAddress(coverURL) {
if got := readBookmark(t, s, key); got.Cover != testCoverBaseURL+"/covers/"+store.CoverAddressForBytes([]byte("cover-bytes")) {
t.Fatalf("wire Cover = %q, want content-addressed URL", got.Cover)
}
}
@@ -914,7 +916,7 @@ func TestRunOnceRejectsInvalidKaganeCover(t *testing.T) {
p.runOnce(context.Background())
p.waitCovers()
if _, _, found, err := s.CoverByAddress(store.CoverAddress(coverURL)); err != nil || found {
if _, _, found, err := s.CoverByAddress(store.CoverAddressForBytes([]byte("not an image"))); err != nil || found {
t.Fatalf("invalid cover persisted = %v, err %v; want missing", found, err)
}
}
@@ -940,7 +942,7 @@ func TestRunOnceWithoutCoverFetcherStillPollsKagane(t *testing.T) {
p.runOnce(context.Background())
p.waitCovers()
if _, _, found, err := s.CoverByAddress(store.CoverAddress(coverURL)); err != nil || found {
if _, _, found, err := s.CoverByAddress(store.CoverAddressForBytes([]byte("cover-bytes"))); err != nil || found {
t.Fatalf("cover after nil CoverFetch = found %v, err %v; want missing", found, err)
}
}
@@ -1019,8 +1021,8 @@ func TestFetchableSeriesURLPinsNovelHosts(t *testing.T) {
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := fetchableSeriesURL(tc.site, tc.url); got != tc.want {
t.Fatalf("fetchableSeriesURL(%q, %q) = %v, want %v", tc.site, tc.url, got, tc.want)
if got := FetchableSeriesURL(tc.site, tc.url); got != tc.want {
t.Fatalf("FetchableSeriesURL(%q, %q) = %v, want %v", tc.site, tc.url, got, tc.want)
}
})
}
@@ -1038,7 +1040,6 @@ func TestRunOnceFillsBlankCoverFromSeriesPage(t *testing.T) {
seriesURL string
kind string
body string
wantCover string
browser bool
}{
{
@@ -1047,14 +1048,12 @@ func TestRunOnceFillsBlankCoverFromSeriesPage(t *testing.T) {
seriesID: "chronicles-of-the-demon-faction-f886a8af",
seriesURL: "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af",
kind: store.KindManga, body: asuraSeriesFixture + asuraCoverFixture,
wantCover: "https://cdn.asurascans.com/asura-images/covers/chronicles-of-the-demon-faction.d4dcb8.webp",
},
{
name: "lightnovelworld novel",
key: "lightnovelworld:all-jobs-and-classes-i-just-wanted-one-skill-not-them-all", site: "lightnovelworld",
seriesID: "all-jobs-and-classes-i-just-wanted-one-skill-not-them-all", seriesURL: "https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/",
kind: store.KindNovel, body: lnwSeriesFixture + lnwCoverFixture,
wantCover: "https://i1.wp.com/lightnovelworld.net/wp-content/uploads/2025/10/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all.jpg",
},
{
name: "kagane manga",
@@ -1062,7 +1061,6 @@ func TestRunOnceFillsBlankCoverFromSeriesPage(t *testing.T) {
seriesID: "019fe11a-8670-7cf3-8343-0b02057d3787",
seriesURL: "https://kagane.to/series/019fe11a-8670-7cf3-8343-0b02057d3787",
kind: store.KindManga, body: kaganeAPIFixtureWithCover, browser: true,
wantCover: "https://kagane.to/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed",
},
}
for _, tc := range cases {
@@ -1086,7 +1084,7 @@ func TestRunOnceFillsBlankCoverFromSeriesPage(t *testing.T) {
p.waitCovers()
got := readBookmark(t, s, tc.key)
wantWire := testCoverBaseURL + "/covers/" + store.CoverAddress(tc.wantCover)
wantWire := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("cover-bytes"))
if got.Cover != wantWire {
t.Fatalf("Cover = %q, want %q", got.Cover, wantWire)
}
@@ -1143,7 +1141,7 @@ func TestRunOnceDoesNotReplaceExistingCover(t *testing.T) {
t.Fatalf("cover fetch calls = %d, want 0", got)
}
got := readBookmark(t, s, key)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(first); got.Cover != want {
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("first")); got.Cover != want {
t.Fatalf("Cover = %q, want the first one %q", got.Cover, want)
}
}
@@ -1190,7 +1188,7 @@ func TestRunOnceRetriesFailedBlankCoverOnNextPoll(t *testing.T) {
t.Fatalf("cover fetch calls after retry = %d, want 2", got)
}
got := readBookmark(t, s, key)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(coverURL); got.Cover != want {
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("cover-bytes")); got.Cover != want {
t.Fatalf("Cover after retry = %q, want %q", got.Cover, want)
}
}
@@ -2337,3 +2335,363 @@ func TestForcedSeriesWakesSleepingBrowser(t *testing.T) {
t.Fatalf("browser fetches with a forced series = %d, want 2 (the lane wakes)", got)
}
}
// A forced pass accepts the page as it now stands, so it writes the Cover
// through the replace path; an ordinary pass still only fills a blank one
// (issue #135).
func TestRunOnceForcedPassReplacesExistingCover(t *testing.T) {
s, _ := newTestStore(t)
const (
key = "asura:chronicles-of-the-demon-faction-f886a8af"
seriesID = "chronicles-of-the-demon-faction-f886a8af"
seriesURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
first = "https://cdn.example/covers/first.jpg"
)
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key, Site: "asura", SeriesID: seriesID, SeriesURL: seriesURL, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if err := s.SetSeriesCover("asura", seriesID, first, []byte("first"), "image/jpeg"); err != nil {
t.Fatalf("seed cover: %v", err)
}
at := time.UnixMilli(5_000_000)
public := &fakeBytesCoverFetcher{body: []byte("second"), contentType: "image/jpeg"}
t.Run("unforced pass leaves the cover alone", func(t *testing.T) {
p := &Poller{
Store: s, Fetch: &fakeFetcher{body: asuraSeriesFixture + asuraCoverFixture, status: 200},
CoverBytesFetch: public,
Now: func() time.Time { return at },
}
p.runOnce(context.Background())
p.waitCovers()
if got := public.callCount(); got != 0 {
t.Fatalf("cover fetch calls = %d, want 0", got)
}
got := readBookmark(t, s, key)
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("first")); got.Cover != want {
t.Fatalf("Cover = %q, want the first one %q", got.Cover, want)
}
})
t.Run("forced pass replaces the cover", func(t *testing.T) {
if err := s.ForceSeriesPoll("asura", seriesID, at.Add(time.Hour).UnixMilli()); err != nil {
t.Fatalf("ForceSeriesPoll: %v", err)
}
p := &Poller{
Store: s, Fetch: &fakeFetcher{body: asuraSeriesFixture + asuraCoverFixture, status: 200},
CoverBytesFetch: public,
Now: func() time.Time { return at },
}
p.runOnce(context.Background())
p.waitCovers()
if got := public.callCount(); got != 1 {
t.Fatalf("cover fetch calls = %d, want 1", got)
}
got := readBookmark(t, s, key)
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("second")); got.Cover != want {
t.Fatalf("Cover = %q, want the second one %q", got.Cover, want)
}
body, _, ok, err := s.CoverByAddress(store.CoverAddressForBytes([]byte("second")))
if err != nil || !ok {
t.Fatalf("CoverByAddress: %v found=%v", err, ok)
}
if string(body) != "second" {
t.Fatalf("stored cover = %q, want second", body)
}
})
}
// Identical artwork re-served is an honest no-op the caller can tell apart
// from a replacement: the address comes from the bytes, so the row cannot
// change in substance, and the replace call site reports the three outcomes
// distinctly (issue #135).
func TestRunOnceForcedPassIdenticalBytesLogsNoOp(t *testing.T) {
s, _ := newTestStore(t)
const (
key = "asura:chronicles-of-the-demon-faction-f886a8af"
seriesID = "chronicles-of-the-demon-faction-f886a8af"
seriesURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
first = "https://cdn.example/covers/first.jpg"
)
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key, Site: "asura", SeriesID: seriesID, SeriesURL: seriesURL, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if err := s.SetSeriesCover("asura", seriesID, first, []byte("cover-bytes"), "image/jpeg"); err != nil {
t.Fatalf("seed cover: %v", err)
}
at := time.UnixMilli(5_000_000)
if err := s.ForceSeriesPoll("asura", seriesID, at.Add(time.Hour).UnixMilli()); err != nil {
t.Fatalf("ForceSeriesPoll: %v", err)
}
var logs strings.Builder
prev := log.Writer()
log.SetOutput(&logs)
t.Cleanup(func() { log.SetOutput(prev) })
p := &Poller{
Store: s, Fetch: &fakeFetcher{body: asuraSeriesFixture + asuraCoverFixture, status: 200},
CoverBytesFetch: &fakeBytesCoverFetcher{body: []byte("cover-bytes"), contentType: "image/jpeg"},
Now: func() time.Time { return at },
}
p.runOnce(context.Background())
p.waitCovers()
got := readBookmark(t, s, key)
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("cover-bytes")); got.Cover != want {
t.Fatalf("Cover = %q, want unchanged %q", got.Cover, want)
}
if body, _, ok, err := s.CoverByAddress(store.CoverAddressForBytes([]byte("cover-bytes"))); err != nil || !ok || string(body) != "cover-bytes" {
t.Fatalf("stored cover after no-op: found=%v err=%v", ok, err)
}
logged := logs.String()
if !strings.Contains(logged, "cover unchanged") {
t.Fatalf("no-op not reported as unchanged; log:\n%s", logged)
}
if strings.Contains(logged, "cover replaced") {
t.Fatalf("no-op reported as a replacement; log:\n%s", logged)
}
}
// A Series whose sharded Cover file was unlinked out from under it is
// repaired by one forced pass: same bytes mean the same address and the file
// re-linked (issue #135, story 32).
func TestRunOnceForcedPassRelinksUnlinkedCoverFile(t *testing.T) {
coverDir := t.TempDir()
url := pgtest.URL(t)
s, err := store.Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}
t.Cleanup(func() { s.Close() })
const (
key = "asura:chronicles-of-the-demon-faction-f886a8af"
seriesID = "chronicles-of-the-demon-faction-f886a8af"
seriesURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
first = "https://cdn.example/covers/first.jpg"
)
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key, Site: "asura", SeriesID: seriesID, SeriesURL: seriesURL, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if err := s.SetSeriesCover("asura", seriesID, first, []byte("cover-bytes"), "image/jpeg"); err != nil {
t.Fatalf("seed cover: %v", err)
}
address := store.CoverAddressForBytes([]byte("cover-bytes"))
coverPath := filepath.Join(coverDir, filepath.FromSlash(address[:2]+"/"+address[2:4]+"/"+address))
if err := os.Remove(coverPath); err != nil {
t.Fatalf("unlink cover file: %v", err)
}
if _, _, ok, err := s.CoverByAddress(address); err != nil || ok {
t.Fatalf("CoverByAddress after unlink = found %v err %v; want missing (file gone)", ok, err)
}
at := time.UnixMilli(5_000_000)
if err := s.ForceSeriesPoll("asura", seriesID, at.Add(time.Hour).UnixMilli()); err != nil {
t.Fatalf("ForceSeriesPoll: %v", err)
}
p := &Poller{
Store: s, Fetch: &fakeFetcher{body: asuraSeriesFixture + asuraCoverFixture, status: 200},
CoverBytesFetch: &fakeBytesCoverFetcher{body: []byte("cover-bytes"), contentType: "image/jpeg"},
Now: func() time.Time { return at },
}
p.runOnce(context.Background())
p.waitCovers()
body, contentType, ok, err := s.CoverByAddress(address)
if err != nil || !ok {
t.Fatalf("CoverByAddress after forced pass: found=%v err=%v; want the file re-linked", ok, err)
}
if string(body) != "cover-bytes" || contentType != "image/jpeg" {
t.Fatalf("re-linked cover = (%q, %q), want (cover-bytes, image/jpeg)", body, contentType)
}
}
// A forced pass degrades exactly like an ordinary one when the cover sidecar
// is unreachable: the fetch is skipped and logged, the chapter poll is
// untouched, and the stored Cover is not moved (issue #135, story 6).
func TestRunOnceForcedPassWithoutCoverFetcherStillPolls(t *testing.T) {
s, _ := newTestStore(t)
const (
key = "kagane:019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"
seriesID = "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"
)
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key, Site: "kagane", SeriesID: seriesID,
SeriesURL: "https://kagane.to/series/" + seriesID, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if err := s.SetSeriesCover("kagane", seriesID, "https://kagane.to/api/v2/image/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b/compressed", []byte("existing"), "image/webp"); err != nil {
t.Fatalf("seed cover: %v", err)
}
at := time.UnixMilli(5_000_000)
if err := s.ForceSeriesPoll("kagane", seriesID, at.Add(time.Hour).UnixMilli()); err != nil {
t.Fatalf("ForceSeriesPoll: %v", err)
}
var logs strings.Builder
prev := log.Writer()
log.SetOutput(&logs)
t.Cleanup(func() { log.SetOutput(prev) })
p := &Poller{
Store: s, BrowserFetch: &fakeFetcher{body: kaganeAPIFixtureWithCover, status: 200},
Now: func() time.Time { return at },
}
p.runOnce(context.Background())
p.waitCovers()
got := readBookmark(t, s, key)
if got.LatestChapterNum == nil || *got.LatestChapterNum != 41 {
t.Fatalf("LatestChapterNum = %v, want 41", got.LatestChapterNum)
}
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("existing")); got.Cover != want {
t.Fatalf("Cover = %q, want the existing one %q untouched", got.Cover, want)
}
if checked := readLatestCheckedAt(t, s, key); checked != at.UnixMilli() {
t.Fatalf("latest_checked_at = %d, want %d", checked, at.UnixMilli())
}
if !strings.Contains(logs.String(), "fetch cover") {
t.Fatalf("missing skipped-cover log; log:\n%s", logs.String())
}
}
// A forced replacement strands the previous address, and the poller reclaims
// it from the stranded branch: the old sharded file and covers row are both
// gone once the pass lands while the new bytes read back (issue #154). The
// identical-bytes no-op that follows reclaims nothing: previous == current
// there, and a reclamation would delete the Cover the pass just wrote.
func TestRunOnceForcedPassReclaimsSupersededCover(t *testing.T) {
coverDir := t.TempDir()
url := pgtest.URL(t)
s, err := store.Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}
t.Cleanup(func() { s.Close() })
const (
key = "asura:chronicles-of-the-demon-faction-f886a8af"
seriesID = "chronicles-of-the-demon-faction-f886a8af"
seriesURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
first = "https://cdn.example/covers/first.jpg"
)
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key, Site: "asura", SeriesID: seriesID, SeriesURL: seriesURL, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if err := s.SetSeriesCover("asura", seriesID, first, []byte("first"), "image/jpeg"); err != nil {
t.Fatalf("seed cover: %v", err)
}
stale := store.CoverAddressForBytes([]byte("first"))
stalePath := filepath.Join(coverDir, filepath.FromSlash(stale[:2]+"/"+stale[2:4]+"/"+stale))
at := time.UnixMilli(5_000_000)
if err := s.ForceSeriesPoll("asura", seriesID, at.Add(time.Hour).UnixMilli()); err != nil {
t.Fatalf("ForceSeriesPoll: %v", err)
}
p := &Poller{
Store: s, Fetch: &fakeFetcher{body: asuraSeriesFixture + asuraCoverFixture, status: 200},
CoverBytesFetch: &fakeBytesCoverFetcher{body: []byte("second"), contentType: "image/jpeg"},
Now: func() time.Time { return at },
}
p.runOnce(context.Background())
p.waitCovers()
// The stranded address is gone on disk and in SQL; the new one reads back.
if _, err := os.Stat(stalePath); !errors.Is(err, fs.ErrNotExist) {
t.Fatalf("stale sharded file after replacement = %v, want fs.ErrNotExist", err)
}
if _, _, ok, err := s.CoverByAddress(stale); err != nil || ok {
t.Fatalf("stale bytes after replacement = found %v err %v, want reclaimed", ok, err)
}
if body, _, ok, err := s.CoverByAddress(store.CoverAddressForBytes([]byte("second"))); err != nil || !ok || string(body) != "second" {
t.Fatalf("new bytes after replacement = found %v err %v, want served", ok, err)
}
// The identical-bytes pass is an honest no-op and reclaims nothing.
if err := s.ForceSeriesPoll("asura", seriesID, at.Add(2*time.Hour).UnixMilli()); err != nil {
t.Fatalf("ForceSeriesPoll: %v", err)
}
p.runOnce(context.Background())
p.waitCovers()
current := store.CoverAddressForBytes([]byte("second"))
currentPath := filepath.Join(coverDir, filepath.FromSlash(current[:2]+"/"+current[2:4]+"/"+current))
if _, err := os.Stat(currentPath); err != nil {
t.Fatalf("live sharded file after no-op pass = %v, want present", err)
}
if body, _, ok, err := s.CoverByAddress(current); err != nil || !ok || string(body) != "second" {
t.Fatalf("bytes after no-op pass = found %v err %v, want still served", ok, err)
}
}
// A reclamation that fails must not fail the Poll: the failure is logged
// against the Series and the replacement still lands, so the stranded bytes
// stay reachable for a retry and the owner's act succeeded (issue #154).
func TestRunOnceForcedPassReclaimFailureDoesNotFailPoll(t *testing.T) {
coverDir := t.TempDir()
url := pgtest.URL(t)
s, err := store.Open(url, testOwner, coverDir, testCoverBaseURL)
if err != nil {
t.Fatalf("Open: %v", err)
}
t.Cleanup(func() { s.Close() })
const (
key = "asura:chronicles-of-the-fallen-f886a8af"
seriesID = "chronicles-of-the-fallen-f886a8af"
seriesURL = "https://asurascans.com/comics/chronicles-of-the-fallen-f886a8af"
first = "https://cdn.example/covers/first.jpg"
)
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key, Site: "asura", SeriesID: seriesID, SeriesURL: seriesURL, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if err := s.SetSeriesCover("asura", seriesID, first, []byte("first"), "image/jpeg"); err != nil {
t.Fatalf("seed cover: %v", err)
}
// Make the stranded file unremovable: a non-empty directory in its place.
stale := store.CoverAddressForBytes([]byte("first"))
stalePath := filepath.Join(coverDir, filepath.FromSlash(stale[:2]+"/"+stale[2:4]+"/"+stale))
if err := os.Remove(stalePath); err != nil {
t.Fatalf("clear file: %v", err)
}
if err := os.Mkdir(stalePath, 0o755); err != nil {
t.Fatalf("replace file with dir: %v", err)
}
if err := os.WriteFile(filepath.Join(stalePath, "blob"), []byte("x"), 0o644); err != nil {
t.Fatalf("fill dir: %v", err)
}
at := time.UnixMilli(5_000_000)
if err := s.ForceSeriesPoll("asura", seriesID, at.Add(time.Hour).UnixMilli()); err != nil {
t.Fatalf("ForceSeriesPoll: %v", err)
}
var logs strings.Builder
prev := log.Writer()
log.SetOutput(&logs)
t.Cleanup(func() { log.SetOutput(prev) })
p := &Poller{
Store: s, Fetch: &fakeFetcher{body: asuraSeriesFixture + asuraCoverFixture, status: 200},
CoverBytesFetch: &fakeBytesCoverFetcher{body: []byte("second"), contentType: "image/jpeg"},
Now: func() time.Time { return at },
}
p.runOnce(context.Background())
p.waitCovers()
logged := logs.String()
if !strings.Contains(logged, "reclaim cover") {
t.Fatalf("failed reclamation not logged; log:\n%s", logged)
}
if !strings.Contains(logged, "cover replaced") {
t.Fatalf("replacement not reported after a failed reclaim; log:\n%s", logged)
}
got := readBookmark(t, s, key)
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes([]byte("second")); got.Cover != want {
t.Fatalf("Cover = %q, want the replacement %q", got.Cover, want)
}
}
+1 -1
View File
@@ -41,7 +41,7 @@ var (
// its own network position to whatever URL a token-holder writes, including
// link-local/internal addresses or non-https schemes.
func readSeriesPage(ctx context.Context, site, seriesURL string, browser, tls Fetcher) (seriesRead, error) {
if !fetchableSeriesURL(site, seriesURL) {
if !FetchableSeriesURL(site, seriesURL) {
return seriesRead{}, fmt.Errorf("%w: site=%q url=%q", errNotFetchable, site, seriesURL)
}
f := fetcherFor(site, browser, tls)
+1 -1
View File
@@ -46,7 +46,7 @@ type site struct {
type browserRead struct {
// Read builds the tab read for seriesURL, refusing (false) an address
// this Site will not open in a browser — the per-Site half of the SSRF
// gate, kept deliberately behind fetchableSeriesURL: a headless browser
// gate, kept deliberately behind FetchableSeriesURL: a headless browser
// executes JavaScript and carries cookies, and series_url is
// client-supplied.
Read func(seriesURL string, out *string) (chromedp.Action, bool)
+12 -8
View File
@@ -4,6 +4,7 @@ import (
"context"
"net/http"
"os"
"strings"
"testing"
"time"
@@ -108,10 +109,7 @@ func TestSmokeAcquireKaganeCover(t *testing.T) {
if ws == "" {
t.Skip("SMOKE_BROWSER_WS_URL unset")
}
const (
seriesID = "019fe11a-8670-7cf3-8343-0b02057d3787"
coverURL = "https://kagane.to/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed"
)
const seriesID = "019fe11a-8670-7cf3-8343-0b02057d3787"
s, _ := newTestStore(t)
bf, err := NewBrowserFetcher(ws)
if err != nil {
@@ -140,12 +138,13 @@ func TestSmokeAcquireKaganeCover(t *testing.T) {
if err != nil || !found {
t.Fatalf("Get: %v found=%v", err, found)
}
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(coverURL); got.Cover != want {
t.Fatalf("Cover = %q, want %q — the acquire path did not store the browser-fetched bytes", got.Cover, want)
addr, ok := strings.CutPrefix(got.Cover, testCoverBaseURL+"/covers/")
if !ok {
t.Fatalf("Cover = %q, want an address on %q — the acquire path did not store the browser-fetched bytes", got.Cover, testCoverBaseURL+"/covers/")
}
body, contentType, ok, err := s.CoverByAddress(store.CoverAddress(coverURL))
body, contentType, ok, err := s.CoverByAddress(addr)
if err != nil || !ok {
t.Fatalf("CoverByAddress: %v found=%v", err, ok)
t.Fatalf("CoverByAddress(%q): %v found=%v", addr, err, ok)
}
if len(body) < 1000 {
t.Fatalf("stored cover is %d bytes, want a real image", len(body))
@@ -153,5 +152,10 @@ func TestSmokeAcquireKaganeCover(t *testing.T) {
if contentType != "image/webp" {
t.Fatalf("content type = %q, want image/webp", contentType)
}
// The address the row carries is the bytes' own SHA-256: a re-art behind
// the same URL would be a different address, which is the whole point.
if want := testCoverBaseURL + "/covers/" + store.CoverAddressForBytes(body); got.Cover != want {
t.Fatalf("Cover = %q, want %q", got.Cover, want)
}
t.Logf("stored %d bytes of %s", len(body), contentType)
}
+1 -1
View File
@@ -40,7 +40,7 @@ func TestSmokeLnwCommentBoundary(t *testing.T) {
if seriesURL == "" {
t.Skip("SMOKE_LNW_SERIES_URL unset")
}
if !fetchableSeriesURL("lightnovelworld", seriesURL) {
if !FetchableSeriesURL("lightnovelworld", seriesURL) {
t.Fatalf("%q is not a fetchable lightnovelworld series URL", seriesURL)
}
+10 -3
View File
@@ -41,7 +41,8 @@ type SeriesFilter struct {
// must never leave the store package — so the projection does not select it,
// and only the anonymous boolean in raisedByReaderAnswer crosses it.
const adminSeriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover_address,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at, s.force_poll_at`
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at, s.force_poll_at,
s.latest_corrected_at`
// raisedByReaderAnswer answers "did a Reader's report set this number" without
// naming which Reader. Kept apart from adminSeriesColumns so the column list —
@@ -72,8 +73,14 @@ type AdminSeries struct {
// meaning never asked. Pending is derived, never stored: a request is
// pending while ForcePollAt is newer than LatestCheckedAt.
ForcePollAt int64
// LatestCorrectedAt is the correction stamp (issue #149): non-zero means
// the Latest Chapter is the owner's, zero means never corrected. The
// provenance line (#152) derives from it, so the zero-means-never meaning
// is load-bearing.
LatestCorrectedAt int64
ReaderCount int
RaisedByReader bool // a Reader's report set LatestChapterNum
}
// SeriesPage is one page of the owner's filtered Series list plus the count
@@ -189,7 +196,7 @@ func (s *Store) SeriesPage(f SeriesFilter) (SeriesPage, error) {
`+where+`
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover_address,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at,
s.force_poll_at, s.latest_raised_by
s.force_poll_at, s.latest_corrected_at, s.latest_raised_by
`+having+`
ORDER BY s.latest_checked_at, s.site, s.series_id
LIMIT $`+strconv.Itoa(base+1)+` OFFSET $`+strconv.Itoa(base+2), args...)
@@ -264,7 +271,7 @@ func scanAdminSeries(scan func(...any) error) (AdminSeries, int, error) {
if err := scan(
&a.Site, &a.SeriesID, &a.Title, &a.SeriesURL, &a.CoverAddress,
&a.Kind, &a.LatestChapter, &latestChapterNum, &a.LatestCheckedAt,
&a.ForcePollAt,
&a.ForcePollAt, &a.LatestCorrectedAt,
&a.RaisedByReader, &a.ReaderCount, &total,
); err != nil {
return AdminSeries{}, 0, err
@@ -1,8 +1,15 @@
-- The Cover splits into two facts. `cover` keeps the third-party address the
-- bytes come from, which is what the acquisition path refetches and dedupes
-- on; `cover_address` is the content address of the bytes once they are
-- actually stored, and is what the wire's absolute URL is built from.
-- bytes come from, which is what the refetch path dedupes on; `cover_address`
-- is the content address of the bytes once they are actually stored, and is
-- what the wire's absolute URL is built from.
--
-- Empty `cover_address` therefore means "no Cover yet" rather than "a Cover
-- that 404s", which is the distinction the API and the UI both depend on.
--
-- The content address was originally the hex SHA-256 of the source URL
-- (ADR-0007). Since ADR-0014 it is the hex SHA-256 of the bytes themselves,
-- so a re-art behind the same URL is a new address. Rows written before
-- ADR-0014 keep their URL-derived addresses; they are never rehashed and heal
-- into byte addressing on their first forced replacement. Both derivations
-- share the 64-hex-digit shape, so the serving guard is unchanged.
ALTER TABLE series ADD COLUMN cover_address text NOT NULL DEFAULT '';
@@ -0,0 +1,4 @@
-- latest_corrected_at is the "the current Latest Chapter is the owner's" stamp
-- (#149). Written by the Correction; zeroed by every machine write of the
-- value. Zero means never corrected.
ALTER TABLE series ADD COLUMN latest_corrected_at bigint NOT NULL DEFAULT 0;
+204 -37
View File
@@ -16,6 +16,7 @@ import (
"strconv"
"strings"
"github.com/jackc/pgx/v5/pgconn"
_ "github.com/jackc/pgx/v5/stdlib"
)
@@ -60,10 +61,11 @@ type Bookmark struct {
// exists once no matter how many bookmarks point at it (ADR-0003).
//
// Title, SeriesURL and Cover are written once, at creation: a PUT naming an
// existing Series has them ignored, and only the backend's own Poll may change
// them. Kind and the latest-chapter fields are last-write-wins like the
// bookmark's own fields. Never serialized: the wire format is the flat
// Bookmark (ADR-0004).
// existing Series has them ignored, and only the backend's own Poll may
// change them. The one exception is SeriesURL, which the owner's
// SetSeriesURL may repair (issue #151). Kind and the latest-chapter fields
// are last-write-wins like the bookmark's own fields. Never serialized: the
// wire format is the flat Bookmark (ADR-0004).
type Series struct {
Site string
SeriesID string
@@ -704,67 +706,112 @@ func (s *Store) getCoverByAddress(address string) ([]byte, string, bool, error)
return body, contentType, true, nil
}
func (s *Store) putCover(sourceURL string, body []byte, contentType string) error {
func (s *Store) putCover(sourceURL string, body []byte, contentType string) (string, error) {
stored, ok := CoverContentType(contentType)
if !ok {
return fmt.Errorf("put cover %q: unsupported content type %q", sourceURL, contentType)
return "", fmt.Errorf("put cover %q: unsupported content type %q", sourceURL, contentType)
}
contentType = stored
address := coverSourceAddress(sourceURL)
address := CoverAddressForBytes(body)
relativePath := coverRelativePath(address)
coverPath := filepath.Join(s.coverDir, filepath.FromSlash(relativePath))
if err := os.MkdirAll(filepath.Dir(coverPath), 0o755); err != nil {
return fmt.Errorf("create cover shard: %w", err)
return "", fmt.Errorf("create cover shard: %w", err)
}
tmp, err := os.CreateTemp(filepath.Dir(coverPath), ".cover-*")
if err != nil {
return fmt.Errorf("create cover temp file: %w", err)
return "", fmt.Errorf("create cover temp file: %w", err)
}
tmpName := tmp.Name()
defer os.Remove(tmpName)
if _, err := tmp.Write(body); err != nil {
tmp.Close()
return fmt.Errorf("write cover temp file: %w", err)
return "", fmt.Errorf("write cover temp file: %w", err)
}
if err := tmp.Sync(); err != nil {
tmp.Close()
return fmt.Errorf("sync cover temp file: %w", err)
return "", fmt.Errorf("sync cover temp file: %w", err)
}
if err := tmp.Close(); err != nil {
return fmt.Errorf("close cover temp file: %w", err)
return "", fmt.Errorf("close cover temp file: %w", err)
}
if err := os.Link(tmpName, coverPath); err != nil && !errors.Is(err, fs.ErrExist) {
return fmt.Errorf("install cover file: %w", err)
return "", fmt.Errorf("install cover file: %w", err)
}
if _, err := s.db.Exec(`
INSERT INTO covers (address, path, content_type)
VALUES ($1, $2, $3)
ON CONFLICT (address) DO NOTHING`, address, relativePath, contentType); err != nil {
return fmt.Errorf("record cover %q: %w", address, err)
return "", fmt.Errorf("record cover %q: %w", address, err)
}
return address, nil
}
// ReclaimCover permanently removes a Cover nothing references: the sharded
// file first, the covers row last. A blank address is a no-op, and so is any
// address a Series row still points at — byte-identical artwork is one row by
// construction (ADR-0014), so reclaiming one Series' stranded bytes must not
// blank another's. The file goes first because the covers row is the handle:
// an interrupted run stays findable in SQL — covers rows unreferenced by any
// series cover_address — and re-running finishes the job, whereas deleting
// the row first would leave a file nothing names. A concurrent Forced Poll
// repointing a live Series at this address between the guard and the unlink
// is the repairable case: the missing file reads as ok=false and the next
// pass re-installs it. Failures are returned, never logged here — the caller
// logs and carries on — and a failed unlink leaves the row in place for a
// retry. A whole-table sweep, if ever wanted, is one SQL query over covers,
// not a tree walk and not this function.
func (s *Store) ReclaimCover(address string) error {
if address == "" {
return nil
}
var referenced int
err := s.db.QueryRow(`SELECT 1 FROM series WHERE cover_address = $1 LIMIT 1`, address).Scan(&referenced)
if err == nil {
return nil
}
if !errors.Is(err, sql.ErrNoRows) {
return fmt.Errorf("guard reclaim of cover %q: %w", address, err)
}
coverPath := filepath.Join(s.coverDir, filepath.FromSlash(coverRelativePath(address)))
if err := os.Remove(coverPath); err != nil && !errors.Is(err, fs.ErrNotExist) {
return fmt.Errorf("remove cover file %q: %w", address, err)
}
if _, err := s.db.Exec(`DELETE FROM covers WHERE address = $1`, address); err != nil {
return fmt.Errorf("delete cover row %q: %w", address, err)
}
return nil
}
// GetCover returns the immutable object addressed by its source URL. Missing
// GetCover returns the immutable object a source URL's own hash names. Rows
// written before byte addressing (ADR-0014) are the only ones that ever reach
// it; it hashes the URL, so a byte-addressed Cover is invisible to it. Missing
// files are reported with ok=false so callers can retry acquisition later.
func (s *Store) GetCover(sourceURL string) ([]byte, string, bool, error) {
return s.getCover(sourceURL)
}
// PutCover persists bytes under the source URL's content address. A later
// write for the same URL cannot replace the immutable object.
// PutCover persists bytes under their own content address (ADR-0014). A later
// write of the same bytes cannot replace the immutable object.
func (s *Store) PutCover(sourceURL string, body []byte, contentType string) error {
return s.putCover(sourceURL, body, contentType)
_, err := s.putCover(sourceURL, body, contentType)
return err
}
// CoverAddress is the content address bytes fetched from sourceURL are stored
// under. It is a pure function of the URL, so the acquisition path can name a
// Cover before it has the bytes.
func CoverAddress(sourceURL string) string { return coverSourceAddress(sourceURL) }
// CoverAddressForBytes is the content address body is stored under: the hex
// SHA-256 of the bytes, so identical artwork is one address and a re-art a
// new one. Legacy rows were addressed from their source URL instead and are
// never rehashed — both derivations coexist (ADR-0014).
func CoverAddressForBytes(body []byte) string {
sum := sha256.Sum256(body)
return hex.EncodeToString(sum[:])
}
// coverAddressRe is the shape of a stored address: the hex SHA-256 of a source
// URL. Request paths reach CoverByAddress, so the shape is checked before the
// value is ever turned into a filesystem path.
// coverAddressRe is the shape of a stored address: 64 lowercase hex digits —
// the hex SHA-256 of the cover bytes, or of the source URL for legacy rows
// (ADR-0014). Request paths reach CoverByAddress, so the shape is checked
// before the value is ever turned into a filesystem path; byte-derived
// addresses keep the same shape, so the guard is unchanged.
var coverAddressRe = regexp.MustCompile(`^[0-9a-f]{64}$`)
// CoverByAddress returns the immutable object at one content address. An
@@ -788,24 +835,68 @@ func (s *Store) CoverWireURL(address string) string {
return s.coverBaseURL + "/covers/" + address
}
// SetSeriesCover stores the bytes and points the Series at them, but only
// while the Series has no Cover: acquisition at creation and the poll both
// call this, and whichever arrives second must not overwrite the first. The
// bytes themselves are content-addressed and immutable, so storing them twice
// is free.
// SetSeriesCover stores the bytes and points the Series at their address, but
// only while the Series has no Cover: acquisition at creation and the poll
// both call this, and whichever arrives second must not overwrite the first.
// The bytes themselves are content-addressed and immutable, so storing them
// twice is free. See ReplaceSeriesCover for the write that may move a Cover
// once one exists (ADR-0014).
func (s *Store) SetSeriesCover(site, seriesID, sourceURL string, body []byte, contentType string) error {
if err := s.putCover(sourceURL, body, contentType); err != nil {
address, err := s.putCover(sourceURL, body, contentType)
if err != nil {
return err
}
if _, err := s.db.Exec(`
UPDATE series SET cover = $3, cover_address = $4
WHERE site = $1 AND series_id = $2 AND cover_address = ''`,
site, seriesID, sourceURL, coverSourceAddress(sourceURL)); err != nil {
site, seriesID, sourceURL, address); err != nil {
return fmt.Errorf("set cover for %q: %w", site+":"+seriesID, err)
}
return nil
}
// ReplaceSeriesCover stores the bytes and points the Series at their address
// whether or not one already exists, writing the current source URL alongside
// — the Forced Poll's installer and the only write that may move a Cover once
// one exists (ADR-0014). previous is the address the row held before the write
// ("" if it had none) and current the address of the bytes just stored; both
// are read and written in one transaction, so a concurrent replacement reports
// the exact displacement. previous == current means the Site served identical
// artwork, an honest no-op; otherwise previous is stranded — the row no
// longer points at it, and reclaiming its bytes is the caller's separate act
// (the poller's replace path calls ReclaimCover on it). This write itself
// removes nothing.
func (s *Store) ReplaceSeriesCover(site, seriesID, sourceURL string, body []byte, contentType string) (previous, current string, err error) {
current, err = s.putCover(sourceURL, body, contentType)
if err != nil {
return "", "", err
}
tx, err := s.db.Begin()
if err != nil {
return "", "", fmt.Errorf("begin replace cover for %q: %w", site+":"+seriesID, err)
}
defer tx.Rollback()
err = tx.QueryRow(`
SELECT cover_address FROM series
WHERE site = $1 AND series_id = $2 FOR UPDATE`,
site, seriesID).Scan(&previous)
if errors.Is(err, sql.ErrNoRows) {
previous = ""
} else if err != nil {
return "", "", fmt.Errorf("read cover for %q: %w", site+":"+seriesID, err)
}
if _, err := tx.Exec(`
UPDATE series SET cover = $3, cover_address = $4
WHERE site = $1 AND series_id = $2`,
site, seriesID, sourceURL, current); err != nil {
return "", "", fmt.Errorf("replace cover for %q: %w", site+":"+seriesID, err)
}
if err := tx.Commit(); err != nil {
return "", "", fmt.Errorf("commit cover replace for %q: %w", site+":"+seriesID, err)
}
return previous, current, nil
}
// List returns every bookmark of one reader, newest activity first.
// Series-owned fields are joined in, so each Bookmark reads back whole and
// flat (ADR-0004).
@@ -904,6 +995,11 @@ func (s *Store) Upsert(readerID int64, b Bookmark) (Bookmark, error) {
// xmax is zero only on a row this statement inserted, which is how a
// Series nobody had bookmarked before is told apart from one that already
// existed — DO UPDATE returns a row either way.
// latest_corrected_at is the one clause conditional on the value moving
// (#149): after a Correction a Reader's cached row holds the corrected
// number and resends it on the next Progress PUT, so unconditional
// zeroing would erase the fact while the value is still the owner's. The
// stamp survives a same-number PUT and dies the moment the number moves.
var created bool
if err := tx.QueryRow(`
INSERT INTO series (site, series_id, title, series_url, kind,
@@ -914,7 +1010,10 @@ func (s *Store) Upsert(readerID int64, b Bookmark) (Bookmark, error) {
ON CONFLICT (site, series_id) DO UPDATE SET
kind=excluded.kind,
latest_chapter=excluded.latest_chapter,
latest_chapter_num=excluded.latest_chapter_num
latest_chapter_num=excluded.latest_chapter_num,
latest_corrected_at = CASE
WHEN series.latest_chapter_num IS DISTINCT FROM excluded.latest_chapter_num
THEN 0 ELSE series.latest_corrected_at END
RETURNING xmax = 0`,
b.Site, b.SeriesID, b.Title, b.SeriesURL, b.Kind,
b.LatestChapter, latestNum).Scan(&created); err != nil {
@@ -984,6 +1083,37 @@ func (s *Store) Delete(readerID int64, key string) error {
return nil
}
// pgForeignKeyViolation is the SQLSTATE the driver surfaces when a Bookmark
// row refuses a Series delete (bookmarks_series_fk). pgconn exports no named
// constant for it, so the store names it here.
const pgForeignKeyViolation = "23503"
// ErrSeriesHasBookmarks is RemoveSeries' refusal: a Reader still holds the
// Series, so the owner's removal must not reach past that record. The
// delete is the check — no NOT EXISTS pre-check that can race the insert —
// and the driver's foreign-key violation is translated here so no driver
// type escapes the store (issue #155).
var ErrSeriesHasBookmarks = errors.New("series has bookmarks")
// RemoveSeries deletes one Series row by (site, series_id). It is refused
// while any Bookmark references the row; deleting an absent key is not an
// error, matching Delete. The caller owns the stranded Cover: read the row's
// cover_address before the delete and call ReclaimCover after it — the
// helper's guard cannot pass while the series row still points at the
// address, so the order is the sequence, not a preference.
func (s *Store) RemoveSeries(site, seriesID string) error {
if _, err := s.db.Exec(
`DELETE FROM series WHERE site = $1 AND series_id = $2`,
site, seriesID); err != nil {
var pgErr *pgconn.PgError
if errors.As(err, &pgErr) && pgErr.Code == pgForeignKeyViolation {
return ErrSeriesHasBookmarks
}
return fmt.Errorf("remove series %s:%s: %w", site, seriesID, err)
}
return nil
}
// RecordLanePass appends one pass and prunes every older row in the same
// transaction. retainBefore is supplied by the poller's clock.
func (s *Store) RecordLanePass(p LanePass, retainBefore int64) error {
@@ -1320,10 +1450,13 @@ func (s *Store) LatestCheckedAt(site, seriesID string) (int64, error) {
// SetLatestChapter records the newest chapter the poll found on a series page.
// The poller walks Series rather than Bookmarks, so this is a series-level
// write: the row is shared, and updating it once refreshes every bookmark that
// joins to it. Touching a missing series is not an error.
// joins to it. Touching a missing series is not an error. The correction stamp
// is zeroed unconditionally: checkOne only calls this when the number differs,
// so a second copy of the condition would drift (#149).
func (s *Store) SetLatestChapter(site, seriesID, label string, num float64) error {
if _, err := s.db.Exec(
`UPDATE series SET latest_chapter = $3, latest_chapter_num = $4
`UPDATE series SET latest_chapter = $3, latest_chapter_num = $4,
latest_corrected_at = 0
WHERE site = $1 AND series_id = $2`,
site, seriesID, label, num); err != nil {
return fmt.Errorf("set latest chapter %s:%s: %w", site, seriesID, err)
@@ -1331,8 +1464,42 @@ func (s *Store) SetLatestChapter(site, seriesID, label string, num float64) erro
return nil
}
// RecordSighting notes that a Reader's browser reported this Series' Latest
// Chapter, which is the half of a Sighting the client body cannot express
// SetSeriesURL stores the owner's repair for a Series' source address
// (issue #151): the one write that lifts the write-once rule documented on
// Series.SeriesURL. It is a store, not a verification — the caller has
// already passed the poller's fetch gate. The handler 404s on an unknown row
// before calling; the write itself is a plain single-column UPDATE like
// MarkLatestChecked.
func (s *Store) SetSeriesURL(site, seriesID, seriesURL string) error {
if _, err := s.db.Exec(
`UPDATE series SET series_url = $3 WHERE site = $1 AND series_id = $2`,
site, seriesID, seriesURL); err != nil {
return fmt.Errorf("set series url %s:%s: %w", site, seriesID, err)
}
return nil
}
// CorrectLatestChapter makes the Latest Chapter the owner's: one UPDATE
// carrying the number, the derived label and the correction stamp. The label
// shape is the poller's and the userscript's ("Chapter " + the number as
// printed), so chapterLeadIn strips it and the UI renders "Ch N" with no
// special case. latest_checked_at is not touched: a Correction is not a check.
// A raising Reader is cleared without judgement: the number is the owner's
// now, and no Sighting counter moves (spec #135).
func (s *Store) CorrectLatestChapter(site, seriesID string, num float64, at int64) error {
if _, err := s.db.Exec(
`UPDATE series SET
latest_chapter = $3,
latest_chapter_num = $4,
latest_corrected_at = $5,
latest_raised_by = NULL
WHERE site = $1 AND series_id = $2`,
site, seriesID, "Chapter "+strconv.FormatFloat(num, 'f', -1, 64), num, at); err != nil {
return fmt.Errorf("correct latest chapter %s:%s: %w", site, seriesID, err)
}
return nil
}
// (issue #103). It must be called *before* the Upsert that stores the reported
// value: the raise test compares against what is still on the row, and after
// the Upsert there is nothing left to compare with. A Series that does not
+562 -12
View File
@@ -5,6 +5,8 @@ import (
"crypto/sha256"
"database/sql"
"encoding/hex"
"errors"
"io/fs"
"os"
"path/filepath"
"strconv"
@@ -850,7 +852,7 @@ func TestSetSeriesCoverDoesNotOverwrite(t *testing.T) {
if err != nil || !ok {
t.Fatalf("Get = %v, %v", ok, err)
}
if want := "https://bookmarks.test/covers/" + CoverAddress(first); got.Cover != want {
if want := "https://bookmarks.test/covers/" + CoverAddressForBytes([]byte("first")); got.Cover != want {
t.Fatalf("Cover = %q, want the first one %q", got.Cover, want)
}
}
@@ -869,7 +871,7 @@ func TestCoverByAddress(t *testing.T) {
t.Fatalf("SetSeriesCover: %v", err)
}
body, contentType, ok, err := store.CoverByAddress(CoverAddress(source))
body, contentType, ok, err := store.CoverByAddress(CoverAddressForBytes([]byte("bytes")))
if err != nil || !ok {
t.Fatalf("CoverByAddress = %v, %v", ok, err)
}
@@ -877,8 +879,8 @@ func TestCoverByAddress(t *testing.T) {
t.Fatalf("CoverByAddress = %q, %q, want the stored bytes", body, contentType)
}
for _, address := range []string{"", "../../etc/passwd", "ZZ" + CoverAddress(source)[2:],
CoverAddress("never stored")} {
for _, address := range []string{"", "../../etc/passwd", "ZZ" + CoverAddressForBytes([]byte("bytes"))[2:],
CoverAddressForBytes([]byte("never stored"))} {
_, _, ok, err := store.CoverByAddress(address)
if err != nil || ok {
t.Fatalf("CoverByAddress(%q) = %v, %v, want a clean miss", address, ok, err)
@@ -913,7 +915,7 @@ func TestUpsertExistingSeriesIgnoresClientTitleCoverURL(t *testing.T) {
if err != nil {
t.Fatalf("Upsert: %v", err)
}
wantCover := "https://bookmarks.test/covers/" + CoverAddress(acquired)
wantCover := "https://bookmarks.test/covers/" + CoverAddressForBytes([]byte("bytes"))
if got.Title != "Solo Leveling" || got.SeriesURL != "https://asurascans.com/comics/solo" ||
got.Cover != wantCover {
t.Fatalf("stored = %+v, want original title/url/cover kept", got)
@@ -981,7 +983,7 @@ func TestDeleteKeepsSeriesRow(t *testing.T) {
if err != nil {
t.Fatalf("re-upsert: %v", err)
}
wantCover := "https://bookmarks.test/covers/" + CoverAddress(acquired)
wantCover := "https://bookmarks.test/covers/" + CoverAddressForBytes([]byte("bytes"))
if stored.Title != "Solo Leveling" || stored.Cover != wantCover {
t.Fatalf("re-bookmark = %+v, want title/cover from the surviving series row", stored)
}
@@ -1498,9 +1500,9 @@ func TestCoverPersistsAcrossReopen(t *testing.T) {
t.Fatalf("reopen: %v", err)
}
defer second.Close()
got, contentType, ok, err := second.GetCover(sourceURL)
got, contentType, ok, err := second.CoverByAddress(CoverAddressForBytes(body))
if err != nil {
t.Fatalf("GetCover: %v", err)
t.Fatalf("CoverByAddress: %v", err)
}
if !ok || !bytes.Equal(got, body) || contentType != "image/webp" {
t.Fatalf("stored cover = (%q, %q, %v), want (%q, image/webp, true)", got, contentType, ok, body)
@@ -1539,7 +1541,7 @@ func TestCoverIsContentAddressedOnFilesystem(t *testing.T) {
}
defer first.Close()
addressBytes := sha256.Sum256([]byte(sourceURL))
addressBytes := sha256.Sum256(body)
address := hex.EncodeToString(addressBytes[:])
wantPath := filepath.Join(address[:2], address[2:4], address)
@@ -1570,9 +1572,9 @@ func TestCoverStoreAcceptsAnySourceURL(t *testing.T) {
if err := s.PutCover(sourceURL, want, "image/jpeg"); err != nil {
t.Fatalf("PutCover: %v", err)
}
got, contentType, ok, err := s.GetCover(sourceURL)
got, contentType, ok, err := s.CoverByAddress(CoverAddressForBytes(want))
if err != nil {
t.Fatalf("GetCover: %v", err)
t.Fatalf("CoverByAddress: %v", err)
}
if !ok || !bytes.Equal(got, want) || contentType != "image/jpeg" {
t.Fatalf("GetCover = (%q, %q, %v), want (%q, image/jpeg, true)", got, contentType, ok, want)
@@ -1580,7 +1582,7 @@ func TestCoverStoreAcceptsAnySourceURL(t *testing.T) {
if err := s.PutCover("https://cdn.example/not-image", []byte("html"), "text/html"); err == nil {
t.Fatal("PutCover accepted a non-image")
}
if _, _, ok, err := s.GetCover("https://cdn.example/not-image"); err != nil || ok {
if _, _, ok, err := s.CoverByAddress(CoverAddressForBytes([]byte("html"))); err != nil || ok {
t.Fatalf("rejected cover = found %v, err %v; want missing", ok, err)
}
}
@@ -1899,3 +1901,551 @@ func TestDueForLatestCheckForcedDoesNotOverrideURLOrJoin(t *testing.T) {
t.Fatalf("due = %v, want neither the URL-less nor the orphan series", due)
}
}
// A Correction writes the number, the derived label and the stamp, clears the
// raising Reader, and never touches either Sighting counter or the check
// stamp — a Correction is not a check and never judges a Reader (#149).
func TestCorrectLatestChapterStampsClearsAndDoesNotTouchCheckOrMarks(t *testing.T) {
s := newTestStore(t)
other := secondReader(t, s)
seedForCheck(t, s, "asura:solo", "https://asurascans.com/comics/solo", 4321_000)
// A Reader raised the number, and carries a mark for it.
if err := s.RecordSighting(other, "asura", "solo", num2(3), 1000); err != nil {
t.Fatalf("RecordSighting: %v", err)
}
if _, err := s.db.Exec(`
UPDATE readers SET sighting_agreements = 5, sighting_disagreements = 2
WHERE id = $1`, other); err != nil {
t.Fatalf("mark reader: %v", err)
}
if err := s.CorrectLatestChapter("asura", "solo", 12.5, 9000); err != nil {
t.Fatalf("CorrectLatestChapter: %v", err)
}
var chapter string
var num float64
var stamp, checkedAt int64
var raisedBy any
if err := s.db.QueryRow(`
SELECT latest_chapter, latest_chapter_num, latest_corrected_at,
latest_checked_at, latest_raised_by
FROM series WHERE site = 'asura' AND series_id = 'solo'`).
Scan(&chapter, &num, &stamp, &checkedAt, &raisedBy); err != nil {
t.Fatalf("read back: %v", err)
}
if chapter != "Chapter 12.5" {
t.Errorf("latest_chapter = %q, want the derived label %q", chapter, "Chapter 12.5")
}
if num != 12.5 {
t.Errorf("latest_chapter_num = %v, want 12.5", num)
}
if stamp != 9000 {
t.Errorf("latest_corrected_at = %d, want 9000", stamp)
}
if checkedAt != 4321_000 {
t.Errorf("latest_checked_at = %d, want the untouched 4321000", checkedAt)
}
if raisedBy != nil {
t.Errorf("latest_raised_by = %v, want the attribution cleared", raisedBy)
}
readers, err := s.Readers()
if err != nil {
t.Fatalf("Readers: %v", err)
}
for _, r := range readers {
if r.ID == other && (r.Agreements != 5 || r.Disagreements != 2) {
t.Errorf("raising reader's marks = %+v, want agreements 5, disagreements 2 unchanged", r)
}
}
}
// The stamp follows the number (spec #135): an Upsert resending the corrected
// value — a Reader's cached row after a correction — keeps it, and an Upsert
// that actually moves the number kills it. Unconditional zeroing would erase
// the fact while the value is still the owner's; that is the whole point of
// the clause.
func TestUpsertCorrectionStampFollowsTheNumber(t *testing.T) {
s := newTestStore(t)
base := Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", Kind: KindManga,
SeriesURL: "https://asurascans.com/comics/solo", UpdatedAt: 1000,
}
if _, err := s.Upsert(s.OwnerID(), base); err != nil {
t.Fatalf("seed: %v", err)
}
if err := s.CorrectLatestChapter("asura", "solo", 5, 9000); err != nil {
t.Fatalf("CorrectLatestChapter: %v", err)
}
// Same number back: the value is still the owner's.
same := base
same.LatestChapterNum = num2(5)
if _, err := s.Upsert(s.OwnerID(), same); err != nil {
t.Fatalf("same-number upsert: %v", err)
}
if got := s.latestCorrectedAt(t, "asura", "solo"); got != 9000 {
t.Fatalf("stamp after same-number Upsert = %d, want 9000 kept", got)
}
// A different number: a machine (or a Reader) wrote the value.
moved := base
moved.LatestChapterNum = num2(7)
if _, err := s.Upsert(s.OwnerID(), moved); err != nil {
t.Fatalf("moved upsert: %v", err)
}
if got := s.latestCorrectedAt(t, "asura", "solo"); got != 0 {
t.Fatalf("stamp after moved Upsert = %d, want zeroed", got)
}
}
// The poller's chapter setter zeroes the stamp unconditionally: checkOne only
// calls it when the number differs, so the condition lives upstream and a
// second copy here would drift (#149).
func TestSetLatestChapterZeroesCorrectionStamp(t *testing.T) {
s := newTestStore(t)
seedForCheck(t, s, "asura:solo", "https://asurascans.com/comics/solo", 0)
if err := s.CorrectLatestChapter("asura", "solo", 5, 9000); err != nil {
t.Fatalf("CorrectLatestChapter: %v", err)
}
if err := s.SetLatestChapter("asura", "solo", "Chapter 6", 6); err != nil {
t.Fatalf("SetLatestChapter: %v", err)
}
if got := s.latestCorrectedAt(t, "asura", "solo"); got != 0 {
t.Fatalf("stamp after a machine write = %d, want zeroed", got)
}
}
// latestCorrectedAt reads the stamp column for the assertion above.
func (s *Store) latestCorrectedAt(t *testing.T, site, seriesID string) int64 {
t.Helper()
var stamp int64
if err := s.db.QueryRow(
`SELECT latest_corrected_at FROM series WHERE site = $1 AND series_id = $2`,
site, seriesID).Scan(&stamp); err != nil {
t.Fatalf("read stamp: %v", err)
}
return stamp
}
// num2 boxes a chapter number for the Bookmark fields that take a pointer.
func num2(f float64) *float64 { return &f }
// --- Cover addressing (ADR-0014): the address is the bytes' SHA-256 ---
// The address is what makes a re-art visible at all, so the same bytes must
// always name the same address and different bytes different ones — and the
// address must keep the 64-hex-digit shape CoverByAddress's guard still checks
// before any request-supplied value becomes a filesystem path.
func TestCoverAddressForBytesIsDeterministicAndDistinct(t *testing.T) {
first := CoverAddressForBytes([]byte("art"))
again := CoverAddressForBytes([]byte("art"))
other := CoverAddressForBytes([]byte("artwork"))
if first != again {
t.Fatalf("same bytes gave %q then %q, want one address", first, again)
}
if first == other {
t.Fatalf("different bytes gave the same address %q", first)
}
if !coverAddressRe.MatchString(first) {
t.Fatalf("address %q is not the 64-hex-digit shape the serving guard checks", first)
}
}
// ReplaceSeriesCover is the forced-replacement installer: it moves a Cover
// whether or not one exists, writes the source URL alongside it, and reports
// the three outcomes the Forced Poll has to tell apart.
func TestReplaceSeriesCover(t *testing.T) {
store := newTestStore(t)
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
// A blank Cover: previous is "", and the row points at the new bytes.
previous, current, err := store.ReplaceSeriesCover("asura", "solo",
"https://cdn.asurascans.com/covers/solo.webp", []byte("first-art"), "image/webp")
if err != nil {
t.Fatalf("ReplaceSeriesCover on a blank: %v", err)
}
if previous != "" {
t.Fatalf("previous on a blank = %q, want empty", previous)
}
if want := CoverAddressForBytes([]byte("first-art")); current != want {
t.Fatalf("current = %q, want %q", current, want)
}
if sr := readSeries(t, store, "asura", "solo"); sr.CoverAddress != current ||
sr.Cover != "https://cdn.asurascans.com/covers/solo.webp" {
t.Fatalf("series after blank fill = %+v, want the new address and source URL", sr)
}
// A re-art: previous is the stranded address, current the new one.
previous, current, err = store.ReplaceSeriesCover("asura", "solo",
"https://cdn.asurascans.com/covers/solo-rebrand.webp", []byte("second-art"), "image/jpeg")
if err != nil {
t.Fatalf("ReplaceSeriesCover over a filled Cover: %v", err)
}
if want := CoverAddressForBytes([]byte("first-art")); previous != want {
t.Fatalf("previous = %q, want the replaced address %q", previous, want)
}
if want := CoverAddressForBytes([]byte("second-art")); current != want {
t.Fatalf("current = %q, want %q", current, want)
}
if sr := readSeries(t, store, "asura", "solo"); sr.CoverAddress != current ||
sr.Cover != "https://cdn.asurascans.com/covers/solo-rebrand.webp" {
t.Fatalf("series after replacement = %+v, want the new address and source URL", sr)
}
// The replaced bytes stay served under their old address: ReplaceSeriesCover
// itself reclaims nothing, reclamation is the caller's separate act (#154).
if _, _, ok, err := store.CoverByAddress(CoverAddressForBytes([]byte("first-art"))); err != nil || !ok {
t.Fatalf("superseded bytes = found %v, err %v, want still served", ok, err)
}
// The Site is serving the same artwork again: previous == current is the
// honest no-op the caller reports as "unchanged".
previous, current, err = store.ReplaceSeriesCover("asura", "solo",
"https://cdn.asurascans.com/covers/solo-rebrand.webp", []byte("second-art"), "image/jpeg")
if err != nil {
t.Fatalf("ReplaceSeriesCover over identical bytes: %v", err)
}
if previous != current {
t.Fatalf("identical bytes: previous = %q, current = %q, want one address", previous, current)
}
if want := CoverAddressForBytes([]byte("second-art")); current != want {
t.Fatalf("current = %q, want %q", current, want)
}
}
// Rows written before byte addressing hold the hash of their source URL and
// are never rehashed: GetCover — the poller's heal path — keeps resolving
// them through coverSourceAddress.
func TestGetCoverResolvesLegacyURLDerivedAddress(t *testing.T) {
store := newTestStore(t)
source := "https://cdn.example/legacy.jpg"
legacy := coverSourceAddress(source)
relativePath := coverRelativePath(legacy)
coverPath := filepath.Join(store.coverDir, filepath.FromSlash(relativePath))
if err := os.MkdirAll(filepath.Dir(coverPath), 0o755); err != nil {
t.Fatalf("create shard dir: %v", err)
}
if err := os.WriteFile(coverPath, []byte("legacy-bytes"), 0o644); err != nil {
t.Fatalf("write legacy file: %v", err)
}
if _, err := store.db.Exec(
`INSERT INTO covers (address, path, content_type) VALUES ($1, $2, $3)`,
legacy, relativePath, "image/jpeg"); err != nil {
t.Fatalf("plant legacy row: %v", err)
}
body, contentType, ok, err := store.GetCover(source)
if err != nil || !ok {
t.Fatalf("GetCover on a legacy row = %v, %v, want found", ok, err)
}
if string(body) != "legacy-bytes" || contentType != "image/jpeg" {
t.Fatalf("legacy cover = (%q, %q), want the planted bytes", body, contentType)
}
}
// SetSeriesURL is the one write that lifts the write-once rule of
// Series.SeriesURL (issue #151): a client PUT naming an existing Series still
// has its new URL dropped, yet the owner's repair lands where the Upsert
// would have ignored it.
func TestSetSeriesURLWritesWhereUpsertIgnores(t *testing.T) {
store := newTestStore(t)
base := Bookmark{
Key: "asura:solo", Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", SeriesURL: "https://asurascans.com/comics/solo",
UpdatedAt: 1000,
}
if _, err := store.Upsert(store.OwnerID(), base); err != nil {
t.Fatalf("seed: %v", err)
}
// A client PUT naming the existing Series is refused: the row is shared,
// so the stored URL stands.
base.SeriesURL = "https://evil.example/solo"
if got, err := store.Upsert(store.OwnerID(), base); err != nil {
t.Fatalf("Upsert: %v", err)
} else if got.SeriesURL != "https://asurascans.com/comics/solo" {
t.Fatalf("Upsert stored %q, want the original URL untouched", got.SeriesURL)
}
// The owner's repair writes where the Upsert would have ignored it.
repair := "https://asurascans.com/comics/solo-renumbered"
if err := store.SetSeriesURL("asura", "solo", repair); err != nil {
t.Fatalf("SetSeriesURL: %v", err)
}
sr := readSeries(t, store, "asura", "solo")
if sr.SeriesURL != repair {
t.Fatalf("stored URL = %q, want %q", sr.SeriesURL, repair)
}
}
// --- Cover byte reclamation (issue #154): one guarded helper, file first ---
// coverShardPath is the on-disk location of one address's bytes, built the
// same way getCoverByAddress reads them.
func coverShardPath(t *testing.T, s *Store, address string) string {
t.Helper()
return filepath.Join(s.coverDir, filepath.FromSlash(coverRelativePath(address)))
}
// ReclaimCover removes a Cover nothing references: the row alone is not the
// point — the sharded file must be gone too, because the file is the reclaimed
// disk space.
func TestReclaimCoverRemovesUnreferencedBytes(t *testing.T) {
store := newTestStore(t)
seedForCheck(t, store, "asura:solo", "https://asurascans.com/comics/solo", 0)
if err := store.SetSeriesCover("asura", "solo", "https://cdn.example/covers/old.jpg", []byte("old-art"), "image/jpeg"); err != nil {
t.Fatalf("put cover: %v", err)
}
old := CoverAddressForBytes([]byte("old-art"))
if _, _, err := store.ReplaceSeriesCover("asura", "solo", "https://cdn.example/covers/new.jpg", []byte("new-art"), "image/jpeg"); err != nil {
t.Fatalf("replace cover: %v", err)
}
if err := store.ReclaimCover(old); err != nil {
t.Fatalf("ReclaimCover: %v", err)
}
if _, err := os.Stat(coverShardPath(t, store, old)); !errors.Is(err, fs.ErrNotExist) {
t.Fatalf("sharded path after reclaim = %v, want fs.ErrNotExist", err)
}
if _, _, ok, err := store.CoverByAddress(old); err != nil || ok {
t.Fatalf("covers row after reclaim = found %v err %v, want gone", ok, err)
}
// The live Cover survives the reclamation of the stranded one.
if body, _, ok, err := store.CoverByAddress(CoverAddressForBytes([]byte("new-art"))); err != nil || !ok || string(body) != "new-art" {
t.Fatalf("new bytes after reclaim = found %v err %v, want still served", ok, err)
}
}
// The guard is the whole design: byte-identical artwork is one covers row by
// construction (ADR-0014), so a second Series pointing at the address must
// keep the bytes — reclaiming one Series' stranded artwork may not blank
// another's.
func TestReclaimCoverSparesReferencedAddress(t *testing.T) {
store := newTestStore(t)
seedForCheck(t, store, "asura:solo", "https://asurascans.com/comics/solo", 0)
const src = "https://cdn.example/covers/shared.jpg"
addr := CoverAddressForBytes([]byte("shared-art"))
if err := store.SetSeriesCover("asura", "solo", src, []byte("shared-art"), "image/jpeg"); err != nil {
t.Fatalf("put cover: %v", err)
}
// One Series pointing at the address is enough for the guard.
if err := store.ReclaimCover(addr); err != nil {
t.Fatalf("ReclaimCover on a referenced address: %v", err)
}
if body, _, ok, err := store.CoverByAddress(addr); err != nil || !ok || string(body) != "shared-art" {
t.Fatalf("bytes after no-op = found %v err %v, want still served", ok, err)
}
if _, err := os.Stat(coverShardPath(t, store, addr)); err != nil {
t.Fatalf("sharded file after no-op: %v, want present", err)
}
// A second Series serving identical bytes shares the row by construction.
seedForCheck(t, store, "asura:second", "https://asurascans.com/comics/second", 0)
if err := store.SetSeriesCover("asura", "second", src, []byte("shared-art"), "image/jpeg"); err != nil {
t.Fatalf("share cover: %v", err)
}
if err := store.ReclaimCover(addr); err != nil {
t.Fatalf("ReclaimCover on a shared address: %v", err)
}
if body, _, ok, err := store.CoverByAddress(addr); err != nil || !ok || string(body) != "shared-art" {
t.Fatalf("shared bytes after no-op = found %v err %v, want still served", ok, err)
}
if _, err := os.Stat(coverShardPath(t, store, addr)); err != nil {
t.Fatalf("sharded file after shared no-op: %v, want present", err)
}
}
// A blank address is the wire value for "no Cover" (ADR-0007), never a
// reclaimable one.
func TestReclaimCoverBlankAddressIsNoOp(t *testing.T) {
store := newTestStore(t)
if err := store.ReclaimCover(""); err != nil {
t.Fatalf("ReclaimCover(\"\") = %v, want nil", err)
}
}
// An interrupted reclamation is the state the file-first order exists for:
// the row is the handle, so the unreferenced-covers query finds the torn
// Cover and re-running ReclaimCover finishes the job — a missing file is
// "already gone", which counts as success.
func TestReclaimCoverInterruptedRunIsFindableAndFinishes(t *testing.T) {
store := newTestStore(t)
seedForCheck(t, store, "asura:solo", "https://asurascans.com/comics/solo", 0)
if err := store.SetSeriesCover("asura", "solo", "https://cdn.example/covers/torn.jpg", []byte("torn-art"), "image/jpeg"); err != nil {
t.Fatalf("put cover: %v", err)
}
torn := CoverAddressForBytes([]byte("torn-art"))
if _, _, err := store.ReplaceSeriesCover("asura", "solo", "https://cdn.example/covers/new.jpg", []byte("new-art"), "image/jpeg"); err != nil {
t.Fatalf("replace cover: %v", err)
}
if err := os.Remove(coverShardPath(t, store, torn)); err != nil {
t.Fatalf("unlink mid-reclamation: %v", err)
}
var found string
err := store.db.QueryRow(`
SELECT address FROM covers c
WHERE NOT EXISTS (SELECT 1 FROM series s WHERE s.cover_address = c.address)
LIMIT 1`).Scan(&found)
if err != nil || found != torn {
t.Fatalf("unreferenced-covers query = (%q, %v), want the torn row %q", found, err, torn)
}
if err := store.ReclaimCover(torn); err != nil {
t.Fatalf("re-run over a missing file: %v", err)
}
if _, _, ok, err := store.CoverByAddress(torn); err != nil || ok {
t.Fatalf("row after re-run = found %v err %v, want gone", ok, err)
}
}
// A failed file removal is the one state that is not self-cleaning: the
// covers row must survive so a retry can finish the job, and the store
// returns the error rather than logging — each caller logs and carries on,
// so the failure has no user-facing surface.
func TestReclaimCoverFailedUnlinkKeepsRow(t *testing.T) {
store := newTestStore(t)
seedForCheck(t, store, "asura:solo", "https://asurascans.com/comics/solo", 0)
if err := store.SetSeriesCover("asura", "solo", "https://cdn.example/covers/stuck.jpg", []byte("stuck-art"), "image/jpeg"); err != nil {
t.Fatalf("put cover: %v", err)
}
stuck := CoverAddressForBytes([]byte("stuck-art"))
if _, _, err := store.ReplaceSeriesCover("asura", "solo", "https://cdn.example/covers/other.jpg", []byte("other-art"), "image/jpeg"); err != nil {
t.Fatalf("replace cover: %v", err)
}
// Make the unlink fail: the sharded path becomes a non-empty directory,
// which os.Remove refuses.
shard := coverShardPath(t, store, stuck)
if err := os.Remove(shard); err != nil {
t.Fatalf("clear file: %v", err)
}
if err := os.Mkdir(shard, 0o755); err != nil {
t.Fatalf("replace file with dir: %v", err)
}
if err := os.WriteFile(filepath.Join(shard, "blob"), []byte("x"), 0o644); err != nil {
t.Fatalf("fill dir: %v", err)
}
if err := store.ReclaimCover(stuck); err == nil {
t.Fatal("ReclaimCover over an unremovable file = nil, want the error")
}
var one int
if err := store.db.QueryRow(`SELECT 1 FROM covers WHERE address = $1`, stuck).Scan(&one); err != nil {
t.Fatal("covers row after failed unlink is gone; want it left for a retry")
}
}
// RemoveSeries is the orphan removal (#155): one Series, one delete, refused
// by the database while any Bookmark points at it. The store translates the
// foreign-key violation into its own sentinel so no driver type escapes, and
// the caller reaps the stranded Cover through ReclaimCover.
func TestRemoveSeriesRemovesOrphanAndReclaimsCover(t *testing.T) {
store := newTestStore(t)
seedForCheck(t, store, "asura:solo", "https://asurascans.com/comics/solo", 0)
if err := store.Delete(store.OwnerID(), "asura:solo"); err != nil {
t.Fatalf("orphan the series: %v", err)
}
if err := store.SetSeriesCover("asura", "solo", "https://cdn.example/covers/old.jpg", []byte("old-art"), "image/jpeg"); err != nil {
t.Fatalf("put cover: %v", err)
}
addr := CoverAddressForBytes([]byte("old-art"))
if err := store.RemoveSeries("asura", "solo"); err != nil {
t.Fatalf("RemoveSeries: %v", err)
}
// The caller's sequence: the row is deleted first, then the address is
// reclaimed — the guard cannot pass while the row still points at it.
if err := store.ReclaimCover(addr); err != nil {
t.Fatalf("ReclaimCover: %v", err)
}
var one int
if err := store.db.QueryRow(`SELECT 1 FROM series WHERE site = $1 AND series_id = $2`, "asura", "solo").Scan(&one); err != sql.ErrNoRows {
t.Fatalf("series row after remove = %v, want sql.ErrNoRows", err)
}
if _, _, ok, err := store.CoverByAddress(addr); err != nil || ok {
t.Fatalf("covers row after remove = found %v err %v, want gone", ok, err)
}
if _, err := os.Stat(coverShardPath(t, store, addr)); !errors.Is(err, fs.ErrNotExist) {
t.Fatalf("sharded file after remove = %v, want fs.ErrNotExist", err)
}
}
// The refusal is the whole point of the sentinel: a Series a Reader still
// holds is not removed, its row is untouched and its Cover keeps serving.
func TestRemoveSeriesRefusedWhileBookmarked(t *testing.T) {
store := newTestStore(t)
seedForCheck(t, store, "asura:solo", "https://asurascans.com/comics/solo", 0)
if err := store.SetSeriesCover("asura", "solo", "https://cdn.example/covers/kept.jpg", []byte("kept-art"), "image/jpeg"); err != nil {
t.Fatalf("put cover: %v", err)
}
addr := CoverAddressForBytes([]byte("kept-art"))
if err := store.RemoveSeries("asura", "solo"); !errors.Is(err, ErrSeriesHasBookmarks) {
t.Fatalf("RemoveSeries on a bookmarked series = %v, want ErrSeriesHasBookmarks", err)
}
var held int
if err := store.db.QueryRow(`SELECT 1 FROM series WHERE site = $1 AND series_id = $2`, "asura", "solo").Scan(&held); err != nil {
t.Fatal("series row after refusal is gone; want it untouched")
}
if body, _, ok, err := store.CoverByAddress(addr); err != nil || !ok || string(body) != "kept-art" {
t.Fatalf("cover after refusal = found %v err %v, want still served", ok, err)
}
}
// A Series sharing its Cover address with a second Series is removed while
// the artwork stays readable through CoverByAddress: the series row stops
// referencing the address first, so ReclaimCover's guard passes for this
// caller without touching the shared bytes (ADR-0014).
func TestRemoveSeriesSparesSharedCover(t *testing.T) {
store := newTestStore(t)
seedForCheck(t, store, "asura:solo", "https://asurascans.com/comics/solo", 0)
seedForCheck(t, store, "asura:second", "https://asurascans.com/comics/second", 0)
if err := store.Delete(store.OwnerID(), "asura:solo"); err != nil {
t.Fatalf("orphan solo: %v", err)
}
if err := store.Delete(store.OwnerID(), "asura:second"); err != nil {
t.Fatalf("orphan second: %v", err)
}
const src = "https://cdn.example/covers/shared.jpg"
if err := store.SetSeriesCover("asura", "solo", src, []byte("shared-art"), "image/jpeg"); err != nil {
t.Fatalf("put cover on solo: %v", err)
}
if err := store.SetSeriesCover("asura", "second", src, []byte("shared-art"), "image/jpeg"); err != nil {
t.Fatalf("put cover on second: %v", err)
}
addr := CoverAddressForBytes([]byte("shared-art"))
if err := store.RemoveSeries("asura", "solo"); err != nil {
t.Fatalf("RemoveSeries: %v", err)
}
// The guard spares the shared bytes even though this caller reclaims.
if err := store.ReclaimCover(addr); err != nil {
t.Fatalf("ReclaimCover over a shared address: %v", err)
}
if body, _, ok, err := store.CoverByAddress(addr); err != nil || !ok || string(body) != "shared-art" {
t.Fatalf("shared bytes after remove = found %v err %v, want still served", ok, err)
}
if _, err := os.Stat(coverShardPath(t, store, addr)); err != nil {
t.Fatalf("sharded file after remove: %v, want present", err)
}
var one int
if err := store.db.QueryRow(`SELECT 1 FROM series WHERE site = $1 AND series_id = $2`, "asura", "second").Scan(&one); err != nil {
t.Fatal("the second series row vanished with the first")
}
}
// Deleting an absent key removes nothing and is not an error, matching the
// Delete precedent — the handler's own lookups turn the absent case into the
// 404 before the store ever sees it.
func TestRemoveSeriesMissingKeyIsCleanNoOp(t *testing.T) {
store := newTestStore(t)
if err := store.RemoveSeries("asura", "ghost"); err != nil {
t.Fatalf("RemoveSeries on a missing key = %v, want nil", err)
}
}
+3
View File
@@ -47,6 +47,9 @@ func (h *Handler) adminRoutes() []adminRoute {
{"GET /admin/series", h.adminSeries},
{"GET /admin/series/{key}", h.adminSeriesDetail},
{"POST /admin/series/{key}/poll", h.adminSeriesPoll},
{"POST /admin/series/{key}/latest", h.adminSeriesCorrectLatest},
{"POST /admin/series/{key}/series-url", h.adminSeriesSetURL},
{"POST /admin/series/{key}/remove", h.adminSeriesRemove},
{"POST /admin/lanes/{site}/pause", h.adminLanePause},
{"POST /admin/lanes/{site}/resume", h.adminLaneResume},
{"GET /ui/admin/lanes", h.uiLanes},
+275 -1
View File
@@ -1,8 +1,10 @@
package web
import (
"errors"
"fmt"
"log"
"math"
"net/http"
"net/url"
"strconv"
@@ -47,7 +49,6 @@ var seriesFilterOrder = []string{
}
// seriesListView is the Series list page's data. The template renders strings
// and flags, and every judgement about what a value means is made here.
type seriesListView struct {
Filters []seriesFilterOption
Sites []string
@@ -56,6 +57,9 @@ type seriesListView struct {
FilterLabel string
Rows []seriesRowView
Total int
// OOB marks the out-of-band copy of the heading the removal answer
// carries; on the page itself it is false (issue #155).
OOB bool
// KindBoth / KindManga / KindNovel are the Library segment links, and
// PrevHref / NextHref the pager's, all carrying the active filter, Site
// and Kind so narrowing never drops state.
@@ -100,6 +104,12 @@ type seriesRowView struct {
CanPoll bool
Pending bool
Requested string // "requested 3m ago", rendered only while pending
// CanRemove is the Remove control's visibility: only a Series no Reader
// holds can be removed, so the owner is never offered a button that the
// database will always refuse (issue #155). RemovalRefused marks the one
// raced answer: the row stays and says a fresh Bookmark caught the press.
CanRemove bool
RemovalRefused bool
}
// adminSeries renders the filterable, bookmarkable Series list: filter, Site,
@@ -172,6 +182,269 @@ func (h *Handler) adminSeriesPoll(w http.ResponseWriter, r *http.Request) {
h.render(w, http.StatusOK, "series-row", seriesRow(a, band, time.Now()))
}
// adminSeriesCorrectLatest is the Latest Chapter correction: the owner types
// one number and the Series' Latest Chapter becomes it, stamped as a
// Correction. The number must be a finite float greater than zero — a
// non-numeric, zero or negative value answers 400 and never reaches the
// store, because a bad value would become every Reader's problem. The press
// answers with the freshly rendered meta fragment, so the figures describe
// the state after the press. The owner gate is the route's, not this
// handler's; the body is capped like the API path caps its bodies.
func (h *Handler) adminSeriesCorrectLatest(w http.ResponseWriter, r *http.Request) {
site, seriesID, ok := strings.Cut(r.PathValue("key"), ":")
if !ok || site == "" || seriesID == "" {
http.Error(w, "bad series key", http.StatusBadRequest)
return
}
r.Body = http.MaxBytesReader(w, r.Body, 1<<16)
if err := r.ParseForm(); err != nil {
http.Error(w, "invalid form", http.StatusBadRequest)
return
}
num, err := strconv.ParseFloat(r.PostFormValue("chapter"), 64)
if err != nil || math.IsNaN(num) || math.IsInf(num, 0) || num <= 0 {
http.Error(w, "chapter must be a finite number greater than zero", http.StatusBadRequest)
return
}
if _, found, err := h.adminSeriesByKey(site, seriesID); err != nil {
log.Printf("series correction %s: %v", site+":"+seriesID, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
} else if !found {
http.NotFound(w, r)
return
}
if err := h.store.CorrectLatestChapter(site, seriesID, num, time.Now().UnixMilli()); err != nil {
log.Printf("series correction %s: %v", site+":"+seriesID, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
// Re-read after the write: the answer must describe the state after the
// press, so the marker reads "corrected just now".
a, found, err := h.adminSeriesByKey(site, seriesID)
if err != nil {
log.Printf("series correction %s: %v", site+":"+seriesID, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if !found {
http.NotFound(w, r)
return
}
h.render(w, http.StatusOK, "series-detail-meta", h.seriesDetailView(a))
}
// adminSeriesSetURL is the series URL repair: the owner types one address
// and the Series' Poll fetches it from then on, verified by the same gate
// the poller uses before it fetches anything — a URL failing
// latest.FetchableSeriesURL answers 400 and never reaches the store. The
// repair is a store, not a verification: it performs no outbound fetch, and
// the owner presses Check now afterwards. This lifts the write-once rule of
// Series.SeriesURL for the owner only — a Reader's PUT is still ignored. The
// owner gate is the route's, not this handler's; the body is capped like the
// API path caps its bodies; the key is validated here — a malformed key is a
// 400 and an unknown one a 404.
func (h *Handler) adminSeriesSetURL(w http.ResponseWriter, r *http.Request) {
site, seriesID, ok := strings.Cut(r.PathValue("key"), ":")
if !ok || site == "" || seriesID == "" {
http.Error(w, "bad series key", http.StatusBadRequest)
return
}
r.Body = http.MaxBytesReader(w, r.Body, 1<<16)
if err := r.ParseForm(); err != nil {
http.Error(w, "invalid form", http.StatusBadRequest)
return
}
seriesURL := r.PostFormValue("series_url")
if !latest.FetchableSeriesURL(site, seriesURL) {
http.Error(w, "series URL must be an https address on this site's host", http.StatusBadRequest)
return
}
if _, found, err := h.adminSeriesByKey(site, seriesID); err != nil {
log.Printf("series url %s: %v", site+":"+seriesID, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
} else if !found {
http.NotFound(w, r)
return
}
if err := h.store.SetSeriesURL(site, seriesID, seriesURL); err != nil {
log.Printf("series url %s: %v", site+":"+seriesID, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
// Re-read after the write: the answer must describe the state after the
// press, like the correction's answer does.
a, found, err := h.adminSeriesByKey(site, seriesID)
if err != nil {
log.Printf("series url %s: %v", site+":"+seriesID, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if !found {
http.NotFound(w, r)
return
}
h.render(w, http.StatusOK, "series-detail-meta", h.seriesDetailView(a))
}
// seriesListHeadView is the list heading's data. The template renders it
// inline at the top of the Series list and out of band in the removal answer
// (OOB true, like the chrome partials' OOB flag): the count and the filter
// label are one fact (issue #155).
type seriesListHeadView struct {
Total int
FilterLabel string
OOB bool
}
// adminSeriesRemove is the orphan removal: one Series, one delete, refused by
// the database while any Bookmark exists (translated by the store, never a
// driver error on the page). The owner gate is the route's, not this
// handler's; the body is capped like the API path caps its bodies; the key is
// validated here — a malformed key is a 400 and an unknown one a 404.
//
// The Cover is read from the row before the delete and reclaimed after it:
// ReclaimCover's guard cannot pass while a series row still points at the
// address, so the order is the sequence, not a preference. A reclamation
// failure is not a removal failure — the row is gone and the covers row
// survives for a retry; the handler logs and answers success, because the
// failure has no user-facing surface.
//
// Two callers, one handler, branched on HX-Target like adminSeriesPoll. The
// detail page's remove answers with a navigation — to the No-Readers list on
// success, back to the detail page when a fresh Bookmark raced the press,
// where the new count is visible. The list row's answers with the removed
// row's fragment and the heading re-rendered with the fresh count out of
// band; HX-Reswap deletes the row through the same button that swaps the
// refusal back in, and the count query runs over the press's own filter
// state, so the heading describes the list the owner is looking at.
func (h *Handler) adminSeriesRemove(w http.ResponseWriter, r *http.Request) {
site, seriesID, ok := strings.Cut(r.PathValue("key"), ":")
if !ok || site == "" || seriesID == "" {
http.Error(w, "bad series key", http.StatusBadRequest)
return
}
r.Body = http.MaxBytesReader(w, r.Body, 1<<16)
if err := r.ParseForm(); err != nil {
http.Error(w, "invalid form", http.StatusBadRequest)
return
}
// The row's Cover address is read before the delete because the delete is
// what makes it reclaimable.
a, found, err := h.adminSeriesByKey(site, seriesID)
if err != nil {
log.Printf("series remove %s: %v", site+":"+seriesID, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if !found {
http.NotFound(w, r)
return
}
cover := a.CoverAddress
if err := h.store.RemoveSeries(site, seriesID); err != nil {
if errors.Is(err, store.ErrSeriesHasBookmarks) {
// A Bookmark landed between the owner's read and the press: the
// row stays, answered at its new count with the fact spelled
// out — never a 500, and never a deleted row.
if r.Header.Get("HX-Target") == "detail-meta" {
seriesRemoveNavigation(w, r, "/admin/series/"+site+":"+seriesID)
return
}
fresh, found, err := h.adminSeriesByKey(site, seriesID)
if err != nil {
log.Printf("series remove %s: %v", site+":"+seriesID, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if !found {
// A second press removed it while this one was refused; the
// row has nothing left to say.
http.NotFound(w, r)
return
}
band := 0
if r.PostFormValue("band") == "1" {
band = 1
}
row := seriesRow(fresh, band, time.Now())
row.RemovalRefused = true
h.render(w, http.StatusOK, "series-row", row)
return
}
log.Printf("series remove %s: %v", site+":"+seriesID, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if err := h.store.ReclaimCover(cover); err != nil {
log.Printf("series remove %s: reclaim cover: %v", site+":"+seriesID, err)
}
if r.Header.Get("HX-Target") == "detail-meta" {
seriesRemoveNavigation(w, r, "/admin/series?filter="+store.SeriesFilterNoReaders)
return
}
// The list answer: the removed row's fragment, plus the heading
// re-rendered with the fresh count. HX-Reswap deletes the row through the
// same button that swaps the refusal back in. The count query failing
// does not undo the removal — log it and answer the row alone.
w.Header().Set("HX-Reswap", "delete")
band := 0
if r.PostFormValue("band") == "1" {
band = 1
}
h.render(w, http.StatusOK, "series-row", seriesRow(a, band, time.Now()))
if head, err := h.seriesListHeadView(r); err != nil {
log.Printf("series remove %s: %v", site+":"+seriesID, err)
} else {
h.render(w, http.StatusOK, "series-list-head", head)
}
}
// seriesListHeadView is the list heading with the count as it stands after a
// removal: the same filter, Site and Kind the press's row carried (the list
// row's button hx-includes the filterbar), so the figure describes the list
// the owner is looking at — the All filter and an unknown one stay the
// absent case. The count is the store's window total, one query.
func (h *Handler) seriesListHeadView(r *http.Request) (seriesListHeadView, error) {
filter := r.PostFormValue("filter")
if _, ok := seriesFilterLabels[filter]; !ok {
filter = store.SeriesFilterAll
}
site := r.PostFormValue("site")
kind := r.PostFormValue("kind")
if kind != store.KindManga && kind != store.KindNovel {
kind = ""
}
data, err := h.store.SeriesPage(store.SeriesFilter{
Site: site,
Kind: kind,
Name: filter,
Cutoff: time.Now().Add(-ownerWindow).UnixMilli(),
Page: 1,
})
if err != nil {
return seriesListHeadView{}, err
}
return seriesListHeadView{
Total: data.Total,
FilterLabel: seriesFilterLabels[filter],
OOB: true,
}, nil
}
// seriesRemoveNavigation answers a removal from the detail page. htmx gets a
// full navigation (HX-Redirect): a bare 303 would be followed by the request
// and the landing page swapped into the press's target, so the header is the
// redirect htmx can see; plain clients get the 303 the ticket names.
func seriesRemoveNavigation(w http.ResponseWriter, r *http.Request, to string) {
if r.Header.Get("HX-Request") != "" {
w.Header().Set("HX-Redirect", to)
return
}
http.Redirect(w, r, to, http.StatusSeeOther)
}
// seriesListView assembles one Series list view from the request's query
// string. An unknown filter value is the absent All case, never an error: the
// select's options are not the only way this URL can be reached.
@@ -296,6 +569,7 @@ func seriesRow(a store.AdminSeries, i int, now time.Time) seriesRowView {
Readers: a.ReaderCount,
Band: i%2 == 1,
CanPoll: canPoll,
CanRemove: a.ReaderCount == 0,
Pending: pending,
Requested: requested,
}
+48 -1
View File
@@ -24,14 +24,26 @@ type seriesDetailView struct {
Cover string
Chapter string // Latest Chapter number, or "—" before the first capture
Checked string // how long ago the poller last checked, or "never"
// URL is the stored source address, prefilled into the repair input —
// the one stored string this page renders back into a form (issue #151).
URL string
Readers int
// Corrected is the correction marker's text, "" while no Correction
// stands: "corrected <age> ago" — the copy that says the value is the
// owner's, and it dies with the stamp (a machine write of the number).
Corrected string
// Provenance is the actor class behind the current Chapter: "correction",
// "sighting" or "machine read"; "" while the Series was never read, when
// the line is not rendered. Derived from the same anonymous stamps the
// marks above read — no Reader identity crosses here.
Provenance string
// Marks, one per hygiene fact, rendered only while it holds.
Unpollable bool // no SeriesURL to fetch
NoCover bool
Orphan bool // no Reader holds the Series
SightingRaised bool // a Reader's Sighting set the Latest Chapter
// Poll is the Check now control and the pending marker (issue #146): the
// same derivation and visibility as the list row. CanPoll is false on a
// Series with no page to fetch and on an orphan; Pending is derived —
@@ -40,6 +52,10 @@ type seriesDetailView struct {
CanPoll bool
Pending bool
Requested string
// CanRemove is the Remove control's visibility (issue #155): only a
// Series no Reader holds can be removed, so the owner is never offered a
// button the database will always refuse.
CanRemove bool
}
// adminSeriesDetail renders one Series' page, keyed by the composite
@@ -103,12 +119,14 @@ func (h *Handler) seriesDetailView(a store.AdminSeries) seriesDetailView {
Kind: a.Kind,
Title: a.Title,
Cover: h.store.CoverWireURL(a.CoverAddress),
URL: a.SeriesURL,
Readers: a.ReaderCount,
Unpollable: a.SeriesURL == "",
NoCover: a.CoverAddress == "",
Orphan: a.ReaderCount == 0,
SightingRaised: a.RaisedByReader,
CanPoll: canPoll,
CanRemove: a.ReaderCount == 0,
Pending: pending,
Requested: requested,
}
@@ -122,5 +140,34 @@ func (h *Handler) seriesDetailView(a store.AdminSeries) seriesDetailView {
} else {
v.Checked = since(time.Now(), time.UnixMilli(a.LatestCheckedAt))
}
v.Corrected = correctedAge(time.Now(), a.LatestCorrectedAt)
// Provenance: the actor class behind the current number, evaluated in the
// order the classes outrank one another — the owner's stamp, which a
// Correction leaves standing and a machine write clears (issue #149); a
// raising Reader, which a Correction drops; then any check stamp at all.
// An Acquisition reads as a machine read because it stamps
// latest_checked_at exactly as a Poll does, so the two are
// indistinguishable the moment it finishes; telling them apart would need
// the column this project declines to add (spec #135), and the one
// actionable case — acquired once, never read again — is already the
// unchecked filter.
if a.LatestCorrectedAt != 0 {
v.Provenance = "correction"
} else if a.RaisedByReader {
v.Provenance = "sighting"
} else if a.LatestCheckedAt != 0 {
v.Provenance = "machine read"
}
return v
}
// correctedAge is the correction marker's text: "corrected <age> ago" while
// the stamp is set, "" when zero — zero means never corrected, and the marker
// must not read as history once a machine wrote the number.
func correctedAge(now time.Time, at int64) string {
if at == 0 {
return ""
}
return "corrected " + since(now, time.UnixMilli(at))
}
@@ -0,0 +1,34 @@
package web
import (
"testing"
"bookmarkmanager/backend/internal/store"
)
// seriesDetailView derives the provenance line from the three stamps the
// admin projection already carries: the correction stamp outranks a raising
// Reader, which outranks a check stamp, and a Series with none of the three
// renders no line at all — it was never read, and no actor class is true of
// it. Acquisition stamps latest_checked_at exactly as a Poll does, so an
// acquired value lands in the same "machine read" class (#152).
func TestSeriesDetailViewProvenance(t *testing.T) {
cases := []struct {
name string
a store.AdminSeries
want string
}{
{"correction stamp", store.AdminSeries{LatestCorrectedAt: 1}, "correction"},
{"raising reader only", store.AdminSeries{RaisedByReader: true}, "sighting"},
{"check stamp only", store.AdminSeries{LatestCheckedAt: 1}, "machine read"},
{"correction outranks sighting", store.AdminSeries{LatestCorrectedAt: 1, RaisedByReader: true}, "correction"},
{"none of the three", store.AdminSeries{}, ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := (&Handler{}).seriesDetailView(tc.a).Provenance; got != tc.want {
t.Fatalf("Provenance = %q, want %q", got, tc.want)
}
})
}
}
+28
View File
@@ -653,6 +653,13 @@
margin-left: auto;
}
.admin-sheet .tbl.series .row-msg {
grid-column: 1 / -1;
margin-top: 6px;
color: var(--danger-soft);
font: 400 13px/1.4 var(--font-body);
}
.admin-sheet .detail-back {
display: inline-block;
margin: 18px 0 0;
@@ -720,6 +727,27 @@
margin-top: 8px;
}
.admin-sheet .dform input {
min-width: 0;
padding: 8px 10px;
border: 1px solid var(--field-line);
background: var(--ink);
color: var(--paper);
font: 500 14px var(--font-mono);
outline: none;
}
/* Focus follows the chapter form's idiom — paper, not heat: a red border on
a valid number field reads as "invalid". */
.admin-sheet .dform input:focus {
border-color: var(--paper);
}
.admin-sheet .dform .hint {
margin: 0;
color: var(--mute-2);
font: 500 12px/1.4 var(--font-mono);
letter-spacing: .04em;
}
.admin-sheet .pausebar {
display: flex;
align-items: center;
@@ -1,8 +1,9 @@
{{/* Per-Series page: one address per Series, keyed "<site>:<series_id>" so the
list row is one hop from it. Everything here is a Series-level fact plus
the anonymous Reader count. The Check now control lands in its own .dform
below the (empty) .detail-grid; the pending marker rides the meta line
with the other marks. */}}
the anonymous Reader count. Check now lands in its own .dform below the
.detail-grid; the correction form is the grid's first column and the URL
repair the second (issue #151). The pending and corrected markers ride the
meta line with the other marks. */}}
{{define "series-detail"}}
<a class="ghost detail-back" href="/admin/series">← Series</a>
<h1 class="detail-title">{{.Title}}</h1>
@@ -10,22 +11,48 @@
{{if .Cover}}<div class="cover"><img src="{{.Cover}}" alt="" loading="lazy"></div>
{{else}}<div class="cover"></div>{{end}}
{{template "series-detail-meta" .}}
<div class="detail-grid"></div>
<div class="detail-grid">
<form class="dform" hx-post="/admin/series/{{.Key}}/latest" hx-target="#detail-meta" hx-swap="outerHTML">
<h3>Correct latest chapter</h3>
<p class="hint">The next successful Poll overwrites this value.</p>
<div class="field">
<input type="number" name="chapter" step="any" placeholder="{{.Chapter}}" required>
<button type="submit" class="ghost">Set</button>
</div>
</form>
<form class="dform" hx-post="/admin/series/{{.Key}}/series-url" hx-target="#detail-meta" hx-swap="outerHTML">
<h3>Repair series URL</h3>
<p class="hint">The Poll fetches this address — storing is not verifying it. A Site-wide host change is a SQL migration, not two hundred forms.</p>
<div class="field">
<input type="url" name="series_url" value="{{.URL}}" required>
<button type="submit" class="ghost">Set</button>
</div>
</form>
</div>
{{if .CanPoll}}
<div class="dform">
<div class="field"><a class="ghost act" hx-post="/admin/series/{{.Key}}/poll" hx-target="#detail-meta" hx-swap="outerHTML" href="#">Check now</a></div>
</div>
{{end}}
{{if .CanRemove}}
<div class="dform">
<div class="field"><button class="ghost danger" hx-post="/admin/series/{{.Key}}/remove" hx-target="#detail-meta" hx-confirm="Removes this series and its stored cover. No Reader has it bookmarked; one re-bookmarking it recreates the row.">Remove</button></div>
</div>
{{end}}
{{end}}
{{/* series-detail-meta is the meta line, and the answer a Check now press on
the detail page swaps into its place: the same marks, re-rendered after
the stamp so the pending marker shows. */}}
{{/* series-detail-meta is the meta line, and the answer a Check now or
correction press on the detail page swaps into its place: the same marks,
re-rendered after the stamp so the pending and corrected markers — and
the provenance line beside the number — describe the value they sit
next to. */}}
{{define "series-detail-meta"}}
<div class="detail-meta" id="detail-meta">
<span>ch {{.Chapter}}</span>
{{if .Provenance}}<span>{{.Provenance}}</span>{{end}}
<span>checked {{.Checked}}</span>
<span>{{.Readers}} readers</span>
{{if .Corrected}}<span class="mark">{{.Corrected}}</span>{{end}}
{{if .Pending}}<span class="mark">{{.Requested}}</span>{{end}}
{{if .Unpollable}}<span class="mark">unpollable</span>{{end}}
{{if .NoCover}}<span class="mark">no cover</span>{{end}}
@@ -4,7 +4,7 @@
that can be bookmarked: the two selects submit the GET form, and the
Library segment links and the pager preserve the filter and Site. */}}
{{define "series-list"}}
<form class="filterbar" method="get" action="/admin/series">
<form class="filterbar" id="filterbar" method="get" action="/admin/series">
<input type="hidden" name="kind" value="{{.Kind}}">
<label class="fsel"><span>Show</span><select name="filter" onchange="this.form.submit()">
{{range .Filters}}<option value="{{.Name}}"{{if .Selected}} selected{{end}}>{{.Label}} ({{.Count}})</option>{{end}}
@@ -19,7 +19,7 @@
<a href="{{.KindNovel}}"{{if eq .Kind "novel"}} class="active"{{end}}>novels</a>
</span>
</form>
<div class="listhead">{{.Total}} series <span class="lbl">· <em>{{.FilterLabel}}</em></span></div>
{{template "series-list-head" .}}
{{if .Rows}}
<div class="tbl series">
<div class="thead"><span>Site</span><span class="c-ch">Ch</span><span>Checked</span><span class="c-rd">Readers</span><span>Notes</span><span></span></div>
@@ -48,6 +48,16 @@
<span>{{.Age}}</span>
<span class="c-rd">{{.Readers}}</span>
<span class="c-note">{{range .Notes}}<span class="mark">{{.}}</span>{{end}}{{if .More}}<span class="mark mark-faint">+{{.More}}</span>{{end}}</span>
<span class="c-act">{{if .CanPoll}}<a class="ghost act" hx-post="/admin/series/{{.Key}}/poll" hx-target="closest .trow" hx-swap="outerHTML" hx-vals='{"band":{{if .Band}}1{{else}}0{{end}}}' href="#">Check now</a>{{end}}</span>
<span class="c-act">{{if .CanPoll}}<a class="ghost act" hx-post="/admin/series/{{.Key}}/poll" hx-target="closest .trow" hx-swap="outerHTML" hx-vals='{"band":{{if .Band}}1{{else}}0{{end}}}' href="#">Check now</a>{{end}}{{if .CanRemove}}<button class="ghost danger" hx-post="/admin/series/{{.Key}}/remove" hx-target="closest .trow" hx-swap="outerHTML" hx-include="#filterbar" hx-vals='{"band":{{if .Band}}1{{else}}0{{end}}}' hx-confirm="Removes this series and its stored cover. No Reader has it bookmarked; one re-bookmarking it recreates the row.">Remove</button>{{end}}</span>
{{if .RemovalRefused}}<span class="row-msg">a Reader has bookmarked this Series again</span>{{end}}
</div>
{{end}}
{{/* series-list-head is the list's heading — the count and the label are one
fact. The removal answer renders it out of band (the OOB flag, like the
chrome partials) so the heading never lies past the row that made it,
and inline here it is the page's own heading. The id is the OOB swap's
hook; hx-swap-oob sits on the element the answer carries. */}}
{{define "series-list-head"}}
<div class="listhead" id="series-listhead"{{if .OOB}} hx-swap-oob="true"{{end}}>{{.Total}} series <span class="lbl">· <em>{{.FilterLabel}}</em></span></div>
{{end}}
+548 -8
View File
@@ -2557,18 +2557,21 @@ func TestSeriesListRowShape(t *testing.T) {
if !strings.Contains(body, `class="c-site site-asura"`) {
t.Errorf("the site cell lacks its site class:\n%s", body)
}
// The action cell carries the Check now control on pollable rows and is
// empty on the orphan (no URL, no Readers — a button that can never do
// anything is not offered), and no Remove control or confirm row renders
// in this batch.
if !strings.Contains(body, `<span class="c-act"></span>`) {
t.Errorf("the orphan's action cell is not present and empty:\n%s", body)
// The action cell carries the Check now control on pollable rows and the
// Remove control on the orphan (#155) — a Series no Reader holds can be
// removed, so the orphan's cell is never empty. No confirm row renders
// in this batch: the confirm gate is htmx's own, not a toggled cell.
if !strings.Contains(body, `<span class="c-act">`) {
t.Errorf("the action cell is not present:\n%s", body)
}
if got := strings.Count(body, "Check now"); got != 2 {
t.Errorf("Check now control count = %d, want 2 (only the two pollable rows):\n%s", got, body)
}
if strings.Contains(body, "Remove") || strings.Contains(body, "confirm-row") {
t.Errorf("a Remove control or confirm row renders in this batch:\n%s", body)
if got := strings.Count(body, ">Remove<"); got != 1 {
t.Errorf("Remove control count = %d, want 1 (only the orphan):\n%s", got, body)
}
if strings.Contains(body, "confirm-row") {
t.Errorf("a confirm row renders in this batch:\n%s", body)
}
// No ember: the new-chapter signal stays off the admin surface. Scoped to
// the page content — the shell's brand mark legitimately wears the ember
@@ -2722,6 +2725,60 @@ func TestAdminSeriesDetailRendersMarks(t *testing.T) {
}
}
// The provenance line beside the chapter names the actor class behind the
// value — "machine read" for a checked Series, "correction" for the owner's
// stamp, "sighting" for a Reader-raised one — and appears nowhere in the
// rendered Series list: an actor class is context for the Series the owner is
// already looking at, never a population to sweep (#152).
func TestAdminSeriesDetailProvenanceLine(t *testing.T) {
router, st := newWebTestServer(t, testConfig())
seed(t, st, store.Bookmark{
Key: "asura:machine", Site: "asura", SeriesID: "machine",
Title: "Machine", SeriesURL: "https://asurascans.com/series/machine",
Kind: "manga", LatestChapter: "45", LatestChapterNum: floatPtr(45),
})
if err := st.MarkLatestChecked("asura", "machine", time.Now().Add(-time.Hour).UnixMilli()); err != nil {
t.Fatalf("MarkLatestChecked: %v", err)
}
seed(t, st, store.Bookmark{
Key: "asura:hand", Site: "asura", SeriesID: "hand",
Title: "Hand", SeriesURL: "https://asurascans.com/series/hand",
Kind: "manga", LatestChapter: "12", LatestChapterNum: floatPtr(12),
})
if err := st.CorrectLatestChapter("asura", "hand", 13, time.Now().UnixMilli()); err != nil {
t.Fatalf("CorrectLatestChapter: %v", err)
}
seed(t, st, store.Bookmark{
Key: "demonic:raised", Site: "demonic", SeriesID: "raised",
Title: "Raised", SeriesURL: "https://demonicscans.org/series/raised",
Kind: "manga",
})
if err := st.RecordSighting(st.OwnerID(), "demonic", "raised", floatPtr(7), time.Now().UnixMilli()); err != nil {
t.Fatalf("RecordSighting: %v", err)
}
for _, tc := range []struct{ key, want string }{
{"asura:machine", "machine read"},
{"asura:hand", "correction"},
{"demonic:raised", "sighting"},
} {
body := seriesDetailPage(t, router, st, tc.key)
if !strings.Contains(body, "<span>"+tc.want+"</span>") {
t.Errorf("detail %s lacks the %q provenance line:\n%s", tc.key, tc.want, body)
}
}
listBody := adminSeriesPage(t, router, st, "")
for _, word := range []string{"machine read", "correction", "sighting"} {
if strings.Contains(listBody, "<span>"+word+"</span>") {
t.Errorf("Series list carries a %q provenance line:\n%s", word, listBody)
}
}
}
// A well-formed key naming no row is a 404, and so is a key with no ":",
// an empty Site or an empty SeriesID — the detail page never answers 500 for
// an address nobody can reach.
@@ -3238,3 +3295,486 @@ func TestSeriesPollCapsBody(t *testing.T) {
t.Errorf("an oversized body still stamped the request:\n%s", body)
}
}
// The correction route validates at the boundary: a non-numeric, zero,
// negative or non-finite chapter answers 400 and never reaches the store, and
// a finite number greater than zero stores the number, the derived label and
// the stamp. The answer is the freshly rendered meta fragment, so the figures
// describe the state after the press (#149).
func TestCorrectLatestChapterRoute(t *testing.T) {
st, dsn := newTestStoreURL(t)
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
seedSeriesRow(t, st, db, seriesRowSeed{
key: "asura:solo", url: "u", checkedAt: 9000, bookmarks: 1, latestNum: floatPtr(3),
})
router := newRouter(st, testConfig())
cookie := sessionCookie(t, st)
for _, body := range []string{
"chapter=abc", "chapter=", "chapter=0", "chapter=-1", "chapter=NaN", "chapter=Inf",
} {
req := httptest.NewRequest(http.MethodPost, "/admin/series/asura:solo/latest", strings.NewReader(body))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(cookie)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusBadRequest {
t.Errorf("POST latest with body %q: status = %d, want 400", body, rr.Code)
}
}
// Nothing reached the store: the seeded number stands, unstamped.
var num float64
var stamp int64
if err := db.QueryRow(`
SELECT latest_chapter_num, latest_corrected_at
FROM series WHERE site = 'asura' AND series_id = 'solo'`).
Scan(&num, &stamp); err != nil {
t.Fatalf("read back: %v", err)
}
if num != 3 || stamp != 0 {
t.Fatalf("after 400s the row is num %v, stamp %d; want 3, 0", num, stamp)
}
// A good press stores the number, the derived label and the stamp, and
// answers with the meta fragment describing the state after the press.
req := httptest.NewRequest(http.MethodPost, "/admin/series/asura:solo/latest", strings.NewReader("chapter=12.5"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(cookie)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("POST latest status = %d, want 200 (body %s)", rr.Code, rr.Body.String())
}
body := rr.Body.String()
if !strings.Contains(body, `id="detail-meta"`) {
t.Errorf("correction answer is not the meta fragment:\n%s", body)
}
if !strings.Contains(body, `<span class="mark">corrected `) {
t.Errorf("correction answer lacks the fresh corrected marker:\n%s", body)
}
var label string
if err := db.QueryRow(`
SELECT latest_chapter, latest_chapter_num, latest_corrected_at
FROM series WHERE site = 'asura' AND series_id = 'solo'`).
Scan(&label, &num, &stamp); err != nil {
t.Fatalf("read back: %v", err)
}
if label != "Chapter 12.5" || num != 12.5 {
t.Errorf("stored = %q, %v; want the derived label and 12.5", label, num)
}
if stamp == 0 {
t.Error("stamp = 0, want the correction stamp written")
}
}
// The detail page offers the one-input correction with the plain copy, and
// the corrected marker rides the meta line while the stamp is set — then
// disappears the moment a machine writes the number (#149).
func TestAdminSeriesDetailCorrectionMarker(t *testing.T) {
st, dsn := newTestStoreURL(t)
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
seedSeriesRow(t, st, db, seriesRowSeed{key: "asura:solo", url: "u", checkedAt: 9000, bookmarks: 1})
router := newRouter(st, testConfig())
cookie := sessionCookie(t, st)
body := seriesDetailPage(t, router, st, "asura:solo")
for _, want := range []string{
`name="chapter"`,
`hx-post="/admin/series/asura:solo/latest"`,
"The next successful Poll overwrites this value.",
} {
if !strings.Contains(body, want) {
t.Errorf("detail page lacks %q:\n%s", want, body)
}
}
if strings.Contains(body, "corrected ") {
t.Errorf("uncorrected detail already carries the marker:\n%s", body)
}
// The press lands the marker on the meta line.
req := httptest.NewRequest(http.MethodPost, "/admin/series/asura:solo/latest", strings.NewReader("chapter=7"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(cookie)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("POST latest status = %d, want 200", rr.Code)
}
body = seriesDetailPage(t, router, st, "asura:solo")
if !strings.Contains(body, `<span class="mark">corrected `) {
t.Errorf("detail page lacks the corrected marker after the press:\n%s", body)
}
if !strings.Contains(body, "ch 7") {
t.Errorf("detail page does not show the corrected number:\n%s", body)
}
// A machine write (the poller's setter) kills the marker.
if err := st.SetLatestChapter("asura", "solo", "Chapter 8", 8); err != nil {
t.Fatalf("SetLatestChapter: %v", err)
}
body = seriesDetailPage(t, router, st, "asura:solo")
if strings.Contains(body, "corrected ") {
t.Errorf("marker survives a machine write:\n%s", body)
}
if !strings.Contains(body, "ch 8") {
t.Errorf("detail page does not show the machine-written number:\n%s", body)
}
}
// The series URL repair validates with the poller's own fetch gate and
// answers 400 before anything reaches the store; a URL that passes the gate
// is stored where an Upsert would have ignored it. The request performs no
// outbound fetch — no fetcher is ever constructed on this path (the web
// router has no fetcher seam at all, and the handler only calls the store),
// so "storing is not verifying" is enforced by construction (#151).
func TestSeriesURLRepairRoute(t *testing.T) {
st, dsn := newTestStoreURL(t)
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
seedSeriesRow(t, st, db, seriesRowSeed{
key: "asura:solo", url: "https://asurascans.com/comics/solo", checkedAt: 9000, bookmarks: 1,
})
router := newRouter(st, testConfig())
cookie := sessionCookie(t, st)
// A URL the gate refuses — foreign host, http scheme, host of another
// Site — answers 400 and never reaches the store.
for _, body := range []string{
"series_url=https://evil.example/solo",
"series_url=http://asurascans.com/stories/solo",
"series_url=https://kagane.to/series/solo",
"series_url=",
} {
req := httptest.NewRequest(http.MethodPost, "/admin/series/asura:solo/series-url", strings.NewReader(body))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(cookie)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusBadRequest {
t.Errorf("POST series-url with body %q: status = %d, want 400", body, rr.Code)
}
}
capReq := httptest.NewRequest(http.MethodPost, "/admin/series/asura:solo/series-url",
strings.NewReader("series_url=https://asurascans.com/stories/"+strings.Repeat("a", 1<<17)))
capReq.Header.Set("Content-Type", "application/x-www-form-urlencoded")
capReq.AddCookie(cookie)
capRR := httptest.NewRecorder()
router.ServeHTTP(capRR, capReq)
if capRR.Code != http.StatusBadRequest {
t.Errorf("POST series-url with an oversized body: status = %d, want 400", capRR.Code)
}
var stored string
if err := db.QueryRow(`SELECT series_url FROM series WHERE site = 'asura' AND series_id = 'solo'`).
Scan(&stored); err != nil {
t.Fatalf("read back: %v", err)
}
if stored != "https://asurascans.com/comics/solo" {
t.Fatalf("after 400s the stored URL = %q, want the seeded one untouched", stored)
}
// A URL that passes the gate lands, and the press answers with the meta
// fragment just like the other detail-page actions.
repair := "https://asurascans.com/stories/solo-renumbered"
req := httptest.NewRequest(http.MethodPost, "/admin/series/asura:solo/series-url",
strings.NewReader("series_url="+repair))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(cookie)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("POST series-url status = %d, want 200 (body %s)", rr.Code, rr.Body.String())
}
if body := rr.Body.String(); !strings.Contains(body, `id="detail-meta"`) {
t.Errorf("repair answer is not the meta fragment:\n%s", body)
}
if err := db.QueryRow(`SELECT series_url FROM series WHERE site = 'asura' AND series_id = 'solo'`).
Scan(&stored); err != nil {
t.Fatalf("read back: %v", err)
}
if stored != repair {
t.Fatalf("stored URL = %q, want %q", stored, repair)
}
}
// The detail page offers the repair input prefilled with the stored address,
// states the honest limit — a Site-wide host change is a SQL migration, not a
// per-Series form — and a stored string renders back into the input escaped
// (issue #151).
func TestAdminSeriesDetailRepairForm(t *testing.T) {
st, dsn := newTestStoreURL(t)
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
seedSeriesRow(t, st, db, seriesRowSeed{
key: "asura:solo", url: "https://asurascans.com/stories/solo", bookmarks: 1,
})
seedSeriesRow(t, st, db, seriesRowSeed{
key: "asura:evil", url: `https://asurascans.com/x"><script>alert(1)</script>`, bookmarks: 1,
})
router := newRouter(st, testConfig())
body := seriesDetailPage(t, router, st, "asura:solo")
for _, want := range []string{
`name="series_url"`,
`hx-post="/admin/series/asura:solo/series-url"`,
`value="https://asurascans.com/stories/solo"`,
"A Site-wide host change", "SQL migration",
} {
if !strings.Contains(body, want) {
t.Errorf("detail page lacks %q:\n%s", want, body)
}
}
// The stored value that is markup stays markup in the input's value
// attribute, never executable HTML.
body = seriesDetailPage(t, router, st, "asura:evil")
if strings.Contains(body, `<script>alert(1)</script>`) {
t.Errorf("repair input renders stored URL unescaped:\n%s", body)
}
if !strings.Contains(body, `value="https://asurascans.com/x&#34;&gt;&lt;script&gt;alert(1)&lt;/script&gt;"`) {
t.Errorf("repair input does not carry the escaped stored URL:\n%s", body)
}
}
// The Remove control is offered only to the owner, and only on a Series no
// Reader holds: on the orphan's list row and on the orphan's detail page,
// nowhere else (#155).
func TestSeriesRemoveRendersOnlyOnOrphans(t *testing.T) {
st, dsn := newTestStoreURL(t)
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
seedSeriesRow(t, st, db, seriesRowSeed{key: "asura:ok", url: "u", checkedAt: 9000, bookmarks: 1})
seedSeriesRow(t, st, db, seriesRowSeed{key: "asura:orphan", url: "u", checkedAt: 9000, bookmarks: 0})
router := newRouter(st, testConfig())
body := adminSeriesPage(t, router, st, "")
if got := strings.Count(body, ">Remove<"); got != 1 {
t.Errorf("list offers Remove %d times, want 1 (only the orphan):\n%s", got, body)
}
for _, tc := range []struct {
key string
want bool
}{
{"asura:ok", false},
{"asura:orphan", true},
} {
body := seriesDetailPage(t, router, st, tc.key)
if got := strings.Contains(body, ">Remove<"); got != tc.want {
t.Errorf("%s detail offers Remove = %v, want %v", tc.key, got, tc.want)
}
}
}
// A removal from the list answers with the removed row's fragment and the
// heading re-rendered out of band with the fresh count: the row and the
// count are one fact. The row's press carries the list's filter state, so
// the count describes the list the owner is looking at, and the HX-Reswap
// header deletes the row through the same button that swaps the refusal in.
func TestRemoveFromListAnswersRowAndFreshHeading(t *testing.T) {
st, dsn := newTestStoreURL(t)
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
seedSeriesRow(t, st, db, seriesRowSeed{key: "asura:a", url: "u", checkedAt: 9000, bookmarks: 0})
seedSeriesRow(t, st, db, seriesRowSeed{key: "asura:b", url: "u", checkedAt: 9000, bookmarks: 0})
seedSeriesRow(t, st, db, seriesRowSeed{key: "asura:held", url: "u", checkedAt: 9000, bookmarks: 1})
router := newRouter(st, testConfig())
req := httptest.NewRequest(http.MethodPost, "/admin/series/asura:a/remove",
strings.NewReader("filter=no_readers&band=0"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("removal status = %d, want 200", rr.Code)
}
if got := rr.Header().Get("HX-Reswap"); got != "delete" {
t.Errorf("response does not ask htmx to delete the row (HX-Reswap = %q)", got)
}
body := rr.Body.String()
if !strings.Contains(body, "Title of asura:a") {
t.Errorf("answer does not carry the removed row's fragment:\n%s", body)
}
if !strings.Contains(body, `hx-swap-oob="true"`) ||
!strings.Contains(body, "1 series") || !strings.Contains(body, "No Readers") {
t.Errorf("answer does not re-render the heading out of band with the fresh count:\n%s", body)
}
// Gone from the store, gone from the list, and the heading lies no longer.
var one int
if err := db.QueryRow(`SELECT 1 FROM series WHERE site = 'asura' AND series_id = 'a'`).Scan(&one); err != sql.ErrNoRows {
t.Fatalf("series row after removal = %v, want sql.ErrNoRows", err)
}
body = adminSeriesPage(t, router, st, "?filter=no_readers")
if strings.Contains(body, "Title of asura:a") || !strings.Contains(body, "1 series") {
t.Errorf("list after removal is not the fresh view:\n%s", body)
}
}
// A removal from the detail page navigates to the No-Readers list: htmx gets
// a full navigation (HX-Redirect — a 303 would be followed by the request
// and the list page swapped into the press's target), plain clients the 303
// the ticket names, to the wire filter the orphan list actually is.
func TestRemoveFromDetailRedirectsToNoReadersList(t *testing.T) {
st, dsn := newTestStoreURL(t)
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
seedSeriesRow(t, st, db, seriesRowSeed{key: "asura:gone", url: "u", checkedAt: 9000, bookmarks: 0})
// A second orphan for the htmx dialect's request, whose row must still
// exist after the first request removed its own.
seedSeriesRow(t, st, db, seriesRowSeed{key: "asura:gone2", url: "u", checkedAt: 9000, bookmarks: 0})
router := newRouter(st, testConfig())
target := "/admin/series?filter=" + store.SeriesFilterNoReaders
req := httptest.NewRequest(http.MethodPost, "/admin/series/asura:gone/remove", nil)
req.Header.Set("HX-Target", "detail-meta")
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusSeeOther {
t.Fatalf("detail removal status = %d, want 303", rr.Code)
}
if got := rr.Header().Get("Location"); got != target {
t.Errorf("Location = %q, want %q", got, target)
}
var one int
if err := db.QueryRow(`SELECT 1 FROM series WHERE site = 'asura' AND series_id = 'gone'`).Scan(&one); err != sql.ErrNoRows {
t.Fatalf("series row after detail removal = %v, want sql.ErrNoRows", err)
}
req = httptest.NewRequest(http.MethodPost, "/admin/series/asura:gone2/remove", nil)
req.Header.Set("HX-Request", "true")
req.Header.Set("HX-Target", "detail-meta")
req.AddCookie(sessionCookie(t, st))
rr = httptest.NewRecorder()
router.ServeHTTP(rr, req)
if got := rr.Header().Get("HX-Redirect"); got != target {
t.Errorf("HX-Redirect = %q, want %q", got, target)
}
}
// A removal that races a fresh Bookmark is a refusal, not an error: the row
// is rendered again at its new count with the fact spelled out, never a 500,
// and it must not vanish from the list — the delete never happened. The
// detail-surface refusal navigates back to the detail page, where the same
// fresh count is visible.
func TestRemoveRacedBookmarkIsRefusedNotError(t *testing.T) {
st, dsn := newTestStoreURL(t)
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
seedSeriesRow(t, st, db, seriesRowSeed{key: "asura:raced", url: "u", checkedAt: 9000, bookmarks: 1})
router := newRouter(st, testConfig())
req := httptest.NewRequest(http.MethodPost, "/admin/series/asura:raced/remove",
strings.NewReader("filter=no_readers&band=0"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("refusal status = %d, want 200 (never a 500)", rr.Code)
}
if got := rr.Header().Get("HX-Reswap"); got != "" {
t.Errorf("refusal carries HX-Reswap = %q, want none (the row must stay)", got)
}
body := rr.Body.String()
if !strings.Contains(body, "a Reader has bookmarked this Series again") {
t.Errorf("refusal does not say what happened:\n%s", body)
}
if !strings.Contains(body, `class="c-rd">1</span>`) {
t.Errorf("refusal does not render the fresh count:\n%s", body)
}
var one int
if err := db.QueryRow(`SELECT 1 FROM series WHERE site = 'asura' AND series_id = 'raced'`).Scan(&one); err != nil {
t.Fatalf("series row after refusal = %v, want present", err)
}
list := adminSeriesPage(t, router, st, "")
if !strings.Contains(list, "Title of asura:raced") {
t.Fatal("row vanished from the list after a refused removal")
}
if strings.Contains(list, ">Remove<") {
t.Errorf("a held series still offers Remove:\n%s", list)
}
// The detail-surface refusal navigates back to the detail page.
req = httptest.NewRequest(http.MethodPost, "/admin/series/asura:raced/remove", nil)
req.Header.Set("HX-Target", "detail-meta")
req.AddCookie(sessionCookie(t, st))
rr = httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusSeeOther {
t.Fatalf("detail refusal status = %d, want 303", rr.Code)
}
if got := rr.Header().Get("Location"); got != "/admin/series/asura:raced" {
t.Errorf("refusal Location = %q, want the detail page", got)
}
}
// The remove route trusts the same way the poll route does: a malformed key
// is a 400 and an unknown Series a 404, and an oversized body is a 400 that
// removes nothing.
func TestRemoveRejectsBadKeysAndCapsBody(t *testing.T) {
router, st := newWebTestServer(t, testConfig())
seed(t, st, store.Bookmark{
Key: "asura:x", Site: "asura", SeriesID: "x",
Title: "Title of asura:x", SeriesURL: "u",
})
for _, tc := range []struct {
path string
want int
}{
{"/admin/series/nocolon/remove", http.StatusBadRequest},
{"/admin/series/:x/remove", http.StatusBadRequest},
{"/admin/series/asura:/remove", http.StatusBadRequest},
{"/admin/series/asura:ghost/remove", http.StatusNotFound},
} {
req := httptest.NewRequest(http.MethodPost, tc.path, nil)
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != tc.want {
t.Errorf("POST %s status = %d, want %d", tc.path, rr.Code, tc.want)
}
}
big := strings.Repeat("a", 1<<17) // 128 KiB, over the 64 KiB cap
req := httptest.NewRequest(http.MethodPost, "/admin/series/asura:x/remove", strings.NewReader(big))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if rr.Code != http.StatusBadRequest {
t.Fatalf("oversized body status = %d, want 400", rr.Code)
}
list := adminSeriesPage(t, router, st, "")
if !strings.Contains(list, "Title of asura:x") {
t.Errorf("an oversized body still removed the row:\n%s", list)
}
}
@@ -0,0 +1,86 @@
# ADR-0014: Cover addresses derived from the bytes, not the source URL
Date: 2026-08-22
Status: accepted
## Decision
A Cover's content address is the hex SHA-256 of its **bytes**, not of the
source URL it was fetched from. `CoverAddressForBytes(body)` names the address
`putCover` stores under, `SetSeriesCover` and `ReplaceSeriesCover` point the
Series row at it, and the wire URL is built from it exactly as before — same
route, same 64-hex-digit shape, same immutability, only the input to the hash
changes. Rows written before this ADR keep their URL-derived addresses
forever: they are never rehashed on read, and they heal into byte addressing
only when a Forced Poll replaces them.
`ReplaceSeriesCover(site, seriesID, sourceURL, body, contentType)`
`(previous, current, error)` is the one write that may move a Cover once one
exists. It stores the bytes, then in one transaction locks the Series row,
reads the old `cover_address`, writes the new one and the source URL, and
reports both addresses: `previous == ""` means there was no Cover,
`previous == current` means the Site served identical artwork, and any other
pair names the stranded address.
## Why a future reader will find this surprising
The address is what makes a re-art visible at all. URL addressing collapses
every image behind a stable URL into one address, so a Series whose Cover
changes (a big-budget CPI blitz on a light novel is the standing example)
keeps serving its original cover bytes: the poll refetches the same URL,
hashes it, and the store records the same address, everyone happy except the
Reader. Nothing in the system can detect the change, because the address is a
pure function of the fetch target, and identical bytes written 1,000 times
are one blob on disk. Storing bytes we already know how to store is only a
few lines of work. **Rejecting that work is the surprising part, and the
answer is the Forced Poll wave**: for a corrupt/blank cover the poll's
fill-if-blank installer already worked, but for a *wrong but non-blank* cover
there was no write that would move it at all — only a manual truth in
`series.cover_address`, which is exactly the thing that must never be set by
hand. Byte addressing gives the replacement write a **new address to write**,
and with it a legitimate, transaction-safe mover.
## Considered options
**Keep URL addressing and add a generic "clear the cover" write.**
Rejected: clearing is a two-phase action (blank it, wait for the poll to
re-fill, hope the bytes changed in between) that cannot report what the
write did, and it makes the Series render cover-less in between. The
replacement write is atomic, reports its displacement, and has one effect:
the Series now points at bytes that actually came from its source URL.
**Address by URL, but salt it so a re-art is a new address.**
Rejected: the salt would have to live somewhere addressable (a stored per-
Series nonce), turning the address from a content fact into a mutable fact —
two rows could then hold identical bytes under different addresses and the
invariant "same bytes object" is gone.
## Consequences
- `store.CoverAddress` (URL-hash) is deleted; `CoverAddressForBytes` is
public so tests and the forced-poll wave can predict addresses from the
bytes fakes serve.
- Legacy URL-addressed rows are read-only facts: `GetCover(sourceURL)` keeps
resolving them (the poll heal path), and they are re-addressed only by a
forced replacement. Until one happens, they are invisible to byte-derived
lookups — the reverse direction was always true, so this side has no
migration and no lookup fan-out.
- A replaced Cover's old bytes stay on disk under their address (the `covers`
row is untouched — only the Series row moves). Nothing reclaims them
today; a later sweep is a small query over `covers` addresses not
referenced by any `series` row.
- `SetSeriesCover` keeps its `cover_address = ''` guard untouched: the
acquisition-at-creation and poll fill paths still may not overwrite a
non-blank Cover. The two installers are now deliberately different
functions instead of one function with a conditional.
- The address is still a filesystem path (≤64 hex chars, no separators), so
`coverAddressRe` and the sharding stay exactly as they are.
## Cost of reversing
The URL-hash side of the current rows is uncomputable from the rows alone: a
rollback would need every stored blob's source URL, a join to a table that
does not store it, or a refetch of every Series. Keeping both derivations
resolvable is cheaper than either, so the two derivations are documented in
the 0009 migration comment: no component may assume which derivation a
stored address came from, because the 64-hex shape hides it.