Spec #136: Finished belongs to the Series — owner-owned poll gate, Lifecycle bucket dropped (#163)

Closes #136.

Spec #136 end to end: `finished` becomes a fact about the Series, written only by the owner, and the reader-facing Lifecycle bucket is gone.

## What landed

- **#157** — `series.finished_at bigint NOT NULL DEFAULT 0` plus the migration whose statement order is load-bearing (seed from the buckets, then flip them); both Lane queries lose the `HAVING COUNT(*) FILTER (WHERE b.status <> 'finished')` clause and gate on `finished_at = 0` instead, with the due-query/eligible-count force asymmetry kept deliberate and commented; `StatusFinished`, its API special-case 400, the web tab and the templates' Finished bucket deleted.
- **#158** — owner Finish control on the Series detail page: confirm-gated finish, instant un-finish, admin accent (never ember, nothing is destroyed), `Store.SetSeriesFinished`, the two routes behind the owner gate, and the state displayed on the list row without offering the control there.
- **#160** — reader side: derived `finished` bool on the flat Bookmark (`s.finished_at > 0`), rendered as a text-only label in both userscripts and on the web card; read-only inbound by omission from `Upsert`'s explicit `series` column list, same mechanism that already protects `cover`.
- **#161** — glossary and the stale Reader-count divergence note catch up.
- **#159** — `finished` joins the admin filter vocabulary (predicate `finished_at > 0`, label `Finished`, own aggregate count, figure last in the stats block as informational); the four clock-driven hygiene predicates (stale, never-checked, no-cover, no-chapter) exclude finished Series while unpollable, orphan and sighting-raised deliberately do not.

## Verification

`go vet ./...` and `go test ./...` green on the merged branch (Docker-backed, throwaway `postgres:17-alpine` per package). Each ticket also passed a two-axis review (spec + standards) on its own branch before merge.

Reviewed-on: #163
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #163.
This commit is contained in:
2026-08-22 17:31:52 +07:00
committed by sulthan
parent faa80c41ea
commit 8e4fa6448e
34 changed files with 1225 additions and 241 deletions
+34 -18
View File
@@ -7,11 +7,13 @@ import (
"strings"
)
// Series filter names (issue #140), ordered permanent-then-fixable — the
// repairs nothing will ever undo first, the ones a Poll can make right after.
// A name is the repair a row needs, not the SQL that finds it; the values are
// the wire form the Series list URL carries (#142). "all" is the absent and
// unknown case: every Series.
// Series filter names (issue #140): the seven repair filters are ordered
// permanent-then-fixable — the repairs nothing will ever undo first, the
// ones a Poll can make right after. SeriesFilterFinished is not part of
// that ordering: a finished Series is a deliberate state, not a repair, so
// it sits last, informational. A name is the repair a row needs, not the
// SQL that finds it; the values are the wire form the Series list URL
// carries (#142). "all" is the absent and unknown case: every Series.
const (
SeriesFilterAll = "all"
SeriesFilterNoURL = "no_series_url"
@@ -21,9 +23,10 @@ const (
SeriesFilterStale = "stale"
SeriesFilterNoCover = "no_cover"
SeriesFilterReaderReport = "reader_report"
SeriesFilterFinished = "finished"
)
// SeriesFilter is one named hygiene predicate over the whole library. Site
// SeriesFilter is one named filter predicate over the whole library. Site
// and Kind narrow the row read; Name picks the predicate; Cutoff is the
// staleness boundary the "stale" filter compares against, supplied by the
// caller's clock — the store has no clock; Page is 1-based.
@@ -42,7 +45,7 @@ type SeriesFilter struct {
// and only the anonymous boolean in raisedByReaderAnswer crosses it.
const adminSeriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover_address,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at, s.force_poll_at,
s.latest_corrected_at`
s.latest_corrected_at, s.finished_at`
// raisedByReaderAnswer answers "did a Reader's report set this number" without
// naming which Reader. Kept apart from adminSeriesColumns so the column list —
@@ -80,7 +83,9 @@ type AdminSeries struct {
LatestCorrectedAt int64
ReaderCount int
RaisedByReader bool // a Reader's report set LatestChapterNum
// FinishedAt is the owner's finish stamp: unix ms, zero while the Series is
// not finished — the same shape as the Correction stamp, and its own undo.
FinishedAt int64
}
// SeriesPage is one page of the owner's filtered Series list plus the count
@@ -113,10 +118,19 @@ func (a AdminSeries) Key() string { return a.Site + ":" + a.SeriesID }
// The WHERE set is: no URL (an empty URL only — the host-failing-the-fetch-
// gate case is invisible to SQL, needs the Site registry in Go, and belongs to
// a later repair), never-read-a-chapter and never-checked as disjoint halves
// (non-zero versus zero check stamp), stale, no cover, and Reader-report.
// (non-zero versus zero check stamp), stale, no cover, finished (the
// retirement stamp, read directly), and Reader-report.
// no_readers is the one HAVING predicate: it is the orphan test, an aggregate
// over the LEFT JOIN, where a bare WHERE has no row to test.
//
// The clock-versus-outcome split decides which predicates exclude finished
// Series (`s.finished_at = 0` in each of the four): the clock-driven one —
// never-checked, stale, no-chapter, no-cover — keep ticking after the last
// Poll, so they would report a retired row as a problem no Poll is coming to
// fix; the three outcome-driven ones — no-URL, no-readers, Reader-report —
// read stored facts that simply stop arriving, so a finished Series needing a
// genuine repair still shows up under them.
//
// stale is the checked-but-old half of the stamp partition — because the
// verdict line wants "not checked in twelve hours" as one figure, and a never
// checked Series is already counted on its own "waiting"/never-checked
@@ -133,16 +147,18 @@ func adminFilter(f SeriesFilter) (where, having string, args []any, err error) {
case SeriesFilterNoURL:
clauses = append(clauses, `s.series_url = ''`)
case SeriesFilterNoChapter:
clauses = append(clauses, `s.latest_checked_at <> 0 AND s.latest_chapter_num IS NULL`)
clauses = append(clauses, `s.latest_checked_at <> 0 AND s.latest_chapter_num IS NULL AND s.finished_at = 0`)
case SeriesFilterNeverChecked:
clauses = append(clauses, `s.latest_checked_at = 0`)
clauses = append(clauses, `s.latest_checked_at = 0 AND s.finished_at = 0`)
case SeriesFilterStale:
clauses = append(clauses, `s.latest_checked_at > 0 AND s.latest_checked_at < $`+strconv.Itoa(len(args)+1))
clauses = append(clauses, `s.latest_checked_at > 0 AND s.latest_checked_at < $`+strconv.Itoa(len(args)+1)+` AND s.finished_at = 0`)
args = append(args, f.Cutoff)
case SeriesFilterNoCover:
clauses = append(clauses, `s.cover_address = ''`)
clauses = append(clauses, `s.cover_address = '' AND s.finished_at = 0`)
case SeriesFilterReaderReport:
clauses = append(clauses, `s.latest_raised_by IS NOT NULL`)
case SeriesFilterFinished:
clauses = append(clauses, `s.finished_at > 0`)
case SeriesFilterNoReaders:
having = `HAVING COUNT(b.reader_id) = 0`
default:
@@ -157,9 +173,9 @@ func adminFilter(f SeriesFilter) (where, having string, args []any, err error) {
// SeriesPage returns one page of the Series matching the filter, least
// recently checked first. The LEFT JOIN to Bookmarks is what surfaces the
// orphans that hygiene has to find — an inner join would hide them, exactly
// as the Lane's join does. ReaderCount is a plain count of every Bookmark on
// the Series, which knowingly disagrees with the two Lane queries for as long
// as the finished lifecycle bucket exists (#140).
// as the Lane's join does. Every bookmark keeps its Series polled now that
// finished is a Series flag, so this plain ReaderCount agrees with the Lane
// queries (issue #157).
//
// The tie-break is mandatory, not decorative: every unpollable Series shares a
// zero check stamp, so ordering on that column alone gives no stable page
@@ -196,7 +212,7 @@ func (s *Store) SeriesPage(f SeriesFilter) (SeriesPage, error) {
`+where+`
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover_address,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at,
s.force_poll_at, s.latest_corrected_at, s.latest_raised_by
s.force_poll_at, s.latest_corrected_at, s.finished_at, s.latest_raised_by
`+having+`
ORDER BY s.latest_checked_at, s.site, s.series_id
LIMIT $`+strconv.Itoa(base+1)+` OFFSET $`+strconv.Itoa(base+2), args...)
@@ -271,7 +287,7 @@ func scanAdminSeries(scan func(...any) error) (AdminSeries, int, error) {
if err := scan(
&a.Site, &a.SeriesID, &a.Title, &a.SeriesURL, &a.CoverAddress,
&a.Kind, &a.LatestChapter, &latestChapterNum, &a.LatestCheckedAt,
&a.ForcePollAt, &a.LatestCorrectedAt,
&a.ForcePollAt, &a.LatestCorrectedAt, &a.FinishedAt,
&a.RaisedByReader, &a.ReaderCount, &total,
); err != nil {
return AdminSeries{}, 0, err
+133
View File
@@ -128,6 +128,85 @@ func TestAdminSeriesFilters(t *testing.T) {
}
}
// A finished Series is the owner's deliberate state, not a problem a Poll
// will fix: the four clock-driven hygiene predicates exclude it (their
// stamps stop advancing at the last Poll, so without the guard a retired row
// is reported forever), the three outcome-driven ones still include it, and
// the finished filter returns exactly the retired rows.
func TestAdminFinishedSeriesFilters(t *testing.T) {
s := newTestStore(t)
// Each fin-* row is shaped to trip exactly one predicate if its guard
// fails: checked-but-old for stale, a zero stamp for never-checked, a
// stamp with no chapter for no-chapter, an empty cover for no-cover, and
// the unguarded three shaped to trip their own. A healthy, unfinished
// neighbour keeps the exclusion checks honest: a filter that regressed to
// matching nothing would pass a bare "no finished rows" assertion.
seedAdminSeries(t, s, seriesSeed{key: "asura:fin-stale", url: "u", cover: "c", checkedAt: 2000, latestNum: new(4.0), bookmarks: 1})
seedAdminSeries(t, s, seriesSeed{key: "asura:fin-neverchecked", url: "u", cover: "c", checkedAt: 0, latestNum: new(4.0), bookmarks: 1})
seedAdminSeries(t, s, seriesSeed{key: "asura:fin-nochapter", url: "u", cover: "c", checkedAt: 9000, bookmarks: 1})
seedAdminSeries(t, s, seriesSeed{key: "asura:fin-nocover", url: "u", checkedAt: 9000, latestNum: new(4.0), bookmarks: 1})
seedAdminSeries(t, s, seriesSeed{key: "asura:fin-nourl", url: "", cover: "c", checkedAt: 9000, latestNum: new(4.0), bookmarks: 1})
seedAdminSeries(t, s, seriesSeed{key: "asura:fin-orphan", url: "u", cover: "c", checkedAt: 9000, latestNum: new(4.0), bookmarks: 0})
seedAdminSeries(t, s, seriesSeed{key: "asura:fin-report", url: "u", cover: "c", checkedAt: 9000, latestNum: new(4.0), bookmarks: 1, raisedBy: true})
seedAdminSeries(t, s, seriesSeed{key: "asura:healthy", url: "u", cover: "c", checkedAt: 9000, latestNum: new(4.0), bookmarks: 1})
finished := []string{
"asura:fin-stale", "asura:fin-neverchecked", "asura:fin-nochapter",
"asura:fin-nocover", "asura:fin-nourl", "asura:fin-orphan", "asura:fin-report",
}
for _, key := range finished {
site, id, _ := strings.Cut(key, ":")
if err := s.SetSeriesFinished(site, id, 1000); err != nil {
t.Fatalf("finish %s: %v", key, err)
}
}
cases := []struct {
name string
f SeriesFilter
want []string
}{
{"stale excludes finished", SeriesFilter{Name: SeriesFilterStale, Cutoff: 5000}, nil},
{"never checked excludes finished", SeriesFilter{Name: SeriesFilterNeverChecked}, nil},
{"no chapter excludes finished", SeriesFilter{Name: SeriesFilterNoChapter}, nil},
{"no cover excludes finished", SeriesFilter{Name: SeriesFilterNoCover}, nil},
{"no url includes finished", SeriesFilter{Name: SeriesFilterNoURL}, []string{"asura:fin-nourl"}},
{"no readers includes finished", SeriesFilter{Name: SeriesFilterNoReaders}, []string{"asura:fin-orphan"}},
{"reader report includes finished", SeriesFilter{Name: SeriesFilterReaderReport}, []string{"asura:fin-report"}},
{"finished returns the retired rows", SeriesFilter{Name: SeriesFilterFinished}, finished},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := pageKeys(t, s, tc.f)
want := map[string]bool{}
for _, k := range tc.want {
want[k] = true
}
if len(got) != len(want) {
t.Fatalf("%+v returned %v, want exactly %v", tc.f, got, want)
}
for k := range want {
if !got[k] {
t.Fatalf("%+v dropped %q (got %v)", tc.f, k, got)
}
}
})
}
// The aggregate's finished total counts every retired row — the same
// predicate the Overview's finished figure is summed from.
shapes, err := s.SeriesShapes(SeriesFilter{Name: SeriesFilterFinished})
if err != nil {
t.Fatalf("SeriesShapes(finished): %v", err)
}
sum := 0
for _, sh := range shapes {
sum += sh.Total
}
if sum != len(finished) {
t.Fatalf("finished aggregate = %d, want %d", sum, len(finished))
}
}
// "Never read a chapter" and "never checked" are disjoint by construction:
// the first requires a non-zero check stamp, the second a zero one. Over a
// mix that should satisfy both, no row may be counted twice.
@@ -394,3 +473,57 @@ func TestAdminSeriesCarriesForcePollAt(t *testing.T) {
t.Fatalf("row = %+v, want ForcePollAt 5000", page.Rows)
}
}
// SetSeriesFinished is the owner's finish stamp write: finishing writes the
// given ms, un-finishing writes zero — the one undo, the same shape as the
// correction stamp. Touching a missing series is not an error: the row may
// have been orphaned, and the caller's read decides what exists.
func TestSetSeriesFinishedStampsAndClears(t *testing.T) {
s := newTestStore(t)
seedAdminSeries(t, s, seriesSeed{key: "asura:x", url: "u", checkedAt: 9000, bookmarks: 1})
if err := s.SetSeriesFinished("asura", "x", 42); err != nil {
t.Fatalf("SetSeriesFinished: %v", err)
}
var got int64
if err := s.db.QueryRow(
`SELECT finished_at FROM series WHERE site = 'asura' AND series_id = 'x'`).Scan(&got); err != nil {
t.Fatalf("read finished_at: %v", err)
}
if got != 42 {
t.Fatalf("finished_at = %d, want 42", got)
}
if err := s.SetSeriesFinished("asura", "x", 0); err != nil {
t.Fatalf("SetSeriesFinished un-finish: %v", err)
}
if err := s.db.QueryRow(
`SELECT finished_at FROM series WHERE site = 'asura' AND series_id = 'x'`).Scan(&got); err != nil {
t.Fatalf("read finished_at after un-finish: %v", err)
}
if got != 0 {
t.Fatalf("finished_at = %d, want 0 (un-finish writes zero)", got)
}
if err := s.SetSeriesFinished("asura", "ghost", 42); err != nil {
t.Fatalf("SetSeriesFinished missing: %v", err)
}
}
// The admin projection carries the finish stamp so the web layer can render
// the finished state without a second read.
func TestAdminSeriesCarriesFinishedAt(t *testing.T) {
s := newTestStore(t)
seedAdminSeries(t, s, seriesSeed{key: "asura:x", url: "u", checkedAt: 1000, bookmarks: 1})
if err := s.SetSeriesFinished("asura", "x", 5000); err != nil {
t.Fatalf("SetSeriesFinished: %v", err)
}
page, err := s.SeriesPage(SeriesFilter{})
if err != nil {
t.Fatalf("SeriesPage: %v", err)
}
if len(page.Rows) != 1 || page.Rows[0].FinishedAt != 5000 {
t.Fatalf("row = %+v, want FinishedAt 5000", page.Rows)
}
}
@@ -0,0 +1,24 @@
-- finished_at is "the owner marked this Series finished" (#157): epoch ms,
-- zero means not finished, and it doubles as the undo (write zero). The poll
-- gate reads it — a Series is polled only while finished_at = 0 — never a
-- bookmark's status.
ALTER TABLE series ADD COLUMN finished_at bigint NOT NULL DEFAULT 0;
-- Column first, seed second, flip third — the order is load-bearing: a seed
-- that ran after the flip would read the buckets it just destroyed, declare
-- nothing finished, and silently resume polling on Series nobody chose to
-- resume. The seed mirrors the pre-cutover due gate exactly: a Series stays
-- polled while any bookmark is outside the finished bucket, so a Series whose
-- every bookmark sits in it is stamped, one click from being read again
-- afterwards. The stamp is the only memory of the bucket the flip is about to
-- erase.
UPDATE series s SET finished_at = (EXTRACT(EPOCH FROM now()) * 1000)::bigint
WHERE EXISTS (SELECT 1 FROM bookmarks b
WHERE b.site = s.site AND b.series_id = s.series_id)
AND NOT EXISTS (SELECT 1 FROM bookmarks b
WHERE b.site = s.site AND b.series_id = s.series_id
AND b.status <> 'finished');
-- The Lifecycle bucket is gone; a finished bookmark is an archived one. The
-- flip must come after the seed, which still reads the bucket.
UPDATE bookmarks SET status = 'archived' WHERE status = 'finished';
+60 -33
View File
@@ -47,13 +47,16 @@ type Bookmark struct {
LatestChapter string `json:"latest_chapter"`
LatestChapterNum *float64 `json:"latest_chapter_num"` // nil until first captured
UpdatedAt int64 `json:"updated_at"` // unix ms; see Upsert
// Status is the lifecycle bucket: reading, archived, or finished.
// Archived series stay polled for new chapters; finished ones do not.
// Status is the lifecycle bucket: reading or archived.
// Archived series stay polled for new chapters.
// Empty on the way in means "no opinion" — see Upsert.
Status string `json:"status"`
// Kind is the library bucket: manga or novel. Empty on the way in means
// "no opinion" — see Upsert.
Kind string `json:"kind"`
// Finished is the owner's retirement of the Series, derived: the flag is a
// Series fact and a client cannot write it — see Upsert.
Finished bool `json:"finished"`
}
// Series is one distinct work, shared by every bookmark that tracks it. It is
@@ -214,10 +217,11 @@ const (
)
// Lifecycle buckets. A bookmark is in exactly one; favorite is orthogonal.
// Finished is not a bucket: it is a fact about the Series (series.finished_at),
// never about a Reader's bookmark.
const (
StatusReading = "reading"
StatusArchived = "archived"
StatusFinished = "finished"
)
//go:embed migrations/*.sql
@@ -229,7 +233,8 @@ var migrations embed.FS
// order, so the flat Bookmark reads back whole despite the split (ADR-0004).
const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover_address,
b.last_chapter, b.last_chapter_num, b.last_chapter_url,
b.favorite, s.latest_chapter, s.latest_chapter_num, b.updated_at, b.status, s.kind`
b.favorite, s.latest_chapter, s.latest_chapter_num, b.updated_at, b.status, s.kind,
s.finished_at > 0`
// seriesColumns is the series row in scanSeries order, used by the poller's
// due query. latest_checked_at lives only on series — see MarkLatestChecked
@@ -608,6 +613,7 @@ func (s *Store) scanBookmark(scan func(...any) error) (Bookmark, error) {
&b.Site, &b.SeriesID, &b.Title, &b.SeriesURL, &coverAddress,
&b.LastChapter, &b.LastChapterNum, &b.LastChapterURL,
&b.Favorite, &b.LatestChapter, &latestChapterNum, &b.UpdatedAt, &b.Status, &b.Kind,
&b.Finished,
); err != nil {
return Bookmark{}, err
}
@@ -619,9 +625,9 @@ func (s *Store) scanBookmark(scan func(...any) error) (Bookmark, error) {
// bookmark is keyed (reader_id, site, series_id) (issue #22).
b.Key = b.Site + ":" + b.SeriesID
// An unrecognised bucket (a hand-edited row) would leave the row in no list
// at all, so anything outside the three known buckets reads as the default
// at all, so anything outside the two known buckets reads as the default
// rather than being passed through.
if b.Status != StatusReading && b.Status != StatusArchived && b.Status != StatusFinished {
if b.Status != StatusReading && b.Status != StatusArchived {
b.Status = StatusReading
}
return b, nil
@@ -1302,10 +1308,12 @@ func (s *Store) LaneGates(site string) (pausedUntil, refuseUntil int64, err erro
//
// A forced Series (force_poll_at newer than latest_checked_at, issue #146)
// overrides exactly three gates: the rest cutoff, the Sighting-deferral
// clause and the finished-only bucket. It never overrides an empty
// series_url or the Bookmarks join — nothing to fetch, and no consumer for
// the result — so those stay unconditional. Forced rows sort to the front of
// the queue; the reader-count-then-age ordering among the rest is ADR-0003.
// clause and a finished Series. It never overrides an empty series_url or the
// Bookmarks join — nothing to fetch, and no consumer for the result — so
// those stay unconditional, and it never clears the finish: nothing here
// writes finished_at, and pending force clears itself when the pass stamps
// the check timestamp. Forced rows sort to the front of the queue; the
// reader-count-then-age ordering among the rest is ADR-0003.
//
// The reader_count ordering is the point of the split (ADR-0003): a series
// shared by several readers is fetched once per due cycle, and the popular
@@ -1313,14 +1321,15 @@ func (s *Store) LaneGates(site string) (pausedUntil, refuseUntil int64, err erro
// reader count, oldest-first keeps the poll fair when the backlog outgrows
// throughput: the most neglected series is always next, so a large collection
// refreshes uniformly slower rather than leaving a tail that never refreshes
// at all. The userscript sorts its own queue the same way (L453).
// at all. The userscript sorts its own queue the same way.
//
// Series with no series_url are skipped — there is nothing to fetch, which is
// the same filter the userscript applies at L452. Series whose only bookmarks
// are finished are skipped too: nothing more is coming, so fetching them only
// burns requests. Archived bookmarks still count — knowing what a shelved
// series is up to is the whole reason for archiving instead of deleting.
// A series with no bookmarks at all never appears: the join excludes it.
// the same filter the userscript applies before refreshing. A finished Series is
// skipped unless forced: nothing more is coming, so fetching it only burns
// requests (issue #157). Archived bookmarks still count — knowing what a
// shelved series is up to is the whole reason for archiving instead of
// deleting. A series with no bookmarks at all never appears: the join
// excludes it.
//
// ceilingMs is the Sighting deferral ceiling (issue #103): a Series whose last
// real Poll is older than it appears however recently it was sighted. That is
@@ -1329,10 +1338,10 @@ func (s *Store) LaneGates(site string) (pausedUntil, refuseUntil int64, err erro
// decided here, from two facts the query already computes, so a Lane gains no
// query per round: a Sighting younger than cutoffMs holds the Series back, but
// only while COUNT(*) is 1. A Series a second Reader bookmarks is Polled on
// schedule, so a wrong value the whole guild can see is corrected by a check
// that was never postponed; on a solitary Series the only person a wrong value
// reaches is the Reader who reported it. Whether the reporting Reader is
// allowed to defer at all was settled when the Sighting was recorded — see
// the schedule, so a wrong value the guild can see is corrected by a check
// that was never postponed; on a solitary Series the only person a wrong
// value reaches is the Reader who reported it. Whether the reporting Reader
// is allowed to defer at all was settled when the Sighting was recorded — see
// RecordSighting.
func (s *Store) DueForLatestCheck(site string, cutoffMs, ceilingMs int64) ([]Series, error) {
rows, err := s.db.Query(`SELECT `+seriesColumns+`,
@@ -1344,12 +1353,11 @@ func (s *Store) DueForLatestCheck(site string, cutoffMs, ceilingMs int64) ([]Ser
AND s.series_url <> ''
AND (s.latest_checked_at <= $2::bigint
OR s.force_poll_at > s.latest_checked_at)
AND (s.finished_at = 0 OR s.force_poll_at > s.latest_checked_at)
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at,
s.force_poll_at
HAVING (COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
OR s.force_poll_at > s.latest_checked_at)
AND (COUNT(*) > 1
s.force_poll_at, s.finished_at
HAVING (COUNT(*) > 1
OR s.latest_sighted_at <= $2::bigint
OR s.latest_checked_at <= $3::bigint
OR s.force_poll_at > s.latest_checked_at)
@@ -1371,14 +1379,18 @@ func (s *Store) DueForLatestCheck(site string, cutoffMs, ceilingMs int64) ([]Ser
return out, rows.Err()
}
// EligibleSeriesCount returns how many of a Site's Series still have at least
// one bookmark outside the finished bucket. It is the denominator of the
// Lane's pace (issue #100): the effective gap is the smaller of the registry
// gap and one hour divided by this count, so Series that will never be Polled
// do not make the Lane faster than it needs to be, and counting every eligible
// Series rather than only those currently due keeps the pace steady — the
// single worst moment to be fastest is startup, when everything is due at
// once.
// EligibleSeriesCount returns how many of a Site's Series are not finished —
// the flag, never a Reader vote. It is the denominator of the Lane's pace
// (issue #100): the effective gap is the smaller of the registry gap and one
// hour divided by this count, so Series that will never be Polled do not make
// the Lane faster than it needs to be, and counting every eligible Series
// rather than only those currently due keeps the pace steady — the single
// worst moment to be fastest is startup, when everything is due at once.
//
// The deliberate asymmetry with DueForLatestCheck's WHERE: a forced Series
// is due but never admitted here, because a forced pass must not speed up
// every other fetch on the Site — one impassioned press is not a reason to
// hammer the Site (issue #157).
func (s *Store) EligibleSeriesCount(site string) (int, error) {
var n int
err := s.db.QueryRow(`SELECT COUNT(*) FROM (
@@ -1386,8 +1398,8 @@ func (s *Store) EligibleSeriesCount(site string) (int, error) {
FROM series s
JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id
WHERE s.site = $1
AND s.finished_at = 0
GROUP BY s.site, s.series_id
HAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
) e`, site).Scan(&n)
if err != nil {
return 0, fmt.Errorf("count eligible series %s: %w", site, err)
@@ -1433,6 +1445,21 @@ func (s *Store) ForceSeriesPoll(site, seriesID string, at int64) error {
return nil
}
// SetSeriesFinished stamps or clears the owner's finish. at is unix ms to
// finish, zero to un-finish. A finished Series drops out of the Lane's reads
// (issue #157), and nothing else writes this column: it is the only writer
// outside migration 0016, so a machine write can never retire a Series
// silently. Touching a missing series is not an error: the row may have been
// orphaned, and the caller's read decides what exists.
func (s *Store) SetSeriesFinished(site, seriesID string, at int64) error {
if _, err := s.db.Exec(
`UPDATE series SET finished_at = $1 WHERE site = $2 AND series_id = $3`,
at, site, seriesID); err != nil {
return fmt.Errorf("set series finished %s:%s: %w", site, seriesID, err)
}
return nil
}
// LatestCheckedAt reads the column MarkLatestChecked writes. It exists for
// tests outside this package (the poller's own tests assert on rest
// bookkeeping) — see MarkLatestChecked for why the field stays off the
+197 -35
View File
@@ -279,6 +279,27 @@ func seedForCheck(t *testing.T, s *Store, key, seriesURL string, checkedAt int64
}
}
// seedFinished inserts a bookmark (and with it its series) and stamps the
// series finished — the series-level fact the due gate reads (issue #157).
func seedFinished(t *testing.T, s *Store, key, seriesURL string) {
t.Helper()
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
if _, err := s.Upsert(s.OwnerID(), Bookmark{
Key: key, Site: site, SeriesID: seriesID, SeriesURL: seriesURL,
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed %q: %v", key, err)
}
if _, err := s.db.Exec(
`UPDATE series SET finished_at = 1000 WHERE site = $1 AND series_id = $2`,
site, seriesID); err != nil {
t.Fatalf("seed finish %q: %v", key, err)
}
}
// noCeiling is a Sighting deferral ceiling no Series can reach, for the tests
// that predate the ceiling and are about rest, ordering or buckets instead.
const noCeiling = int64(-1)
@@ -486,24 +507,15 @@ func TestUpsertStatusChangeKeepsUpdatedAt(t *testing.T) {
t.Fatalf("UpdatedAt = %d, want it frozen at %d", stored.UpdatedAt, first.UpdatedAt)
}
}
// Archiving is the reason to keep polling — the point is to come back to a
// series that has moved on. A finished series has nothing left to publish.
// series that has moved on. A finished series has nothing left to publish,
// whichever Reader marked it (issue #157).
func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
store := newTestStore(t)
for _, tc := range []struct{ key, status string }{
{"asura:reading", StatusReading},
{"asura:archived", StatusArchived},
{"asura:finished", StatusFinished},
} {
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: tc.key, Site: "asura", SeriesID: strings.TrimPrefix(tc.key, "asura:"),
SeriesURL: "https://asurascans.com/comics/" + tc.key,
Status: tc.status, UpdatedAt: time.Now().UnixMilli(),
}); err != nil {
t.Fatalf("seed %s: %v", tc.key, err)
}
for _, key := range []string{"asura:reading", "asura:archived"} {
seedForCheck(t, store, key, "https://asurascans.com/comics/"+key, 0)
}
seedFinished(t, store, "asura:finished", "https://asurascans.com/comics/asura:finished")
due, err := store.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
if err != nil {
@@ -522,27 +534,26 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
}
// The gap's denominator counts every Series the Lane will ever Poll: a
// finished Series must not make the Lane faster than it needs to be, and
// another Site's Series must not leak into this Site's count.
// finished Series must not make the Lane faster than it needs to be, another
// Site's Series must not leak into this Site's count, and a forced Series is
// not admitted either — it is due once, not a reason to tighten the pace for
// every other fetch on the Site (issue #157).
func TestEligibleSeriesCount(t *testing.T) {
store := newTestStore(t)
seedForCheck(t, store, "asura:reading", "https://asurascans.com/comics/reading", 0)
seedForCheck(t, store, "asura:archived", "https://asurascans.com/comics/archived", 0)
if _, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:finished", Site: "asura", SeriesID: "finished",
SeriesURL: "https://asurascans.com/comics/finished",
Status: StatusFinished, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed finished: %v", err)
}
seedFinished(t, store, "asura:finished", "https://asurascans.com/comics/asura:finished")
seedForCheck(t, store, "demonic:z", "https://demonicscans.org/manga/z", 0)
if err := store.ForceSeriesPoll("asura", "archived", 5000); err != nil {
t.Fatalf("ForceSeriesPoll: %v", err)
}
n, err := store.EligibleSeriesCount("asura")
if err != nil {
t.Fatalf("EligibleSeriesCount: %v", err)
}
if n != 2 {
t.Fatalf("eligible = %d, want 2 (finished excluded, demonic excluded)", n)
t.Fatalf("eligible = %d, want 2 (finished and forced excluded, demonic excluded)", n)
}
n, err = store.EligibleSeriesCount("demonic")
if err != nil {
@@ -755,6 +766,133 @@ func TestMigration0008DropsLegacyCoverRows(t *testing.T) {
}
}
// 0016's three statements are order-dependent: the finished_at seed must see
// the pre-flip 'finished' buckets, and the flip must come after it. A seed
// that ran after the flip would read bookmarks that no longer say
// 'finished', declare nothing finished, and silently resume polling series
// somebody marked done. This seeds a pre-0016 database the way every
// deployment looked — finished a bookmark bucket, no finished_at column —
// runs the migration, and asserts on what it preserved.
func TestMigration0016SeedsFinishedAtBeforeFlippingBucket(t *testing.T) {
url := pgtest.URL(t)
db, err := sql.Open("pgx", url)
if err != nil {
t.Fatalf("open: %v", err)
}
defer db.Close()
if err := migrate(db, 15); err != nil {
t.Fatalf("migrate to 0015: %v", err)
}
if err := seedOwner(db, testOwner); err != nil {
t.Fatalf("seed owner: %v", err)
}
// A second reader, so "mixed" can carry two bookmarks on one series.
if _, err := db.Exec(
`INSERT INTO readers (discord_id, token_sha256) VALUES ('second', '\x01'::bytea)`); err != nil {
t.Fatalf("seed second reader: %v", err)
}
owner := func() int64 {
t.Helper()
var id int64
if err := db.QueryRow(
`SELECT id FROM readers WHERE discord_id = $1`, testOwner.DiscordID).Scan(&id); err != nil {
t.Fatalf("owner id: %v", err)
}
return id
}()
seedBookmark := func(readerID int64, site, seriesID, status string) {
t.Helper()
// The bookmark FK demands the series row; Upsert would create it on
// the fly, raw SQL has to spell it out.
if _, err := db.Exec(
`INSERT INTO series (site, series_id) VALUES ($1, $2) ON CONFLICT DO NOTHING`,
site, seriesID); err != nil {
t.Fatalf("seed series %s:%s: %v", site, seriesID, err)
}
if _, err := db.Exec(
`INSERT INTO bookmarks (reader_id, site, series_id, status, updated_at)
VALUES ($1, $2, $3, $4, 1000)`, readerID, site, seriesID, status); err != nil {
t.Fatalf("seed %s:%s/%s: %v", site, seriesID, status, err)
}
}
// done: the one-bookmark series every finished series looked like.
seedBookmark(owner, "asura", "done", "finished")
// shared: every reader finished — the multi-reader equivalent.
seedBookmark(owner, "asura", "shared", "finished")
// mixed: a second reader still reading keeps the series alive.
seedBookmark(owner, "asura", "mixed", "finished")
seedBookmark(owner+1, "asura", "mixed", "archived")
// live: nobody finished it.
seedBookmark(owner, "asura", "live", "reading")
// orphan: no bookmarks at all, nothing to decide from.
if _, err := db.Exec(
`INSERT INTO series (site, series_id) VALUES ('asura', 'orphan')`); err != nil {
t.Fatalf("seed orphan: %v", err)
}
if err := migrate(db, 0); err != nil {
t.Fatalf("migrate 0016: %v", err)
}
finishedAt := func(site, seriesID string) int64 {
t.Helper()
var at int64
if err := db.QueryRow(
`SELECT finished_at FROM series WHERE site = $1 AND series_id = $2`,
site, seriesID).Scan(&at); err != nil {
t.Fatalf("finished_at %s:%s: %v", site, seriesID, err)
}
return at
}
status := func(readerID int64, site, seriesID string) string {
t.Helper()
var s string
if err := db.QueryRow(
`SELECT status FROM bookmarks WHERE reader_id = $1
AND site = $2 AND series_id = $3`, readerID, site, seriesID).Scan(&s); err != nil {
t.Fatalf("status %s:%s: %v", site, seriesID, err)
}
return s
}
// The seed ran before the flip: had the flip gone first, done's bookmarks
// would read archived and nothing would be stamped. The stamp is a real
// timestamp, not a sentinel.
before := time.Now().Add(-time.Minute).UnixMilli()
after := time.Now().Add(time.Minute).UnixMilli()
for _, sr := range []struct{ site, seriesID string }{
{"asura", "done"}, {"asura", "shared"},
} {
if at := finishedAt(sr.site, sr.seriesID); at < before || at > after {
t.Fatalf("%s:%s finished_at = %d, want now-ish", sr.site, sr.seriesID, at)
}
// The flip followed the seed: every finished bookmark is archived.
if s := status(owner, sr.site, sr.seriesID); s != StatusArchived {
t.Fatalf("%s:%s status = %q, want archived after the flip", sr.site, sr.seriesID, s)
}
}
// The seed ignores a series any bookmark keeps alive.
if at := finishedAt("asura", "mixed"); at != 0 {
t.Fatalf("mixed finished_at = %d, want 0", at)
}
// The flip is a bucket rewrite, not a series-wide one: the finished
// bookmark becomes archived and the sibling rows keep their buckets.
if s := status(owner, "asura", "mixed"); s != StatusArchived {
t.Fatalf("mixed finished bookmark = %q, want archived", s)
}
if s := status(owner+1, "asura", "mixed"); s != StatusArchived {
t.Fatalf("mixed archived bookmark = %q, want untouched archived", s)
}
if s := status(owner, "asura", "live"); s != StatusReading {
t.Fatalf("live status = %q, want untouched reading", s)
}
if at := finishedAt("asura", "orphan"); at != 0 {
t.Fatalf("orphan finished_at = %d, want 0", at)
}
}
// readSeries reads the series row directly, for asserting on what Upsert
// actually stored rather than what the joined Bookmark reports.
func readSeries(t *testing.T, s *Store, site, seriesID string) Series {
@@ -796,6 +934,37 @@ func TestUpsertCreatesSeriesFromClient(t *testing.T) {
}
}
// The wire's finished flag is derived, never stored: the Upsert's explicit
// column list does not name finished_at — the same omission that protects
// cover — so a client echoing a cached flag, true or false, cannot change the
// Series' retired state (issues #157, #160).
func TestUpsertCannotWriteSeriesFinished(t *testing.T) {
store := newTestStore(t)
seedFinished(t, store, "asura:done", "https://asurascans.com/comics/asura:done")
for _, sent := range []bool{true, false} {
stored, err := store.Upsert(store.OwnerID(), Bookmark{
Key: "asura:done", Site: "asura", SeriesID: "done",
Finished: sent, UpdatedAt: 2000,
})
if err != nil {
t.Fatalf("Upsert(Finished: %v): %v", sent, err)
}
if !stored.Finished {
t.Fatalf("stored.Finished = false after echoing %v, want true", sent)
}
}
var finishedAt int64
if err := store.db.QueryRow(
`SELECT finished_at FROM series WHERE site = $1 AND series_id = $2`,
"asura", "done").Scan(&finishedAt); err != nil {
t.Fatalf("read finished_at: %v", err)
}
if finishedAt != 1000 {
t.Fatalf("series.finished_at = %v, want the seeded 1000 untouched", finishedAt)
}
}
// The hook is what starts creation-time acquisition, so it must fire exactly
// once per Series — on the PUT that created it, and on no later one, whichever
// Reader sends it.
@@ -1809,19 +1978,12 @@ func TestDueForLatestCheckForcedOverridesSightingDeferral(t *testing.T) {
}
}
// The finished-only bucket excludes a series whose only bookmarks are
// finished; a forced request overrides it — the owner asked, so the Lane
// looks.
func TestDueForLatestCheckForcedOverridesFinishedBucket(t *testing.T) {
// The finished flag excludes a series; a forced request overrides it — the
// owner asked, so the Lane looks.
func TestDueForLatestCheckForcedOverridesFinishedFlag(t *testing.T) {
s := newTestStore(t)
seedForCheck(t, s, "asura:reading", "https://asurascans.com/comics/reading", 0)
if _, err := s.Upsert(s.OwnerID(), Bookmark{
Key: "asura:finished", Site: "asura", SeriesID: "finished",
SeriesURL: "https://asurascans.com/comics/finished",
Status: StatusFinished, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed finished: %v", err)
}
seedFinished(t, s, "asura:finished", "https://asurascans.com/comics/asura:finished")
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
if err != nil {