Sightings: a Reader report defers a Poll of a solitary Series (#103) #108

Merged
sulthan merged 2 commits from feat/103-sightings into main 2026-08-16 20:11:00 +07:00
17 changed files with 3709 additions and 963 deletions
+28 -2
View File
@@ -27,7 +27,8 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
`series` keyed `(site, series_id)` `series` keyed `(site, series_id)`
(`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the (`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the
shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`), shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`),
Latest Chapter, `latest_checked_at` — and `bookmarks` holds only what Latest Chapter, `latest_checked_at`, and the Sighting pair
`latest_sighted_at`/`latest_raised_by` (issue #103) — and `bookmarks` holds only what
differs between readers: progress, favourite, lifecycle bucket, differs between readers: progress, favourite, lifecycle bucket,
`updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no `updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no
surrogate id; the wire `key` is derived as `site:series_id` on read — and surrogate id; the wire `key` is derived as `site:series_id` on read — and
@@ -78,7 +79,9 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
each re-checking that Site's bookmarked series' newest published chapter from each re-checking that Site's bookmarked series' newest published chapter from
backend's own network access, so `latest_chapter` stays fresh when the user backend's own network access, so `latest_chapter` stays fresh when the user
isn't browsing. Second, parallel signal — the userscript keeps its own isn't browsing. Second, parallel signal — the userscript keeps its own
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged. `maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` schedule, and its
`reportLatestChapter` PUTs every read, unchanged numbers included, because an
unchanged read is exactly the Sighting worth deferring a Poll on (#103).
Two independent clocks: per-series rest (`series.latest_checked_at`, Two independent clocks: per-series rest (`series.latest_checked_at`,
enforced by `Store.DueForLatestCheck`'s WHERE clause — `now - Rest`) and enforced by `Store.DueForLatestCheck`'s WHERE clause — `now - Rest`) and
per-Lane gap (the Lane sleeping between fetches, `effectiveGap`). Both live per-Lane gap (the Lane sleeping between fetches, `effectiveGap`). Both live
@@ -91,6 +94,29 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
every tick; found chapter written straight to the series row via every tick; found chapter written straight to the series row via
`Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list `Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
order — is never touched. order — is never touched.
**Sightings** (issue #103, ADR-0011) let a Reader's own page read defer a
Poll: `Store.RecordSighting` — called by the PUT handler *before* the Upsert,
because the raise test needs the row as it stands — stamps
`series.latest_sighted_at` and, when the report raises the stored number,
names its Reader in `series.latest_raised_by`. The due query's HAVING clause
is where deferral lives: a Series is skipped only while it has exactly one
Bookmark, was sighted within one Rest, and is under the ceiling
(`sightingCeilingRests`, six of that Site's rests) since its last Poll. So a
shared Series is never deferred, and no Series goes six hours unpolled
whatever arrives. `checkOne` judges the named Reader off the comparison it
already makes: a lower number is a contradiction (logged with the Reader and
both numbers), the same number an agreement, a higher number the Site
publishing and neither — that last one clears the attribution instead, since
the value the Poll then stores is its own and a later retraction is not the
Reader's fault. Three contradictions
(`store.SightingDisagreementLimit`) stop that Reader deferring — their
reports still write the Latest Chapter — and twenty consecutive agreements
(`store.SightingAgreementsToClear`) forgive them, as does the owner's
clear-marks control. Deferral is recomputed from live facts every round, so
nothing needs invalidating when a Series gains a second Bookmark; the one
input read earlier is the Reader's marks, checked when the Sighting is
recorded, so crossing the threshold or being cleared takes effect from that
Reader's next Sighting and the standing already bought lasts out its rest.
Refusals and browser loss are Lane-local: two `errChallengeHeld` in one pass Refusals and browser loss are Lane-local: two `errChallengeHeld` in one pass
stop that Site for `refuseBackoff` (15m) while other Lanes continue; an stop that Site for `refuseBackoff` (15m) while other Lanes continue; an
`errBrowserInterrupted` (remote Chrome restart) sets a shared Poller flag `errBrowserInterrupted` (remote Chrome restart) sets a shared Poller flag
+40
View File
@@ -621,6 +621,46 @@ func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) {
} }
} }
// Sighting deferral (issue #103) only reaches production through the PUT
// handler: the store and poller can be right and the feature still dead if the
// handler never records the report. Asserted where a client can see it - the
// series stops being due the moment the PUT lands.
func TestPutRecordsASighting(t *testing.T) {
s := newTestStore(t)
srv := newRouter(s, testConfig(), nil)
now := time.Now().UnixMilli()
hour := time.Hour.Milliseconds()
seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", now-2*hour)
due, err := s.DueForLatestCheck("asura", now-hour, now-6*hour)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
if len(due) != 1 {
t.Fatalf("due before the PUT = %d series, want 1", len(due))
}
body := `{"key":"asura:x","site":"asura","series_id":"x",
"series_url":"https://asurascans.com/comics/x",
"last_chapter":"Chapter 5","last_chapter_num":5,
"latest_chapter":"Chapter 9","latest_chapter_num":9}`
req := httptest.NewRequest(http.MethodPut, "/bookmarks/asura:x", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
srv.ServeHTTP(rec, auth(req))
if rec.Code != http.StatusOK {
t.Fatalf("PUT status = %d, want 200 (body %s)", rec.Code, rec.Body.String())
}
due, err = s.DueForLatestCheck("asura", now-hour, now-6*hour)
if err != nil {
t.Fatalf("DueForLatestCheck: %v", err)
}
if len(due) != 0 {
t.Fatalf("due after the PUT = %d series, want 0: the handler recorded no Sighting", len(due))
}
}
// The userscript route is registered outside the web UI's Discord auth, so it // The userscript route is registered outside the web UI's Discord auth, so it
// must keep working whatever the web config — see internal/userscript for the // must keep working whatever the web config — see internal/userscript for the
// handler's own behaviour. The credential in the path is the owner's derived // handler's own behaviour. The credential in the path is the owner's derived
+12 -1
View File
@@ -99,7 +99,18 @@ func (h *Handler) Put(w http.ResponseWriter, r *http.Request) {
// reading progress actually moved. Any client value is ignored. // reading progress actually moved. Any client value is ignored.
b.UpdatedAt = time.Now().UnixMilli() b.UpdatedAt = time.Now().UnixMilli()
stored, err := h.Store.Upsert(httpmw.ReaderID(r), b) // A userscript PUT is a Sighting: the Reader's browser was on the Series
// page and read its Latest Chapter (issue #103). Recorded before the
// Upsert, which is what makes the raise comparison possible, and never
// from the web UI's own read-modify-write — a Reader toggling a favourite
// has not looked at the Site and must not postpone a Poll. A failure here
// costs a deferral, not the write, so it is logged and dropped.
readerID := httpmw.ReaderID(r)
if err := h.Store.RecordSighting(readerID, b.Site, b.SeriesID, b.LatestChapterNum, b.UpdatedAt); err != nil {
log.Printf("record sighting: %v", err)
}
stored, err := h.Store.Upsert(readerID, b)
if err != nil { if err != nil {
log.Printf("upsert: %v", err) log.Printf("upsert: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError) http.Error(w, "internal error", http.StatusInternalServerError)
+44 -1
View File
@@ -250,7 +250,8 @@ func (p *Poller) runLanePass(ctx context.Context, name string, paced bool) time.
return defaultGap return defaultGap
} }
due, err := p.Store.DueForLatestCheck(name, now.Add(-s.Rest).UnixMilli()) due, err := p.Store.DueForLatestCheck(name, now.Add(-s.Rest).UnixMilli(),
now.Add(-sightingCeilingRests*s.Rest).UnixMilli())
if err != nil { if err != nil {
log.Printf("latest poll %s: due query: %v", name, err) log.Printf("latest poll %s: due query: %v", name, err)
st.Gap = defaultGap st.Gap = defaultGap
@@ -461,6 +462,11 @@ func (p *Poller) checkOne(ctx context.Context, sr store.Series) error {
return nil return nil
} }
// The Poll is the oracle for whatever Sighting last raised this Series
// (issue #103), and the judgement is free: the comparison below already
// exists, and no extra request is made to reach it.
p.judgeSighting(sr, facts.Latest.Num)
// Equality, not >, mirroring the userscript (L427): a site that retracts a // Equality, not >, mirroring the userscript (L427): a site that retracts a
// chapter should correct the stored number downward. The comparison is // chapter should correct the stored number downward. The comparison is
// against the due-query snapshot; a concurrent write in between only costs // against the due-query snapshot; a concurrent write in between only costs
@@ -481,6 +487,43 @@ func (p *Poller) checkOne(ctx context.Context, sr store.Series) error {
return nil return nil
} }
// judgeSighting settles the Sighting the Series' stored Latest Chapter is owed
// to, if any, against what the Site actually publishes. The asymmetry is the
// whole of the detection rule and is what keeps it free of false alarms: a Poll
// finding a *lower* number than stored means the Reader who raised it reported
// a chapter that does not exist, while a Poll finding a higher one is only the
// Site publishing since and means nothing about the report. Equality confirms
// the report, which is how an honest Reader earns back a mark.
//
// A Series with no attribution — the stored value is a Poll's own, or a
// previous Poll already judged the report — is nobody's to answer for.
func (p *Poller) judgeSighting(sr store.Series, found float64) {
if sr.LatestRaisedBy == nil || sr.LatestChapterNum == nil {
return
}
stored := *sr.LatestChapterNum
if found > stored {
// The report is neither confirmed nor contradicted, but it is answered:
// the value about to be stored is the Poll's own, so leaving the
// attribution would credit this Reader with the next Poll's agreement
// and blame them if the Site later retracts.
if err := p.Store.ClearSightingAttribution(sr.Site, sr.SeriesID, *sr.LatestRaisedBy); err != nil {
log.Printf("latest poll %q: clear sighting attribution: %v", sr.Key(), err)
}
return
}
if found < stored {
// Logged with both numbers and the Reader, because that is what tells a
// broken Site adapter (which marks every Reader of that Site at once)
// from one Reader deliberately lying.
log.Printf("latest poll %q: sighting contradicted: reader %d raised it to %v, site publishes %v",
sr.Key(), *sr.LatestRaisedBy, stored, found)
}
if err := p.Store.RecordSightingOutcome(sr.Site, sr.SeriesID, *sr.LatestRaisedBy, found == stored); err != nil {
log.Printf("latest poll %q: record sighting outcome: %v", sr.Key(), err)
}
}
// healCover runs prefetchCover in the background. Cover bytes come from a // healCover runs prefetchCover in the background. Cover bytes come from a
// different host — often a CDN — and heal once in a Series's life, so they // different host — often a CDN — and heal once in a Series's life, so they
// must not consume a Lane's gap: a large import with many blanks would // must not consume a Lane's gap: a large import with many blanks would
+494
View File
@@ -0,0 +1,494 @@
package latest
import (
"context"
"crypto/sha256"
"fmt"
"log"
"strings"
"testing"
"time"
"bookmarkmanager/backend/internal/store"
)
// Sightings (issue #103) are specified at the Poller seam, with the store as
// the way in: a Sighting is seeded the way handlers.Put performs one, a round
// is run against the injected fetcher and a frozen clock, and the assertions
// are the two observable facts — whether the Series was fetched, and what the
// stored Latest Chapter is afterwards. Nothing here asserts counter arithmetic
// through an internal call or reads how a deferral is represented in a row.
const (
sightingSlug = "chronicles-of-the-demon-faction-f886a8af"
sightingKey = "asura:" + sightingSlug
sightingURL = "https://asurascans.com/comics/" + sightingSlug
)
// sightingFixtureLatest is the newest chapter asuraSeriesFixture publishes.
const sightingFixtureLatest = 181.0
// sight performs one Sighting exactly as the JSON API does (handlers.Put):
// RecordSighting against the row as stored, then the Upsert that stores the
// reported value. The order is load-bearing — the raise comparison has nothing
// to compare against once the Upsert has landed — and the bookmark's own fields
// are carried over untouched, which is what a userscript PUT does when it
// echoes back the row it cached.
func sight(t *testing.T, s *store.Store, readerID int64, key string, num float64, at time.Time) {
t.Helper()
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
b, found, err := s.Get(readerID, key)
if err != nil || !found {
t.Fatalf("sight %q: get: %v found=%v", key, err, found)
}
if err := s.RecordSighting(readerID, site, seriesID, &num, at.UnixMilli()); err != nil {
t.Fatalf("sight %q: %v", key, err)
}
b.LatestChapter = fmt.Sprintf("Chapter %v", num)
b.LatestChapterNum = &num
b.UpdatedAt = at.UnixMilli()
if _, err := s.Upsert(readerID, b); err != nil {
t.Fatalf("sight %q: upsert: %v", key, err)
}
}
// secondReader is another Reader on the same database. The owner seed is the
// only reader-creation path in this package, so a second Open as a different
// owner is how a test gets one (as TestRunOnceFetchesSharedSeriesOnce does).
func secondReader(t *testing.T, dbURL string) *store.Store {
t.Helper()
other, err := store.Open(dbURL,
store.Owner{DiscordID: "second-reader", TokenHash: sha256.Sum256([]byte("second-token-hash"))},
t.TempDir(), testCoverBaseURL)
if err != nil {
t.Fatalf("Open second reader: %v", err)
}
t.Cleanup(func() { other.Close() })
return other
}
func readLatestNum(t *testing.T, s *store.Store, readerID int64, key string) float64 {
t.Helper()
b, ok, err := s.Get(readerID, key)
if err != nil || !ok {
t.Fatalf("Get %q: %v ok=%v", key, err, ok)
}
if b.LatestChapterNum == nil {
t.Fatalf("%q has no latest chapter", key)
}
return *b.LatestChapterNum
}
// A Series only one Reader bookmarks is the case where being wrong can hurt
// nobody but the Reader who reported it, so their Sighting stands in for the
// Poll and the round leaves the Series alone.
func TestSightingOnSolitarySeriesDefersPoll(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).runOnce(context.Background())
if got := f.callCount(); got != 0 {
t.Fatalf("fetched %d times after a Sighting on a solitary Series, want 0", got)
}
}
// On a shared Series the Sighting still writes the Latest Chapter for everyone,
// but the Poll happens on schedule anyway — which is what corrects a wrong
// value within the hour instead of letting it persist.
func TestSightingOnSharedSeriesDoesNotDeferPoll(t *testing.T) {
s, dbURL := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
other := secondReader(t, dbURL)
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
}); err != nil {
t.Fatalf("seed second reader: %v", err)
}
sight(t, s, s.OwnerID(), sightingKey, 200, now.Add(-10*time.Minute))
// The Sighting updated the shared row immediately, before any Poll.
if got := readLatestNum(t, s, s.OwnerID(), sightingKey); got != 200 {
t.Fatalf("latest after the Sighting = %v, want 200", got)
}
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times after a Sighting on a shared Series, want 1", got)
}
if got := readLatestNum(t, s, s.OwnerID(), sightingKey); got != sightingFixtureLatest {
t.Fatalf("latest after the Poll = %v, want the Site's own %v", got, sightingFixtureLatest)
}
}
// Reporting a chapter is not reading one: a Sighting may move the Latest
// Chapter and nothing else. Both the solitary and the shared case, because the
// deferral branch must not be where this guarantee lives.
func TestSightingLeavesProgressAndOrderingUntouched(t *testing.T) {
for _, shared := range []bool{false, true} {
name := "solitary"
if shared {
name = "shared"
}
t.Run(name, func(t *testing.T) {
s, dbURL := newTestStore(t)
read := 5.0
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, SeriesURL: sightingURL,
LastChapter: "Chapter 5", LastChapterNum: read,
LastChapterURL: sightingURL + "/chapter/5", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if shared {
other := secondReader(t, dbURL)
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
}); err != nil {
t.Fatalf("seed second reader: %v", err)
}
}
sight(t, s, s.OwnerID(), sightingKey, 200, time.UnixMilli(9_000_000))
b, ok, err := s.Get(s.OwnerID(), sightingKey)
if err != nil || !ok {
t.Fatalf("Get: %v ok=%v", err, ok)
}
if b.LatestChapterNum == nil || *b.LatestChapterNum != 200 {
t.Fatalf("LatestChapterNum = %v, want 200", b.LatestChapterNum)
}
if b.LastChapterNum != read {
t.Fatalf("LastChapterNum = %v, want %v: a Sighting is not Progress", b.LastChapterNum, read)
}
if b.UpdatedAt != 1000 {
t.Fatalf("updated_at moved to %d: a Sighting must not reorder the list", b.UpdatedAt)
}
})
}
}
// The ceiling is what makes trusting a client report safe: however recently a
// Series was sighted, one that has not been Polled in six hours is Polled.
func TestSightingCeilingForcesPoll(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-7*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-time.Minute))
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times past the %s ceiling, want 1", got, sightingCeilingRests*defaultRest)
}
}
// Deferral is decided from live facts every round, so a Series that gains a
// second Bookmark stops deferring at once — and one that loses it defers again.
func TestDeferralFollowsTheBookmarkCount(t *testing.T) {
s, dbURL := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
p := newTestPoller(t, s, f, now)
p.runOnce(context.Background())
if got := f.callCount(); got != 0 {
t.Fatalf("solitary Series fetched %d times, want 0", got)
}
other := secondReader(t, dbURL)
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
}); err != nil {
t.Fatalf("seed second reader: %v", err)
}
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("shared Series fetched %d times, want 1", got)
}
// The Poll above consumed the rest, so move past it before asking again.
if err := other.Delete(other.OwnerID(), sightingKey); err != nil {
t.Fatalf("delete second bookmark: %v", err)
}
later := now.Add(2 * time.Hour)
p.Now = func() time.Time { return later }
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, later.Add(-time.Minute))
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("Series fetched %d times after returning to one Bookmark, want 1", got)
}
}
// A Series nobody reports any more returns to the normal schedule on its own:
// the Sighting's standing lasts one rest, not forever.
func TestDeferralExpiresWithoutFurtherSightings(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
p := newTestPoller(t, s, f, now)
p.runOnce(context.Background())
if got := f.callCount(); got != 0 {
t.Fatalf("fetched %d times while the Sighting stood, want 0", got)
}
p.Now = func() time.Time { return now.Add(90 * time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times once the Sighting aged out, want 1", got)
}
}
// demonicFixture publishes one chapter in demonicscans' live page shape, so a
// test can make a Site publish an arbitrary number rather than the one the
// captured fixture froze.
func demonicFixture(num float64) string {
return fmt.Sprintf(
`<a href="/chaptered.php?manga=11799&chapter=%v" class="chplinks" title="Catastrophic Necromancer %v">Chapter %v</a>`,
num, num, num)
}
const (
demonicKey = "demonic:Catastrophic-Necromancer"
demonicURL = "https://demonicscans.org/manga/Catastrophic-Necromancer"
)
// contradictOnce reports a chapter that does not exist and then runs the round
// that catches it, returning when that round ran so a caller can chain the
// next one. The wait is one rest and a minute: a Sighting stands in for exactly
// one rest, so that is the first moment this solitary Series is Polled again.
func contradictOnce(t *testing.T, s *store.Store, p *Poller, sightAt time.Time, real float64) time.Time {
t.Helper()
sight(t, s, s.OwnerID(), demonicKey, real+500, sightAt)
at := sightAt.Add(defaultRest + time.Minute)
p.Now = func() time.Time { return at }
p.runOnce(context.Background())
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != real {
t.Fatalf("latest after the Poll = %v, want the Site's own %v", got, real)
}
return at
}
func seedDemonic(t *testing.T, s *store.Store, checkedAt int64) {
t.Helper()
seedForCheck(t, s, demonicKey, demonicURL, checkedAt)
}
// A Poll finding a lower number than stored means the Sighting that raised it
// was false. The Reader is named — not the Series flagged — and both numbers are
// logged, because that is what tells a broken adapter from a deliberate lie.
func TestPollContradictingASightingNamesTheReaderAndBothNumbers(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
var logs strings.Builder
prev := log.Writer()
log.SetOutput(&logs)
t.Cleanup(func() { log.SetOutput(prev) })
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, now)
contradictOnce(t, s, p, now, 296)
got := logs.String()
for _, want := range []string{
fmt.Sprintf("reader %d", s.OwnerID()), "796", "296", demonicKey,
} {
if !strings.Contains(got, want) {
t.Fatalf("contradiction log = %q, want it to name %q", got, want)
}
}
}
// Three contradictions cost the Reader the right to defer. Nothing here writes
// a counter: the marks are earned through Polls, which is the only way
// production produces them.
func TestThreeContradictionsStopDeferral(t *testing.T) {
s, _ := newTestStore(t)
start := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, start)
at := start
for range store.SightingDisagreementLimit {
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
}
fetchesSoFar := f.callCount()
// The marked Reader sights the same solitary Series again. It still writes
// the Latest Chapter — the penalty removes a privilege, it does not silence
// anyone — but the Poll is no longer postponed: the round below runs while a
// trusted Reader's Sighting would still be standing, and fetches anyway.
sight(t, s, s.OwnerID(), demonicKey, 900, at.Add(31*time.Minute))
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != 900 {
t.Fatalf("latest after a marked Reader's Sighting = %v, want 900", got)
}
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != fetchesSoFar+1 {
t.Fatalf("marked Reader's Sighting still deferred the Poll (fetches %d, want %d)",
got, fetchesSoFar+1)
}
}
// The owner's remedy for a mark a broken Site adapter produced restores the
// privilege without a wait and without SQL.
func TestClearingMarksRestoresDeferral(t *testing.T) {
s, _ := newTestStore(t)
start := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, start)
at := start
for range store.SightingDisagreementLimit {
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
}
if err := s.ClearReaderMarks(s.OwnerID()); err != nil {
t.Fatalf("ClearReaderMarks: %v", err)
}
fetchesSoFar := f.callCount()
sight(t, s, s.OwnerID(), demonicKey, 900, at.Add(31*time.Minute))
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != fetchesSoFar {
t.Fatalf("fetched %d times after the marks were cleared, want %d: deferral must resume",
got, fetchesSoFar)
}
}
// Recovery is automatic but expensive: twenty Polls that each confirm a
// Sighting of this Reader's clear the marks. Each round needs a new chapter,
// because only a report that raises the stored number is attributed and so only
// that one can be confirmed.
func TestTwentyAgreementsClearTheMarks(t *testing.T) {
s, _ := newTestStore(t)
start := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, start)
at := start
for range store.SightingDisagreementLimit {
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
}
chapter := 296.0
for range store.SightingAgreementsToClear {
chapter++
sight(t, s, s.OwnerID(), demonicKey, chapter, at.Add(time.Minute))
f.body = demonicFixture(chapter) // the Site publishes what was reported
at = at.Add(defaultRest + time.Minute)
p.Now = func() time.Time { return at }
p.runOnce(context.Background())
}
fetchesSoFar := f.callCount()
chapter++
sight(t, s, s.OwnerID(), demonicKey, chapter, at.Add(31*time.Minute))
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != fetchesSoFar {
t.Fatalf("fetched %d times after %d confirmations, want %d: the marks must be forgiven",
got, store.SightingAgreementsToClear, fetchesSoFar)
}
}
// A Poll finding a higher number is the Site publishing since the Sighting and
// means nothing about the Reader — no mark, and no credit either.
func TestPollFindingHigherNumberIsNotAContradiction(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
// Reported truthfully, then the Site published one more.
sight(t, s, s.OwnerID(), demonicKey, 295, now.Add(-10*time.Minute))
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, now)
// One rest on, the Sighting has lapsed and the Poll happens.
p.Now = func() time.Time { return now.Add(7 * time.Hour) }
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times past the ceiling, want 1", got)
}
// Unmarked, so a fresh Sighting still defers.
at := now.Add(9 * time.Hour)
sight(t, s, s.OwnerID(), demonicKey, 296, at.Add(-time.Minute))
p.Now = func() time.Time { return at }
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("a Reader whose report the Site overtook lost the right to defer (fetches %d, want 1)", got)
}
}
// A Poll that overtakes a Sighting takes ownership of the row: the value stored
// afterwards is the Poll's own, so a later retraction is not the Reader's fault
// and must not be charged to them.
func TestAttributionDoesNotSurviveAPollThatOvertookIt(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
sight(t, s, s.OwnerID(), demonicKey, 295, now.Add(-10*time.Minute))
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, now)
at := now.Add(defaultRest + time.Minute)
p.Now = func() time.Time { return at }
p.runOnce(context.Background())
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != 296 {
t.Fatalf("latest after the Poll = %v, want the Site's own 296", got)
}
var logs strings.Builder
prev := log.Writer()
log.SetOutput(&logs)
t.Cleanup(func() { log.SetOutput(prev) })
f.body = demonicFixture(290) // the Site retracts what only the Poll wrote
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
p.runOnce(context.Background())
if strings.Contains(logs.String(), "sighting contradicted") {
t.Fatalf("a retraction of the Poll's own value was charged to a Reader: %s", logs.String())
}
}
// A PUT with no Latest Chapter in it — a favourite toggle, progress written
// from a chapter page — is nobody looking at the Series page, so it buys no
// deferral. Otherwise a client could suppress a Series' Polls while reporting
// nothing, and with nothing reported there would be nothing to judge.
func TestPutWithoutALatestChapterDoesNotDefer(t *testing.T) {
s, _ := newTestStore(t)
now := time.UnixMilli(20 * time.Hour.Milliseconds())
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
// The handler's own call, with the field the client omitted.
if err := s.RecordSighting(s.OwnerID(), "demonic", "Catastrophic-Necromancer",
nil, now.Add(-time.Minute).UnixMilli()); err != nil {
t.Fatalf("RecordSighting: %v", err)
}
f := &fakeFetcher{body: demonicFixture(296), status: 200}
p := newTestPoller(t, s, f, now)
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times after a PUT carrying no chapter, want 1", got)
}
}
+8
View File
@@ -409,6 +409,14 @@ const (
// asleep (ADR-0005 on-demand browser). // asleep (ADR-0005 on-demand browser).
browserWakeCount = 5 browserWakeCount = 5
browserWakeAge = 15 * time.Minute browserWakeAge = 15 * time.Minute
// sightingCeilingRests caps Sighting deferral (issue #103): however many
// Sightings arrive, a Series unpolled for this many of its Site's rests is
// Polled. It is what makes a client report safe to trust — a wrong Latest
// Chapter dies within the ceiling deterministically, rather than in
// expectation the way a randomised audit would have it. Six, so a Series a
// Reader visits constantly still gets one authoritative check per working
// day-part.
sightingCeilingRests = 6
) )
// effectiveGap is a Site's pace: the registry gap, or one rest divided by the // effectiveGap is a Site's pace: the registry gap, or one rest divided by the
@@ -0,0 +1,10 @@
-- Sighting deferral (issue #103). latest_sighted_at is when a Reader's report
-- last stood in for a Poll; it is separate from latest_checked_at because the
-- six-hour ceiling has to know when the Series was last really fetched, and a
-- Sighting writing the Poll's own column would erase that.
-- latest_raised_by is attribution: whoever last raised this Series' Latest
-- Chapter by Sighting, so a Poll that contradicts the value downwards names a
-- Reader rather than flagging a row. Cleared by the Poll that judges it, NULL
-- whenever the stored value is the Poll's own.
ALTER TABLE series ADD COLUMN latest_sighted_at bigint NOT NULL DEFAULT 0;
ALTER TABLE series ADD COLUMN latest_raised_by bigint REFERENCES readers(id) ON DELETE SET NULL;
+139 -6
View File
@@ -79,6 +79,11 @@ type Series struct {
LatestChapter string LatestChapter string
LatestChapterNum *float64 // nil until first captured LatestChapterNum *float64 // nil until first captured
LatestCheckedAt int64 // unix ms; see MarkLatestChecked LatestCheckedAt int64 // unix ms; see MarkLatestChecked
// LatestRaisedBy is the Reader whose Sighting last raised LatestChapter,
// and nil when the stored value is a Poll's own finding. It is what lets a
// Poll that contradicts the value downwards name a Reader instead of
// merely flagging the row (issue #103); the Poll that judges it clears it.
LatestRaisedBy *int64
// readerCount is the number of bookmarks referencing this series, filled // readerCount is the number of bookmarks referencing this series, filled
// only by the due-queue query that orders on it. // only by the due-queue query that orders on it.
@@ -195,7 +200,7 @@ const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover_add
// due query. latest_checked_at lives only on series — see MarkLatestChecked // due query. latest_checked_at lives only on series — see MarkLatestChecked
// for why it stays off every client-visible write. // for why it stays off every client-visible write.
const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover, s.cover_address, const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover, s.cover_address,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at` s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at, s.latest_raised_by`
// Owner is the person running the service: the first Reader, seeded at startup // Owner is the person running the service: the first Reader, seeded at startup
// so a fresh deployment has a library before anyone logs in. The seed makes // so a fresh deployment has a library before anyone logs in. The seed makes
@@ -584,16 +589,17 @@ func (s *Store) scanBookmark(scan func(...any) error) (Bookmark, error) {
} }
// scanSeries reads one row in seriesColumns order, plus the due query's // scanSeries reads one row in seriesColumns order, plus the due query's
// reader_count column. latest_chapter_num is NULL until the first capture, // reader_count column. latest_chapter_num and latest_raised_by are both
// same as on the bookmark read path. // nullable, same as latest_chapter_num on the bookmark read path.
func scanSeries(scan func(...any) error) (Series, error) { func scanSeries(scan func(...any) error) (Series, error) {
var ( var (
sr Series sr Series
latestChapterNum sql.NullFloat64 latestChapterNum sql.NullFloat64
latestRaisedBy sql.NullInt64
) )
if err := scan( if err := scan(
&sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover, &sr.CoverAddress, &sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover, &sr.CoverAddress,
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt, &sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt, &latestRaisedBy,
&sr.readerCount, &sr.readerCount,
); err != nil { ); err != nil {
return Series{}, err return Series{}, err
@@ -601,6 +607,9 @@ func scanSeries(scan func(...any) error) (Series, error) {
if latestChapterNum.Valid { if latestChapterNum.Valid {
sr.LatestChapterNum = &latestChapterNum.Float64 sr.LatestChapterNum = &latestChapterNum.Float64
} }
if latestRaisedBy.Valid {
sr.LatestRaisedBy = &latestRaisedBy.Int64
}
return sr, nil return sr, nil
} }
@@ -947,7 +956,20 @@ func (s *Store) Delete(readerID int64, key string) error {
// burns requests. Archived bookmarks still count — knowing what a shelved // burns requests. Archived bookmarks still count — knowing what a shelved
// series is up to is the whole reason for archiving instead of deleting. // series is up to is the whole reason for archiving instead of deleting.
// A series with no bookmarks at all never appears: the join excludes it. // A series with no bookmarks at all never appears: the join excludes it.
func (s *Store) DueForLatestCheck(site string, cutoffMs int64) ([]Series, error) { //
// ceilingMs is the Sighting deferral ceiling (issue #103): a Series whose last
// real Poll is older than it appears however recently it was sighted. That is
// what bounds the whole mechanism — a wrong Latest Chapter dies within the
// ceiling deterministically rather than in expectation. Deferral itself is
// decided here, from two facts the query already computes, so a Lane gains no
// query per round: a Sighting younger than cutoffMs holds the Series back, but
// only while COUNT(*) is 1. A Series a second Reader bookmarks is Polled on
// schedule, so a wrong value the whole guild can see is corrected by a check
// that was never postponed; on a solitary Series the only person a wrong value
// reaches is the Reader who reported it. Whether the reporting Reader is
// allowed to defer at all was settled when the Sighting was recorded — see
// RecordSighting.
func (s *Store) DueForLatestCheck(site string, cutoffMs, ceilingMs int64) ([]Series, error) {
rows, err := s.db.Query(`SELECT `+seriesColumns+`, COUNT(*) AS reader_count rows, err := s.db.Query(`SELECT `+seriesColumns+`, COUNT(*) AS reader_count
FROM series s FROM series s
JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id
@@ -957,7 +979,10 @@ func (s *Store) DueForLatestCheck(site string, cutoffMs int64) ([]Series, error)
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover, GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at
HAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0 HAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
ORDER BY reader_count DESC, s.latest_checked_at ASC`, site, cutoffMs) AND (COUNT(*) > 1
OR s.latest_sighted_at <= $2::bigint
OR s.latest_checked_at <= $3::bigint)
ORDER BY reader_count DESC, s.latest_checked_at ASC`, site, cutoffMs, ceilingMs)
if err != nil { if err != nil {
return nil, fmt.Errorf("query due series: %w", err) return nil, fmt.Errorf("query due series: %w", err)
} }
@@ -1044,3 +1069,111 @@ func (s *Store) SetLatestChapter(site, seriesID, label string, num float64) erro
} }
return nil return nil
} }
// RecordSighting notes that a Reader's browser reported this Series' Latest
// Chapter, which is the half of a Sighting the client body cannot express
// (issue #103). It must be called *before* the Upsert that stores the reported
// value: the raise test compares against what is still on the row, and after
// the Upsert there is nothing left to compare with. A Series that does not
// exist yet — the first Bookmark of it — is not a Sighting at all: nothing has
// ever been Polled, so there is nothing to defer and nobody to attribute.
//
// Two independent effects, hence the two CASE arms. The deferral stamp is only
// written for a Reader below the disagreement limit, so a marked Reader's
// reports keep updating the Latest Chapter but stop postponing anything, and
// clearing their marks restores the privilege on their next Sighting. The
// attribution is written whenever the report raises the stored number,
// including for a marked Reader — their Sightings are still judged, which is
// how they earn the privilege back.
//
// num is the reported chapter number. A PUT that carries none — a favourite
// toggle, or progress written from a chapter page — is no Sighting at all:
// nobody read the Series page, so there is nothing to stand in for a Poll and
// nothing that could later be judged.
func (s *Store) RecordSighting(readerID int64, site, seriesID string, num *float64, ts int64) error {
if num == nil {
return nil
}
if _, err := s.db.Exec(`
UPDATE series SET
latest_sighted_at = CASE
WHEN (SELECT sighting_disagreements FROM readers WHERE id = $3) < $6
THEN $4::bigint ELSE latest_sighted_at END,
latest_raised_by = CASE
WHEN latest_chapter_num IS NULL OR $5::double precision > latest_chapter_num
THEN $3::bigint ELSE latest_raised_by END
WHERE site = $1 AND series_id = $2`,
site, seriesID, readerID, ts, *num, SightingDisagreementLimit); err != nil {
return fmt.Errorf("record sighting %s:%s: %w", site, seriesID, err)
}
return nil
}
// SightingAgreementsToClear is how many Polls must confirm a Reader's
// Sightings in a row before their disagreements are forgiven. An agreement is
// only recorded when a Poll later confirms a Sighting, so this is twenty Polls
// of Series that Reader bookmarks — hours to days, not twenty page views. That
// is the intended price: recovery is automatic but cannot be outwaited, and a
// disagreement resets the run to zero, so credit cannot be banked in advance.
const SightingAgreementsToClear = 20
// RecordSightingOutcome settles what a Poll decided about the Reader whose
// Sighting last raised this Series' Latest Chapter, and clears the attribution
// in the same transaction so one Sighting is judged exactly once. agreed is
// the Poll confirming the stored value; its opposite is the Poll finding a
// lower number, which means the raise was false.
//
// A Poll finding a *higher* number is neither — the Site published — and takes
// ClearSightingAttribution instead.
func (s *Store) RecordSightingOutcome(site, seriesID string, readerID int64, agreed bool) error {
tx, err := s.db.Begin()
if err != nil {
return fmt.Errorf("begin sighting outcome %s:%s: %w", site, seriesID, err)
}
defer tx.Rollback()
// The run length is what "consecutive" means: a disagreement zeroes the
// agreements, and completing a run zeroes both, so the next run starts
// from nothing rather than forgiving every later disagreement instantly.
q := `UPDATE readers SET sighting_disagreements = sighting_disagreements + 1,
sighting_agreements = 0
WHERE id = $1`
args := []any{readerID}
if agreed {
q = `UPDATE readers SET
sighting_agreements = CASE WHEN sighting_agreements + 1 >= $2 THEN 0
ELSE sighting_agreements + 1 END,
sighting_disagreements = CASE WHEN sighting_agreements + 1 >= $2 THEN 0
ELSE sighting_disagreements END
WHERE id = $1`
args = append(args, SightingAgreementsToClear)
}
if _, err := tx.Exec(q, args...); err != nil {
return fmt.Errorf("record sighting outcome for reader %d: %w", readerID, err)
}
if _, err := tx.Exec(clearAttributionSQL, site, seriesID, readerID); err != nil {
return fmt.Errorf("clear sighting attribution %s:%s: %w", site, seriesID, err)
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("commit sighting outcome %s:%s: %w", site, seriesID, err)
}
return nil
}
// ClearSightingAttribution answers a Sighting without judging it: the Poll
// found a higher number, so the value about to be stored is its own and this
// Reader is no longer answerable for the row. Without it the next Poll's
// agreement would be credited to a Reader who did not earn it.
func (s *Store) ClearSightingAttribution(site, seriesID string, readerID int64) error {
if _, err := s.db.Exec(clearAttributionSQL, site, seriesID, readerID); err != nil {
return fmt.Errorf("clear sighting attribution %s:%s: %w", site, seriesID, err)
}
return nil
}
// clearAttributionSQL drops the attribution only while it still names the
// Reader being judged: a Sighting landing between the due query's snapshot and
// this write is a fresh, unjudged one and must not be erased by the previous
// one's verdict.
const clearAttributionSQL = `UPDATE series SET latest_raised_by = NULL
WHERE site = $1 AND series_id = $2 AND latest_raised_by = $3`
+11 -7
View File
@@ -277,6 +277,10 @@ func seedForCheck(t *testing.T, s *Store, key, seriesURL string, checkedAt int64
} }
} }
// noCeiling is a Sighting deferral ceiling no Series can reach, for the tests
// that predate the ceiling and are about rest, ordering or buckets instead.
const noCeiling = int64(-1)
func TestDueForLatestCheck(t *testing.T) { func TestDueForLatestCheck(t *testing.T) {
const hour = int64(3600_000) const hour = int64(3600_000)
now := 10 * hour now := 10 * hour
@@ -298,7 +302,7 @@ func TestDueForLatestCheck(t *testing.T) {
s := newTestStore(t) s := newTestStore(t)
seedForCheck(t, s, "asura:x", tt.seriesURL, tt.checkedAt) seedForCheck(t, s, "asura:x", tt.seriesURL, tt.checkedAt)
due, err := s.DueForLatestCheck("asura", now-hour) due, err := s.DueForLatestCheck("asura", now-hour, noCeiling)
if err != nil { if err != nil {
t.Fatalf("DueForLatestCheck: %v", err) t.Fatalf("DueForLatestCheck: %v", err)
} }
@@ -319,7 +323,7 @@ func TestDueForLatestCheckOldestFirstAndScopedToSite(t *testing.T) {
// asks for one Site, and no Lane may see another's queue. // asks for one Site, and no Lane may see another's queue.
seedForCheck(t, s, "demonic:z", "https://demonicscans.org/manga/z", 0) seedForCheck(t, s, "demonic:z", "https://demonicscans.org/manga/z", 0)
due, err := s.DueForLatestCheck("asura", 1000) due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
if err != nil { if err != nil {
t.Fatalf("DueForLatestCheck: %v", err) t.Fatalf("DueForLatestCheck: %v", err)
} }
@@ -499,7 +503,7 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
} }
} }
due, err := store.DueForLatestCheck("asura", time.Now().UnixMilli()) due, err := store.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
if err != nil { if err != nil {
t.Fatalf("DueForLatestCheck: %v", err) t.Fatalf("DueForLatestCheck: %v", err)
} }
@@ -1005,7 +1009,7 @@ func TestDueForLatestCheckOrdersByReaderCountThenAge(t *testing.T) {
seedSecondReader(t, s, "asura:pop:2", "asura", "pop", 1001) seedSecondReader(t, s, "asura:pop:2", "asura", "pop", 1001)
seedForCheck(t, s, "asura:solo", "https://asurascans.com/comics/solo", 100) seedForCheck(t, s, "asura:solo", "https://asurascans.com/comics/solo", 100)
due, err := s.DueForLatestCheck("asura", 1000) due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
if err != nil { if err != nil {
t.Fatalf("DueForLatestCheck: %v", err) t.Fatalf("DueForLatestCheck: %v", err)
} }
@@ -1031,7 +1035,7 @@ func TestDueForLatestCheckExcludesOrphanSeries(t *testing.T) {
t.Fatalf("seed orphan series: %v", err) t.Fatalf("seed orphan series: %v", err)
} }
due, err := s.DueForLatestCheck("asura", 1000) due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
if err != nil { if err != nil {
t.Fatalf("DueForLatestCheck: %v", err) t.Fatalf("DueForLatestCheck: %v", err)
} }
@@ -1449,7 +1453,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
t.Fatalf("series rows = %d, want 1 shared row for two bookmarks", series) t.Fatalf("series rows = %d, want 1 shared row for two bookmarks", series)
} }
due, err := s.DueForLatestCheck("asura", time.Now().UnixMilli()) due, err := s.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
if err != nil { if err != nil {
t.Fatalf("DueForLatestCheck: %v", err) t.Fatalf("DueForLatestCheck: %v", err)
} }
@@ -1465,7 +1469,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
if b, ok, err := s.Get(s.OwnerID(), "asura:solo"); err != nil || !ok || b.LastChapterNum != 200 { if b, ok, err := s.Get(s.OwnerID(), "asura:solo"); err != nil || !ok || b.LastChapterNum != 200 {
t.Fatalf("owner's bookmark after the other's delete = %+v ok=%v err=%v, want it intact", b, ok, err) t.Fatalf("owner's bookmark after the other's delete = %+v ok=%v err=%v, want it intact", b, ok, err)
} }
due, err = s.DueForLatestCheck("asura", time.Now().UnixMilli()) due, err = s.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
if err != nil { if err != nil {
t.Fatalf("DueForLatestCheck after delete: %v", err) t.Fatalf("DueForLatestCheck after delete: %v", err)
} }
@@ -0,0 +1,139 @@
# ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them
Date: 2026-08-16
Status: accepted
## Decision
A **Sighting** is the Latest Chapter the Reader's own browser read off the
Series page and PUT to the backend. It is now allowed to stand in for a Poll,
under one restriction and one ceiling:
- **Solitary Series only.** A Sighting defers the Poll of a Series exactly one
Bookmark points at. A Series two Readers share is Polled on schedule no matter
how recently it was sighted.
- **One rest of standing.** A Sighting postpones Polls for one Rest
(`defaultRest`, an hour), not forever: a Series nobody visits again returns to
the normal schedule by itself.
- **Six-rest ceiling.** `sightingCeilingRests = 6`, counted in the Site's own
Rest — six hours everywhere today. However many Sightings arrive, a Series
unpolled that long is Polled.
Both live in the due query's HAVING clause (`store.DueForLatestCheck`), beside
the Rest cutoff — the same place the schedule has always been decided, so no
timer and no second code path can disagree with it.
Attribution and judgement:
- `Store.RecordSighting` runs *before* the Upsert that stores the reported
value, because the raise test needs the row as it stands. A report that raises
the stored Latest Chapter names its Reader in `series.latest_raised_by`.
- The Poll is the oracle. `Poller.checkOne` already compares what the Site
publishes against what is stored, so judgement costs no extra request: a lower
number contradicts the Sighting (`sighting_disagreements + 1`, both numbers and
the Reader logged), the same number confirms it (`sighting_agreements + 1`), a
**higher** number is the Site publishing and means nothing either way — but it
does clear the attribution (`Store.ClearSightingAttribution`), because the
value stored afterwards is the Poll's own and nobody must answer for it.
- At `SightingDisagreementLimit` (3) that Reader's Sightings stop deferring
anything. They still write the Latest Chapter — the penalty removes a
privilege, it does not silence anyone.
- `SightingAgreementsToClear` (20) consecutive confirmations forgive the
disagreements. A disagreement resets the run to zero.
- The owner clears marks from the administration page (issue #102, shipped
first precisely so a false mark has a remedy the day the mechanism lands).
One client change was required, and only one. Both userscripts stopped short of
PUTting a read whose number had not moved (`applyLatestChapterIfChanged`), so
the case this whole mechanism exists for — visiting a Series with nothing new —
never reached the backend. `reportLatestChapter` now sends it, skipping only the
local write and the re-render. A numberless PUT (favourite toggle, progress from
a chapter page) is not a Sighting and defers nothing: nobody read the Series
page, so there would be nothing to judge later.
## Why
Most of the backend's work was redundant. The userscript reads the Latest
Chapter on every Series page visit; minutes later the Poll Lane fetches the same
page for the same number. Deferring on a report converts a visit into a Poll
saved, which is Lane capacity handed back to Series nobody is
reading.
The restriction is the whole safety argument, and it is about **blast radius**,
not about trust arithmetic:
- On a solitary Series, a wrong report can only mislead the Reader who made it.
There is nobody else's ember to falsify.
- On a shared Series it could mislead someone else, so a report never postpones
anything there.
The ceiling bounds the damage in time: a false value dies within six hours
whatever happens, because the Poll that finds it is guaranteed. That is also
what makes lying pointless — the six-hour audit is certain, not sampled, so a
determined attacker buys at most three ceilings' worth of a wrong number on
their own Series and then loses deferral entirely.
The cost of recovery is deliberate. An agreement is only recorded when a later
Poll confirms a Sighting, so twenty agreements are twenty Polls of Series that
Reader bookmarks — hours to days of real time, not twenty page views. Waiting is
therefore not a strategy, and credit cannot be banked in advance.
## Tradeoffs and rejections
- **Trusting a Sighting on a shared Series** rejected: it is the only case where
one Reader's mistake reaches another Reader's list, and no amount of
reputation makes that recoverable within the six-hour window.
- **Cross-Reader agreement, voting, weighting, consensus scoring** rejected on
evidence: every truth-discovery method estimates source reliability by
comparing sources on the same object, and the standard survey states outright
that an object provided by very few sources cannot have its confidence
evaluated — Li, Gao, Meng, Li, Su, Zhao, Fan, Han, *A Survey on Truth
Discovery*, SIGMOD Record 45(1), 2016 (arXiv:1505.02463), §"Challenges" on
sparse sources. With the two Readers this backend actually has, a
disagreement is a coin flip. The Poll is an authoritative oracle, so it is
the only judge.
- **A randomised audit** (Poll a fraction of deferred Series) rejected in favour
of the fixed ceiling. Sampling an oracle against untrusted reports is the
gold-question technique from crowdsourcing quality control — Le, Edmonds,
Hester, Biewald, *Ensuring quality in crowdsourced search relevance
evaluation: the effects of training question distribution*, SIGIR 2010
Workshop on Crowdsourcing for Search Evaluation, which inserts known answers
sporadically and adjusts each worker's trust from them. The ceiling is the
same idea made deterministic: sampling prices an attack in expectation, a
guaranteed six-hour audit prices it as a certainty, which is what makes the
solitary-Series rule defensible in one sentence.
- **A trust *ratio*** (agreements over judgements, as that same gold-question
scheme uses) rejected for two thresholds: a ratio lets an attacker bank
credit first and spend it on lies later, and it needs the owner watching a
score to act. Three-and-twenty is a threshold both ways — a disagreement
resets the run to zero, so credit cannot be pre-bought, and recovery happens
without the owner in the loop.
- **Blocking a marked Reader's writes** rejected: the Latest Chapter they report
is still the best available value, and their Sightings must keep being judged
or they could never earn the privilege back.
- **Per-Series flagging** rejected in favour of per-Reader marks: a Series is
not the thing that can be wrong. Naming the Reader and logging both numbers is
also what distinguishes a broken Site adapter (every Reader of that Site
contradicted at once) from one bad actor.
- **Timers or a background reputation job** rejected: deferral is recomputed
from live facts every round — Bookmark count and sighting timestamp — so a
Series that gains a second Bookmark stops deferring at once, with nothing to
invalidate. The Reader's marks are the one input read earlier, when the
Sighting is recorded rather than when the round runs: a Reader who crosses
the threshold, or has their marks cleared, changes behaviour from their next
Sighting on, and the standing they already bought lasts out its rest. That is
bounded by one rest and costs one subselect instead of joining `readers` into
the due query on every round.
## Constraints preserved
- A Sighting is not Progress: it may move the Latest Chapter and nothing else.
`updated_at` never moves, so a report cannot reorder the list (ADR-0004).
- The Latest Chapter is a Series-level fact (ADR-0003): a Sighting writes the
shared row, so every Reader of a shared Series sees it immediately — deferral
is the only thing the solitary rule withholds.
- Ember means new chapter only (`docs/design-system.md`): a marked Reader
renders no differently in their own list, and nothing about the trust model
reaches the Series list's colour.
- The Poll remains authoritative. Where a Sighting and a Poll disagree, the
Poll's value is what gets stored.
+2
View File
@@ -40,6 +40,7 @@
"38": "novel-logic.test.js", "38": "novel-logic.test.js",
"39": "UI Critique 2026-07-26A", "39": "UI Critique 2026-07-26A",
"40": "UI Critique 2026-07-26B", "40": "UI Critique 2026-07-26B",
"41": "Handler",
"42": "pgtest.go", "42": "pgtest.go",
"43": "Issue Tracker & Triage", "43": "Issue Tracker & Triage",
"44": "Ticket Workflow", "44": "Ticket Workflow",
@@ -151,6 +152,7 @@
"150": "Reviewer Subagent (opencode)", "150": "Reviewer Subagent (opencode)",
"151": "Finding Severity Rubric", "151": "Finding Severity Rubric",
"152": "Spec Compliance Review", "152": "Spec Compliance Review",
"154": "ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them",
"156": "ResponseWriter", "156": "ResponseWriter",
"161": "Why Use samber/oops", "161": "Why Use samber/oops",
"162": "singleflight Cache Stampede Prevention", "162": "singleflight Cache Stampede Prevention",
+40 -34
View File
@@ -1,16 +1,16 @@
# Graph Report - mangaBookmark (2026-08-16) # Graph Report - mangaBookmark (2026-08-16)
## Corpus Check ## Corpus Check
- 116 files · ~284,419 words - 119 files · ~290,953 words
- Verdict: corpus is large enough that graph structure adds value. - Verdict: corpus is large enough that graph structure adds value.
## Summary ## Summary
- 1701 nodes · 3463 edges · 219 communities (69 shown, 150 thin omitted) - 1736 nodes · 3597 edges · 221 communities (70 shown, 151 thin omitted)
- Extraction: 91% EXTRACTED · 9% INFERRED · 0% AMBIGUOUS · INFERRED: 326 edges (avg confidence: 0.77) - Extraction: 90% EXTRACTED · 10% INFERRED · 0% AMBIGUOUS · INFERRED: 361 edges (avg confidence: 0.78)
- Token cost: 0 input · 0 output - Token cost: 0 input · 0 output
## Graph Freshness ## Graph Freshness
- Built from commit: `58014eb8` - Built from commit: `56afb9f2`
- Run `git rev-parse HEAD` and compare to check if the graph is stale. - Run `git rev-parse HEAD` and compare to check if the graph is stale.
- Run `graphify update .` after code changes (no API cost). - Run `graphify update .` after code changes (no API cost).
@@ -55,6 +55,7 @@
- [[_COMMUNITY_novel-logic.test.js|novel-logic.test.js]] - [[_COMMUNITY_novel-logic.test.js|novel-logic.test.js]]
- [[_COMMUNITY_UI Critique 2026-07-26A|UI Critique 2026-07-26A]] - [[_COMMUNITY_UI Critique 2026-07-26A|UI Critique 2026-07-26A]]
- [[_COMMUNITY_UI Critique 2026-07-26B|UI Critique 2026-07-26B]] - [[_COMMUNITY_UI Critique 2026-07-26B|UI Critique 2026-07-26B]]
- [[_COMMUNITY_Handler|Handler]]
- [[_COMMUNITY_pgtest.go|pgtest.go]] - [[_COMMUNITY_pgtest.go|pgtest.go]]
- [[_COMMUNITY_Issue Tracker & Triage|Issue Tracker & Triage]] - [[_COMMUNITY_Issue Tracker & Triage|Issue Tracker & Triage]]
- [[_COMMUNITY_Ticket Workflow|Ticket Workflow]] - [[_COMMUNITY_Ticket Workflow|Ticket Workflow]]
@@ -166,6 +167,7 @@
- [[_COMMUNITY_Reviewer Subagent (opencode)|Reviewer Subagent (opencode)]] - [[_COMMUNITY_Reviewer Subagent (opencode)|Reviewer Subagent (opencode)]]
- [[_COMMUNITY_Finding Severity Rubric|Finding Severity Rubric]] - [[_COMMUNITY_Finding Severity Rubric|Finding Severity Rubric]]
- [[_COMMUNITY_Spec Compliance Review|Spec Compliance Review]] - [[_COMMUNITY_Spec Compliance Review|Spec Compliance Review]]
- [[_COMMUNITY_ADR-0011 Sightings — a Reader report defers a Poll where being wrong hurts only them|ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them]]
- [[_COMMUNITY_ResponseWriter|ResponseWriter]] - [[_COMMUNITY_ResponseWriter|ResponseWriter]]
- [[_COMMUNITY_Why Use samberoops|Why Use samber/oops]] - [[_COMMUNITY_Why Use samberoops|Why Use samber/oops]]
- [[_COMMUNITY_singleflight Cache Stampede Prevention|singleflight Cache Stampede Prevention]] - [[_COMMUNITY_singleflight Cache Stampede Prevention|singleflight Cache Stampede Prevention]]
@@ -235,16 +237,16 @@
- [[_COMMUNITY_Userscript CLAUDE guidance|Userscript CLAUDE guidance]] - [[_COMMUNITY_Userscript CLAUDE guidance|Userscript CLAUDE guidance]]
## God Nodes (most connected - your core abstractions) ## God Nodes (most connected - your core abstractions)
1. `testConfig()` - 54 edges 1. `newTestStore()` - 62 edges
2. `newTestStore()` - 49 edges 2. `testConfig()` - 55 edges
3. `newWebTestServer()` - 49 edges 3. `newWebTestServer()` - 49 edges
4. `newTestStore()` - 44 edges 4. `newTestStore()` - 44 edges
5. `e()` - 33 edges 5. `e()` - 33 edges
6. `Open()` - 30 edges 6. `Store` - 31 edges
7. `Handler` - 29 edges 7. `Open()` - 31 edges
8. `Store` - 28 edges 8. `newTestPoller()` - 30 edges
9. `ne()` - 28 edges 9. `Handler` - 29 edges
10. `De()` - 28 edges 10. `ne()` - 28 edges
## Surprising Connections (you probably didn't know these) ## Surprising Connections (you probably didn't know these)
- `el()` --indirect_call--> `c()` [INFERRED] - `el()` --indirect_call--> `c()` [INFERRED]
@@ -268,7 +270,7 @@
- **Headless browser infrastructure (sidecar, on-demand, home deployment)** — docs_adr_0005_on_demand_browser_headless_shell, docs_adr_0005_on_demand_browser_cdp, docs_adr_0005_on_demand_browser_on_demand_start, docs_adr_0006_browser_on_the_home_machine_home_machine_rationale [INFERRED 0.85] - **Headless browser infrastructure (sidecar, on-demand, home deployment)** — docs_adr_0005_on_demand_browser_headless_shell, docs_adr_0005_on_demand_browser_cdp, docs_adr_0005_on_demand_browser_on_demand_start, docs_adr_0006_browser_on_the_home_machine_home_machine_rationale [INFERRED 0.85]
- **lightnovelworld series-identity investigation and fix** — docs_research_lightnovelworld_chapter_vs_series_slug_issue_77, docs_research_lightnovelworld_chapter_vs_series_slug_unscoped_regex, docs_adr_0008_series_identity_is_discovered_not_derived_discovered_identity [INFERRED 0.85] - **lightnovelworld series-identity investigation and fix** — docs_research_lightnovelworld_chapter_vs_series_slug_issue_77, docs_research_lightnovelworld_chapter_vs_series_slug_unscoped_regex, docs_adr_0008_series_identity_is_discovered_not_derived_discovered_identity [INFERRED 0.85]
## Communities (219 total, 150 thin omitted) ## Communities (221 total, 151 thin omitted)
### Community 0 - "HTMX Library Internals" ### Community 0 - "HTMX Library Internals"
Cohesion: 0.08 Cohesion: 0.08
@@ -280,23 +282,23 @@ Nodes (89): floatPtr(), testConfig(), getCover(), Cookie, Handler, ResponseRecor
### Community 2 - "Manga Userscript Adapters" ### Community 2 - "Manga Userscript Adapters"
Cohesion: 0.06 Cohesion: 0.06
Nodes (76): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+68 more) Nodes (77): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), armDwell() (+69 more)
### Community 3 - "Novel Userscript Adapters" ### Community 3 - "Novel Userscript Adapters"
Cohesion: 0.06 Cohesion: 0.06
Nodes (79): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+71 more) Nodes (78): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLnwStaleRowRepair() (+70 more)
### Community 4 - "Series Acquisition Tests" ### Community 4 - "Series Acquisition Tests"
Cohesion: 0.08 Cohesion: 0.07
Nodes (78): bookmarkNewKaganeSeries(), bookmarkNewNovelfullSeries(), bookmarkNewSeries(), Context, Store, T, newAcquirer(), readBookmark() (+70 more) Nodes (103): bookmarkNewKaganeSeries(), bookmarkNewNovelfullSeries(), bookmarkNewSeries(), Context, Store, T, newAcquirer(), readBookmark() (+95 more)
### Community 5 - "Bookmarks API Tests" ### Community 5 - "Bookmarks API Tests"
Cohesion: 0.07 Cohesion: 0.07
Nodes (70): auth(), getBookmarks(), Handler, Request, Store, T, newTestServer(), newTestStore() (+62 more) Nodes (71): auth(), getBookmarks(), Handler, Request, Store, T, newTestServer(), newTestStore() (+63 more)
### Community 7 - "Cover & Acquire Internals" ### Community 7 - "Cover & Acquire Internals"
Cohesion: 0.07 Cohesion: 0.06
Nodes (43): Addr, fakeLanes, Context, Store, WaitGroup, defaultCoverResolver(), fetchCoverBytes(), Client (+35 more) Nodes (43): Addr, fakeLanes, Context, Store, WaitGroup, isInterstitial(), defaultCoverResolver(), fetchCoverBytes() (+35 more)
### Community 8 - "System Architecture Concepts" ### Community 8 - "System Architecture Concepts"
Cohesion: 0.15 Cohesion: 0.15
@@ -318,10 +320,6 @@ Nodes (45): Store, T, newTestStore(), readLatestCheckedAt(), readSeries(), secon
Cohesion: 0.08 Cohesion: 0.08
Nodes (32): Handler, Request, ResponseWriter, Store, Healthz(), writeJSON(), Auth(), compressible() (+24 more) Nodes (32): Handler, Request, ResponseWriter, Store, Healthz(), writeJSON(), Auth(), compressible() (+24 more)
### Community 13 - "Web UI Handlers"
Cohesion: 0.14
Nodes (18): currentLib(), currentTab(), filterBookmarks(), Client, HandlerFunc, Request, ResponseWriter, Store (+10 more)
### Community 14 - "Go Error Handling" ### Community 14 - "Go Error Handling"
Cohesion: 0.06 Cohesion: 0.06
Nodes (33): Creating Errors, Custom Error Types, Custom types that wrap other errors, Decision table: which error strategy to use, Error Creation, Error String Conventions, Errors as Values, `errors.New` — static error messages (+25 more) Nodes (33): Creating Errors, Custom Error Types, Custom types that wrap other errors, Decision table: which error strategy to use, Error Creation, Error String Conventions, Errors as Values, `errors.New` — static error messages (+25 more)
@@ -368,7 +366,7 @@ Nodes (12): coverResponse(), Request, T, TestCoverFetcherCanonicalisesJpgAlias()
### Community 26 - "Open" ### Community 26 - "Open"
Cohesion: 0.05 Cohesion: 0.05
Nodes (62): awaitPromise(), browserConnectionLost(), classifyBrowserError(), comixRead(), comixSeriesPageURL(), Action, Context, Mutex (+54 more) Nodes (61): awaitPromise(), browserConnectionLost(), classifyBrowserError(), comixRead(), comixSeriesPageURL(), Action, Context, Mutex (+53 more)
### Community 27 - "Find Skills Guide" ### Community 27 - "Find Skills Guide"
Cohesion: 0.14 Cohesion: 0.14
@@ -387,8 +385,8 @@ Cohesion: 0.40
Nodes (5): Map of pointers for large, frequently updated structs, Memory Layout, Pointer receivers for large structs, Struct field alignment, Zero-size field at end of struct Nodes (5): Map of pointers for large, frequently updated structs, Memory Layout, Pointer receivers for large structs, Struct field alignment, Zero-size field at end of struct
### Community 31 - "Store" ### Community 31 - "Store"
Cohesion: 0.08 Cohesion: 0.09
Nodes (8): coverRelativePath(), coverSourceAddress(), displayChapter(), Store, scanSeries(), TestDisplayChapter(), Bookmark, ReaderSummary Nodes (5): coverRelativePath(), coverSourceAddress(), Store, scanSeries(), ReaderSummary
### Community 33 - "Go Testing Guide" ### Community 33 - "Go Testing Guide"
Cohesion: 0.20 Cohesion: 0.20
@@ -418,6 +416,10 @@ Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations,
Cohesion: 0.29 Cohesion: 0.29
Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations, Persona Red Flags, Priority Issues, Questions to Consider Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations, Persona Red Flags, Priority Issues, Questions to Consider
### Community 41 - "Handler"
Cohesion: 0.14
Nodes (18): currentLib(), currentTab(), filterBookmarks(), Client, HandlerFunc, Request, ResponseWriter, Store (+10 more)
### Community 42 - "pgtest.go" ### Community 42 - "pgtest.go"
Cohesion: 0.18 Cohesion: 0.18
Nodes (12): M, TestMain(), M, TestMain(), M, Main(), start(), URL() (+4 more) Nodes (12): M, TestMain(), M, TestMain(), M, Main(), start(), URL() (+4 more)
@@ -534,6 +536,10 @@ Nodes (3): ADR-0005: On-demand browser sidecar, Constraints, Decision
Cohesion: 0.48 Cohesion: 0.48
Nodes (6): T, TestCreateAndGetSession(), TestDeleteSessionIsPerReader(), TestDeleteSessionRevokes(), TestExpiredSessionIsGone(), TestGetSessionUnknownID() Nodes (6): T, TestCreateAndGetSession(), TestDeleteSessionIsPerReader(), TestDeleteSessionRevokes(), TestExpiredSessionIsGone(), TestGetSessionUnknownID()
### Community 154 - "ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them"
Cohesion: 0.33
Nodes (5): ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them, Constraints preserved, Decision, Tradeoffs and rejections, Why
### Community 156 - "ResponseWriter" ### Community 156 - "ResponseWriter"
Cohesion: 0.18 Cohesion: 0.18
Nodes (13): AdminPatterns(), HandlerFunc, Request, ResponseWriter, Time, Handler, readerPathID(), since() (+5 more) Nodes (13): AdminPatterns(), HandlerFunc, Request, ResponseWriter, Time, Handler, readerPathID(), since() (+5 more)
@@ -543,24 +549,24 @@ Cohesion: 0.50
Nodes (3): Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust), Second script: `novel-bookmark.user.js`, Userscript structure (single IIFE, `manga-bookmark.user.js`) Nodes (3): Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust), Second script: `novel-bookmark.user.js`, Userscript structure (single IIFE, `manga-bookmark.user.js`)
## Knowledge Gaps ## Knowledge Gaps
- **520 isolated node(s):** `bookmarkmanager/backend`, `ctxKey`, `loginView`, `ctxKey`, `test` (+515 more) - **524 isolated node(s):** `bookmarkmanager/backend`, `ctxKey`, `loginView`, `ctxKey`, `test` (+519 more)
These have ≤1 connection - possible missing edges or undocumented components. These have ≤1 connection - possible missing edges or undocumented components.
- **150 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes. - **151 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes.
## Suggested Questions ## Suggested Questions
_Questions this graph is uniquely positioned to answer:_ _Questions this graph is uniquely positioned to answer:_
- **Why does `Open()` connect `CDP Browser Client` to `Cover Fetch Test Helpers`, `Series Acquisition Tests`, `Bookmarks API Tests`, `pgtest.go`, `Store Tests`, `Store`?** - **Why does `Open()` connect `CDP Browser Client` to `Cover Fetch Test Helpers`, `Series Acquisition Tests`, `Bookmarks API Tests`, `pgtest.go`, `Store Tests`, `Store`?**
_High betweenness centrality (0.060) - this node is a cross-community bridge._
- **Why does `New()` connect `Series Acquisition Tests` to `Bookmarks API Tests`, `Cover & Acquire Internals`, `Handler`, `Session Middleware`, `CDP Browser Client`, `ResponseWriter`?**
_High betweenness centrality (0.050) - this node is a cross-community bridge._ _High betweenness centrality (0.050) - this node is a cross-community bridge._
- **Why does `New()` connect `Series Acquisition Tests` to `Bookmarks API Tests`, `Cover & Acquire Internals`, `Session Middleware`, `Web UI Handlers`, `CDP Browser Client`, `ResponseWriter`?**
_High betweenness centrality (0.048) - this node is a cross-community bridge._
- **Why does `newRouter()` connect `Bookmarks API Tests` to `Cover Fetch Test Helpers`, `Bookmarks API Handler`, `Series Acquisition Tests`, `ResponseWriter`?** - **Why does `newRouter()` connect `Bookmarks API Tests` to `Cover Fetch Test Helpers`, `Bookmarks API Handler`, `Series Acquisition Tests`, `ResponseWriter`?**
_High betweenness centrality (0.031) - this node is a cross-community bridge._ _High betweenness centrality (0.030) - this node is a cross-community bridge._
- **Are the 25 inferred relationships involving `newTestStore()` (e.g. with `TestAcquireDoesNotBlockTheWrite()` and `TestAcquireFailureLeavesTheBookmarkIntact()`) actually correct?**
_`newTestStore()` has 25 INFERRED edges - model-reasoned connections that need verification._
- **Are the 48 inferred relationships involving `testConfig()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverNeverEchoesNonImage()`) actually correct?** - **Are the 48 inferred relationships involving `testConfig()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverNeverEchoesNonImage()`) actually correct?**
_`testConfig()` has 48 INFERRED edges - model-reasoned connections that need verification._ _`testConfig()` has 48 INFERRED edges - model-reasoned connections that need verification._
- **Are the 12 inferred relationships involving `newTestStore()` (e.g. with `TestAcquireDoesNotBlockTheWrite()` and `TestAcquireFailureLeavesTheBookmarkIntact()`) actually correct?**
_`newTestStore()` has 12 INFERRED edges - model-reasoned connections that need verification._
- **Are the 8 inferred relationships involving `newWebTestServer()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverRejectsUnknownAddress()`) actually correct?** - **Are the 8 inferred relationships involving `newWebTestServer()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverRejectsUnknownAddress()`) actually correct?**
_`newWebTestServer()` has 8 INFERRED edges - model-reasoned connections that need verification._ _`newWebTestServer()` has 8 INFERRED edges - model-reasoned connections that need verification._
- **What connects `bookmarkmanager/backend`, `ctxKey`, `loginView` to the rest of the system?** - **What connects `bookmarkmanager/backend`, `ctxKey`, `loginView` to the rest of the system?**
_552 weakly-connected nodes found - possible documentation gaps or missing edges._ _556 weakly-connected nodes found - possible documentation gaps or missing edges._
File diff suppressed because one or more lines are too long
+2654 -859
View File
File diff suppressed because it is too large Load Diff
+36 -21
View File
@@ -35,8 +35,8 @@
"semantic_hash": "4a94ac85bad6bce330d085bcc0ae3ffd" "semantic_hash": "4a94ac85bad6bce330d085bcc0ae3ffd"
}, },
"userscript/manga-bookmark.user.js": { "userscript/manga-bookmark.user.js": {
"mtime": 1786499529.779988, "mtime": 1786872888.3482232,
"ast_hash": "1f8bcddd3632d709f058a8401af8f127", "ast_hash": "4e79684942234b26f4a1fb03fccd6d35",
"semantic_hash": "" "semantic_hash": ""
}, },
".agents/skills/find-skills/SKILL.md": { ".agents/skills/find-skills/SKILL.md": {
@@ -225,8 +225,8 @@
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/api_test.go": { "backend/api_test.go": {
"mtime": 1786863966.7056258, "mtime": 1786868080.2317674,
"ast_hash": "0d443b4fe3c04ab057505d73b8a88d07", "ast_hash": "b64372df5429fb14e48b37b3bab10402",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/cover_test.go": { "backend/cover_test.go": {
@@ -235,9 +235,9 @@
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/api/handlers.go": { "backend/internal/api/handlers.go": {
"mtime": 1786363889.552731, "mtime": 1786867745.788687,
"ast_hash": "59e6b8767ab19839bb8f82891a7e4616", "ast_hash": "ae4f3b961450f06a06e0632eb273a5b2",
"semantic_hash": "59e6b8767ab19839bb8f82891a7e4616" "semantic_hash": ""
}, },
"backend/internal/httpmw/middleware.go": { "backend/internal/httpmw/middleware.go": {
"mtime": 1786216141.672644, "mtime": 1786216141.672644,
@@ -260,8 +260,8 @@
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/poller.go": { "backend/internal/latest/poller.go": {
"mtime": 1786865046.096795, "mtime": 1786872774.641225,
"ast_hash": "2bfb51177e3df474b80ea1b05523377e", "ast_hash": "5e55e2ac5cb7d347807754f92425d29b",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/poller_test.go": { "backend/internal/latest/poller_test.go": {
@@ -270,8 +270,8 @@
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/sites.go": { "backend/internal/latest/sites.go": {
"mtime": 1786861012.026504, "mtime": 1786868771.0663204,
"ast_hash": "9b90f9b710ccbd0d0a0a6a0dd721ff3c", "ast_hash": "5786000dd0dfd5b4b3bd2b87b1fc100b",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/latest/sites_test.go": { "backend/internal/latest/sites_test.go": {
@@ -340,13 +340,13 @@
"semantic_hash": "0b6764a0ee20f5cb7748eecd31a1d220" "semantic_hash": "0b6764a0ee20f5cb7748eecd31a1d220"
}, },
"backend/internal/store/store.go": { "backend/internal/store/store.go": {
"mtime": 1786863704.5112119, "mtime": 1786868763.646761,
"ast_hash": "989629af18232e35b12f1eef2a8cb422", "ast_hash": "34f55617b7409f03f02e925bec92fac0",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/store/store_test.go": { "backend/internal/store/store_test.go": {
"mtime": 1786863712.075631, "mtime": 1786867781.301985,
"ast_hash": "e48f21e34238ba46874cc88cf96e022f", "ast_hash": "16f69d97eb63bcec7409294466642411",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/userscript/userscript.go": { "backend/internal/userscript/userscript.go": {
@@ -395,8 +395,8 @@
"semantic_hash": "8008a187690764436540fab47ba0cfcc" "semantic_hash": "8008a187690764436540fab47ba0cfcc"
}, },
"userscript/novel-bookmark.user.js": { "userscript/novel-bookmark.user.js": {
"mtime": 1786499529.779988, "mtime": 1786872902.809149,
"ast_hash": "834effb0821f8d6c9f57f6554a5db462", "ast_hash": "36c81462919719f0e053ca5d633a243a",
"semantic_hash": "" "semantic_hash": ""
}, },
"userscript/test/novel-logic.test.js": { "userscript/test/novel-logic.test.js": {
@@ -425,13 +425,13 @@
"semantic_hash": "6c6f3e4c4c2f57867894280bce728c50" "semantic_hash": "6c6f3e4c4c2f57867894280bce728c50"
}, },
"backend/AGENTS.md": { "backend/AGENTS.md": {
"mtime": 1786865319.314815, "mtime": 1786872951.3127444,
"ast_hash": "5bbabcd7dd425302cb0d989d8897baf6", "ast_hash": "68bc86af9593adad1e37fa05188fe63e",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/CLAUDE.md": { "backend/CLAUDE.md": {
"mtime": 1786865319.314815, "mtime": 1786872951.3127444,
"ast_hash": "5bbabcd7dd425302cb0d989d8897baf6", "ast_hash": "68bc86af9593adad1e37fa05188fe63e",
"semantic_hash": "" "semantic_hash": ""
}, },
"backend/internal/web/templates/app.html": { "backend/internal/web/templates/app.html": {
@@ -658,5 +658,20 @@
"mtime": 1786865113.5362902, "mtime": 1786865113.5362902,
"ast_hash": "8b9f9c7a56e0ec3bf950aa70acd95e92", "ast_hash": "8b9f9c7a56e0ec3bf950aa70acd95e92",
"semantic_hash": "" "semantic_hash": ""
},
"backend/internal/latest/sighting_test.go": {
"mtime": 1786868815.9760568,
"ast_hash": "3b1372bbeeb538b73f86f57da210b4cc",
"semantic_hash": ""
},
"backend/internal/store/migrations/0011_series_sightings.sql": {
"mtime": 1786867605.3728056,
"ast_hash": "d0ade928a6e951eb179e4d0255cca526",
"semantic_hash": ""
},
"docs/adr/0011-sighting-deferral-trust-model.md": {
"mtime": 1786872970.473592,
"ast_hash": "0b6ca704f62185c9d629320f2d8475b5",
"semantic_hash": ""
} }
} }
+16 -6
View File
@@ -905,27 +905,37 @@
} }
} }
// Records the newest chapter a site has published. Silent: this fires from // Reports the newest chapter a site has published. Silent: this fires from
// page visits and background checks the user did not ask for, and it never // page visits and background checks the user did not ask for, and it never
// reorders the list — updated_at is a candidate the server discards unless // reorders the list — updated_at is a candidate the server discards unless
// reading progress moved. // reading progress moved.
async function applyLatestChapterIfChanged(existing, latest) { //
// An unchanged number is still sent. It is the read that lets the backend
// skip its own poll of this series (a Sighting, issue #103), so the common
// case — visiting a series with nothing new — is exactly the one worth
// reporting. Only the local write and the re-render are skipped.
async function reportLatestChapter(existing, latest) {
if (!existing || !latest) return; if (!existing || !latest) return;
if (existing.latest_chapter_num === latest.num) return; const changed = existing.latest_chapter_num !== latest.num;
const bm = Object.assign({}, existing, { let bm = existing;
if (changed) {
bm = Object.assign({}, existing, {
latest_chapter: latest.label, latest_chapter: latest.label,
latest_chapter_num: latest.num, latest_chapter_num: latest.num,
updated_at: Date.now(), updated_at: Date.now(),
}); });
upsertLocal(bm); upsertLocal(bm);
render(); render();
}
// A queued write owns this row; the drain sends latest_chapter // A queued write owns this row; the drain sends latest_chapter
// with it, carrying the correct bucket. // with it, carrying the correct bucket.
if (queueGet(bm.key)) return; if (queueGet(bm.key)) return;
try { try {
const saved = await apiPut(bm.key, bm); const saved = await apiPut(bm.key, bm);
if (changed) {
upsertLocal(saved); upsertLocal(saved);
render(); render();
}
} catch (e) { } catch (e) {
/* offline — the local cache still shows it, retried on a later visit */ /* offline — the local cache still shows it, retried on a later visit */
} }
@@ -937,7 +947,7 @@
if (p.type !== "series") return; if (p.type !== "series") return;
const existing = state.byKey[keyOf(p)]; const existing = state.byKey[keyOf(p)];
if (!existing) return; if (!existing) return;
applyLatestChapterIfChanged( reportLatestChapter(
existing, existing,
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId) computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
); );
@@ -977,7 +987,7 @@
const html = await res.text(); const html = await res.text();
latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id); latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
} }
await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest); await reportLatestChapter(state.byKey[bm.key] || bm, latest);
} catch (e) { } catch (e) {
/* offline or blocked — try again after the throttle window */ /* offline or blocked — try again after the throttle window */
} }
+16 -6
View File
@@ -809,27 +809,37 @@
} }
} }
// Records the newest chapter a site has published. Silent: this fires from // Reports the newest chapter a site has published. Silent: this fires from
// page visits and background checks the user did not ask for, and it never // page visits and background checks the user did not ask for, and it never
// reorders the list — updated_at is a candidate the server discards unless // reorders the list — updated_at is a candidate the server discards unless
// reading progress moved. // reading progress moved.
async function applyLatestChapterIfChanged(existing, latest) { //
// An unchanged number is still sent. It is the read that lets the backend
// skip its own poll of this series (a Sighting, issue #103), so the common
// case — visiting a series with nothing new — is exactly the one worth
// reporting. Only the local write and the re-render are skipped.
async function reportLatestChapter(existing, latest) {
if (!existing || !latest) return; if (!existing || !latest) return;
if (existing.latest_chapter_num === latest.num) return; const changed = existing.latest_chapter_num !== latest.num;
const bm = Object.assign({}, existing, { let bm = existing;
if (changed) {
bm = Object.assign({}, existing, {
latest_chapter: latest.label, latest_chapter: latest.label,
latest_chapter_num: latest.num, latest_chapter_num: latest.num,
updated_at: Date.now(), updated_at: Date.now(),
}); });
upsertLocal(bm); upsertLocal(bm);
render(); render();
}
// A queued write owns this row; the drain sends latest_chapter // A queued write owns this row; the drain sends latest_chapter
// with it, carrying the correct bucket. // with it, carrying the correct bucket.
if (queueGet(bm.key)) return; if (queueGet(bm.key)) return;
try { try {
const saved = await apiPut(bm.key, bm); const saved = await apiPut(bm.key, bm);
if (changed) {
upsertLocal(saved); upsertLocal(saved);
render(); render();
}
} catch (e) { } catch (e) {
/* offline — the local cache still shows it, retried on a later visit */ /* offline — the local cache still shows it, retried on a later visit */
} }
@@ -841,7 +851,7 @@
if (p.type !== "series") return; if (p.type !== "series") return;
const existing = state.byKey[keyOf(p)]; const existing = state.byKey[keyOf(p)];
if (!existing) return; if (!existing) return;
applyLatestChapterIfChanged( reportLatestChapter(
existing, existing,
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId) computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
); );
@@ -883,7 +893,7 @@
if (!res.ok) continue; if (!res.ok) continue;
const html = await res.text(); const html = await res.text();
const latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id); const latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest); await reportLatestChapter(state.byKey[bm.key] || bm, latest);
} catch (e) { } catch (e) {
/* offline or blocked — try again after the throttle window */ /* offline or blocked — try again after the throttle window */
} }