Closes #103. A userscript PUT already carries the Latest Chapter the Reader's own browser read off the Series page. It may now stand in for a Poll, under one restriction and one ceiling: - **Solitary Series only** — a Series two Readers share is Polled on schedule however recently it was sighted, so one Reader's mistake can never reach another's list. - **One rest of standing**, and a **six-rest ceiling** (`sightingCeilingRests`, counted in the Site's own Rest): however many Sightings arrive, an unpolled Series is Polled. Both live in the due query's HAVING clause (`Store.DueForLatestCheck`) — the same place the schedule has always been decided, so no timer and no second code path can disagree with it. No new query per scheduler round. Judgement costs no extra request. `Poller.checkOne` already compares what the Site publishes against what is stored: a lower number contradicts the Sighting (Reader and both numbers logged), the same number confirms it, a higher number is the Site publishing and clears the attribution instead. Three contradictions stop that Reader deferring — their reports still write the Latest Chapter — and twenty consecutive confirmations forgive them, as does the owner's clear-marks control from #102. One client change was required: both userscripts skipped the PUT when the number had not moved, so the case the whole mechanism exists for — visiting a Series with nothing new — never reached the backend. `reportLatestChapter` sends it, skipping only the local write and the re-render. A numberless PUT (favourite toggle, progress from a chapter page) is no Sighting and defers nothing. Schema: migration `0011_series_sightings.sql` adds `series.latest_sighted_at` and `series.latest_raised_by`. Trust model, thresholds, and rejected alternatives with their citations: `docs/adr/0011-sighting-deferral-trust-model.md`. Reviewed on both axes (spec against #103, standards against the repo's rules); the blocker — attribution surviving a Poll that overtook the report — is fixed and has a test that fails without the fix. Verification: `go test ./...` green (needs Docker), `node --test userscript/test/*.test.js` 66 pass. Reviewed-on: #108 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #108.
This commit is contained in:
+28
-2
@@ -27,7 +27,8 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
|
||||
`series` keyed `(site, series_id)`
|
||||
(`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the
|
||||
shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`),
|
||||
Latest Chapter, `latest_checked_at` — and `bookmarks` holds only what
|
||||
Latest Chapter, `latest_checked_at`, and the Sighting pair
|
||||
`latest_sighted_at`/`latest_raised_by` (issue #103) — and `bookmarks` holds only what
|
||||
differs between readers: progress, favourite, lifecycle bucket,
|
||||
`updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no
|
||||
surrogate id; the wire `key` is derived as `site:series_id` on read — and
|
||||
@@ -78,7 +79,9 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
|
||||
each re-checking that Site's bookmarked series' newest published chapter from
|
||||
backend's own network access, so `latest_chapter` stays fresh when the user
|
||||
isn't browsing. Second, parallel signal — the userscript keeps its own
|
||||
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
|
||||
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` schedule, and its
|
||||
`reportLatestChapter` PUTs every read, unchanged numbers included, because an
|
||||
unchanged read is exactly the Sighting worth deferring a Poll on (#103).
|
||||
Two independent clocks: per-series rest (`series.latest_checked_at`,
|
||||
enforced by `Store.DueForLatestCheck`'s WHERE clause — `now - Rest`) and
|
||||
per-Lane gap (the Lane sleeping between fetches, `effectiveGap`). Both live
|
||||
@@ -91,6 +94,29 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
|
||||
every tick; found chapter written straight to the series row via
|
||||
`Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
|
||||
order — is never touched.
|
||||
**Sightings** (issue #103, ADR-0011) let a Reader's own page read defer a
|
||||
Poll: `Store.RecordSighting` — called by the PUT handler *before* the Upsert,
|
||||
because the raise test needs the row as it stands — stamps
|
||||
`series.latest_sighted_at` and, when the report raises the stored number,
|
||||
names its Reader in `series.latest_raised_by`. The due query's HAVING clause
|
||||
is where deferral lives: a Series is skipped only while it has exactly one
|
||||
Bookmark, was sighted within one Rest, and is under the ceiling
|
||||
(`sightingCeilingRests`, six of that Site's rests) since its last Poll. So a
|
||||
shared Series is never deferred, and no Series goes six hours unpolled
|
||||
whatever arrives. `checkOne` judges the named Reader off the comparison it
|
||||
already makes: a lower number is a contradiction (logged with the Reader and
|
||||
both numbers), the same number an agreement, a higher number the Site
|
||||
publishing and neither — that last one clears the attribution instead, since
|
||||
the value the Poll then stores is its own and a later retraction is not the
|
||||
Reader's fault. Three contradictions
|
||||
(`store.SightingDisagreementLimit`) stop that Reader deferring — their
|
||||
reports still write the Latest Chapter — and twenty consecutive agreements
|
||||
(`store.SightingAgreementsToClear`) forgive them, as does the owner's
|
||||
clear-marks control. Deferral is recomputed from live facts every round, so
|
||||
nothing needs invalidating when a Series gains a second Bookmark; the one
|
||||
input read earlier is the Reader's marks, checked when the Sighting is
|
||||
recorded, so crossing the threshold or being cleared takes effect from that
|
||||
Reader's next Sighting and the standing already bought lasts out its rest.
|
||||
Refusals and browser loss are Lane-local: two `errChallengeHeld` in one pass
|
||||
stop that Site for `refuseBackoff` (15m) while other Lanes continue; an
|
||||
`errBrowserInterrupted` (remote Chrome restart) sets a shared Poller flag
|
||||
|
||||
@@ -621,6 +621,46 @@ func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// Sighting deferral (issue #103) only reaches production through the PUT
|
||||
// handler: the store and poller can be right and the feature still dead if the
|
||||
// handler never records the report. Asserted where a client can see it - the
|
||||
// series stops being due the moment the PUT lands.
|
||||
func TestPutRecordsASighting(t *testing.T) {
|
||||
s := newTestStore(t)
|
||||
srv := newRouter(s, testConfig(), nil)
|
||||
|
||||
now := time.Now().UnixMilli()
|
||||
hour := time.Hour.Milliseconds()
|
||||
seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", now-2*hour)
|
||||
due, err := s.DueForLatestCheck("asura", now-hour, now-6*hour)
|
||||
if err != nil {
|
||||
t.Fatalf("DueForLatestCheck: %v", err)
|
||||
}
|
||||
if len(due) != 1 {
|
||||
t.Fatalf("due before the PUT = %d series, want 1", len(due))
|
||||
}
|
||||
|
||||
body := `{"key":"asura:x","site":"asura","series_id":"x",
|
||||
"series_url":"https://asurascans.com/comics/x",
|
||||
"last_chapter":"Chapter 5","last_chapter_num":5,
|
||||
"latest_chapter":"Chapter 9","latest_chapter_num":9}`
|
||||
req := httptest.NewRequest(http.MethodPut, "/bookmarks/asura:x", strings.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
rec := httptest.NewRecorder()
|
||||
srv.ServeHTTP(rec, auth(req))
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("PUT status = %d, want 200 (body %s)", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
due, err = s.DueForLatestCheck("asura", now-hour, now-6*hour)
|
||||
if err != nil {
|
||||
t.Fatalf("DueForLatestCheck: %v", err)
|
||||
}
|
||||
if len(due) != 0 {
|
||||
t.Fatalf("due after the PUT = %d series, want 0: the handler recorded no Sighting", len(due))
|
||||
}
|
||||
}
|
||||
|
||||
// The userscript route is registered outside the web UI's Discord auth, so it
|
||||
// must keep working whatever the web config — see internal/userscript for the
|
||||
// handler's own behaviour. The credential in the path is the owner's derived
|
||||
|
||||
@@ -99,7 +99,18 @@ func (h *Handler) Put(w http.ResponseWriter, r *http.Request) {
|
||||
// reading progress actually moved. Any client value is ignored.
|
||||
b.UpdatedAt = time.Now().UnixMilli()
|
||||
|
||||
stored, err := h.Store.Upsert(httpmw.ReaderID(r), b)
|
||||
// A userscript PUT is a Sighting: the Reader's browser was on the Series
|
||||
// page and read its Latest Chapter (issue #103). Recorded before the
|
||||
// Upsert, which is what makes the raise comparison possible, and never
|
||||
// from the web UI's own read-modify-write — a Reader toggling a favourite
|
||||
// has not looked at the Site and must not postpone a Poll. A failure here
|
||||
// costs a deferral, not the write, so it is logged and dropped.
|
||||
readerID := httpmw.ReaderID(r)
|
||||
if err := h.Store.RecordSighting(readerID, b.Site, b.SeriesID, b.LatestChapterNum, b.UpdatedAt); err != nil {
|
||||
log.Printf("record sighting: %v", err)
|
||||
}
|
||||
|
||||
stored, err := h.Store.Upsert(readerID, b)
|
||||
if err != nil {
|
||||
log.Printf("upsert: %v", err)
|
||||
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||
|
||||
@@ -250,7 +250,8 @@ func (p *Poller) runLanePass(ctx context.Context, name string, paced bool) time.
|
||||
return defaultGap
|
||||
}
|
||||
|
||||
due, err := p.Store.DueForLatestCheck(name, now.Add(-s.Rest).UnixMilli())
|
||||
due, err := p.Store.DueForLatestCheck(name, now.Add(-s.Rest).UnixMilli(),
|
||||
now.Add(-sightingCeilingRests*s.Rest).UnixMilli())
|
||||
if err != nil {
|
||||
log.Printf("latest poll %s: due query: %v", name, err)
|
||||
st.Gap = defaultGap
|
||||
@@ -461,6 +462,11 @@ func (p *Poller) checkOne(ctx context.Context, sr store.Series) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// The Poll is the oracle for whatever Sighting last raised this Series
|
||||
// (issue #103), and the judgement is free: the comparison below already
|
||||
// exists, and no extra request is made to reach it.
|
||||
p.judgeSighting(sr, facts.Latest.Num)
|
||||
|
||||
// Equality, not >, mirroring the userscript (L427): a site that retracts a
|
||||
// chapter should correct the stored number downward. The comparison is
|
||||
// against the due-query snapshot; a concurrent write in between only costs
|
||||
@@ -481,6 +487,43 @@ func (p *Poller) checkOne(ctx context.Context, sr store.Series) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// judgeSighting settles the Sighting the Series' stored Latest Chapter is owed
|
||||
// to, if any, against what the Site actually publishes. The asymmetry is the
|
||||
// whole of the detection rule and is what keeps it free of false alarms: a Poll
|
||||
// finding a *lower* number than stored means the Reader who raised it reported
|
||||
// a chapter that does not exist, while a Poll finding a higher one is only the
|
||||
// Site publishing since and means nothing about the report. Equality confirms
|
||||
// the report, which is how an honest Reader earns back a mark.
|
||||
//
|
||||
// A Series with no attribution — the stored value is a Poll's own, or a
|
||||
// previous Poll already judged the report — is nobody's to answer for.
|
||||
func (p *Poller) judgeSighting(sr store.Series, found float64) {
|
||||
if sr.LatestRaisedBy == nil || sr.LatestChapterNum == nil {
|
||||
return
|
||||
}
|
||||
stored := *sr.LatestChapterNum
|
||||
if found > stored {
|
||||
// The report is neither confirmed nor contradicted, but it is answered:
|
||||
// the value about to be stored is the Poll's own, so leaving the
|
||||
// attribution would credit this Reader with the next Poll's agreement
|
||||
// and blame them if the Site later retracts.
|
||||
if err := p.Store.ClearSightingAttribution(sr.Site, sr.SeriesID, *sr.LatestRaisedBy); err != nil {
|
||||
log.Printf("latest poll %q: clear sighting attribution: %v", sr.Key(), err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if found < stored {
|
||||
// Logged with both numbers and the Reader, because that is what tells a
|
||||
// broken Site adapter (which marks every Reader of that Site at once)
|
||||
// from one Reader deliberately lying.
|
||||
log.Printf("latest poll %q: sighting contradicted: reader %d raised it to %v, site publishes %v",
|
||||
sr.Key(), *sr.LatestRaisedBy, stored, found)
|
||||
}
|
||||
if err := p.Store.RecordSightingOutcome(sr.Site, sr.SeriesID, *sr.LatestRaisedBy, found == stored); err != nil {
|
||||
log.Printf("latest poll %q: record sighting outcome: %v", sr.Key(), err)
|
||||
}
|
||||
}
|
||||
|
||||
// healCover runs prefetchCover in the background. Cover bytes come from a
|
||||
// different host — often a CDN — and heal once in a Series's life, so they
|
||||
// must not consume a Lane's gap: a large import with many blanks would
|
||||
|
||||
@@ -0,0 +1,494 @@
|
||||
package latest
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"fmt"
|
||||
"log"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookmarkmanager/backend/internal/store"
|
||||
)
|
||||
|
||||
// Sightings (issue #103) are specified at the Poller seam, with the store as
|
||||
// the way in: a Sighting is seeded the way handlers.Put performs one, a round
|
||||
// is run against the injected fetcher and a frozen clock, and the assertions
|
||||
// are the two observable facts — whether the Series was fetched, and what the
|
||||
// stored Latest Chapter is afterwards. Nothing here asserts counter arithmetic
|
||||
// through an internal call or reads how a deferral is represented in a row.
|
||||
|
||||
const (
|
||||
sightingSlug = "chronicles-of-the-demon-faction-f886a8af"
|
||||
sightingKey = "asura:" + sightingSlug
|
||||
sightingURL = "https://asurascans.com/comics/" + sightingSlug
|
||||
)
|
||||
|
||||
// sightingFixtureLatest is the newest chapter asuraSeriesFixture publishes.
|
||||
const sightingFixtureLatest = 181.0
|
||||
|
||||
// sight performs one Sighting exactly as the JSON API does (handlers.Put):
|
||||
// RecordSighting against the row as stored, then the Upsert that stores the
|
||||
// reported value. The order is load-bearing — the raise comparison has nothing
|
||||
// to compare against once the Upsert has landed — and the bookmark's own fields
|
||||
// are carried over untouched, which is what a userscript PUT does when it
|
||||
// echoes back the row it cached.
|
||||
func sight(t *testing.T, s *store.Store, readerID int64, key string, num float64, at time.Time) {
|
||||
t.Helper()
|
||||
site, seriesID, ok := strings.Cut(key, ":")
|
||||
if !ok {
|
||||
t.Fatalf("key %q: no ':' separator", key)
|
||||
}
|
||||
b, found, err := s.Get(readerID, key)
|
||||
if err != nil || !found {
|
||||
t.Fatalf("sight %q: get: %v found=%v", key, err, found)
|
||||
}
|
||||
if err := s.RecordSighting(readerID, site, seriesID, &num, at.UnixMilli()); err != nil {
|
||||
t.Fatalf("sight %q: %v", key, err)
|
||||
}
|
||||
b.LatestChapter = fmt.Sprintf("Chapter %v", num)
|
||||
b.LatestChapterNum = &num
|
||||
b.UpdatedAt = at.UnixMilli()
|
||||
if _, err := s.Upsert(readerID, b); err != nil {
|
||||
t.Fatalf("sight %q: upsert: %v", key, err)
|
||||
}
|
||||
}
|
||||
|
||||
// secondReader is another Reader on the same database. The owner seed is the
|
||||
// only reader-creation path in this package, so a second Open as a different
|
||||
// owner is how a test gets one (as TestRunOnceFetchesSharedSeriesOnce does).
|
||||
func secondReader(t *testing.T, dbURL string) *store.Store {
|
||||
t.Helper()
|
||||
other, err := store.Open(dbURL,
|
||||
store.Owner{DiscordID: "second-reader", TokenHash: sha256.Sum256([]byte("second-token-hash"))},
|
||||
t.TempDir(), testCoverBaseURL)
|
||||
if err != nil {
|
||||
t.Fatalf("Open second reader: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { other.Close() })
|
||||
return other
|
||||
}
|
||||
|
||||
func readLatestNum(t *testing.T, s *store.Store, readerID int64, key string) float64 {
|
||||
t.Helper()
|
||||
b, ok, err := s.Get(readerID, key)
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("Get %q: %v ok=%v", key, err, ok)
|
||||
}
|
||||
if b.LatestChapterNum == nil {
|
||||
t.Fatalf("%q has no latest chapter", key)
|
||||
}
|
||||
return *b.LatestChapterNum
|
||||
}
|
||||
|
||||
// A Series only one Reader bookmarks is the case where being wrong can hurt
|
||||
// nobody but the Reader who reported it, so their Sighting stands in for the
|
||||
// Poll and the round leaves the Series alone.
|
||||
func TestSightingOnSolitarySeriesDefersPoll(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
|
||||
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
|
||||
|
||||
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||
newTestPoller(t, s, f, now).runOnce(context.Background())
|
||||
|
||||
if got := f.callCount(); got != 0 {
|
||||
t.Fatalf("fetched %d times after a Sighting on a solitary Series, want 0", got)
|
||||
}
|
||||
}
|
||||
|
||||
// On a shared Series the Sighting still writes the Latest Chapter for everyone,
|
||||
// but the Poll happens on schedule anyway — which is what corrects a wrong
|
||||
// value within the hour instead of letting it persist.
|
||||
func TestSightingOnSharedSeriesDoesNotDeferPoll(t *testing.T) {
|
||||
s, dbURL := newTestStore(t)
|
||||
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
|
||||
other := secondReader(t, dbURL)
|
||||
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
|
||||
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed second reader: %v", err)
|
||||
}
|
||||
sight(t, s, s.OwnerID(), sightingKey, 200, now.Add(-10*time.Minute))
|
||||
|
||||
// The Sighting updated the shared row immediately, before any Poll.
|
||||
if got := readLatestNum(t, s, s.OwnerID(), sightingKey); got != 200 {
|
||||
t.Fatalf("latest after the Sighting = %v, want 200", got)
|
||||
}
|
||||
|
||||
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||
newTestPoller(t, s, f, now).runOnce(context.Background())
|
||||
|
||||
if got := f.callCount(); got != 1 {
|
||||
t.Fatalf("fetched %d times after a Sighting on a shared Series, want 1", got)
|
||||
}
|
||||
if got := readLatestNum(t, s, s.OwnerID(), sightingKey); got != sightingFixtureLatest {
|
||||
t.Fatalf("latest after the Poll = %v, want the Site's own %v", got, sightingFixtureLatest)
|
||||
}
|
||||
}
|
||||
|
||||
// Reporting a chapter is not reading one: a Sighting may move the Latest
|
||||
// Chapter and nothing else. Both the solitary and the shared case, because the
|
||||
// deferral branch must not be where this guarantee lives.
|
||||
func TestSightingLeavesProgressAndOrderingUntouched(t *testing.T) {
|
||||
for _, shared := range []bool{false, true} {
|
||||
name := "solitary"
|
||||
if shared {
|
||||
name = "shared"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
s, dbURL := newTestStore(t)
|
||||
read := 5.0
|
||||
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
|
||||
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, SeriesURL: sightingURL,
|
||||
LastChapter: "Chapter 5", LastChapterNum: read,
|
||||
LastChapterURL: sightingURL + "/chapter/5", UpdatedAt: 1000,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed: %v", err)
|
||||
}
|
||||
if shared {
|
||||
other := secondReader(t, dbURL)
|
||||
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
|
||||
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed second reader: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
sight(t, s, s.OwnerID(), sightingKey, 200, time.UnixMilli(9_000_000))
|
||||
|
||||
b, ok, err := s.Get(s.OwnerID(), sightingKey)
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("Get: %v ok=%v", err, ok)
|
||||
}
|
||||
if b.LatestChapterNum == nil || *b.LatestChapterNum != 200 {
|
||||
t.Fatalf("LatestChapterNum = %v, want 200", b.LatestChapterNum)
|
||||
}
|
||||
if b.LastChapterNum != read {
|
||||
t.Fatalf("LastChapterNum = %v, want %v: a Sighting is not Progress", b.LastChapterNum, read)
|
||||
}
|
||||
if b.UpdatedAt != 1000 {
|
||||
t.Fatalf("updated_at moved to %d: a Sighting must not reorder the list", b.UpdatedAt)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The ceiling is what makes trusting a client report safe: however recently a
|
||||
// Series was sighted, one that has not been Polled in six hours is Polled.
|
||||
func TestSightingCeilingForcesPoll(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-7*time.Hour).UnixMilli())
|
||||
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-time.Minute))
|
||||
|
||||
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||
newTestPoller(t, s, f, now).runOnce(context.Background())
|
||||
|
||||
if got := f.callCount(); got != 1 {
|
||||
t.Fatalf("fetched %d times past the %s ceiling, want 1", got, sightingCeilingRests*defaultRest)
|
||||
}
|
||||
}
|
||||
|
||||
// Deferral is decided from live facts every round, so a Series that gains a
|
||||
// second Bookmark stops deferring at once — and one that loses it defers again.
|
||||
func TestDeferralFollowsTheBookmarkCount(t *testing.T) {
|
||||
s, dbURL := newTestStore(t)
|
||||
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
|
||||
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
|
||||
|
||||
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||
p := newTestPoller(t, s, f, now)
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != 0 {
|
||||
t.Fatalf("solitary Series fetched %d times, want 0", got)
|
||||
}
|
||||
|
||||
other := secondReader(t, dbURL)
|
||||
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
|
||||
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed second reader: %v", err)
|
||||
}
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != 1 {
|
||||
t.Fatalf("shared Series fetched %d times, want 1", got)
|
||||
}
|
||||
|
||||
// The Poll above consumed the rest, so move past it before asking again.
|
||||
if err := other.Delete(other.OwnerID(), sightingKey); err != nil {
|
||||
t.Fatalf("delete second bookmark: %v", err)
|
||||
}
|
||||
later := now.Add(2 * time.Hour)
|
||||
p.Now = func() time.Time { return later }
|
||||
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, later.Add(-time.Minute))
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != 1 {
|
||||
t.Fatalf("Series fetched %d times after returning to one Bookmark, want 1", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A Series nobody reports any more returns to the normal schedule on its own:
|
||||
// the Sighting's standing lasts one rest, not forever.
|
||||
func TestDeferralExpiresWithoutFurtherSightings(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
|
||||
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
|
||||
|
||||
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||
p := newTestPoller(t, s, f, now)
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != 0 {
|
||||
t.Fatalf("fetched %d times while the Sighting stood, want 0", got)
|
||||
}
|
||||
|
||||
p.Now = func() time.Time { return now.Add(90 * time.Minute) }
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != 1 {
|
||||
t.Fatalf("fetched %d times once the Sighting aged out, want 1", got)
|
||||
}
|
||||
}
|
||||
|
||||
// demonicFixture publishes one chapter in demonicscans' live page shape, so a
|
||||
// test can make a Site publish an arbitrary number rather than the one the
|
||||
// captured fixture froze.
|
||||
func demonicFixture(num float64) string {
|
||||
return fmt.Sprintf(
|
||||
`<a href="/chaptered.php?manga=11799&chapter=%v" class="chplinks" title="Catastrophic Necromancer %v">Chapter %v</a>`,
|
||||
num, num, num)
|
||||
}
|
||||
|
||||
const (
|
||||
demonicKey = "demonic:Catastrophic-Necromancer"
|
||||
demonicURL = "https://demonicscans.org/manga/Catastrophic-Necromancer"
|
||||
)
|
||||
|
||||
// contradictOnce reports a chapter that does not exist and then runs the round
|
||||
// that catches it, returning when that round ran so a caller can chain the
|
||||
// next one. The wait is one rest and a minute: a Sighting stands in for exactly
|
||||
// one rest, so that is the first moment this solitary Series is Polled again.
|
||||
func contradictOnce(t *testing.T, s *store.Store, p *Poller, sightAt time.Time, real float64) time.Time {
|
||||
t.Helper()
|
||||
sight(t, s, s.OwnerID(), demonicKey, real+500, sightAt)
|
||||
at := sightAt.Add(defaultRest + time.Minute)
|
||||
p.Now = func() time.Time { return at }
|
||||
p.runOnce(context.Background())
|
||||
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != real {
|
||||
t.Fatalf("latest after the Poll = %v, want the Site's own %v", got, real)
|
||||
}
|
||||
return at
|
||||
}
|
||||
|
||||
func seedDemonic(t *testing.T, s *store.Store, checkedAt int64) {
|
||||
t.Helper()
|
||||
seedForCheck(t, s, demonicKey, demonicURL, checkedAt)
|
||||
}
|
||||
|
||||
// A Poll finding a lower number than stored means the Sighting that raised it
|
||||
// was false. The Reader is named — not the Series flagged — and both numbers are
|
||||
// logged, because that is what tells a broken adapter from a deliberate lie.
|
||||
func TestPollContradictingASightingNamesTheReaderAndBothNumbers(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
|
||||
|
||||
var logs strings.Builder
|
||||
prev := log.Writer()
|
||||
log.SetOutput(&logs)
|
||||
t.Cleanup(func() { log.SetOutput(prev) })
|
||||
|
||||
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||
p := newTestPoller(t, s, f, now)
|
||||
contradictOnce(t, s, p, now, 296)
|
||||
|
||||
got := logs.String()
|
||||
for _, want := range []string{
|
||||
fmt.Sprintf("reader %d", s.OwnerID()), "796", "296", demonicKey,
|
||||
} {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Fatalf("contradiction log = %q, want it to name %q", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Three contradictions cost the Reader the right to defer. Nothing here writes
|
||||
// a counter: the marks are earned through Polls, which is the only way
|
||||
// production produces them.
|
||||
func TestThreeContradictionsStopDeferral(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
start := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
|
||||
|
||||
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||
p := newTestPoller(t, s, f, start)
|
||||
at := start
|
||||
for range store.SightingDisagreementLimit {
|
||||
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
|
||||
}
|
||||
fetchesSoFar := f.callCount()
|
||||
|
||||
// The marked Reader sights the same solitary Series again. It still writes
|
||||
// the Latest Chapter — the penalty removes a privilege, it does not silence
|
||||
// anyone — but the Poll is no longer postponed: the round below runs while a
|
||||
// trusted Reader's Sighting would still be standing, and fetches anyway.
|
||||
sight(t, s, s.OwnerID(), demonicKey, 900, at.Add(31*time.Minute))
|
||||
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != 900 {
|
||||
t.Fatalf("latest after a marked Reader's Sighting = %v, want 900", got)
|
||||
}
|
||||
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != fetchesSoFar+1 {
|
||||
t.Fatalf("marked Reader's Sighting still deferred the Poll (fetches %d, want %d)",
|
||||
got, fetchesSoFar+1)
|
||||
}
|
||||
}
|
||||
|
||||
// The owner's remedy for a mark a broken Site adapter produced restores the
|
||||
// privilege without a wait and without SQL.
|
||||
func TestClearingMarksRestoresDeferral(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
start := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
|
||||
|
||||
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||
p := newTestPoller(t, s, f, start)
|
||||
at := start
|
||||
for range store.SightingDisagreementLimit {
|
||||
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
|
||||
}
|
||||
if err := s.ClearReaderMarks(s.OwnerID()); err != nil {
|
||||
t.Fatalf("ClearReaderMarks: %v", err)
|
||||
}
|
||||
|
||||
fetchesSoFar := f.callCount()
|
||||
sight(t, s, s.OwnerID(), demonicKey, 900, at.Add(31*time.Minute))
|
||||
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != fetchesSoFar {
|
||||
t.Fatalf("fetched %d times after the marks were cleared, want %d: deferral must resume",
|
||||
got, fetchesSoFar)
|
||||
}
|
||||
}
|
||||
|
||||
// Recovery is automatic but expensive: twenty Polls that each confirm a
|
||||
// Sighting of this Reader's clear the marks. Each round needs a new chapter,
|
||||
// because only a report that raises the stored number is attributed and so only
|
||||
// that one can be confirmed.
|
||||
func TestTwentyAgreementsClearTheMarks(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
start := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
|
||||
|
||||
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||
p := newTestPoller(t, s, f, start)
|
||||
at := start
|
||||
for range store.SightingDisagreementLimit {
|
||||
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
|
||||
}
|
||||
|
||||
chapter := 296.0
|
||||
for range store.SightingAgreementsToClear {
|
||||
chapter++
|
||||
sight(t, s, s.OwnerID(), demonicKey, chapter, at.Add(time.Minute))
|
||||
f.body = demonicFixture(chapter) // the Site publishes what was reported
|
||||
at = at.Add(defaultRest + time.Minute)
|
||||
p.Now = func() time.Time { return at }
|
||||
p.runOnce(context.Background())
|
||||
}
|
||||
|
||||
fetchesSoFar := f.callCount()
|
||||
chapter++
|
||||
sight(t, s, s.OwnerID(), demonicKey, chapter, at.Add(31*time.Minute))
|
||||
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != fetchesSoFar {
|
||||
t.Fatalf("fetched %d times after %d confirmations, want %d: the marks must be forgiven",
|
||||
got, store.SightingAgreementsToClear, fetchesSoFar)
|
||||
}
|
||||
}
|
||||
|
||||
// A Poll finding a higher number is the Site publishing since the Sighting and
|
||||
// means nothing about the Reader — no mark, and no credit either.
|
||||
func TestPollFindingHigherNumberIsNotAContradiction(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
|
||||
// Reported truthfully, then the Site published one more.
|
||||
sight(t, s, s.OwnerID(), demonicKey, 295, now.Add(-10*time.Minute))
|
||||
|
||||
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||
p := newTestPoller(t, s, f, now)
|
||||
// One rest on, the Sighting has lapsed and the Poll happens.
|
||||
p.Now = func() time.Time { return now.Add(7 * time.Hour) }
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != 1 {
|
||||
t.Fatalf("fetched %d times past the ceiling, want 1", got)
|
||||
}
|
||||
|
||||
// Unmarked, so a fresh Sighting still defers.
|
||||
at := now.Add(9 * time.Hour)
|
||||
sight(t, s, s.OwnerID(), demonicKey, 296, at.Add(-time.Minute))
|
||||
p.Now = func() time.Time { return at }
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != 1 {
|
||||
t.Fatalf("a Reader whose report the Site overtook lost the right to defer (fetches %d, want 1)", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A Poll that overtakes a Sighting takes ownership of the row: the value stored
|
||||
// afterwards is the Poll's own, so a later retraction is not the Reader's fault
|
||||
// and must not be charged to them.
|
||||
func TestAttributionDoesNotSurviveAPollThatOvertookIt(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
|
||||
sight(t, s, s.OwnerID(), demonicKey, 295, now.Add(-10*time.Minute))
|
||||
|
||||
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||
p := newTestPoller(t, s, f, now)
|
||||
at := now.Add(defaultRest + time.Minute)
|
||||
p.Now = func() time.Time { return at }
|
||||
p.runOnce(context.Background())
|
||||
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != 296 {
|
||||
t.Fatalf("latest after the Poll = %v, want the Site's own 296", got)
|
||||
}
|
||||
|
||||
var logs strings.Builder
|
||||
prev := log.Writer()
|
||||
log.SetOutput(&logs)
|
||||
t.Cleanup(func() { log.SetOutput(prev) })
|
||||
|
||||
f.body = demonicFixture(290) // the Site retracts what only the Poll wrote
|
||||
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
|
||||
p.runOnce(context.Background())
|
||||
if strings.Contains(logs.String(), "sighting contradicted") {
|
||||
t.Fatalf("a retraction of the Poll's own value was charged to a Reader: %s", logs.String())
|
||||
}
|
||||
}
|
||||
|
||||
// A PUT with no Latest Chapter in it — a favourite toggle, progress written
|
||||
// from a chapter page — is nobody looking at the Series page, so it buys no
|
||||
// deferral. Otherwise a client could suppress a Series' Polls while reporting
|
||||
// nothing, and with nothing reported there would be nothing to judge.
|
||||
func TestPutWithoutALatestChapterDoesNotDefer(t *testing.T) {
|
||||
s, _ := newTestStore(t)
|
||||
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
|
||||
// The handler's own call, with the field the client omitted.
|
||||
if err := s.RecordSighting(s.OwnerID(), "demonic", "Catastrophic-Necromancer",
|
||||
nil, now.Add(-time.Minute).UnixMilli()); err != nil {
|
||||
t.Fatalf("RecordSighting: %v", err)
|
||||
}
|
||||
|
||||
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||
p := newTestPoller(t, s, f, now)
|
||||
p.runOnce(context.Background())
|
||||
if got := f.callCount(); got != 1 {
|
||||
t.Fatalf("fetched %d times after a PUT carrying no chapter, want 1", got)
|
||||
}
|
||||
}
|
||||
@@ -409,6 +409,14 @@ const (
|
||||
// asleep (ADR-0005 on-demand browser).
|
||||
browserWakeCount = 5
|
||||
browserWakeAge = 15 * time.Minute
|
||||
// sightingCeilingRests caps Sighting deferral (issue #103): however many
|
||||
// Sightings arrive, a Series unpolled for this many of its Site's rests is
|
||||
// Polled. It is what makes a client report safe to trust — a wrong Latest
|
||||
// Chapter dies within the ceiling deterministically, rather than in
|
||||
// expectation the way a randomised audit would have it. Six, so a Series a
|
||||
// Reader visits constantly still gets one authoritative check per working
|
||||
// day-part.
|
||||
sightingCeilingRests = 6
|
||||
)
|
||||
|
||||
// effectiveGap is a Site's pace: the registry gap, or one rest divided by the
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
-- Sighting deferral (issue #103). latest_sighted_at is when a Reader's report
|
||||
-- last stood in for a Poll; it is separate from latest_checked_at because the
|
||||
-- six-hour ceiling has to know when the Series was last really fetched, and a
|
||||
-- Sighting writing the Poll's own column would erase that.
|
||||
-- latest_raised_by is attribution: whoever last raised this Series' Latest
|
||||
-- Chapter by Sighting, so a Poll that contradicts the value downwards names a
|
||||
-- Reader rather than flagging a row. Cleared by the Poll that judges it, NULL
|
||||
-- whenever the stored value is the Poll's own.
|
||||
ALTER TABLE series ADD COLUMN latest_sighted_at bigint NOT NULL DEFAULT 0;
|
||||
ALTER TABLE series ADD COLUMN latest_raised_by bigint REFERENCES readers(id) ON DELETE SET NULL;
|
||||
@@ -79,6 +79,11 @@ type Series struct {
|
||||
LatestChapter string
|
||||
LatestChapterNum *float64 // nil until first captured
|
||||
LatestCheckedAt int64 // unix ms; see MarkLatestChecked
|
||||
// LatestRaisedBy is the Reader whose Sighting last raised LatestChapter,
|
||||
// and nil when the stored value is a Poll's own finding. It is what lets a
|
||||
// Poll that contradicts the value downwards name a Reader instead of
|
||||
// merely flagging the row (issue #103); the Poll that judges it clears it.
|
||||
LatestRaisedBy *int64
|
||||
|
||||
// readerCount is the number of bookmarks referencing this series, filled
|
||||
// only by the due-queue query that orders on it.
|
||||
@@ -195,7 +200,7 @@ const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover_add
|
||||
// due query. latest_checked_at lives only on series — see MarkLatestChecked
|
||||
// for why it stays off every client-visible write.
|
||||
const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover, s.cover_address,
|
||||
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at`
|
||||
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at, s.latest_raised_by`
|
||||
|
||||
// Owner is the person running the service: the first Reader, seeded at startup
|
||||
// so a fresh deployment has a library before anyone logs in. The seed makes
|
||||
@@ -584,16 +589,17 @@ func (s *Store) scanBookmark(scan func(...any) error) (Bookmark, error) {
|
||||
}
|
||||
|
||||
// scanSeries reads one row in seriesColumns order, plus the due query's
|
||||
// reader_count column. latest_chapter_num is NULL until the first capture,
|
||||
// same as on the bookmark read path.
|
||||
// reader_count column. latest_chapter_num and latest_raised_by are both
|
||||
// nullable, same as latest_chapter_num on the bookmark read path.
|
||||
func scanSeries(scan func(...any) error) (Series, error) {
|
||||
var (
|
||||
sr Series
|
||||
latestChapterNum sql.NullFloat64
|
||||
latestRaisedBy sql.NullInt64
|
||||
)
|
||||
if err := scan(
|
||||
&sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover, &sr.CoverAddress,
|
||||
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt,
|
||||
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt, &latestRaisedBy,
|
||||
&sr.readerCount,
|
||||
); err != nil {
|
||||
return Series{}, err
|
||||
@@ -601,6 +607,9 @@ func scanSeries(scan func(...any) error) (Series, error) {
|
||||
if latestChapterNum.Valid {
|
||||
sr.LatestChapterNum = &latestChapterNum.Float64
|
||||
}
|
||||
if latestRaisedBy.Valid {
|
||||
sr.LatestRaisedBy = &latestRaisedBy.Int64
|
||||
}
|
||||
return sr, nil
|
||||
}
|
||||
|
||||
@@ -947,7 +956,20 @@ func (s *Store) Delete(readerID int64, key string) error {
|
||||
// burns requests. Archived bookmarks still count — knowing what a shelved
|
||||
// series is up to is the whole reason for archiving instead of deleting.
|
||||
// A series with no bookmarks at all never appears: the join excludes it.
|
||||
func (s *Store) DueForLatestCheck(site string, cutoffMs int64) ([]Series, error) {
|
||||
//
|
||||
// ceilingMs is the Sighting deferral ceiling (issue #103): a Series whose last
|
||||
// real Poll is older than it appears however recently it was sighted. That is
|
||||
// what bounds the whole mechanism — a wrong Latest Chapter dies within the
|
||||
// ceiling deterministically rather than in expectation. Deferral itself is
|
||||
// decided here, from two facts the query already computes, so a Lane gains no
|
||||
// query per round: a Sighting younger than cutoffMs holds the Series back, but
|
||||
// only while COUNT(*) is 1. A Series a second Reader bookmarks is Polled on
|
||||
// schedule, so a wrong value the whole guild can see is corrected by a check
|
||||
// that was never postponed; on a solitary Series the only person a wrong value
|
||||
// reaches is the Reader who reported it. Whether the reporting Reader is
|
||||
// allowed to defer at all was settled when the Sighting was recorded — see
|
||||
// RecordSighting.
|
||||
func (s *Store) DueForLatestCheck(site string, cutoffMs, ceilingMs int64) ([]Series, error) {
|
||||
rows, err := s.db.Query(`SELECT `+seriesColumns+`, COUNT(*) AS reader_count
|
||||
FROM series s
|
||||
JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id
|
||||
@@ -957,7 +979,10 @@ func (s *Store) DueForLatestCheck(site string, cutoffMs int64) ([]Series, error)
|
||||
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover,
|
||||
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at
|
||||
HAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
|
||||
ORDER BY reader_count DESC, s.latest_checked_at ASC`, site, cutoffMs)
|
||||
AND (COUNT(*) > 1
|
||||
OR s.latest_sighted_at <= $2::bigint
|
||||
OR s.latest_checked_at <= $3::bigint)
|
||||
ORDER BY reader_count DESC, s.latest_checked_at ASC`, site, cutoffMs, ceilingMs)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("query due series: %w", err)
|
||||
}
|
||||
@@ -1044,3 +1069,111 @@ func (s *Store) SetLatestChapter(site, seriesID, label string, num float64) erro
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RecordSighting notes that a Reader's browser reported this Series' Latest
|
||||
// Chapter, which is the half of a Sighting the client body cannot express
|
||||
// (issue #103). It must be called *before* the Upsert that stores the reported
|
||||
// value: the raise test compares against what is still on the row, and after
|
||||
// the Upsert there is nothing left to compare with. A Series that does not
|
||||
// exist yet — the first Bookmark of it — is not a Sighting at all: nothing has
|
||||
// ever been Polled, so there is nothing to defer and nobody to attribute.
|
||||
//
|
||||
// Two independent effects, hence the two CASE arms. The deferral stamp is only
|
||||
// written for a Reader below the disagreement limit, so a marked Reader's
|
||||
// reports keep updating the Latest Chapter but stop postponing anything, and
|
||||
// clearing their marks restores the privilege on their next Sighting. The
|
||||
// attribution is written whenever the report raises the stored number,
|
||||
// including for a marked Reader — their Sightings are still judged, which is
|
||||
// how they earn the privilege back.
|
||||
//
|
||||
// num is the reported chapter number. A PUT that carries none — a favourite
|
||||
// toggle, or progress written from a chapter page — is no Sighting at all:
|
||||
// nobody read the Series page, so there is nothing to stand in for a Poll and
|
||||
// nothing that could later be judged.
|
||||
func (s *Store) RecordSighting(readerID int64, site, seriesID string, num *float64, ts int64) error {
|
||||
if num == nil {
|
||||
return nil
|
||||
}
|
||||
if _, err := s.db.Exec(`
|
||||
UPDATE series SET
|
||||
latest_sighted_at = CASE
|
||||
WHEN (SELECT sighting_disagreements FROM readers WHERE id = $3) < $6
|
||||
THEN $4::bigint ELSE latest_sighted_at END,
|
||||
latest_raised_by = CASE
|
||||
WHEN latest_chapter_num IS NULL OR $5::double precision > latest_chapter_num
|
||||
THEN $3::bigint ELSE latest_raised_by END
|
||||
WHERE site = $1 AND series_id = $2`,
|
||||
site, seriesID, readerID, ts, *num, SightingDisagreementLimit); err != nil {
|
||||
return fmt.Errorf("record sighting %s:%s: %w", site, seriesID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// SightingAgreementsToClear is how many Polls must confirm a Reader's
|
||||
// Sightings in a row before their disagreements are forgiven. An agreement is
|
||||
// only recorded when a Poll later confirms a Sighting, so this is twenty Polls
|
||||
// of Series that Reader bookmarks — hours to days, not twenty page views. That
|
||||
// is the intended price: recovery is automatic but cannot be outwaited, and a
|
||||
// disagreement resets the run to zero, so credit cannot be banked in advance.
|
||||
const SightingAgreementsToClear = 20
|
||||
|
||||
// RecordSightingOutcome settles what a Poll decided about the Reader whose
|
||||
// Sighting last raised this Series' Latest Chapter, and clears the attribution
|
||||
// in the same transaction so one Sighting is judged exactly once. agreed is
|
||||
// the Poll confirming the stored value; its opposite is the Poll finding a
|
||||
// lower number, which means the raise was false.
|
||||
//
|
||||
// A Poll finding a *higher* number is neither — the Site published — and takes
|
||||
// ClearSightingAttribution instead.
|
||||
func (s *Store) RecordSightingOutcome(site, seriesID string, readerID int64, agreed bool) error {
|
||||
tx, err := s.db.Begin()
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin sighting outcome %s:%s: %w", site, seriesID, err)
|
||||
}
|
||||
defer tx.Rollback()
|
||||
|
||||
// The run length is what "consecutive" means: a disagreement zeroes the
|
||||
// agreements, and completing a run zeroes both, so the next run starts
|
||||
// from nothing rather than forgiving every later disagreement instantly.
|
||||
q := `UPDATE readers SET sighting_disagreements = sighting_disagreements + 1,
|
||||
sighting_agreements = 0
|
||||
WHERE id = $1`
|
||||
args := []any{readerID}
|
||||
if agreed {
|
||||
q = `UPDATE readers SET
|
||||
sighting_agreements = CASE WHEN sighting_agreements + 1 >= $2 THEN 0
|
||||
ELSE sighting_agreements + 1 END,
|
||||
sighting_disagreements = CASE WHEN sighting_agreements + 1 >= $2 THEN 0
|
||||
ELSE sighting_disagreements END
|
||||
WHERE id = $1`
|
||||
args = append(args, SightingAgreementsToClear)
|
||||
}
|
||||
if _, err := tx.Exec(q, args...); err != nil {
|
||||
return fmt.Errorf("record sighting outcome for reader %d: %w", readerID, err)
|
||||
}
|
||||
if _, err := tx.Exec(clearAttributionSQL, site, seriesID, readerID); err != nil {
|
||||
return fmt.Errorf("clear sighting attribution %s:%s: %w", site, seriesID, err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return fmt.Errorf("commit sighting outcome %s:%s: %w", site, seriesID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ClearSightingAttribution answers a Sighting without judging it: the Poll
|
||||
// found a higher number, so the value about to be stored is its own and this
|
||||
// Reader is no longer answerable for the row. Without it the next Poll's
|
||||
// agreement would be credited to a Reader who did not earn it.
|
||||
func (s *Store) ClearSightingAttribution(site, seriesID string, readerID int64) error {
|
||||
if _, err := s.db.Exec(clearAttributionSQL, site, seriesID, readerID); err != nil {
|
||||
return fmt.Errorf("clear sighting attribution %s:%s: %w", site, seriesID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// clearAttributionSQL drops the attribution only while it still names the
|
||||
// Reader being judged: a Sighting landing between the due query's snapshot and
|
||||
// this write is a fresh, unjudged one and must not be erased by the previous
|
||||
// one's verdict.
|
||||
const clearAttributionSQL = `UPDATE series SET latest_raised_by = NULL
|
||||
WHERE site = $1 AND series_id = $2 AND latest_raised_by = $3`
|
||||
|
||||
@@ -277,6 +277,10 @@ func seedForCheck(t *testing.T, s *Store, key, seriesURL string, checkedAt int64
|
||||
}
|
||||
}
|
||||
|
||||
// noCeiling is a Sighting deferral ceiling no Series can reach, for the tests
|
||||
// that predate the ceiling and are about rest, ordering or buckets instead.
|
||||
const noCeiling = int64(-1)
|
||||
|
||||
func TestDueForLatestCheck(t *testing.T) {
|
||||
const hour = int64(3600_000)
|
||||
now := 10 * hour
|
||||
@@ -298,7 +302,7 @@ func TestDueForLatestCheck(t *testing.T) {
|
||||
s := newTestStore(t)
|
||||
seedForCheck(t, s, "asura:x", tt.seriesURL, tt.checkedAt)
|
||||
|
||||
due, err := s.DueForLatestCheck("asura", now-hour)
|
||||
due, err := s.DueForLatestCheck("asura", now-hour, noCeiling)
|
||||
if err != nil {
|
||||
t.Fatalf("DueForLatestCheck: %v", err)
|
||||
}
|
||||
@@ -319,7 +323,7 @@ func TestDueForLatestCheckOldestFirstAndScopedToSite(t *testing.T) {
|
||||
// asks for one Site, and no Lane may see another's queue.
|
||||
seedForCheck(t, s, "demonic:z", "https://demonicscans.org/manga/z", 0)
|
||||
|
||||
due, err := s.DueForLatestCheck("asura", 1000)
|
||||
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
|
||||
if err != nil {
|
||||
t.Fatalf("DueForLatestCheck: %v", err)
|
||||
}
|
||||
@@ -499,7 +503,7 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
due, err := store.DueForLatestCheck("asura", time.Now().UnixMilli())
|
||||
due, err := store.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
|
||||
if err != nil {
|
||||
t.Fatalf("DueForLatestCheck: %v", err)
|
||||
}
|
||||
@@ -1005,7 +1009,7 @@ func TestDueForLatestCheckOrdersByReaderCountThenAge(t *testing.T) {
|
||||
seedSecondReader(t, s, "asura:pop:2", "asura", "pop", 1001)
|
||||
seedForCheck(t, s, "asura:solo", "https://asurascans.com/comics/solo", 100)
|
||||
|
||||
due, err := s.DueForLatestCheck("asura", 1000)
|
||||
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
|
||||
if err != nil {
|
||||
t.Fatalf("DueForLatestCheck: %v", err)
|
||||
}
|
||||
@@ -1031,7 +1035,7 @@ func TestDueForLatestCheckExcludesOrphanSeries(t *testing.T) {
|
||||
t.Fatalf("seed orphan series: %v", err)
|
||||
}
|
||||
|
||||
due, err := s.DueForLatestCheck("asura", 1000)
|
||||
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
|
||||
if err != nil {
|
||||
t.Fatalf("DueForLatestCheck: %v", err)
|
||||
}
|
||||
@@ -1449,7 +1453,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
|
||||
t.Fatalf("series rows = %d, want 1 shared row for two bookmarks", series)
|
||||
}
|
||||
|
||||
due, err := s.DueForLatestCheck("asura", time.Now().UnixMilli())
|
||||
due, err := s.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
|
||||
if err != nil {
|
||||
t.Fatalf("DueForLatestCheck: %v", err)
|
||||
}
|
||||
@@ -1465,7 +1469,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
|
||||
if b, ok, err := s.Get(s.OwnerID(), "asura:solo"); err != nil || !ok || b.LastChapterNum != 200 {
|
||||
t.Fatalf("owner's bookmark after the other's delete = %+v ok=%v err=%v, want it intact", b, ok, err)
|
||||
}
|
||||
due, err = s.DueForLatestCheck("asura", time.Now().UnixMilli())
|
||||
due, err = s.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
|
||||
if err != nil {
|
||||
t.Fatalf("DueForLatestCheck after delete: %v", err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
# ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them
|
||||
|
||||
Date: 2026-08-16
|
||||
Status: accepted
|
||||
|
||||
## Decision
|
||||
|
||||
A **Sighting** is the Latest Chapter the Reader's own browser read off the
|
||||
Series page and PUT to the backend. It is now allowed to stand in for a Poll,
|
||||
under one restriction and one ceiling:
|
||||
|
||||
- **Solitary Series only.** A Sighting defers the Poll of a Series exactly one
|
||||
Bookmark points at. A Series two Readers share is Polled on schedule no matter
|
||||
how recently it was sighted.
|
||||
- **One rest of standing.** A Sighting postpones Polls for one Rest
|
||||
(`defaultRest`, an hour), not forever: a Series nobody visits again returns to
|
||||
the normal schedule by itself.
|
||||
- **Six-rest ceiling.** `sightingCeilingRests = 6`, counted in the Site's own
|
||||
Rest — six hours everywhere today. However many Sightings arrive, a Series
|
||||
unpolled that long is Polled.
|
||||
|
||||
Both live in the due query's HAVING clause (`store.DueForLatestCheck`), beside
|
||||
the Rest cutoff — the same place the schedule has always been decided, so no
|
||||
timer and no second code path can disagree with it.
|
||||
|
||||
Attribution and judgement:
|
||||
|
||||
- `Store.RecordSighting` runs *before* the Upsert that stores the reported
|
||||
value, because the raise test needs the row as it stands. A report that raises
|
||||
the stored Latest Chapter names its Reader in `series.latest_raised_by`.
|
||||
- The Poll is the oracle. `Poller.checkOne` already compares what the Site
|
||||
publishes against what is stored, so judgement costs no extra request: a lower
|
||||
number contradicts the Sighting (`sighting_disagreements + 1`, both numbers and
|
||||
the Reader logged), the same number confirms it (`sighting_agreements + 1`), a
|
||||
**higher** number is the Site publishing and means nothing either way — but it
|
||||
does clear the attribution (`Store.ClearSightingAttribution`), because the
|
||||
value stored afterwards is the Poll's own and nobody must answer for it.
|
||||
- At `SightingDisagreementLimit` (3) that Reader's Sightings stop deferring
|
||||
anything. They still write the Latest Chapter — the penalty removes a
|
||||
privilege, it does not silence anyone.
|
||||
- `SightingAgreementsToClear` (20) consecutive confirmations forgive the
|
||||
disagreements. A disagreement resets the run to zero.
|
||||
- The owner clears marks from the administration page (issue #102, shipped
|
||||
first precisely so a false mark has a remedy the day the mechanism lands).
|
||||
|
||||
One client change was required, and only one. Both userscripts stopped short of
|
||||
PUTting a read whose number had not moved (`applyLatestChapterIfChanged`), so
|
||||
the case this whole mechanism exists for — visiting a Series with nothing new —
|
||||
never reached the backend. `reportLatestChapter` now sends it, skipping only the
|
||||
local write and the re-render. A numberless PUT (favourite toggle, progress from
|
||||
a chapter page) is not a Sighting and defers nothing: nobody read the Series
|
||||
page, so there would be nothing to judge later.
|
||||
|
||||
## Why
|
||||
|
||||
Most of the backend's work was redundant. The userscript reads the Latest
|
||||
Chapter on every Series page visit; minutes later the Poll Lane fetches the same
|
||||
page for the same number. Deferring on a report converts a visit into a Poll
|
||||
saved, which is Lane capacity handed back to Series nobody is
|
||||
reading.
|
||||
|
||||
The restriction is the whole safety argument, and it is about **blast radius**,
|
||||
not about trust arithmetic:
|
||||
|
||||
- On a solitary Series, a wrong report can only mislead the Reader who made it.
|
||||
There is nobody else's ember to falsify.
|
||||
- On a shared Series it could mislead someone else, so a report never postpones
|
||||
anything there.
|
||||
|
||||
The ceiling bounds the damage in time: a false value dies within six hours
|
||||
whatever happens, because the Poll that finds it is guaranteed. That is also
|
||||
what makes lying pointless — the six-hour audit is certain, not sampled, so a
|
||||
determined attacker buys at most three ceilings' worth of a wrong number on
|
||||
their own Series and then loses deferral entirely.
|
||||
|
||||
The cost of recovery is deliberate. An agreement is only recorded when a later
|
||||
Poll confirms a Sighting, so twenty agreements are twenty Polls of Series that
|
||||
Reader bookmarks — hours to days of real time, not twenty page views. Waiting is
|
||||
therefore not a strategy, and credit cannot be banked in advance.
|
||||
|
||||
## Tradeoffs and rejections
|
||||
|
||||
- **Trusting a Sighting on a shared Series** rejected: it is the only case where
|
||||
one Reader's mistake reaches another Reader's list, and no amount of
|
||||
reputation makes that recoverable within the six-hour window.
|
||||
- **Cross-Reader agreement, voting, weighting, consensus scoring** rejected on
|
||||
evidence: every truth-discovery method estimates source reliability by
|
||||
comparing sources on the same object, and the standard survey states outright
|
||||
that an object provided by very few sources cannot have its confidence
|
||||
evaluated — Li, Gao, Meng, Li, Su, Zhao, Fan, Han, *A Survey on Truth
|
||||
Discovery*, SIGMOD Record 45(1), 2016 (arXiv:1505.02463), §"Challenges" on
|
||||
sparse sources. With the two Readers this backend actually has, a
|
||||
disagreement is a coin flip. The Poll is an authoritative oracle, so it is
|
||||
the only judge.
|
||||
- **A randomised audit** (Poll a fraction of deferred Series) rejected in favour
|
||||
of the fixed ceiling. Sampling an oracle against untrusted reports is the
|
||||
gold-question technique from crowdsourcing quality control — Le, Edmonds,
|
||||
Hester, Biewald, *Ensuring quality in crowdsourced search relevance
|
||||
evaluation: the effects of training question distribution*, SIGIR 2010
|
||||
Workshop on Crowdsourcing for Search Evaluation, which inserts known answers
|
||||
sporadically and adjusts each worker's trust from them. The ceiling is the
|
||||
same idea made deterministic: sampling prices an attack in expectation, a
|
||||
guaranteed six-hour audit prices it as a certainty, which is what makes the
|
||||
solitary-Series rule defensible in one sentence.
|
||||
- **A trust *ratio*** (agreements over judgements, as that same gold-question
|
||||
scheme uses) rejected for two thresholds: a ratio lets an attacker bank
|
||||
credit first and spend it on lies later, and it needs the owner watching a
|
||||
score to act. Three-and-twenty is a threshold both ways — a disagreement
|
||||
resets the run to zero, so credit cannot be pre-bought, and recovery happens
|
||||
without the owner in the loop.
|
||||
- **Blocking a marked Reader's writes** rejected: the Latest Chapter they report
|
||||
is still the best available value, and their Sightings must keep being judged
|
||||
or they could never earn the privilege back.
|
||||
- **Per-Series flagging** rejected in favour of per-Reader marks: a Series is
|
||||
not the thing that can be wrong. Naming the Reader and logging both numbers is
|
||||
also what distinguishes a broken Site adapter (every Reader of that Site
|
||||
contradicted at once) from one bad actor.
|
||||
- **Timers or a background reputation job** rejected: deferral is recomputed
|
||||
from live facts every round — Bookmark count and sighting timestamp — so a
|
||||
Series that gains a second Bookmark stops deferring at once, with nothing to
|
||||
invalidate. The Reader's marks are the one input read earlier, when the
|
||||
Sighting is recorded rather than when the round runs: a Reader who crosses
|
||||
the threshold, or has their marks cleared, changes behaviour from their next
|
||||
Sighting on, and the standing they already bought lasts out its rest. That is
|
||||
bounded by one rest and costs one subselect instead of joining `readers` into
|
||||
the due query on every round.
|
||||
|
||||
## Constraints preserved
|
||||
|
||||
- A Sighting is not Progress: it may move the Latest Chapter and nothing else.
|
||||
`updated_at` never moves, so a report cannot reorder the list (ADR-0004).
|
||||
- The Latest Chapter is a Series-level fact (ADR-0003): a Sighting writes the
|
||||
shared row, so every Reader of a shared Series sees it immediately — deferral
|
||||
is the only thing the solitary rule withholds.
|
||||
- Ember means new chapter only (`docs/design-system.md`): a marked Reader
|
||||
renders no differently in their own list, and nothing about the trust model
|
||||
reaches the Series list's colour.
|
||||
- The Poll remains authoritative. Where a Sighting and a Poll disagree, the
|
||||
Poll's value is what gets stored.
|
||||
@@ -40,6 +40,7 @@
|
||||
"38": "novel-logic.test.js",
|
||||
"39": "UI Critique 2026-07-26A",
|
||||
"40": "UI Critique 2026-07-26B",
|
||||
"41": "Handler",
|
||||
"42": "pgtest.go",
|
||||
"43": "Issue Tracker & Triage",
|
||||
"44": "Ticket Workflow",
|
||||
@@ -151,6 +152,7 @@
|
||||
"150": "Reviewer Subagent (opencode)",
|
||||
"151": "Finding Severity Rubric",
|
||||
"152": "Spec Compliance Review",
|
||||
"154": "ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them",
|
||||
"156": "ResponseWriter",
|
||||
"161": "Why Use samber/oops",
|
||||
"162": "singleflight Cache Stampede Prevention",
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
# Graph Report - mangaBookmark (2026-08-16)
|
||||
|
||||
## Corpus Check
|
||||
- 116 files · ~284,419 words
|
||||
- 119 files · ~290,953 words
|
||||
- Verdict: corpus is large enough that graph structure adds value.
|
||||
|
||||
## Summary
|
||||
- 1701 nodes · 3463 edges · 219 communities (69 shown, 150 thin omitted)
|
||||
- Extraction: 91% EXTRACTED · 9% INFERRED · 0% AMBIGUOUS · INFERRED: 326 edges (avg confidence: 0.77)
|
||||
- 1736 nodes · 3597 edges · 221 communities (70 shown, 151 thin omitted)
|
||||
- Extraction: 90% EXTRACTED · 10% INFERRED · 0% AMBIGUOUS · INFERRED: 361 edges (avg confidence: 0.78)
|
||||
- Token cost: 0 input · 0 output
|
||||
|
||||
## Graph Freshness
|
||||
- Built from commit: `58014eb8`
|
||||
- Built from commit: `56afb9f2`
|
||||
- Run `git rev-parse HEAD` and compare to check if the graph is stale.
|
||||
- Run `graphify update .` after code changes (no API cost).
|
||||
|
||||
@@ -55,6 +55,7 @@
|
||||
- [[_COMMUNITY_novel-logic.test.js|novel-logic.test.js]]
|
||||
- [[_COMMUNITY_UI Critique 2026-07-26A|UI Critique 2026-07-26A]]
|
||||
- [[_COMMUNITY_UI Critique 2026-07-26B|UI Critique 2026-07-26B]]
|
||||
- [[_COMMUNITY_Handler|Handler]]
|
||||
- [[_COMMUNITY_pgtest.go|pgtest.go]]
|
||||
- [[_COMMUNITY_Issue Tracker & Triage|Issue Tracker & Triage]]
|
||||
- [[_COMMUNITY_Ticket Workflow|Ticket Workflow]]
|
||||
@@ -166,6 +167,7 @@
|
||||
- [[_COMMUNITY_Reviewer Subagent (opencode)|Reviewer Subagent (opencode)]]
|
||||
- [[_COMMUNITY_Finding Severity Rubric|Finding Severity Rubric]]
|
||||
- [[_COMMUNITY_Spec Compliance Review|Spec Compliance Review]]
|
||||
- [[_COMMUNITY_ADR-0011 Sightings — a Reader report defers a Poll where being wrong hurts only them|ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them]]
|
||||
- [[_COMMUNITY_ResponseWriter|ResponseWriter]]
|
||||
- [[_COMMUNITY_Why Use samberoops|Why Use samber/oops]]
|
||||
- [[_COMMUNITY_singleflight Cache Stampede Prevention|singleflight Cache Stampede Prevention]]
|
||||
@@ -235,16 +237,16 @@
|
||||
- [[_COMMUNITY_Userscript CLAUDE guidance|Userscript CLAUDE guidance]]
|
||||
|
||||
## God Nodes (most connected - your core abstractions)
|
||||
1. `testConfig()` - 54 edges
|
||||
2. `newTestStore()` - 49 edges
|
||||
1. `newTestStore()` - 62 edges
|
||||
2. `testConfig()` - 55 edges
|
||||
3. `newWebTestServer()` - 49 edges
|
||||
4. `newTestStore()` - 44 edges
|
||||
5. `e()` - 33 edges
|
||||
6. `Open()` - 30 edges
|
||||
7. `Handler` - 29 edges
|
||||
8. `Store` - 28 edges
|
||||
9. `ne()` - 28 edges
|
||||
10. `De()` - 28 edges
|
||||
6. `Store` - 31 edges
|
||||
7. `Open()` - 31 edges
|
||||
8. `newTestPoller()` - 30 edges
|
||||
9. `Handler` - 29 edges
|
||||
10. `ne()` - 28 edges
|
||||
|
||||
## Surprising Connections (you probably didn't know these)
|
||||
- `el()` --indirect_call--> `c()` [INFERRED]
|
||||
@@ -268,7 +270,7 @@
|
||||
- **Headless browser infrastructure (sidecar, on-demand, home deployment)** — docs_adr_0005_on_demand_browser_headless_shell, docs_adr_0005_on_demand_browser_cdp, docs_adr_0005_on_demand_browser_on_demand_start, docs_adr_0006_browser_on_the_home_machine_home_machine_rationale [INFERRED 0.85]
|
||||
- **lightnovelworld series-identity investigation and fix** — docs_research_lightnovelworld_chapter_vs_series_slug_issue_77, docs_research_lightnovelworld_chapter_vs_series_slug_unscoped_regex, docs_adr_0008_series_identity_is_discovered_not_derived_discovered_identity [INFERRED 0.85]
|
||||
|
||||
## Communities (219 total, 150 thin omitted)
|
||||
## Communities (221 total, 151 thin omitted)
|
||||
|
||||
### Community 0 - "HTMX Library Internals"
|
||||
Cohesion: 0.08
|
||||
@@ -280,23 +282,23 @@ Nodes (89): floatPtr(), testConfig(), getCover(), Cookie, Handler, ResponseRecor
|
||||
|
||||
### Community 2 - "Manga Userscript Adapters"
|
||||
Cohesion: 0.06
|
||||
Nodes (76): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+68 more)
|
||||
Nodes (77): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), armDwell() (+69 more)
|
||||
|
||||
### Community 3 - "Novel Userscript Adapters"
|
||||
Cohesion: 0.06
|
||||
Nodes (79): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+71 more)
|
||||
Nodes (78): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLnwStaleRowRepair() (+70 more)
|
||||
|
||||
### Community 4 - "Series Acquisition Tests"
|
||||
Cohesion: 0.08
|
||||
Nodes (78): bookmarkNewKaganeSeries(), bookmarkNewNovelfullSeries(), bookmarkNewSeries(), Context, Store, T, newAcquirer(), readBookmark() (+70 more)
|
||||
Cohesion: 0.07
|
||||
Nodes (103): bookmarkNewKaganeSeries(), bookmarkNewNovelfullSeries(), bookmarkNewSeries(), Context, Store, T, newAcquirer(), readBookmark() (+95 more)
|
||||
|
||||
### Community 5 - "Bookmarks API Tests"
|
||||
Cohesion: 0.07
|
||||
Nodes (70): auth(), getBookmarks(), Handler, Request, Store, T, newTestServer(), newTestStore() (+62 more)
|
||||
Nodes (71): auth(), getBookmarks(), Handler, Request, Store, T, newTestServer(), newTestStore() (+63 more)
|
||||
|
||||
### Community 7 - "Cover & Acquire Internals"
|
||||
Cohesion: 0.07
|
||||
Nodes (43): Addr, fakeLanes, Context, Store, WaitGroup, defaultCoverResolver(), fetchCoverBytes(), Client (+35 more)
|
||||
Cohesion: 0.06
|
||||
Nodes (43): Addr, fakeLanes, Context, Store, WaitGroup, isInterstitial(), defaultCoverResolver(), fetchCoverBytes() (+35 more)
|
||||
|
||||
### Community 8 - "System Architecture Concepts"
|
||||
Cohesion: 0.15
|
||||
@@ -318,10 +320,6 @@ Nodes (45): Store, T, newTestStore(), readLatestCheckedAt(), readSeries(), secon
|
||||
Cohesion: 0.08
|
||||
Nodes (32): Handler, Request, ResponseWriter, Store, Healthz(), writeJSON(), Auth(), compressible() (+24 more)
|
||||
|
||||
### Community 13 - "Web UI Handlers"
|
||||
Cohesion: 0.14
|
||||
Nodes (18): currentLib(), currentTab(), filterBookmarks(), Client, HandlerFunc, Request, ResponseWriter, Store (+10 more)
|
||||
|
||||
### Community 14 - "Go Error Handling"
|
||||
Cohesion: 0.06
|
||||
Nodes (33): Creating Errors, Custom Error Types, Custom types that wrap other errors, Decision table: which error strategy to use, Error Creation, Error String Conventions, Errors as Values, `errors.New` — static error messages (+25 more)
|
||||
@@ -368,7 +366,7 @@ Nodes (12): coverResponse(), Request, T, TestCoverFetcherCanonicalisesJpgAlias()
|
||||
|
||||
### Community 26 - "Open"
|
||||
Cohesion: 0.05
|
||||
Nodes (62): awaitPromise(), browserConnectionLost(), classifyBrowserError(), comixRead(), comixSeriesPageURL(), Action, Context, Mutex (+54 more)
|
||||
Nodes (61): awaitPromise(), browserConnectionLost(), classifyBrowserError(), comixRead(), comixSeriesPageURL(), Action, Context, Mutex (+53 more)
|
||||
|
||||
### Community 27 - "Find Skills Guide"
|
||||
Cohesion: 0.14
|
||||
@@ -387,8 +385,8 @@ Cohesion: 0.40
|
||||
Nodes (5): Map of pointers for large, frequently updated structs, Memory Layout, Pointer receivers for large structs, Struct field alignment, Zero-size field at end of struct
|
||||
|
||||
### Community 31 - "Store"
|
||||
Cohesion: 0.08
|
||||
Nodes (8): coverRelativePath(), coverSourceAddress(), displayChapter(), Store, scanSeries(), TestDisplayChapter(), Bookmark, ReaderSummary
|
||||
Cohesion: 0.09
|
||||
Nodes (5): coverRelativePath(), coverSourceAddress(), Store, scanSeries(), ReaderSummary
|
||||
|
||||
### Community 33 - "Go Testing Guide"
|
||||
Cohesion: 0.20
|
||||
@@ -418,6 +416,10 @@ Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations,
|
||||
Cohesion: 0.29
|
||||
Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations, Persona Red Flags, Priority Issues, Questions to Consider
|
||||
|
||||
### Community 41 - "Handler"
|
||||
Cohesion: 0.14
|
||||
Nodes (18): currentLib(), currentTab(), filterBookmarks(), Client, HandlerFunc, Request, ResponseWriter, Store (+10 more)
|
||||
|
||||
### Community 42 - "pgtest.go"
|
||||
Cohesion: 0.18
|
||||
Nodes (12): M, TestMain(), M, TestMain(), M, Main(), start(), URL() (+4 more)
|
||||
@@ -534,6 +536,10 @@ Nodes (3): ADR-0005: On-demand browser sidecar, Constraints, Decision
|
||||
Cohesion: 0.48
|
||||
Nodes (6): T, TestCreateAndGetSession(), TestDeleteSessionIsPerReader(), TestDeleteSessionRevokes(), TestExpiredSessionIsGone(), TestGetSessionUnknownID()
|
||||
|
||||
### Community 154 - "ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them"
|
||||
Cohesion: 0.33
|
||||
Nodes (5): ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them, Constraints preserved, Decision, Tradeoffs and rejections, Why
|
||||
|
||||
### Community 156 - "ResponseWriter"
|
||||
Cohesion: 0.18
|
||||
Nodes (13): AdminPatterns(), HandlerFunc, Request, ResponseWriter, Time, Handler, readerPathID(), since() (+5 more)
|
||||
@@ -543,24 +549,24 @@ Cohesion: 0.50
|
||||
Nodes (3): Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust), Second script: `novel-bookmark.user.js`, Userscript structure (single IIFE, `manga-bookmark.user.js`)
|
||||
|
||||
## Knowledge Gaps
|
||||
- **520 isolated node(s):** `bookmarkmanager/backend`, `ctxKey`, `loginView`, `ctxKey`, `test` (+515 more)
|
||||
- **524 isolated node(s):** `bookmarkmanager/backend`, `ctxKey`, `loginView`, `ctxKey`, `test` (+519 more)
|
||||
These have ≤1 connection - possible missing edges or undocumented components.
|
||||
- **150 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes.
|
||||
- **151 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes.
|
||||
|
||||
## Suggested Questions
|
||||
_Questions this graph is uniquely positioned to answer:_
|
||||
|
||||
- **Why does `Open()` connect `CDP Browser Client` to `Cover Fetch Test Helpers`, `Series Acquisition Tests`, `Bookmarks API Tests`, `pgtest.go`, `Store Tests`, `Store`?**
|
||||
_High betweenness centrality (0.060) - this node is a cross-community bridge._
|
||||
- **Why does `New()` connect `Series Acquisition Tests` to `Bookmarks API Tests`, `Cover & Acquire Internals`, `Handler`, `Session Middleware`, `CDP Browser Client`, `ResponseWriter`?**
|
||||
_High betweenness centrality (0.050) - this node is a cross-community bridge._
|
||||
- **Why does `New()` connect `Series Acquisition Tests` to `Bookmarks API Tests`, `Cover & Acquire Internals`, `Session Middleware`, `Web UI Handlers`, `CDP Browser Client`, `ResponseWriter`?**
|
||||
_High betweenness centrality (0.048) - this node is a cross-community bridge._
|
||||
- **Why does `newRouter()` connect `Bookmarks API Tests` to `Cover Fetch Test Helpers`, `Bookmarks API Handler`, `Series Acquisition Tests`, `ResponseWriter`?**
|
||||
_High betweenness centrality (0.031) - this node is a cross-community bridge._
|
||||
_High betweenness centrality (0.030) - this node is a cross-community bridge._
|
||||
- **Are the 25 inferred relationships involving `newTestStore()` (e.g. with `TestAcquireDoesNotBlockTheWrite()` and `TestAcquireFailureLeavesTheBookmarkIntact()`) actually correct?**
|
||||
_`newTestStore()` has 25 INFERRED edges - model-reasoned connections that need verification._
|
||||
- **Are the 48 inferred relationships involving `testConfig()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverNeverEchoesNonImage()`) actually correct?**
|
||||
_`testConfig()` has 48 INFERRED edges - model-reasoned connections that need verification._
|
||||
- **Are the 12 inferred relationships involving `newTestStore()` (e.g. with `TestAcquireDoesNotBlockTheWrite()` and `TestAcquireFailureLeavesTheBookmarkIntact()`) actually correct?**
|
||||
_`newTestStore()` has 12 INFERRED edges - model-reasoned connections that need verification._
|
||||
- **Are the 8 inferred relationships involving `newWebTestServer()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverRejectsUnknownAddress()`) actually correct?**
|
||||
_`newWebTestServer()` has 8 INFERRED edges - model-reasoned connections that need verification._
|
||||
- **What connects `bookmarkmanager/backend`, `ctxKey`, `loginView` to the rest of the system?**
|
||||
_552 weakly-connected nodes found - possible documentation gaps or missing edges._
|
||||
_556 weakly-connected nodes found - possible documentation gaps or missing edges._
|
||||
File diff suppressed because one or more lines are too long
+2654
-859
File diff suppressed because it is too large
Load Diff
+36
-21
@@ -35,8 +35,8 @@
|
||||
"semantic_hash": "4a94ac85bad6bce330d085bcc0ae3ffd"
|
||||
},
|
||||
"userscript/manga-bookmark.user.js": {
|
||||
"mtime": 1786499529.779988,
|
||||
"ast_hash": "1f8bcddd3632d709f058a8401af8f127",
|
||||
"mtime": 1786872888.3482232,
|
||||
"ast_hash": "4e79684942234b26f4a1fb03fccd6d35",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
".agents/skills/find-skills/SKILL.md": {
|
||||
@@ -225,8 +225,8 @@
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/api_test.go": {
|
||||
"mtime": 1786863966.7056258,
|
||||
"ast_hash": "0d443b4fe3c04ab057505d73b8a88d07",
|
||||
"mtime": 1786868080.2317674,
|
||||
"ast_hash": "b64372df5429fb14e48b37b3bab10402",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/cover_test.go": {
|
||||
@@ -235,9 +235,9 @@
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/api/handlers.go": {
|
||||
"mtime": 1786363889.552731,
|
||||
"ast_hash": "59e6b8767ab19839bb8f82891a7e4616",
|
||||
"semantic_hash": "59e6b8767ab19839bb8f82891a7e4616"
|
||||
"mtime": 1786867745.788687,
|
||||
"ast_hash": "ae4f3b961450f06a06e0632eb273a5b2",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/httpmw/middleware.go": {
|
||||
"mtime": 1786216141.672644,
|
||||
@@ -260,8 +260,8 @@
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/latest/poller.go": {
|
||||
"mtime": 1786865046.096795,
|
||||
"ast_hash": "2bfb51177e3df474b80ea1b05523377e",
|
||||
"mtime": 1786872774.641225,
|
||||
"ast_hash": "5e55e2ac5cb7d347807754f92425d29b",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/latest/poller_test.go": {
|
||||
@@ -270,8 +270,8 @@
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/latest/sites.go": {
|
||||
"mtime": 1786861012.026504,
|
||||
"ast_hash": "9b90f9b710ccbd0d0a0a6a0dd721ff3c",
|
||||
"mtime": 1786868771.0663204,
|
||||
"ast_hash": "5786000dd0dfd5b4b3bd2b87b1fc100b",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/latest/sites_test.go": {
|
||||
@@ -340,13 +340,13 @@
|
||||
"semantic_hash": "0b6764a0ee20f5cb7748eecd31a1d220"
|
||||
},
|
||||
"backend/internal/store/store.go": {
|
||||
"mtime": 1786863704.5112119,
|
||||
"ast_hash": "989629af18232e35b12f1eef2a8cb422",
|
||||
"mtime": 1786868763.646761,
|
||||
"ast_hash": "34f55617b7409f03f02e925bec92fac0",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/store/store_test.go": {
|
||||
"mtime": 1786863712.075631,
|
||||
"ast_hash": "e48f21e34238ba46874cc88cf96e022f",
|
||||
"mtime": 1786867781.301985,
|
||||
"ast_hash": "16f69d97eb63bcec7409294466642411",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/userscript/userscript.go": {
|
||||
@@ -395,8 +395,8 @@
|
||||
"semantic_hash": "8008a187690764436540fab47ba0cfcc"
|
||||
},
|
||||
"userscript/novel-bookmark.user.js": {
|
||||
"mtime": 1786499529.779988,
|
||||
"ast_hash": "834effb0821f8d6c9f57f6554a5db462",
|
||||
"mtime": 1786872902.809149,
|
||||
"ast_hash": "36c81462919719f0e053ca5d633a243a",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"userscript/test/novel-logic.test.js": {
|
||||
@@ -425,13 +425,13 @@
|
||||
"semantic_hash": "6c6f3e4c4c2f57867894280bce728c50"
|
||||
},
|
||||
"backend/AGENTS.md": {
|
||||
"mtime": 1786865319.314815,
|
||||
"ast_hash": "5bbabcd7dd425302cb0d989d8897baf6",
|
||||
"mtime": 1786872951.3127444,
|
||||
"ast_hash": "68bc86af9593adad1e37fa05188fe63e",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/CLAUDE.md": {
|
||||
"mtime": 1786865319.314815,
|
||||
"ast_hash": "5bbabcd7dd425302cb0d989d8897baf6",
|
||||
"mtime": 1786872951.3127444,
|
||||
"ast_hash": "68bc86af9593adad1e37fa05188fe63e",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/web/templates/app.html": {
|
||||
@@ -658,5 +658,20 @@
|
||||
"mtime": 1786865113.5362902,
|
||||
"ast_hash": "8b9f9c7a56e0ec3bf950aa70acd95e92",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/latest/sighting_test.go": {
|
||||
"mtime": 1786868815.9760568,
|
||||
"ast_hash": "3b1372bbeeb538b73f86f57da210b4cc",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"backend/internal/store/migrations/0011_series_sightings.sql": {
|
||||
"mtime": 1786867605.3728056,
|
||||
"ast_hash": "d0ade928a6e951eb179e4d0255cca526",
|
||||
"semantic_hash": ""
|
||||
},
|
||||
"docs/adr/0011-sighting-deferral-trust-model.md": {
|
||||
"mtime": 1786872970.473592,
|
||||
"ast_hash": "0b6ca704f62185c9d629320f2d8475b5",
|
||||
"semantic_hash": ""
|
||||
}
|
||||
}
|
||||
@@ -905,27 +905,37 @@
|
||||
}
|
||||
}
|
||||
|
||||
// Records the newest chapter a site has published. Silent: this fires from
|
||||
// Reports the newest chapter a site has published. Silent: this fires from
|
||||
// page visits and background checks the user did not ask for, and it never
|
||||
// reorders the list — updated_at is a candidate the server discards unless
|
||||
// reading progress moved.
|
||||
async function applyLatestChapterIfChanged(existing, latest) {
|
||||
//
|
||||
// An unchanged number is still sent. It is the read that lets the backend
|
||||
// skip its own poll of this series (a Sighting, issue #103), so the common
|
||||
// case — visiting a series with nothing new — is exactly the one worth
|
||||
// reporting. Only the local write and the re-render are skipped.
|
||||
async function reportLatestChapter(existing, latest) {
|
||||
if (!existing || !latest) return;
|
||||
if (existing.latest_chapter_num === latest.num) return;
|
||||
const bm = Object.assign({}, existing, {
|
||||
latest_chapter: latest.label,
|
||||
latest_chapter_num: latest.num,
|
||||
updated_at: Date.now(),
|
||||
});
|
||||
upsertLocal(bm);
|
||||
render();
|
||||
const changed = existing.latest_chapter_num !== latest.num;
|
||||
let bm = existing;
|
||||
if (changed) {
|
||||
bm = Object.assign({}, existing, {
|
||||
latest_chapter: latest.label,
|
||||
latest_chapter_num: latest.num,
|
||||
updated_at: Date.now(),
|
||||
});
|
||||
upsertLocal(bm);
|
||||
render();
|
||||
}
|
||||
// A queued write owns this row; the drain sends latest_chapter
|
||||
// with it, carrying the correct bucket.
|
||||
if (queueGet(bm.key)) return;
|
||||
try {
|
||||
const saved = await apiPut(bm.key, bm);
|
||||
upsertLocal(saved);
|
||||
render();
|
||||
if (changed) {
|
||||
upsertLocal(saved);
|
||||
render();
|
||||
}
|
||||
} catch (e) {
|
||||
/* offline — the local cache still shows it, retried on a later visit */
|
||||
}
|
||||
@@ -937,7 +947,7 @@
|
||||
if (p.type !== "series") return;
|
||||
const existing = state.byKey[keyOf(p)];
|
||||
if (!existing) return;
|
||||
applyLatestChapterIfChanged(
|
||||
reportLatestChapter(
|
||||
existing,
|
||||
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
|
||||
);
|
||||
@@ -977,7 +987,7 @@
|
||||
const html = await res.text();
|
||||
latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
|
||||
}
|
||||
await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest);
|
||||
await reportLatestChapter(state.byKey[bm.key] || bm, latest);
|
||||
} catch (e) {
|
||||
/* offline or blocked — try again after the throttle window */
|
||||
}
|
||||
|
||||
@@ -809,27 +809,37 @@
|
||||
}
|
||||
}
|
||||
|
||||
// Records the newest chapter a site has published. Silent: this fires from
|
||||
// Reports the newest chapter a site has published. Silent: this fires from
|
||||
// page visits and background checks the user did not ask for, and it never
|
||||
// reorders the list — updated_at is a candidate the server discards unless
|
||||
// reading progress moved.
|
||||
async function applyLatestChapterIfChanged(existing, latest) {
|
||||
//
|
||||
// An unchanged number is still sent. It is the read that lets the backend
|
||||
// skip its own poll of this series (a Sighting, issue #103), so the common
|
||||
// case — visiting a series with nothing new — is exactly the one worth
|
||||
// reporting. Only the local write and the re-render are skipped.
|
||||
async function reportLatestChapter(existing, latest) {
|
||||
if (!existing || !latest) return;
|
||||
if (existing.latest_chapter_num === latest.num) return;
|
||||
const bm = Object.assign({}, existing, {
|
||||
latest_chapter: latest.label,
|
||||
latest_chapter_num: latest.num,
|
||||
updated_at: Date.now(),
|
||||
});
|
||||
upsertLocal(bm);
|
||||
render();
|
||||
const changed = existing.latest_chapter_num !== latest.num;
|
||||
let bm = existing;
|
||||
if (changed) {
|
||||
bm = Object.assign({}, existing, {
|
||||
latest_chapter: latest.label,
|
||||
latest_chapter_num: latest.num,
|
||||
updated_at: Date.now(),
|
||||
});
|
||||
upsertLocal(bm);
|
||||
render();
|
||||
}
|
||||
// A queued write owns this row; the drain sends latest_chapter
|
||||
// with it, carrying the correct bucket.
|
||||
if (queueGet(bm.key)) return;
|
||||
try {
|
||||
const saved = await apiPut(bm.key, bm);
|
||||
upsertLocal(saved);
|
||||
render();
|
||||
if (changed) {
|
||||
upsertLocal(saved);
|
||||
render();
|
||||
}
|
||||
} catch (e) {
|
||||
/* offline — the local cache still shows it, retried on a later visit */
|
||||
}
|
||||
@@ -841,7 +851,7 @@
|
||||
if (p.type !== "series") return;
|
||||
const existing = state.byKey[keyOf(p)];
|
||||
if (!existing) return;
|
||||
applyLatestChapterIfChanged(
|
||||
reportLatestChapter(
|
||||
existing,
|
||||
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
|
||||
);
|
||||
@@ -883,7 +893,7 @@
|
||||
if (!res.ok) continue;
|
||||
const html = await res.text();
|
||||
const latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
|
||||
await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest);
|
||||
await reportLatestChapter(state.byKey[bm.key] || bm, latest);
|
||||
} catch (e) {
|
||||
/* offline or blocked — try again after the throttle window */
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user