Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2b597921f4 | |||
| 56afb9f237 |
+28
-2
@@ -27,7 +27,8 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
|
|||||||
`series` keyed `(site, series_id)`
|
`series` keyed `(site, series_id)`
|
||||||
(`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the
|
(`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the
|
||||||
shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`),
|
shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`),
|
||||||
Latest Chapter, `latest_checked_at` — and `bookmarks` holds only what
|
Latest Chapter, `latest_checked_at`, and the Sighting pair
|
||||||
|
`latest_sighted_at`/`latest_raised_by` (issue #103) — and `bookmarks` holds only what
|
||||||
differs between readers: progress, favourite, lifecycle bucket,
|
differs between readers: progress, favourite, lifecycle bucket,
|
||||||
`updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no
|
`updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no
|
||||||
surrogate id; the wire `key` is derived as `site:series_id` on read — and
|
surrogate id; the wire `key` is derived as `site:series_id` on read — and
|
||||||
@@ -78,7 +79,9 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
|
|||||||
each re-checking that Site's bookmarked series' newest published chapter from
|
each re-checking that Site's bookmarked series' newest published chapter from
|
||||||
backend's own network access, so `latest_chapter` stays fresh when the user
|
backend's own network access, so `latest_chapter` stays fresh when the user
|
||||||
isn't browsing. Second, parallel signal — the userscript keeps its own
|
isn't browsing. Second, parallel signal — the userscript keeps its own
|
||||||
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
|
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` schedule, and its
|
||||||
|
`reportLatestChapter` PUTs every read, unchanged numbers included, because an
|
||||||
|
unchanged read is exactly the Sighting worth deferring a Poll on (#103).
|
||||||
Two independent clocks: per-series rest (`series.latest_checked_at`,
|
Two independent clocks: per-series rest (`series.latest_checked_at`,
|
||||||
enforced by `Store.DueForLatestCheck`'s WHERE clause — `now - Rest`) and
|
enforced by `Store.DueForLatestCheck`'s WHERE clause — `now - Rest`) and
|
||||||
per-Lane gap (the Lane sleeping between fetches, `effectiveGap`). Both live
|
per-Lane gap (the Lane sleeping between fetches, `effectiveGap`). Both live
|
||||||
@@ -91,6 +94,29 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
|
|||||||
every tick; found chapter written straight to the series row via
|
every tick; found chapter written straight to the series row via
|
||||||
`Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
|
`Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
|
||||||
order — is never touched.
|
order — is never touched.
|
||||||
|
**Sightings** (issue #103, ADR-0011) let a Reader's own page read defer a
|
||||||
|
Poll: `Store.RecordSighting` — called by the PUT handler *before* the Upsert,
|
||||||
|
because the raise test needs the row as it stands — stamps
|
||||||
|
`series.latest_sighted_at` and, when the report raises the stored number,
|
||||||
|
names its Reader in `series.latest_raised_by`. The due query's HAVING clause
|
||||||
|
is where deferral lives: a Series is skipped only while it has exactly one
|
||||||
|
Bookmark, was sighted within one Rest, and is under the ceiling
|
||||||
|
(`sightingCeilingRests`, six of that Site's rests) since its last Poll. So a
|
||||||
|
shared Series is never deferred, and no Series goes six hours unpolled
|
||||||
|
whatever arrives. `checkOne` judges the named Reader off the comparison it
|
||||||
|
already makes: a lower number is a contradiction (logged with the Reader and
|
||||||
|
both numbers), the same number an agreement, a higher number the Site
|
||||||
|
publishing and neither — that last one clears the attribution instead, since
|
||||||
|
the value the Poll then stores is its own and a later retraction is not the
|
||||||
|
Reader's fault. Three contradictions
|
||||||
|
(`store.SightingDisagreementLimit`) stop that Reader deferring — their
|
||||||
|
reports still write the Latest Chapter — and twenty consecutive agreements
|
||||||
|
(`store.SightingAgreementsToClear`) forgive them, as does the owner's
|
||||||
|
clear-marks control. Deferral is recomputed from live facts every round, so
|
||||||
|
nothing needs invalidating when a Series gains a second Bookmark; the one
|
||||||
|
input read earlier is the Reader's marks, checked when the Sighting is
|
||||||
|
recorded, so crossing the threshold or being cleared takes effect from that
|
||||||
|
Reader's next Sighting and the standing already bought lasts out its rest.
|
||||||
Refusals and browser loss are Lane-local: two `errChallengeHeld` in one pass
|
Refusals and browser loss are Lane-local: two `errChallengeHeld` in one pass
|
||||||
stop that Site for `refuseBackoff` (15m) while other Lanes continue; an
|
stop that Site for `refuseBackoff` (15m) while other Lanes continue; an
|
||||||
`errBrowserInterrupted` (remote Chrome restart) sets a shared Poller flag
|
`errBrowserInterrupted` (remote Chrome restart) sets a shared Poller flag
|
||||||
|
|||||||
@@ -621,6 +621,46 @@ func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Sighting deferral (issue #103) only reaches production through the PUT
|
||||||
|
// handler: the store and poller can be right and the feature still dead if the
|
||||||
|
// handler never records the report. Asserted where a client can see it - the
|
||||||
|
// series stops being due the moment the PUT lands.
|
||||||
|
func TestPutRecordsASighting(t *testing.T) {
|
||||||
|
s := newTestStore(t)
|
||||||
|
srv := newRouter(s, testConfig(), nil)
|
||||||
|
|
||||||
|
now := time.Now().UnixMilli()
|
||||||
|
hour := time.Hour.Milliseconds()
|
||||||
|
seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", now-2*hour)
|
||||||
|
due, err := s.DueForLatestCheck("asura", now-hour, now-6*hour)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("DueForLatestCheck: %v", err)
|
||||||
|
}
|
||||||
|
if len(due) != 1 {
|
||||||
|
t.Fatalf("due before the PUT = %d series, want 1", len(due))
|
||||||
|
}
|
||||||
|
|
||||||
|
body := `{"key":"asura:x","site":"asura","series_id":"x",
|
||||||
|
"series_url":"https://asurascans.com/comics/x",
|
||||||
|
"last_chapter":"Chapter 5","last_chapter_num":5,
|
||||||
|
"latest_chapter":"Chapter 9","latest_chapter_num":9}`
|
||||||
|
req := httptest.NewRequest(http.MethodPut, "/bookmarks/asura:x", strings.NewReader(body))
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
srv.ServeHTTP(rec, auth(req))
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("PUT status = %d, want 200 (body %s)", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
due, err = s.DueForLatestCheck("asura", now-hour, now-6*hour)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("DueForLatestCheck: %v", err)
|
||||||
|
}
|
||||||
|
if len(due) != 0 {
|
||||||
|
t.Fatalf("due after the PUT = %d series, want 0: the handler recorded no Sighting", len(due))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// The userscript route is registered outside the web UI's Discord auth, so it
|
// The userscript route is registered outside the web UI's Discord auth, so it
|
||||||
// must keep working whatever the web config — see internal/userscript for the
|
// must keep working whatever the web config — see internal/userscript for the
|
||||||
// handler's own behaviour. The credential in the path is the owner's derived
|
// handler's own behaviour. The credential in the path is the owner's derived
|
||||||
|
|||||||
@@ -99,7 +99,18 @@ func (h *Handler) Put(w http.ResponseWriter, r *http.Request) {
|
|||||||
// reading progress actually moved. Any client value is ignored.
|
// reading progress actually moved. Any client value is ignored.
|
||||||
b.UpdatedAt = time.Now().UnixMilli()
|
b.UpdatedAt = time.Now().UnixMilli()
|
||||||
|
|
||||||
stored, err := h.Store.Upsert(httpmw.ReaderID(r), b)
|
// A userscript PUT is a Sighting: the Reader's browser was on the Series
|
||||||
|
// page and read its Latest Chapter (issue #103). Recorded before the
|
||||||
|
// Upsert, which is what makes the raise comparison possible, and never
|
||||||
|
// from the web UI's own read-modify-write — a Reader toggling a favourite
|
||||||
|
// has not looked at the Site and must not postpone a Poll. A failure here
|
||||||
|
// costs a deferral, not the write, so it is logged and dropped.
|
||||||
|
readerID := httpmw.ReaderID(r)
|
||||||
|
if err := h.Store.RecordSighting(readerID, b.Site, b.SeriesID, b.LatestChapterNum, b.UpdatedAt); err != nil {
|
||||||
|
log.Printf("record sighting: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
stored, err := h.Store.Upsert(readerID, b)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Printf("upsert: %v", err)
|
log.Printf("upsert: %v", err)
|
||||||
http.Error(w, "internal error", http.StatusInternalServerError)
|
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||||
|
|||||||
@@ -250,7 +250,8 @@ func (p *Poller) runLanePass(ctx context.Context, name string, paced bool) time.
|
|||||||
return defaultGap
|
return defaultGap
|
||||||
}
|
}
|
||||||
|
|
||||||
due, err := p.Store.DueForLatestCheck(name, now.Add(-s.Rest).UnixMilli())
|
due, err := p.Store.DueForLatestCheck(name, now.Add(-s.Rest).UnixMilli(),
|
||||||
|
now.Add(-sightingCeilingRests*s.Rest).UnixMilli())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Printf("latest poll %s: due query: %v", name, err)
|
log.Printf("latest poll %s: due query: %v", name, err)
|
||||||
st.Gap = defaultGap
|
st.Gap = defaultGap
|
||||||
@@ -461,6 +462,11 @@ func (p *Poller) checkOne(ctx context.Context, sr store.Series) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The Poll is the oracle for whatever Sighting last raised this Series
|
||||||
|
// (issue #103), and the judgement is free: the comparison below already
|
||||||
|
// exists, and no extra request is made to reach it.
|
||||||
|
p.judgeSighting(sr, facts.Latest.Num)
|
||||||
|
|
||||||
// Equality, not >, mirroring the userscript (L427): a site that retracts a
|
// Equality, not >, mirroring the userscript (L427): a site that retracts a
|
||||||
// chapter should correct the stored number downward. The comparison is
|
// chapter should correct the stored number downward. The comparison is
|
||||||
// against the due-query snapshot; a concurrent write in between only costs
|
// against the due-query snapshot; a concurrent write in between only costs
|
||||||
@@ -481,6 +487,43 @@ func (p *Poller) checkOne(ctx context.Context, sr store.Series) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// judgeSighting settles the Sighting the Series' stored Latest Chapter is owed
|
||||||
|
// to, if any, against what the Site actually publishes. The asymmetry is the
|
||||||
|
// whole of the detection rule and is what keeps it free of false alarms: a Poll
|
||||||
|
// finding a *lower* number than stored means the Reader who raised it reported
|
||||||
|
// a chapter that does not exist, while a Poll finding a higher one is only the
|
||||||
|
// Site publishing since and means nothing about the report. Equality confirms
|
||||||
|
// the report, which is how an honest Reader earns back a mark.
|
||||||
|
//
|
||||||
|
// A Series with no attribution — the stored value is a Poll's own, or a
|
||||||
|
// previous Poll already judged the report — is nobody's to answer for.
|
||||||
|
func (p *Poller) judgeSighting(sr store.Series, found float64) {
|
||||||
|
if sr.LatestRaisedBy == nil || sr.LatestChapterNum == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
stored := *sr.LatestChapterNum
|
||||||
|
if found > stored {
|
||||||
|
// The report is neither confirmed nor contradicted, but it is answered:
|
||||||
|
// the value about to be stored is the Poll's own, so leaving the
|
||||||
|
// attribution would credit this Reader with the next Poll's agreement
|
||||||
|
// and blame them if the Site later retracts.
|
||||||
|
if err := p.Store.ClearSightingAttribution(sr.Site, sr.SeriesID, *sr.LatestRaisedBy); err != nil {
|
||||||
|
log.Printf("latest poll %q: clear sighting attribution: %v", sr.Key(), err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if found < stored {
|
||||||
|
// Logged with both numbers and the Reader, because that is what tells a
|
||||||
|
// broken Site adapter (which marks every Reader of that Site at once)
|
||||||
|
// from one Reader deliberately lying.
|
||||||
|
log.Printf("latest poll %q: sighting contradicted: reader %d raised it to %v, site publishes %v",
|
||||||
|
sr.Key(), *sr.LatestRaisedBy, stored, found)
|
||||||
|
}
|
||||||
|
if err := p.Store.RecordSightingOutcome(sr.Site, sr.SeriesID, *sr.LatestRaisedBy, found == stored); err != nil {
|
||||||
|
log.Printf("latest poll %q: record sighting outcome: %v", sr.Key(), err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// healCover runs prefetchCover in the background. Cover bytes come from a
|
// healCover runs prefetchCover in the background. Cover bytes come from a
|
||||||
// different host — often a CDN — and heal once in a Series's life, so they
|
// different host — often a CDN — and heal once in a Series's life, so they
|
||||||
// must not consume a Lane's gap: a large import with many blanks would
|
// must not consume a Lane's gap: a large import with many blanks would
|
||||||
|
|||||||
@@ -0,0 +1,494 @@
|
|||||||
|
package latest
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"bookmarkmanager/backend/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Sightings (issue #103) are specified at the Poller seam, with the store as
|
||||||
|
// the way in: a Sighting is seeded the way handlers.Put performs one, a round
|
||||||
|
// is run against the injected fetcher and a frozen clock, and the assertions
|
||||||
|
// are the two observable facts — whether the Series was fetched, and what the
|
||||||
|
// stored Latest Chapter is afterwards. Nothing here asserts counter arithmetic
|
||||||
|
// through an internal call or reads how a deferral is represented in a row.
|
||||||
|
|
||||||
|
const (
|
||||||
|
sightingSlug = "chronicles-of-the-demon-faction-f886a8af"
|
||||||
|
sightingKey = "asura:" + sightingSlug
|
||||||
|
sightingURL = "https://asurascans.com/comics/" + sightingSlug
|
||||||
|
)
|
||||||
|
|
||||||
|
// sightingFixtureLatest is the newest chapter asuraSeriesFixture publishes.
|
||||||
|
const sightingFixtureLatest = 181.0
|
||||||
|
|
||||||
|
// sight performs one Sighting exactly as the JSON API does (handlers.Put):
|
||||||
|
// RecordSighting against the row as stored, then the Upsert that stores the
|
||||||
|
// reported value. The order is load-bearing — the raise comparison has nothing
|
||||||
|
// to compare against once the Upsert has landed — and the bookmark's own fields
|
||||||
|
// are carried over untouched, which is what a userscript PUT does when it
|
||||||
|
// echoes back the row it cached.
|
||||||
|
func sight(t *testing.T, s *store.Store, readerID int64, key string, num float64, at time.Time) {
|
||||||
|
t.Helper()
|
||||||
|
site, seriesID, ok := strings.Cut(key, ":")
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("key %q: no ':' separator", key)
|
||||||
|
}
|
||||||
|
b, found, err := s.Get(readerID, key)
|
||||||
|
if err != nil || !found {
|
||||||
|
t.Fatalf("sight %q: get: %v found=%v", key, err, found)
|
||||||
|
}
|
||||||
|
if err := s.RecordSighting(readerID, site, seriesID, &num, at.UnixMilli()); err != nil {
|
||||||
|
t.Fatalf("sight %q: %v", key, err)
|
||||||
|
}
|
||||||
|
b.LatestChapter = fmt.Sprintf("Chapter %v", num)
|
||||||
|
b.LatestChapterNum = &num
|
||||||
|
b.UpdatedAt = at.UnixMilli()
|
||||||
|
if _, err := s.Upsert(readerID, b); err != nil {
|
||||||
|
t.Fatalf("sight %q: upsert: %v", key, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// secondReader is another Reader on the same database. The owner seed is the
|
||||||
|
// only reader-creation path in this package, so a second Open as a different
|
||||||
|
// owner is how a test gets one (as TestRunOnceFetchesSharedSeriesOnce does).
|
||||||
|
func secondReader(t *testing.T, dbURL string) *store.Store {
|
||||||
|
t.Helper()
|
||||||
|
other, err := store.Open(dbURL,
|
||||||
|
store.Owner{DiscordID: "second-reader", TokenHash: sha256.Sum256([]byte("second-token-hash"))},
|
||||||
|
t.TempDir(), testCoverBaseURL)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Open second reader: %v", err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { other.Close() })
|
||||||
|
return other
|
||||||
|
}
|
||||||
|
|
||||||
|
func readLatestNum(t *testing.T, s *store.Store, readerID int64, key string) float64 {
|
||||||
|
t.Helper()
|
||||||
|
b, ok, err := s.Get(readerID, key)
|
||||||
|
if err != nil || !ok {
|
||||||
|
t.Fatalf("Get %q: %v ok=%v", key, err, ok)
|
||||||
|
}
|
||||||
|
if b.LatestChapterNum == nil {
|
||||||
|
t.Fatalf("%q has no latest chapter", key)
|
||||||
|
}
|
||||||
|
return *b.LatestChapterNum
|
||||||
|
}
|
||||||
|
|
||||||
|
// A Series only one Reader bookmarks is the case where being wrong can hurt
|
||||||
|
// nobody but the Reader who reported it, so their Sighting stands in for the
|
||||||
|
// Poll and the round leaves the Series alone.
|
||||||
|
func TestSightingOnSolitarySeriesDefersPoll(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
|
||||||
|
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||||
|
newTestPoller(t, s, f, now).runOnce(context.Background())
|
||||||
|
|
||||||
|
if got := f.callCount(); got != 0 {
|
||||||
|
t.Fatalf("fetched %d times after a Sighting on a solitary Series, want 0", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// On a shared Series the Sighting still writes the Latest Chapter for everyone,
|
||||||
|
// but the Poll happens on schedule anyway — which is what corrects a wrong
|
||||||
|
// value within the hour instead of letting it persist.
|
||||||
|
func TestSightingOnSharedSeriesDoesNotDeferPoll(t *testing.T) {
|
||||||
|
s, dbURL := newTestStore(t)
|
||||||
|
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
|
||||||
|
other := secondReader(t, dbURL)
|
||||||
|
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
|
||||||
|
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("seed second reader: %v", err)
|
||||||
|
}
|
||||||
|
sight(t, s, s.OwnerID(), sightingKey, 200, now.Add(-10*time.Minute))
|
||||||
|
|
||||||
|
// The Sighting updated the shared row immediately, before any Poll.
|
||||||
|
if got := readLatestNum(t, s, s.OwnerID(), sightingKey); got != 200 {
|
||||||
|
t.Fatalf("latest after the Sighting = %v, want 200", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||||
|
newTestPoller(t, s, f, now).runOnce(context.Background())
|
||||||
|
|
||||||
|
if got := f.callCount(); got != 1 {
|
||||||
|
t.Fatalf("fetched %d times after a Sighting on a shared Series, want 1", got)
|
||||||
|
}
|
||||||
|
if got := readLatestNum(t, s, s.OwnerID(), sightingKey); got != sightingFixtureLatest {
|
||||||
|
t.Fatalf("latest after the Poll = %v, want the Site's own %v", got, sightingFixtureLatest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reporting a chapter is not reading one: a Sighting may move the Latest
|
||||||
|
// Chapter and nothing else. Both the solitary and the shared case, because the
|
||||||
|
// deferral branch must not be where this guarantee lives.
|
||||||
|
func TestSightingLeavesProgressAndOrderingUntouched(t *testing.T) {
|
||||||
|
for _, shared := range []bool{false, true} {
|
||||||
|
name := "solitary"
|
||||||
|
if shared {
|
||||||
|
name = "shared"
|
||||||
|
}
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
s, dbURL := newTestStore(t)
|
||||||
|
read := 5.0
|
||||||
|
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
|
||||||
|
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, SeriesURL: sightingURL,
|
||||||
|
LastChapter: "Chapter 5", LastChapterNum: read,
|
||||||
|
LastChapterURL: sightingURL + "/chapter/5", UpdatedAt: 1000,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("seed: %v", err)
|
||||||
|
}
|
||||||
|
if shared {
|
||||||
|
other := secondReader(t, dbURL)
|
||||||
|
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
|
||||||
|
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("seed second reader: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
sight(t, s, s.OwnerID(), sightingKey, 200, time.UnixMilli(9_000_000))
|
||||||
|
|
||||||
|
b, ok, err := s.Get(s.OwnerID(), sightingKey)
|
||||||
|
if err != nil || !ok {
|
||||||
|
t.Fatalf("Get: %v ok=%v", err, ok)
|
||||||
|
}
|
||||||
|
if b.LatestChapterNum == nil || *b.LatestChapterNum != 200 {
|
||||||
|
t.Fatalf("LatestChapterNum = %v, want 200", b.LatestChapterNum)
|
||||||
|
}
|
||||||
|
if b.LastChapterNum != read {
|
||||||
|
t.Fatalf("LastChapterNum = %v, want %v: a Sighting is not Progress", b.LastChapterNum, read)
|
||||||
|
}
|
||||||
|
if b.UpdatedAt != 1000 {
|
||||||
|
t.Fatalf("updated_at moved to %d: a Sighting must not reorder the list", b.UpdatedAt)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The ceiling is what makes trusting a client report safe: however recently a
|
||||||
|
// Series was sighted, one that has not been Polled in six hours is Polled.
|
||||||
|
func TestSightingCeilingForcesPoll(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-7*time.Hour).UnixMilli())
|
||||||
|
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-time.Minute))
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||||
|
newTestPoller(t, s, f, now).runOnce(context.Background())
|
||||||
|
|
||||||
|
if got := f.callCount(); got != 1 {
|
||||||
|
t.Fatalf("fetched %d times past the %s ceiling, want 1", got, sightingCeilingRests*defaultRest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Deferral is decided from live facts every round, so a Series that gains a
|
||||||
|
// second Bookmark stops deferring at once — and one that loses it defers again.
|
||||||
|
func TestDeferralFollowsTheBookmarkCount(t *testing.T) {
|
||||||
|
s, dbURL := newTestStore(t)
|
||||||
|
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
|
||||||
|
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||||
|
p := newTestPoller(t, s, f, now)
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != 0 {
|
||||||
|
t.Fatalf("solitary Series fetched %d times, want 0", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
other := secondReader(t, dbURL)
|
||||||
|
if _, err := s.Upsert(other.OwnerID(), store.Bookmark{
|
||||||
|
Key: sightingKey, Site: "asura", SeriesID: sightingSlug, UpdatedAt: 2000,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("seed second reader: %v", err)
|
||||||
|
}
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != 1 {
|
||||||
|
t.Fatalf("shared Series fetched %d times, want 1", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The Poll above consumed the rest, so move past it before asking again.
|
||||||
|
if err := other.Delete(other.OwnerID(), sightingKey); err != nil {
|
||||||
|
t.Fatalf("delete second bookmark: %v", err)
|
||||||
|
}
|
||||||
|
later := now.Add(2 * time.Hour)
|
||||||
|
p.Now = func() time.Time { return later }
|
||||||
|
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, later.Add(-time.Minute))
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != 1 {
|
||||||
|
t.Fatalf("Series fetched %d times after returning to one Bookmark, want 1", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A Series nobody reports any more returns to the normal schedule on its own:
|
||||||
|
// the Sighting's standing lasts one rest, not forever.
|
||||||
|
func TestDeferralExpiresWithoutFurtherSightings(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedForCheck(t, s, sightingKey, sightingURL, now.Add(-2*time.Hour).UnixMilli())
|
||||||
|
sight(t, s, s.OwnerID(), sightingKey, sightingFixtureLatest, now.Add(-10*time.Minute))
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
|
||||||
|
p := newTestPoller(t, s, f, now)
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != 0 {
|
||||||
|
t.Fatalf("fetched %d times while the Sighting stood, want 0", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
p.Now = func() time.Time { return now.Add(90 * time.Minute) }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != 1 {
|
||||||
|
t.Fatalf("fetched %d times once the Sighting aged out, want 1", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// demonicFixture publishes one chapter in demonicscans' live page shape, so a
|
||||||
|
// test can make a Site publish an arbitrary number rather than the one the
|
||||||
|
// captured fixture froze.
|
||||||
|
func demonicFixture(num float64) string {
|
||||||
|
return fmt.Sprintf(
|
||||||
|
`<a href="/chaptered.php?manga=11799&chapter=%v" class="chplinks" title="Catastrophic Necromancer %v">Chapter %v</a>`,
|
||||||
|
num, num, num)
|
||||||
|
}
|
||||||
|
|
||||||
|
const (
|
||||||
|
demonicKey = "demonic:Catastrophic-Necromancer"
|
||||||
|
demonicURL = "https://demonicscans.org/manga/Catastrophic-Necromancer"
|
||||||
|
)
|
||||||
|
|
||||||
|
// contradictOnce reports a chapter that does not exist and then runs the round
|
||||||
|
// that catches it, returning when that round ran so a caller can chain the
|
||||||
|
// next one. The wait is one rest and a minute: a Sighting stands in for exactly
|
||||||
|
// one rest, so that is the first moment this solitary Series is Polled again.
|
||||||
|
func contradictOnce(t *testing.T, s *store.Store, p *Poller, sightAt time.Time, real float64) time.Time {
|
||||||
|
t.Helper()
|
||||||
|
sight(t, s, s.OwnerID(), demonicKey, real+500, sightAt)
|
||||||
|
at := sightAt.Add(defaultRest + time.Minute)
|
||||||
|
p.Now = func() time.Time { return at }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != real {
|
||||||
|
t.Fatalf("latest after the Poll = %v, want the Site's own %v", got, real)
|
||||||
|
}
|
||||||
|
return at
|
||||||
|
}
|
||||||
|
|
||||||
|
func seedDemonic(t *testing.T, s *store.Store, checkedAt int64) {
|
||||||
|
t.Helper()
|
||||||
|
seedForCheck(t, s, demonicKey, demonicURL, checkedAt)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A Poll finding a lower number than stored means the Sighting that raised it
|
||||||
|
// was false. The Reader is named — not the Series flagged — and both numbers are
|
||||||
|
// logged, because that is what tells a broken adapter from a deliberate lie.
|
||||||
|
func TestPollContradictingASightingNamesTheReaderAndBothNumbers(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
|
||||||
|
|
||||||
|
var logs strings.Builder
|
||||||
|
prev := log.Writer()
|
||||||
|
log.SetOutput(&logs)
|
||||||
|
t.Cleanup(func() { log.SetOutput(prev) })
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||||
|
p := newTestPoller(t, s, f, now)
|
||||||
|
contradictOnce(t, s, p, now, 296)
|
||||||
|
|
||||||
|
got := logs.String()
|
||||||
|
for _, want := range []string{
|
||||||
|
fmt.Sprintf("reader %d", s.OwnerID()), "796", "296", demonicKey,
|
||||||
|
} {
|
||||||
|
if !strings.Contains(got, want) {
|
||||||
|
t.Fatalf("contradiction log = %q, want it to name %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Three contradictions cost the Reader the right to defer. Nothing here writes
|
||||||
|
// a counter: the marks are earned through Polls, which is the only way
|
||||||
|
// production produces them.
|
||||||
|
func TestThreeContradictionsStopDeferral(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
start := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||||
|
p := newTestPoller(t, s, f, start)
|
||||||
|
at := start
|
||||||
|
for range store.SightingDisagreementLimit {
|
||||||
|
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
|
||||||
|
}
|
||||||
|
fetchesSoFar := f.callCount()
|
||||||
|
|
||||||
|
// The marked Reader sights the same solitary Series again. It still writes
|
||||||
|
// the Latest Chapter — the penalty removes a privilege, it does not silence
|
||||||
|
// anyone — but the Poll is no longer postponed: the round below runs while a
|
||||||
|
// trusted Reader's Sighting would still be standing, and fetches anyway.
|
||||||
|
sight(t, s, s.OwnerID(), demonicKey, 900, at.Add(31*time.Minute))
|
||||||
|
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != 900 {
|
||||||
|
t.Fatalf("latest after a marked Reader's Sighting = %v, want 900", got)
|
||||||
|
}
|
||||||
|
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != fetchesSoFar+1 {
|
||||||
|
t.Fatalf("marked Reader's Sighting still deferred the Poll (fetches %d, want %d)",
|
||||||
|
got, fetchesSoFar+1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The owner's remedy for a mark a broken Site adapter produced restores the
|
||||||
|
// privilege without a wait and without SQL.
|
||||||
|
func TestClearingMarksRestoresDeferral(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
start := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||||
|
p := newTestPoller(t, s, f, start)
|
||||||
|
at := start
|
||||||
|
for range store.SightingDisagreementLimit {
|
||||||
|
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
|
||||||
|
}
|
||||||
|
if err := s.ClearReaderMarks(s.OwnerID()); err != nil {
|
||||||
|
t.Fatalf("ClearReaderMarks: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
fetchesSoFar := f.callCount()
|
||||||
|
sight(t, s, s.OwnerID(), demonicKey, 900, at.Add(31*time.Minute))
|
||||||
|
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != fetchesSoFar {
|
||||||
|
t.Fatalf("fetched %d times after the marks were cleared, want %d: deferral must resume",
|
||||||
|
got, fetchesSoFar)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Recovery is automatic but expensive: twenty Polls that each confirm a
|
||||||
|
// Sighting of this Reader's clear the marks. Each round needs a new chapter,
|
||||||
|
// because only a report that raises the stored number is attributed and so only
|
||||||
|
// that one can be confirmed.
|
||||||
|
func TestTwentyAgreementsClearTheMarks(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
start := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedDemonic(t, s, start.Add(-2*time.Hour).UnixMilli())
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||||
|
p := newTestPoller(t, s, f, start)
|
||||||
|
at := start
|
||||||
|
for range store.SightingDisagreementLimit {
|
||||||
|
at = contradictOnce(t, s, p, at.Add(time.Minute), 296)
|
||||||
|
}
|
||||||
|
|
||||||
|
chapter := 296.0
|
||||||
|
for range store.SightingAgreementsToClear {
|
||||||
|
chapter++
|
||||||
|
sight(t, s, s.OwnerID(), demonicKey, chapter, at.Add(time.Minute))
|
||||||
|
f.body = demonicFixture(chapter) // the Site publishes what was reported
|
||||||
|
at = at.Add(defaultRest + time.Minute)
|
||||||
|
p.Now = func() time.Time { return at }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
}
|
||||||
|
|
||||||
|
fetchesSoFar := f.callCount()
|
||||||
|
chapter++
|
||||||
|
sight(t, s, s.OwnerID(), demonicKey, chapter, at.Add(31*time.Minute))
|
||||||
|
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != fetchesSoFar {
|
||||||
|
t.Fatalf("fetched %d times after %d confirmations, want %d: the marks must be forgiven",
|
||||||
|
got, store.SightingAgreementsToClear, fetchesSoFar)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A Poll finding a higher number is the Site publishing since the Sighting and
|
||||||
|
// means nothing about the Reader — no mark, and no credit either.
|
||||||
|
func TestPollFindingHigherNumberIsNotAContradiction(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
|
||||||
|
// Reported truthfully, then the Site published one more.
|
||||||
|
sight(t, s, s.OwnerID(), demonicKey, 295, now.Add(-10*time.Minute))
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||||
|
p := newTestPoller(t, s, f, now)
|
||||||
|
// One rest on, the Sighting has lapsed and the Poll happens.
|
||||||
|
p.Now = func() time.Time { return now.Add(7 * time.Hour) }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != 1 {
|
||||||
|
t.Fatalf("fetched %d times past the ceiling, want 1", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Unmarked, so a fresh Sighting still defers.
|
||||||
|
at := now.Add(9 * time.Hour)
|
||||||
|
sight(t, s, s.OwnerID(), demonicKey, 296, at.Add(-time.Minute))
|
||||||
|
p.Now = func() time.Time { return at }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != 1 {
|
||||||
|
t.Fatalf("a Reader whose report the Site overtook lost the right to defer (fetches %d, want 1)", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A Poll that overtakes a Sighting takes ownership of the row: the value stored
|
||||||
|
// afterwards is the Poll's own, so a later retraction is not the Reader's fault
|
||||||
|
// and must not be charged to them.
|
||||||
|
func TestAttributionDoesNotSurviveAPollThatOvertookIt(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
|
||||||
|
sight(t, s, s.OwnerID(), demonicKey, 295, now.Add(-10*time.Minute))
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||||
|
p := newTestPoller(t, s, f, now)
|
||||||
|
at := now.Add(defaultRest + time.Minute)
|
||||||
|
p.Now = func() time.Time { return at }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := readLatestNum(t, s, s.OwnerID(), demonicKey); got != 296 {
|
||||||
|
t.Fatalf("latest after the Poll = %v, want the Site's own 296", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
var logs strings.Builder
|
||||||
|
prev := log.Writer()
|
||||||
|
log.SetOutput(&logs)
|
||||||
|
t.Cleanup(func() { log.SetOutput(prev) })
|
||||||
|
|
||||||
|
f.body = demonicFixture(290) // the Site retracts what only the Poll wrote
|
||||||
|
p.Now = func() time.Time { return at.Add(defaultRest + time.Minute) }
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if strings.Contains(logs.String(), "sighting contradicted") {
|
||||||
|
t.Fatalf("a retraction of the Poll's own value was charged to a Reader: %s", logs.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A PUT with no Latest Chapter in it — a favourite toggle, progress written
|
||||||
|
// from a chapter page — is nobody looking at the Series page, so it buys no
|
||||||
|
// deferral. Otherwise a client could suppress a Series' Polls while reporting
|
||||||
|
// nothing, and with nothing reported there would be nothing to judge.
|
||||||
|
func TestPutWithoutALatestChapterDoesNotDefer(t *testing.T) {
|
||||||
|
s, _ := newTestStore(t)
|
||||||
|
now := time.UnixMilli(20 * time.Hour.Milliseconds())
|
||||||
|
seedDemonic(t, s, now.Add(-2*time.Hour).UnixMilli())
|
||||||
|
// The handler's own call, with the field the client omitted.
|
||||||
|
if err := s.RecordSighting(s.OwnerID(), "demonic", "Catastrophic-Necromancer",
|
||||||
|
nil, now.Add(-time.Minute).UnixMilli()); err != nil {
|
||||||
|
t.Fatalf("RecordSighting: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
f := &fakeFetcher{body: demonicFixture(296), status: 200}
|
||||||
|
p := newTestPoller(t, s, f, now)
|
||||||
|
p.runOnce(context.Background())
|
||||||
|
if got := f.callCount(); got != 1 {
|
||||||
|
t.Fatalf("fetched %d times after a PUT carrying no chapter, want 1", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -409,6 +409,14 @@ const (
|
|||||||
// asleep (ADR-0005 on-demand browser).
|
// asleep (ADR-0005 on-demand browser).
|
||||||
browserWakeCount = 5
|
browserWakeCount = 5
|
||||||
browserWakeAge = 15 * time.Minute
|
browserWakeAge = 15 * time.Minute
|
||||||
|
// sightingCeilingRests caps Sighting deferral (issue #103): however many
|
||||||
|
// Sightings arrive, a Series unpolled for this many of its Site's rests is
|
||||||
|
// Polled. It is what makes a client report safe to trust — a wrong Latest
|
||||||
|
// Chapter dies within the ceiling deterministically, rather than in
|
||||||
|
// expectation the way a randomised audit would have it. Six, so a Series a
|
||||||
|
// Reader visits constantly still gets one authoritative check per working
|
||||||
|
// day-part.
|
||||||
|
sightingCeilingRests = 6
|
||||||
)
|
)
|
||||||
|
|
||||||
// effectiveGap is a Site's pace: the registry gap, or one rest divided by the
|
// effectiveGap is a Site's pace: the registry gap, or one rest divided by the
|
||||||
|
|||||||
@@ -0,0 +1,10 @@
|
|||||||
|
-- Sighting deferral (issue #103). latest_sighted_at is when a Reader's report
|
||||||
|
-- last stood in for a Poll; it is separate from latest_checked_at because the
|
||||||
|
-- six-hour ceiling has to know when the Series was last really fetched, and a
|
||||||
|
-- Sighting writing the Poll's own column would erase that.
|
||||||
|
-- latest_raised_by is attribution: whoever last raised this Series' Latest
|
||||||
|
-- Chapter by Sighting, so a Poll that contradicts the value downwards names a
|
||||||
|
-- Reader rather than flagging a row. Cleared by the Poll that judges it, NULL
|
||||||
|
-- whenever the stored value is the Poll's own.
|
||||||
|
ALTER TABLE series ADD COLUMN latest_sighted_at bigint NOT NULL DEFAULT 0;
|
||||||
|
ALTER TABLE series ADD COLUMN latest_raised_by bigint REFERENCES readers(id) ON DELETE SET NULL;
|
||||||
@@ -79,6 +79,11 @@ type Series struct {
|
|||||||
LatestChapter string
|
LatestChapter string
|
||||||
LatestChapterNum *float64 // nil until first captured
|
LatestChapterNum *float64 // nil until first captured
|
||||||
LatestCheckedAt int64 // unix ms; see MarkLatestChecked
|
LatestCheckedAt int64 // unix ms; see MarkLatestChecked
|
||||||
|
// LatestRaisedBy is the Reader whose Sighting last raised LatestChapter,
|
||||||
|
// and nil when the stored value is a Poll's own finding. It is what lets a
|
||||||
|
// Poll that contradicts the value downwards name a Reader instead of
|
||||||
|
// merely flagging the row (issue #103); the Poll that judges it clears it.
|
||||||
|
LatestRaisedBy *int64
|
||||||
|
|
||||||
// readerCount is the number of bookmarks referencing this series, filled
|
// readerCount is the number of bookmarks referencing this series, filled
|
||||||
// only by the due-queue query that orders on it.
|
// only by the due-queue query that orders on it.
|
||||||
@@ -195,7 +200,7 @@ const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover_add
|
|||||||
// due query. latest_checked_at lives only on series — see MarkLatestChecked
|
// due query. latest_checked_at lives only on series — see MarkLatestChecked
|
||||||
// for why it stays off every client-visible write.
|
// for why it stays off every client-visible write.
|
||||||
const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover, s.cover_address,
|
const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover, s.cover_address,
|
||||||
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at`
|
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at, s.latest_raised_by`
|
||||||
|
|
||||||
// Owner is the person running the service: the first Reader, seeded at startup
|
// Owner is the person running the service: the first Reader, seeded at startup
|
||||||
// so a fresh deployment has a library before anyone logs in. The seed makes
|
// so a fresh deployment has a library before anyone logs in. The seed makes
|
||||||
@@ -584,16 +589,17 @@ func (s *Store) scanBookmark(scan func(...any) error) (Bookmark, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// scanSeries reads one row in seriesColumns order, plus the due query's
|
// scanSeries reads one row in seriesColumns order, plus the due query's
|
||||||
// reader_count column. latest_chapter_num is NULL until the first capture,
|
// reader_count column. latest_chapter_num and latest_raised_by are both
|
||||||
// same as on the bookmark read path.
|
// nullable, same as latest_chapter_num on the bookmark read path.
|
||||||
func scanSeries(scan func(...any) error) (Series, error) {
|
func scanSeries(scan func(...any) error) (Series, error) {
|
||||||
var (
|
var (
|
||||||
sr Series
|
sr Series
|
||||||
latestChapterNum sql.NullFloat64
|
latestChapterNum sql.NullFloat64
|
||||||
|
latestRaisedBy sql.NullInt64
|
||||||
)
|
)
|
||||||
if err := scan(
|
if err := scan(
|
||||||
&sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover, &sr.CoverAddress,
|
&sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover, &sr.CoverAddress,
|
||||||
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt,
|
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt, &latestRaisedBy,
|
||||||
&sr.readerCount,
|
&sr.readerCount,
|
||||||
); err != nil {
|
); err != nil {
|
||||||
return Series{}, err
|
return Series{}, err
|
||||||
@@ -601,6 +607,9 @@ func scanSeries(scan func(...any) error) (Series, error) {
|
|||||||
if latestChapterNum.Valid {
|
if latestChapterNum.Valid {
|
||||||
sr.LatestChapterNum = &latestChapterNum.Float64
|
sr.LatestChapterNum = &latestChapterNum.Float64
|
||||||
}
|
}
|
||||||
|
if latestRaisedBy.Valid {
|
||||||
|
sr.LatestRaisedBy = &latestRaisedBy.Int64
|
||||||
|
}
|
||||||
return sr, nil
|
return sr, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -947,7 +956,20 @@ func (s *Store) Delete(readerID int64, key string) error {
|
|||||||
// burns requests. Archived bookmarks still count — knowing what a shelved
|
// burns requests. Archived bookmarks still count — knowing what a shelved
|
||||||
// series is up to is the whole reason for archiving instead of deleting.
|
// series is up to is the whole reason for archiving instead of deleting.
|
||||||
// A series with no bookmarks at all never appears: the join excludes it.
|
// A series with no bookmarks at all never appears: the join excludes it.
|
||||||
func (s *Store) DueForLatestCheck(site string, cutoffMs int64) ([]Series, error) {
|
//
|
||||||
|
// ceilingMs is the Sighting deferral ceiling (issue #103): a Series whose last
|
||||||
|
// real Poll is older than it appears however recently it was sighted. That is
|
||||||
|
// what bounds the whole mechanism — a wrong Latest Chapter dies within the
|
||||||
|
// ceiling deterministically rather than in expectation. Deferral itself is
|
||||||
|
// decided here, from two facts the query already computes, so a Lane gains no
|
||||||
|
// query per round: a Sighting younger than cutoffMs holds the Series back, but
|
||||||
|
// only while COUNT(*) is 1. A Series a second Reader bookmarks is Polled on
|
||||||
|
// schedule, so a wrong value the whole guild can see is corrected by a check
|
||||||
|
// that was never postponed; on a solitary Series the only person a wrong value
|
||||||
|
// reaches is the Reader who reported it. Whether the reporting Reader is
|
||||||
|
// allowed to defer at all was settled when the Sighting was recorded — see
|
||||||
|
// RecordSighting.
|
||||||
|
func (s *Store) DueForLatestCheck(site string, cutoffMs, ceilingMs int64) ([]Series, error) {
|
||||||
rows, err := s.db.Query(`SELECT `+seriesColumns+`, COUNT(*) AS reader_count
|
rows, err := s.db.Query(`SELECT `+seriesColumns+`, COUNT(*) AS reader_count
|
||||||
FROM series s
|
FROM series s
|
||||||
JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id
|
JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id
|
||||||
@@ -957,7 +979,10 @@ func (s *Store) DueForLatestCheck(site string, cutoffMs int64) ([]Series, error)
|
|||||||
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover,
|
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover,
|
||||||
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at
|
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at
|
||||||
HAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
|
HAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
|
||||||
ORDER BY reader_count DESC, s.latest_checked_at ASC`, site, cutoffMs)
|
AND (COUNT(*) > 1
|
||||||
|
OR s.latest_sighted_at <= $2::bigint
|
||||||
|
OR s.latest_checked_at <= $3::bigint)
|
||||||
|
ORDER BY reader_count DESC, s.latest_checked_at ASC`, site, cutoffMs, ceilingMs)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("query due series: %w", err)
|
return nil, fmt.Errorf("query due series: %w", err)
|
||||||
}
|
}
|
||||||
@@ -1044,3 +1069,111 @@ func (s *Store) SetLatestChapter(site, seriesID, label string, num float64) erro
|
|||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RecordSighting notes that a Reader's browser reported this Series' Latest
|
||||||
|
// Chapter, which is the half of a Sighting the client body cannot express
|
||||||
|
// (issue #103). It must be called *before* the Upsert that stores the reported
|
||||||
|
// value: the raise test compares against what is still on the row, and after
|
||||||
|
// the Upsert there is nothing left to compare with. A Series that does not
|
||||||
|
// exist yet — the first Bookmark of it — is not a Sighting at all: nothing has
|
||||||
|
// ever been Polled, so there is nothing to defer and nobody to attribute.
|
||||||
|
//
|
||||||
|
// Two independent effects, hence the two CASE arms. The deferral stamp is only
|
||||||
|
// written for a Reader below the disagreement limit, so a marked Reader's
|
||||||
|
// reports keep updating the Latest Chapter but stop postponing anything, and
|
||||||
|
// clearing their marks restores the privilege on their next Sighting. The
|
||||||
|
// attribution is written whenever the report raises the stored number,
|
||||||
|
// including for a marked Reader — their Sightings are still judged, which is
|
||||||
|
// how they earn the privilege back.
|
||||||
|
//
|
||||||
|
// num is the reported chapter number. A PUT that carries none — a favourite
|
||||||
|
// toggle, or progress written from a chapter page — is no Sighting at all:
|
||||||
|
// nobody read the Series page, so there is nothing to stand in for a Poll and
|
||||||
|
// nothing that could later be judged.
|
||||||
|
func (s *Store) RecordSighting(readerID int64, site, seriesID string, num *float64, ts int64) error {
|
||||||
|
if num == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if _, err := s.db.Exec(`
|
||||||
|
UPDATE series SET
|
||||||
|
latest_sighted_at = CASE
|
||||||
|
WHEN (SELECT sighting_disagreements FROM readers WHERE id = $3) < $6
|
||||||
|
THEN $4::bigint ELSE latest_sighted_at END,
|
||||||
|
latest_raised_by = CASE
|
||||||
|
WHEN latest_chapter_num IS NULL OR $5::double precision > latest_chapter_num
|
||||||
|
THEN $3::bigint ELSE latest_raised_by END
|
||||||
|
WHERE site = $1 AND series_id = $2`,
|
||||||
|
site, seriesID, readerID, ts, *num, SightingDisagreementLimit); err != nil {
|
||||||
|
return fmt.Errorf("record sighting %s:%s: %w", site, seriesID, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SightingAgreementsToClear is how many Polls must confirm a Reader's
|
||||||
|
// Sightings in a row before their disagreements are forgiven. An agreement is
|
||||||
|
// only recorded when a Poll later confirms a Sighting, so this is twenty Polls
|
||||||
|
// of Series that Reader bookmarks — hours to days, not twenty page views. That
|
||||||
|
// is the intended price: recovery is automatic but cannot be outwaited, and a
|
||||||
|
// disagreement resets the run to zero, so credit cannot be banked in advance.
|
||||||
|
const SightingAgreementsToClear = 20
|
||||||
|
|
||||||
|
// RecordSightingOutcome settles what a Poll decided about the Reader whose
|
||||||
|
// Sighting last raised this Series' Latest Chapter, and clears the attribution
|
||||||
|
// in the same transaction so one Sighting is judged exactly once. agreed is
|
||||||
|
// the Poll confirming the stored value; its opposite is the Poll finding a
|
||||||
|
// lower number, which means the raise was false.
|
||||||
|
//
|
||||||
|
// A Poll finding a *higher* number is neither — the Site published — and takes
|
||||||
|
// ClearSightingAttribution instead.
|
||||||
|
func (s *Store) RecordSightingOutcome(site, seriesID string, readerID int64, agreed bool) error {
|
||||||
|
tx, err := s.db.Begin()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("begin sighting outcome %s:%s: %w", site, seriesID, err)
|
||||||
|
}
|
||||||
|
defer tx.Rollback()
|
||||||
|
|
||||||
|
// The run length is what "consecutive" means: a disagreement zeroes the
|
||||||
|
// agreements, and completing a run zeroes both, so the next run starts
|
||||||
|
// from nothing rather than forgiving every later disagreement instantly.
|
||||||
|
q := `UPDATE readers SET sighting_disagreements = sighting_disagreements + 1,
|
||||||
|
sighting_agreements = 0
|
||||||
|
WHERE id = $1`
|
||||||
|
args := []any{readerID}
|
||||||
|
if agreed {
|
||||||
|
q = `UPDATE readers SET
|
||||||
|
sighting_agreements = CASE WHEN sighting_agreements + 1 >= $2 THEN 0
|
||||||
|
ELSE sighting_agreements + 1 END,
|
||||||
|
sighting_disagreements = CASE WHEN sighting_agreements + 1 >= $2 THEN 0
|
||||||
|
ELSE sighting_disagreements END
|
||||||
|
WHERE id = $1`
|
||||||
|
args = append(args, SightingAgreementsToClear)
|
||||||
|
}
|
||||||
|
if _, err := tx.Exec(q, args...); err != nil {
|
||||||
|
return fmt.Errorf("record sighting outcome for reader %d: %w", readerID, err)
|
||||||
|
}
|
||||||
|
if _, err := tx.Exec(clearAttributionSQL, site, seriesID, readerID); err != nil {
|
||||||
|
return fmt.Errorf("clear sighting attribution %s:%s: %w", site, seriesID, err)
|
||||||
|
}
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
return fmt.Errorf("commit sighting outcome %s:%s: %w", site, seriesID, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ClearSightingAttribution answers a Sighting without judging it: the Poll
|
||||||
|
// found a higher number, so the value about to be stored is its own and this
|
||||||
|
// Reader is no longer answerable for the row. Without it the next Poll's
|
||||||
|
// agreement would be credited to a Reader who did not earn it.
|
||||||
|
func (s *Store) ClearSightingAttribution(site, seriesID string, readerID int64) error {
|
||||||
|
if _, err := s.db.Exec(clearAttributionSQL, site, seriesID, readerID); err != nil {
|
||||||
|
return fmt.Errorf("clear sighting attribution %s:%s: %w", site, seriesID, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// clearAttributionSQL drops the attribution only while it still names the
|
||||||
|
// Reader being judged: a Sighting landing between the due query's snapshot and
|
||||||
|
// this write is a fresh, unjudged one and must not be erased by the previous
|
||||||
|
// one's verdict.
|
||||||
|
const clearAttributionSQL = `UPDATE series SET latest_raised_by = NULL
|
||||||
|
WHERE site = $1 AND series_id = $2 AND latest_raised_by = $3`
|
||||||
|
|||||||
@@ -277,6 +277,10 @@ func seedForCheck(t *testing.T, s *Store, key, seriesURL string, checkedAt int64
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// noCeiling is a Sighting deferral ceiling no Series can reach, for the tests
|
||||||
|
// that predate the ceiling and are about rest, ordering or buckets instead.
|
||||||
|
const noCeiling = int64(-1)
|
||||||
|
|
||||||
func TestDueForLatestCheck(t *testing.T) {
|
func TestDueForLatestCheck(t *testing.T) {
|
||||||
const hour = int64(3600_000)
|
const hour = int64(3600_000)
|
||||||
now := 10 * hour
|
now := 10 * hour
|
||||||
@@ -298,7 +302,7 @@ func TestDueForLatestCheck(t *testing.T) {
|
|||||||
s := newTestStore(t)
|
s := newTestStore(t)
|
||||||
seedForCheck(t, s, "asura:x", tt.seriesURL, tt.checkedAt)
|
seedForCheck(t, s, "asura:x", tt.seriesURL, tt.checkedAt)
|
||||||
|
|
||||||
due, err := s.DueForLatestCheck("asura", now-hour)
|
due, err := s.DueForLatestCheck("asura", now-hour, noCeiling)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("DueForLatestCheck: %v", err)
|
t.Fatalf("DueForLatestCheck: %v", err)
|
||||||
}
|
}
|
||||||
@@ -319,7 +323,7 @@ func TestDueForLatestCheckOldestFirstAndScopedToSite(t *testing.T) {
|
|||||||
// asks for one Site, and no Lane may see another's queue.
|
// asks for one Site, and no Lane may see another's queue.
|
||||||
seedForCheck(t, s, "demonic:z", "https://demonicscans.org/manga/z", 0)
|
seedForCheck(t, s, "demonic:z", "https://demonicscans.org/manga/z", 0)
|
||||||
|
|
||||||
due, err := s.DueForLatestCheck("asura", 1000)
|
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("DueForLatestCheck: %v", err)
|
t.Fatalf("DueForLatestCheck: %v", err)
|
||||||
}
|
}
|
||||||
@@ -499,7 +503,7 @@ func TestDueForLatestCheckSkipsFinishedKeepsArchived(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
due, err := store.DueForLatestCheck("asura", time.Now().UnixMilli())
|
due, err := store.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("DueForLatestCheck: %v", err)
|
t.Fatalf("DueForLatestCheck: %v", err)
|
||||||
}
|
}
|
||||||
@@ -1005,7 +1009,7 @@ func TestDueForLatestCheckOrdersByReaderCountThenAge(t *testing.T) {
|
|||||||
seedSecondReader(t, s, "asura:pop:2", "asura", "pop", 1001)
|
seedSecondReader(t, s, "asura:pop:2", "asura", "pop", 1001)
|
||||||
seedForCheck(t, s, "asura:solo", "https://asurascans.com/comics/solo", 100)
|
seedForCheck(t, s, "asura:solo", "https://asurascans.com/comics/solo", 100)
|
||||||
|
|
||||||
due, err := s.DueForLatestCheck("asura", 1000)
|
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("DueForLatestCheck: %v", err)
|
t.Fatalf("DueForLatestCheck: %v", err)
|
||||||
}
|
}
|
||||||
@@ -1031,7 +1035,7 @@ func TestDueForLatestCheckExcludesOrphanSeries(t *testing.T) {
|
|||||||
t.Fatalf("seed orphan series: %v", err)
|
t.Fatalf("seed orphan series: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
due, err := s.DueForLatestCheck("asura", 1000)
|
due, err := s.DueForLatestCheck("asura", 1000, noCeiling)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("DueForLatestCheck: %v", err)
|
t.Fatalf("DueForLatestCheck: %v", err)
|
||||||
}
|
}
|
||||||
@@ -1449,7 +1453,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
|
|||||||
t.Fatalf("series rows = %d, want 1 shared row for two bookmarks", series)
|
t.Fatalf("series rows = %d, want 1 shared row for two bookmarks", series)
|
||||||
}
|
}
|
||||||
|
|
||||||
due, err := s.DueForLatestCheck("asura", time.Now().UnixMilli())
|
due, err := s.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("DueForLatestCheck: %v", err)
|
t.Fatalf("DueForLatestCheck: %v", err)
|
||||||
}
|
}
|
||||||
@@ -1465,7 +1469,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
|
|||||||
if b, ok, err := s.Get(s.OwnerID(), "asura:solo"); err != nil || !ok || b.LastChapterNum != 200 {
|
if b, ok, err := s.Get(s.OwnerID(), "asura:solo"); err != nil || !ok || b.LastChapterNum != 200 {
|
||||||
t.Fatalf("owner's bookmark after the other's delete = %+v ok=%v err=%v, want it intact", b, ok, err)
|
t.Fatalf("owner's bookmark after the other's delete = %+v ok=%v err=%v, want it intact", b, ok, err)
|
||||||
}
|
}
|
||||||
due, err = s.DueForLatestCheck("asura", time.Now().UnixMilli())
|
due, err = s.DueForLatestCheck("asura", time.Now().UnixMilli(), noCeiling)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("DueForLatestCheck after delete: %v", err)
|
t.Fatalf("DueForLatestCheck after delete: %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,139 @@
|
|||||||
|
# ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them
|
||||||
|
|
||||||
|
Date: 2026-08-16
|
||||||
|
Status: accepted
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
A **Sighting** is the Latest Chapter the Reader's own browser read off the
|
||||||
|
Series page and PUT to the backend. It is now allowed to stand in for a Poll,
|
||||||
|
under one restriction and one ceiling:
|
||||||
|
|
||||||
|
- **Solitary Series only.** A Sighting defers the Poll of a Series exactly one
|
||||||
|
Bookmark points at. A Series two Readers share is Polled on schedule no matter
|
||||||
|
how recently it was sighted.
|
||||||
|
- **One rest of standing.** A Sighting postpones Polls for one Rest
|
||||||
|
(`defaultRest`, an hour), not forever: a Series nobody visits again returns to
|
||||||
|
the normal schedule by itself.
|
||||||
|
- **Six-rest ceiling.** `sightingCeilingRests = 6`, counted in the Site's own
|
||||||
|
Rest — six hours everywhere today. However many Sightings arrive, a Series
|
||||||
|
unpolled that long is Polled.
|
||||||
|
|
||||||
|
Both live in the due query's HAVING clause (`store.DueForLatestCheck`), beside
|
||||||
|
the Rest cutoff — the same place the schedule has always been decided, so no
|
||||||
|
timer and no second code path can disagree with it.
|
||||||
|
|
||||||
|
Attribution and judgement:
|
||||||
|
|
||||||
|
- `Store.RecordSighting` runs *before* the Upsert that stores the reported
|
||||||
|
value, because the raise test needs the row as it stands. A report that raises
|
||||||
|
the stored Latest Chapter names its Reader in `series.latest_raised_by`.
|
||||||
|
- The Poll is the oracle. `Poller.checkOne` already compares what the Site
|
||||||
|
publishes against what is stored, so judgement costs no extra request: a lower
|
||||||
|
number contradicts the Sighting (`sighting_disagreements + 1`, both numbers and
|
||||||
|
the Reader logged), the same number confirms it (`sighting_agreements + 1`), a
|
||||||
|
**higher** number is the Site publishing and means nothing either way — but it
|
||||||
|
does clear the attribution (`Store.ClearSightingAttribution`), because the
|
||||||
|
value stored afterwards is the Poll's own and nobody must answer for it.
|
||||||
|
- At `SightingDisagreementLimit` (3) that Reader's Sightings stop deferring
|
||||||
|
anything. They still write the Latest Chapter — the penalty removes a
|
||||||
|
privilege, it does not silence anyone.
|
||||||
|
- `SightingAgreementsToClear` (20) consecutive confirmations forgive the
|
||||||
|
disagreements. A disagreement resets the run to zero.
|
||||||
|
- The owner clears marks from the administration page (issue #102, shipped
|
||||||
|
first precisely so a false mark has a remedy the day the mechanism lands).
|
||||||
|
|
||||||
|
One client change was required, and only one. Both userscripts stopped short of
|
||||||
|
PUTting a read whose number had not moved (`applyLatestChapterIfChanged`), so
|
||||||
|
the case this whole mechanism exists for — visiting a Series with nothing new —
|
||||||
|
never reached the backend. `reportLatestChapter` now sends it, skipping only the
|
||||||
|
local write and the re-render. A numberless PUT (favourite toggle, progress from
|
||||||
|
a chapter page) is not a Sighting and defers nothing: nobody read the Series
|
||||||
|
page, so there would be nothing to judge later.
|
||||||
|
|
||||||
|
## Why
|
||||||
|
|
||||||
|
Most of the backend's work was redundant. The userscript reads the Latest
|
||||||
|
Chapter on every Series page visit; minutes later the Poll Lane fetches the same
|
||||||
|
page for the same number. Deferring on a report converts a visit into a Poll
|
||||||
|
saved, which is Lane capacity handed back to Series nobody is
|
||||||
|
reading.
|
||||||
|
|
||||||
|
The restriction is the whole safety argument, and it is about **blast radius**,
|
||||||
|
not about trust arithmetic:
|
||||||
|
|
||||||
|
- On a solitary Series, a wrong report can only mislead the Reader who made it.
|
||||||
|
There is nobody else's ember to falsify.
|
||||||
|
- On a shared Series it could mislead someone else, so a report never postpones
|
||||||
|
anything there.
|
||||||
|
|
||||||
|
The ceiling bounds the damage in time: a false value dies within six hours
|
||||||
|
whatever happens, because the Poll that finds it is guaranteed. That is also
|
||||||
|
what makes lying pointless — the six-hour audit is certain, not sampled, so a
|
||||||
|
determined attacker buys at most three ceilings' worth of a wrong number on
|
||||||
|
their own Series and then loses deferral entirely.
|
||||||
|
|
||||||
|
The cost of recovery is deliberate. An agreement is only recorded when a later
|
||||||
|
Poll confirms a Sighting, so twenty agreements are twenty Polls of Series that
|
||||||
|
Reader bookmarks — hours to days of real time, not twenty page views. Waiting is
|
||||||
|
therefore not a strategy, and credit cannot be banked in advance.
|
||||||
|
|
||||||
|
## Tradeoffs and rejections
|
||||||
|
|
||||||
|
- **Trusting a Sighting on a shared Series** rejected: it is the only case where
|
||||||
|
one Reader's mistake reaches another Reader's list, and no amount of
|
||||||
|
reputation makes that recoverable within the six-hour window.
|
||||||
|
- **Cross-Reader agreement, voting, weighting, consensus scoring** rejected on
|
||||||
|
evidence: every truth-discovery method estimates source reliability by
|
||||||
|
comparing sources on the same object, and the standard survey states outright
|
||||||
|
that an object provided by very few sources cannot have its confidence
|
||||||
|
evaluated — Li, Gao, Meng, Li, Su, Zhao, Fan, Han, *A Survey on Truth
|
||||||
|
Discovery*, SIGMOD Record 45(1), 2016 (arXiv:1505.02463), §"Challenges" on
|
||||||
|
sparse sources. With the two Readers this backend actually has, a
|
||||||
|
disagreement is a coin flip. The Poll is an authoritative oracle, so it is
|
||||||
|
the only judge.
|
||||||
|
- **A randomised audit** (Poll a fraction of deferred Series) rejected in favour
|
||||||
|
of the fixed ceiling. Sampling an oracle against untrusted reports is the
|
||||||
|
gold-question technique from crowdsourcing quality control — Le, Edmonds,
|
||||||
|
Hester, Biewald, *Ensuring quality in crowdsourced search relevance
|
||||||
|
evaluation: the effects of training question distribution*, SIGIR 2010
|
||||||
|
Workshop on Crowdsourcing for Search Evaluation, which inserts known answers
|
||||||
|
sporadically and adjusts each worker's trust from them. The ceiling is the
|
||||||
|
same idea made deterministic: sampling prices an attack in expectation, a
|
||||||
|
guaranteed six-hour audit prices it as a certainty, which is what makes the
|
||||||
|
solitary-Series rule defensible in one sentence.
|
||||||
|
- **A trust *ratio*** (agreements over judgements, as that same gold-question
|
||||||
|
scheme uses) rejected for two thresholds: a ratio lets an attacker bank
|
||||||
|
credit first and spend it on lies later, and it needs the owner watching a
|
||||||
|
score to act. Three-and-twenty is a threshold both ways — a disagreement
|
||||||
|
resets the run to zero, so credit cannot be pre-bought, and recovery happens
|
||||||
|
without the owner in the loop.
|
||||||
|
- **Blocking a marked Reader's writes** rejected: the Latest Chapter they report
|
||||||
|
is still the best available value, and their Sightings must keep being judged
|
||||||
|
or they could never earn the privilege back.
|
||||||
|
- **Per-Series flagging** rejected in favour of per-Reader marks: a Series is
|
||||||
|
not the thing that can be wrong. Naming the Reader and logging both numbers is
|
||||||
|
also what distinguishes a broken Site adapter (every Reader of that Site
|
||||||
|
contradicted at once) from one bad actor.
|
||||||
|
- **Timers or a background reputation job** rejected: deferral is recomputed
|
||||||
|
from live facts every round — Bookmark count and sighting timestamp — so a
|
||||||
|
Series that gains a second Bookmark stops deferring at once, with nothing to
|
||||||
|
invalidate. The Reader's marks are the one input read earlier, when the
|
||||||
|
Sighting is recorded rather than when the round runs: a Reader who crosses
|
||||||
|
the threshold, or has their marks cleared, changes behaviour from their next
|
||||||
|
Sighting on, and the standing they already bought lasts out its rest. That is
|
||||||
|
bounded by one rest and costs one subselect instead of joining `readers` into
|
||||||
|
the due query on every round.
|
||||||
|
|
||||||
|
## Constraints preserved
|
||||||
|
|
||||||
|
- A Sighting is not Progress: it may move the Latest Chapter and nothing else.
|
||||||
|
`updated_at` never moves, so a report cannot reorder the list (ADR-0004).
|
||||||
|
- The Latest Chapter is a Series-level fact (ADR-0003): a Sighting writes the
|
||||||
|
shared row, so every Reader of a shared Series sees it immediately — deferral
|
||||||
|
is the only thing the solitary rule withholds.
|
||||||
|
- Ember means new chapter only (`docs/design-system.md`): a marked Reader
|
||||||
|
renders no differently in their own list, and nothing about the trust model
|
||||||
|
reaches the Series list's colour.
|
||||||
|
- The Poll remains authoritative. Where a Sighting and a Poll disagree, the
|
||||||
|
Poll's value is what gets stored.
|
||||||
@@ -40,6 +40,7 @@
|
|||||||
"38": "novel-logic.test.js",
|
"38": "novel-logic.test.js",
|
||||||
"39": "UI Critique 2026-07-26A",
|
"39": "UI Critique 2026-07-26A",
|
||||||
"40": "UI Critique 2026-07-26B",
|
"40": "UI Critique 2026-07-26B",
|
||||||
|
"41": "Handler",
|
||||||
"42": "pgtest.go",
|
"42": "pgtest.go",
|
||||||
"43": "Issue Tracker & Triage",
|
"43": "Issue Tracker & Triage",
|
||||||
"44": "Ticket Workflow",
|
"44": "Ticket Workflow",
|
||||||
@@ -151,6 +152,7 @@
|
|||||||
"150": "Reviewer Subagent (opencode)",
|
"150": "Reviewer Subagent (opencode)",
|
||||||
"151": "Finding Severity Rubric",
|
"151": "Finding Severity Rubric",
|
||||||
"152": "Spec Compliance Review",
|
"152": "Spec Compliance Review",
|
||||||
|
"154": "ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them",
|
||||||
"156": "ResponseWriter",
|
"156": "ResponseWriter",
|
||||||
"161": "Why Use samber/oops",
|
"161": "Why Use samber/oops",
|
||||||
"162": "singleflight Cache Stampede Prevention",
|
"162": "singleflight Cache Stampede Prevention",
|
||||||
|
|||||||
@@ -1,16 +1,16 @@
|
|||||||
# Graph Report - mangaBookmark (2026-08-16)
|
# Graph Report - mangaBookmark (2026-08-16)
|
||||||
|
|
||||||
## Corpus Check
|
## Corpus Check
|
||||||
- 116 files · ~284,419 words
|
- 119 files · ~290,953 words
|
||||||
- Verdict: corpus is large enough that graph structure adds value.
|
- Verdict: corpus is large enough that graph structure adds value.
|
||||||
|
|
||||||
## Summary
|
## Summary
|
||||||
- 1701 nodes · 3463 edges · 219 communities (69 shown, 150 thin omitted)
|
- 1736 nodes · 3597 edges · 221 communities (70 shown, 151 thin omitted)
|
||||||
- Extraction: 91% EXTRACTED · 9% INFERRED · 0% AMBIGUOUS · INFERRED: 326 edges (avg confidence: 0.77)
|
- Extraction: 90% EXTRACTED · 10% INFERRED · 0% AMBIGUOUS · INFERRED: 361 edges (avg confidence: 0.78)
|
||||||
- Token cost: 0 input · 0 output
|
- Token cost: 0 input · 0 output
|
||||||
|
|
||||||
## Graph Freshness
|
## Graph Freshness
|
||||||
- Built from commit: `58014eb8`
|
- Built from commit: `56afb9f2`
|
||||||
- Run `git rev-parse HEAD` and compare to check if the graph is stale.
|
- Run `git rev-parse HEAD` and compare to check if the graph is stale.
|
||||||
- Run `graphify update .` after code changes (no API cost).
|
- Run `graphify update .` after code changes (no API cost).
|
||||||
|
|
||||||
@@ -55,6 +55,7 @@
|
|||||||
- [[_COMMUNITY_novel-logic.test.js|novel-logic.test.js]]
|
- [[_COMMUNITY_novel-logic.test.js|novel-logic.test.js]]
|
||||||
- [[_COMMUNITY_UI Critique 2026-07-26A|UI Critique 2026-07-26A]]
|
- [[_COMMUNITY_UI Critique 2026-07-26A|UI Critique 2026-07-26A]]
|
||||||
- [[_COMMUNITY_UI Critique 2026-07-26B|UI Critique 2026-07-26B]]
|
- [[_COMMUNITY_UI Critique 2026-07-26B|UI Critique 2026-07-26B]]
|
||||||
|
- [[_COMMUNITY_Handler|Handler]]
|
||||||
- [[_COMMUNITY_pgtest.go|pgtest.go]]
|
- [[_COMMUNITY_pgtest.go|pgtest.go]]
|
||||||
- [[_COMMUNITY_Issue Tracker & Triage|Issue Tracker & Triage]]
|
- [[_COMMUNITY_Issue Tracker & Triage|Issue Tracker & Triage]]
|
||||||
- [[_COMMUNITY_Ticket Workflow|Ticket Workflow]]
|
- [[_COMMUNITY_Ticket Workflow|Ticket Workflow]]
|
||||||
@@ -166,6 +167,7 @@
|
|||||||
- [[_COMMUNITY_Reviewer Subagent (opencode)|Reviewer Subagent (opencode)]]
|
- [[_COMMUNITY_Reviewer Subagent (opencode)|Reviewer Subagent (opencode)]]
|
||||||
- [[_COMMUNITY_Finding Severity Rubric|Finding Severity Rubric]]
|
- [[_COMMUNITY_Finding Severity Rubric|Finding Severity Rubric]]
|
||||||
- [[_COMMUNITY_Spec Compliance Review|Spec Compliance Review]]
|
- [[_COMMUNITY_Spec Compliance Review|Spec Compliance Review]]
|
||||||
|
- [[_COMMUNITY_ADR-0011 Sightings — a Reader report defers a Poll where being wrong hurts only them|ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them]]
|
||||||
- [[_COMMUNITY_ResponseWriter|ResponseWriter]]
|
- [[_COMMUNITY_ResponseWriter|ResponseWriter]]
|
||||||
- [[_COMMUNITY_Why Use samberoops|Why Use samber/oops]]
|
- [[_COMMUNITY_Why Use samberoops|Why Use samber/oops]]
|
||||||
- [[_COMMUNITY_singleflight Cache Stampede Prevention|singleflight Cache Stampede Prevention]]
|
- [[_COMMUNITY_singleflight Cache Stampede Prevention|singleflight Cache Stampede Prevention]]
|
||||||
@@ -235,16 +237,16 @@
|
|||||||
- [[_COMMUNITY_Userscript CLAUDE guidance|Userscript CLAUDE guidance]]
|
- [[_COMMUNITY_Userscript CLAUDE guidance|Userscript CLAUDE guidance]]
|
||||||
|
|
||||||
## God Nodes (most connected - your core abstractions)
|
## God Nodes (most connected - your core abstractions)
|
||||||
1. `testConfig()` - 54 edges
|
1. `newTestStore()` - 62 edges
|
||||||
2. `newTestStore()` - 49 edges
|
2. `testConfig()` - 55 edges
|
||||||
3. `newWebTestServer()` - 49 edges
|
3. `newWebTestServer()` - 49 edges
|
||||||
4. `newTestStore()` - 44 edges
|
4. `newTestStore()` - 44 edges
|
||||||
5. `e()` - 33 edges
|
5. `e()` - 33 edges
|
||||||
6. `Open()` - 30 edges
|
6. `Store` - 31 edges
|
||||||
7. `Handler` - 29 edges
|
7. `Open()` - 31 edges
|
||||||
8. `Store` - 28 edges
|
8. `newTestPoller()` - 30 edges
|
||||||
9. `ne()` - 28 edges
|
9. `Handler` - 29 edges
|
||||||
10. `De()` - 28 edges
|
10. `ne()` - 28 edges
|
||||||
|
|
||||||
## Surprising Connections (you probably didn't know these)
|
## Surprising Connections (you probably didn't know these)
|
||||||
- `el()` --indirect_call--> `c()` [INFERRED]
|
- `el()` --indirect_call--> `c()` [INFERRED]
|
||||||
@@ -268,7 +270,7 @@
|
|||||||
- **Headless browser infrastructure (sidecar, on-demand, home deployment)** — docs_adr_0005_on_demand_browser_headless_shell, docs_adr_0005_on_demand_browser_cdp, docs_adr_0005_on_demand_browser_on_demand_start, docs_adr_0006_browser_on_the_home_machine_home_machine_rationale [INFERRED 0.85]
|
- **Headless browser infrastructure (sidecar, on-demand, home deployment)** — docs_adr_0005_on_demand_browser_headless_shell, docs_adr_0005_on_demand_browser_cdp, docs_adr_0005_on_demand_browser_on_demand_start, docs_adr_0006_browser_on_the_home_machine_home_machine_rationale [INFERRED 0.85]
|
||||||
- **lightnovelworld series-identity investigation and fix** — docs_research_lightnovelworld_chapter_vs_series_slug_issue_77, docs_research_lightnovelworld_chapter_vs_series_slug_unscoped_regex, docs_adr_0008_series_identity_is_discovered_not_derived_discovered_identity [INFERRED 0.85]
|
- **lightnovelworld series-identity investigation and fix** — docs_research_lightnovelworld_chapter_vs_series_slug_issue_77, docs_research_lightnovelworld_chapter_vs_series_slug_unscoped_regex, docs_adr_0008_series_identity_is_discovered_not_derived_discovered_identity [INFERRED 0.85]
|
||||||
|
|
||||||
## Communities (219 total, 150 thin omitted)
|
## Communities (221 total, 151 thin omitted)
|
||||||
|
|
||||||
### Community 0 - "HTMX Library Internals"
|
### Community 0 - "HTMX Library Internals"
|
||||||
Cohesion: 0.08
|
Cohesion: 0.08
|
||||||
@@ -280,23 +282,23 @@ Nodes (89): floatPtr(), testConfig(), getCover(), Cookie, Handler, ResponseRecor
|
|||||||
|
|
||||||
### Community 2 - "Manga Userscript Adapters"
|
### Community 2 - "Manga Userscript Adapters"
|
||||||
Cohesion: 0.06
|
Cohesion: 0.06
|
||||||
Nodes (76): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+68 more)
|
Nodes (77): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), armDwell() (+69 more)
|
||||||
|
|
||||||
### Community 3 - "Novel Userscript Adapters"
|
### Community 3 - "Novel Userscript Adapters"
|
||||||
Cohesion: 0.06
|
Cohesion: 0.06
|
||||||
Nodes (79): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLatestChapterIfChanged() (+71 more)
|
Nodes (78): adapterFor(), anchorsFromDocument(), anchorsFromHTML(), apiDelete(), apiGet(), apiPut(), applyFabPos(), applyLnwStaleRowRepair() (+70 more)
|
||||||
|
|
||||||
### Community 4 - "Series Acquisition Tests"
|
### Community 4 - "Series Acquisition Tests"
|
||||||
Cohesion: 0.08
|
Cohesion: 0.07
|
||||||
Nodes (78): bookmarkNewKaganeSeries(), bookmarkNewNovelfullSeries(), bookmarkNewSeries(), Context, Store, T, newAcquirer(), readBookmark() (+70 more)
|
Nodes (103): bookmarkNewKaganeSeries(), bookmarkNewNovelfullSeries(), bookmarkNewSeries(), Context, Store, T, newAcquirer(), readBookmark() (+95 more)
|
||||||
|
|
||||||
### Community 5 - "Bookmarks API Tests"
|
### Community 5 - "Bookmarks API Tests"
|
||||||
Cohesion: 0.07
|
Cohesion: 0.07
|
||||||
Nodes (70): auth(), getBookmarks(), Handler, Request, Store, T, newTestServer(), newTestStore() (+62 more)
|
Nodes (71): auth(), getBookmarks(), Handler, Request, Store, T, newTestServer(), newTestStore() (+63 more)
|
||||||
|
|
||||||
### Community 7 - "Cover & Acquire Internals"
|
### Community 7 - "Cover & Acquire Internals"
|
||||||
Cohesion: 0.07
|
Cohesion: 0.06
|
||||||
Nodes (43): Addr, fakeLanes, Context, Store, WaitGroup, defaultCoverResolver(), fetchCoverBytes(), Client (+35 more)
|
Nodes (43): Addr, fakeLanes, Context, Store, WaitGroup, isInterstitial(), defaultCoverResolver(), fetchCoverBytes() (+35 more)
|
||||||
|
|
||||||
### Community 8 - "System Architecture Concepts"
|
### Community 8 - "System Architecture Concepts"
|
||||||
Cohesion: 0.15
|
Cohesion: 0.15
|
||||||
@@ -318,10 +320,6 @@ Nodes (45): Store, T, newTestStore(), readLatestCheckedAt(), readSeries(), secon
|
|||||||
Cohesion: 0.08
|
Cohesion: 0.08
|
||||||
Nodes (32): Handler, Request, ResponseWriter, Store, Healthz(), writeJSON(), Auth(), compressible() (+24 more)
|
Nodes (32): Handler, Request, ResponseWriter, Store, Healthz(), writeJSON(), Auth(), compressible() (+24 more)
|
||||||
|
|
||||||
### Community 13 - "Web UI Handlers"
|
|
||||||
Cohesion: 0.14
|
|
||||||
Nodes (18): currentLib(), currentTab(), filterBookmarks(), Client, HandlerFunc, Request, ResponseWriter, Store (+10 more)
|
|
||||||
|
|
||||||
### Community 14 - "Go Error Handling"
|
### Community 14 - "Go Error Handling"
|
||||||
Cohesion: 0.06
|
Cohesion: 0.06
|
||||||
Nodes (33): Creating Errors, Custom Error Types, Custom types that wrap other errors, Decision table: which error strategy to use, Error Creation, Error String Conventions, Errors as Values, `errors.New` — static error messages (+25 more)
|
Nodes (33): Creating Errors, Custom Error Types, Custom types that wrap other errors, Decision table: which error strategy to use, Error Creation, Error String Conventions, Errors as Values, `errors.New` — static error messages (+25 more)
|
||||||
@@ -368,7 +366,7 @@ Nodes (12): coverResponse(), Request, T, TestCoverFetcherCanonicalisesJpgAlias()
|
|||||||
|
|
||||||
### Community 26 - "Open"
|
### Community 26 - "Open"
|
||||||
Cohesion: 0.05
|
Cohesion: 0.05
|
||||||
Nodes (62): awaitPromise(), browserConnectionLost(), classifyBrowserError(), comixRead(), comixSeriesPageURL(), Action, Context, Mutex (+54 more)
|
Nodes (61): awaitPromise(), browserConnectionLost(), classifyBrowserError(), comixRead(), comixSeriesPageURL(), Action, Context, Mutex (+53 more)
|
||||||
|
|
||||||
### Community 27 - "Find Skills Guide"
|
### Community 27 - "Find Skills Guide"
|
||||||
Cohesion: 0.14
|
Cohesion: 0.14
|
||||||
@@ -387,8 +385,8 @@ Cohesion: 0.40
|
|||||||
Nodes (5): Map of pointers for large, frequently updated structs, Memory Layout, Pointer receivers for large structs, Struct field alignment, Zero-size field at end of struct
|
Nodes (5): Map of pointers for large, frequently updated structs, Memory Layout, Pointer receivers for large structs, Struct field alignment, Zero-size field at end of struct
|
||||||
|
|
||||||
### Community 31 - "Store"
|
### Community 31 - "Store"
|
||||||
Cohesion: 0.08
|
Cohesion: 0.09
|
||||||
Nodes (8): coverRelativePath(), coverSourceAddress(), displayChapter(), Store, scanSeries(), TestDisplayChapter(), Bookmark, ReaderSummary
|
Nodes (5): coverRelativePath(), coverSourceAddress(), Store, scanSeries(), ReaderSummary
|
||||||
|
|
||||||
### Community 33 - "Go Testing Guide"
|
### Community 33 - "Go Testing Guide"
|
||||||
Cohesion: 0.20
|
Cohesion: 0.20
|
||||||
@@ -418,6 +416,10 @@ Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations,
|
|||||||
Cohesion: 0.29
|
Cohesion: 0.29
|
||||||
Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations, Persona Red Flags, Priority Issues, Questions to Consider
|
Nodes (6): Design Health Score, Design Specificity Verdict, Minor Observations, Persona Red Flags, Priority Issues, Questions to Consider
|
||||||
|
|
||||||
|
### Community 41 - "Handler"
|
||||||
|
Cohesion: 0.14
|
||||||
|
Nodes (18): currentLib(), currentTab(), filterBookmarks(), Client, HandlerFunc, Request, ResponseWriter, Store (+10 more)
|
||||||
|
|
||||||
### Community 42 - "pgtest.go"
|
### Community 42 - "pgtest.go"
|
||||||
Cohesion: 0.18
|
Cohesion: 0.18
|
||||||
Nodes (12): M, TestMain(), M, TestMain(), M, Main(), start(), URL() (+4 more)
|
Nodes (12): M, TestMain(), M, TestMain(), M, Main(), start(), URL() (+4 more)
|
||||||
@@ -534,6 +536,10 @@ Nodes (3): ADR-0005: On-demand browser sidecar, Constraints, Decision
|
|||||||
Cohesion: 0.48
|
Cohesion: 0.48
|
||||||
Nodes (6): T, TestCreateAndGetSession(), TestDeleteSessionIsPerReader(), TestDeleteSessionRevokes(), TestExpiredSessionIsGone(), TestGetSessionUnknownID()
|
Nodes (6): T, TestCreateAndGetSession(), TestDeleteSessionIsPerReader(), TestDeleteSessionRevokes(), TestExpiredSessionIsGone(), TestGetSessionUnknownID()
|
||||||
|
|
||||||
|
### Community 154 - "ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them"
|
||||||
|
Cohesion: 0.33
|
||||||
|
Nodes (5): ADR-0011: Sightings — a Reader report defers a Poll where being wrong hurts only them, Constraints preserved, Decision, Tradeoffs and rejections, Why
|
||||||
|
|
||||||
### Community 156 - "ResponseWriter"
|
### Community 156 - "ResponseWriter"
|
||||||
Cohesion: 0.18
|
Cohesion: 0.18
|
||||||
Nodes (13): AdminPatterns(), HandlerFunc, Request, ResponseWriter, Time, Handler, readerPathID(), since() (+5 more)
|
Nodes (13): AdminPatterns(), HandlerFunc, Request, ResponseWriter, Time, Handler, readerPathID(), since() (+5 more)
|
||||||
@@ -543,24 +549,24 @@ Cohesion: 0.50
|
|||||||
Nodes (3): Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust), Second script: `novel-bookmark.user.js`, Userscript structure (single IIFE, `manga-bookmark.user.js`)
|
Nodes (3): Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trust), Second script: `novel-bookmark.user.js`, Userscript structure (single IIFE, `manga-bookmark.user.js`)
|
||||||
|
|
||||||
## Knowledge Gaps
|
## Knowledge Gaps
|
||||||
- **520 isolated node(s):** `bookmarkmanager/backend`, `ctxKey`, `loginView`, `ctxKey`, `test` (+515 more)
|
- **524 isolated node(s):** `bookmarkmanager/backend`, `ctxKey`, `loginView`, `ctxKey`, `test` (+519 more)
|
||||||
These have ≤1 connection - possible missing edges or undocumented components.
|
These have ≤1 connection - possible missing edges or undocumented components.
|
||||||
- **150 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes.
|
- **151 thin communities (<3 nodes) omitted from report** — run `graphify query` to explore isolated nodes.
|
||||||
|
|
||||||
## Suggested Questions
|
## Suggested Questions
|
||||||
_Questions this graph is uniquely positioned to answer:_
|
_Questions this graph is uniquely positioned to answer:_
|
||||||
|
|
||||||
- **Why does `Open()` connect `CDP Browser Client` to `Cover Fetch Test Helpers`, `Series Acquisition Tests`, `Bookmarks API Tests`, `pgtest.go`, `Store Tests`, `Store`?**
|
- **Why does `Open()` connect `CDP Browser Client` to `Cover Fetch Test Helpers`, `Series Acquisition Tests`, `Bookmarks API Tests`, `pgtest.go`, `Store Tests`, `Store`?**
|
||||||
|
_High betweenness centrality (0.060) - this node is a cross-community bridge._
|
||||||
|
- **Why does `New()` connect `Series Acquisition Tests` to `Bookmarks API Tests`, `Cover & Acquire Internals`, `Handler`, `Session Middleware`, `CDP Browser Client`, `ResponseWriter`?**
|
||||||
_High betweenness centrality (0.050) - this node is a cross-community bridge._
|
_High betweenness centrality (0.050) - this node is a cross-community bridge._
|
||||||
- **Why does `New()` connect `Series Acquisition Tests` to `Bookmarks API Tests`, `Cover & Acquire Internals`, `Session Middleware`, `Web UI Handlers`, `CDP Browser Client`, `ResponseWriter`?**
|
|
||||||
_High betweenness centrality (0.048) - this node is a cross-community bridge._
|
|
||||||
- **Why does `newRouter()` connect `Bookmarks API Tests` to `Cover Fetch Test Helpers`, `Bookmarks API Handler`, `Series Acquisition Tests`, `ResponseWriter`?**
|
- **Why does `newRouter()` connect `Bookmarks API Tests` to `Cover Fetch Test Helpers`, `Bookmarks API Handler`, `Series Acquisition Tests`, `ResponseWriter`?**
|
||||||
_High betweenness centrality (0.031) - this node is a cross-community bridge._
|
_High betweenness centrality (0.030) - this node is a cross-community bridge._
|
||||||
|
- **Are the 25 inferred relationships involving `newTestStore()` (e.g. with `TestAcquireDoesNotBlockTheWrite()` and `TestAcquireFailureLeavesTheBookmarkIntact()`) actually correct?**
|
||||||
|
_`newTestStore()` has 25 INFERRED edges - model-reasoned connections that need verification._
|
||||||
- **Are the 48 inferred relationships involving `testConfig()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverNeverEchoesNonImage()`) actually correct?**
|
- **Are the 48 inferred relationships involving `testConfig()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverNeverEchoesNonImage()`) actually correct?**
|
||||||
_`testConfig()` has 48 INFERRED edges - model-reasoned connections that need verification._
|
_`testConfig()` has 48 INFERRED edges - model-reasoned connections that need verification._
|
||||||
- **Are the 12 inferred relationships involving `newTestStore()` (e.g. with `TestAcquireDoesNotBlockTheWrite()` and `TestAcquireFailureLeavesTheBookmarkIntact()`) actually correct?**
|
|
||||||
_`newTestStore()` has 12 INFERRED edges - model-reasoned connections that need verification._
|
|
||||||
- **Are the 8 inferred relationships involving `newWebTestServer()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverRejectsUnknownAddress()`) actually correct?**
|
- **Are the 8 inferred relationships involving `newWebTestServer()` (e.g. with `TestListRendersAcquiredCover()` and `TestPublicCoverRejectsUnknownAddress()`) actually correct?**
|
||||||
_`newWebTestServer()` has 8 INFERRED edges - model-reasoned connections that need verification._
|
_`newWebTestServer()` has 8 INFERRED edges - model-reasoned connections that need verification._
|
||||||
- **What connects `bookmarkmanager/backend`, `ctxKey`, `loginView` to the rest of the system?**
|
- **What connects `bookmarkmanager/backend`, `ctxKey`, `loginView` to the rest of the system?**
|
||||||
_552 weakly-connected nodes found - possible documentation gaps or missing edges._
|
_556 weakly-connected nodes found - possible documentation gaps or missing edges._
|
||||||
File diff suppressed because one or more lines are too long
+2654
-859
File diff suppressed because it is too large
Load Diff
+36
-21
@@ -35,8 +35,8 @@
|
|||||||
"semantic_hash": "4a94ac85bad6bce330d085bcc0ae3ffd"
|
"semantic_hash": "4a94ac85bad6bce330d085bcc0ae3ffd"
|
||||||
},
|
},
|
||||||
"userscript/manga-bookmark.user.js": {
|
"userscript/manga-bookmark.user.js": {
|
||||||
"mtime": 1786499529.779988,
|
"mtime": 1786872888.3482232,
|
||||||
"ast_hash": "1f8bcddd3632d709f058a8401af8f127",
|
"ast_hash": "4e79684942234b26f4a1fb03fccd6d35",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
".agents/skills/find-skills/SKILL.md": {
|
".agents/skills/find-skills/SKILL.md": {
|
||||||
@@ -225,8 +225,8 @@
|
|||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/api_test.go": {
|
"backend/api_test.go": {
|
||||||
"mtime": 1786863966.7056258,
|
"mtime": 1786868080.2317674,
|
||||||
"ast_hash": "0d443b4fe3c04ab057505d73b8a88d07",
|
"ast_hash": "b64372df5429fb14e48b37b3bab10402",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/cover_test.go": {
|
"backend/cover_test.go": {
|
||||||
@@ -235,9 +235,9 @@
|
|||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/internal/api/handlers.go": {
|
"backend/internal/api/handlers.go": {
|
||||||
"mtime": 1786363889.552731,
|
"mtime": 1786867745.788687,
|
||||||
"ast_hash": "59e6b8767ab19839bb8f82891a7e4616",
|
"ast_hash": "ae4f3b961450f06a06e0632eb273a5b2",
|
||||||
"semantic_hash": "59e6b8767ab19839bb8f82891a7e4616"
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/internal/httpmw/middleware.go": {
|
"backend/internal/httpmw/middleware.go": {
|
||||||
"mtime": 1786216141.672644,
|
"mtime": 1786216141.672644,
|
||||||
@@ -260,8 +260,8 @@
|
|||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/internal/latest/poller.go": {
|
"backend/internal/latest/poller.go": {
|
||||||
"mtime": 1786865046.096795,
|
"mtime": 1786872774.641225,
|
||||||
"ast_hash": "2bfb51177e3df474b80ea1b05523377e",
|
"ast_hash": "5e55e2ac5cb7d347807754f92425d29b",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/internal/latest/poller_test.go": {
|
"backend/internal/latest/poller_test.go": {
|
||||||
@@ -270,8 +270,8 @@
|
|||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/internal/latest/sites.go": {
|
"backend/internal/latest/sites.go": {
|
||||||
"mtime": 1786861012.026504,
|
"mtime": 1786868771.0663204,
|
||||||
"ast_hash": "9b90f9b710ccbd0d0a0a6a0dd721ff3c",
|
"ast_hash": "5786000dd0dfd5b4b3bd2b87b1fc100b",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/internal/latest/sites_test.go": {
|
"backend/internal/latest/sites_test.go": {
|
||||||
@@ -340,13 +340,13 @@
|
|||||||
"semantic_hash": "0b6764a0ee20f5cb7748eecd31a1d220"
|
"semantic_hash": "0b6764a0ee20f5cb7748eecd31a1d220"
|
||||||
},
|
},
|
||||||
"backend/internal/store/store.go": {
|
"backend/internal/store/store.go": {
|
||||||
"mtime": 1786863704.5112119,
|
"mtime": 1786868763.646761,
|
||||||
"ast_hash": "989629af18232e35b12f1eef2a8cb422",
|
"ast_hash": "34f55617b7409f03f02e925bec92fac0",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/internal/store/store_test.go": {
|
"backend/internal/store/store_test.go": {
|
||||||
"mtime": 1786863712.075631,
|
"mtime": 1786867781.301985,
|
||||||
"ast_hash": "e48f21e34238ba46874cc88cf96e022f",
|
"ast_hash": "16f69d97eb63bcec7409294466642411",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/internal/userscript/userscript.go": {
|
"backend/internal/userscript/userscript.go": {
|
||||||
@@ -395,8 +395,8 @@
|
|||||||
"semantic_hash": "8008a187690764436540fab47ba0cfcc"
|
"semantic_hash": "8008a187690764436540fab47ba0cfcc"
|
||||||
},
|
},
|
||||||
"userscript/novel-bookmark.user.js": {
|
"userscript/novel-bookmark.user.js": {
|
||||||
"mtime": 1786499529.779988,
|
"mtime": 1786872902.809149,
|
||||||
"ast_hash": "834effb0821f8d6c9f57f6554a5db462",
|
"ast_hash": "36c81462919719f0e053ca5d633a243a",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"userscript/test/novel-logic.test.js": {
|
"userscript/test/novel-logic.test.js": {
|
||||||
@@ -425,13 +425,13 @@
|
|||||||
"semantic_hash": "6c6f3e4c4c2f57867894280bce728c50"
|
"semantic_hash": "6c6f3e4c4c2f57867894280bce728c50"
|
||||||
},
|
},
|
||||||
"backend/AGENTS.md": {
|
"backend/AGENTS.md": {
|
||||||
"mtime": 1786865319.314815,
|
"mtime": 1786872951.3127444,
|
||||||
"ast_hash": "5bbabcd7dd425302cb0d989d8897baf6",
|
"ast_hash": "68bc86af9593adad1e37fa05188fe63e",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/CLAUDE.md": {
|
"backend/CLAUDE.md": {
|
||||||
"mtime": 1786865319.314815,
|
"mtime": 1786872951.3127444,
|
||||||
"ast_hash": "5bbabcd7dd425302cb0d989d8897baf6",
|
"ast_hash": "68bc86af9593adad1e37fa05188fe63e",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
},
|
},
|
||||||
"backend/internal/web/templates/app.html": {
|
"backend/internal/web/templates/app.html": {
|
||||||
@@ -658,5 +658,20 @@
|
|||||||
"mtime": 1786865113.5362902,
|
"mtime": 1786865113.5362902,
|
||||||
"ast_hash": "8b9f9c7a56e0ec3bf950aa70acd95e92",
|
"ast_hash": "8b9f9c7a56e0ec3bf950aa70acd95e92",
|
||||||
"semantic_hash": ""
|
"semantic_hash": ""
|
||||||
|
},
|
||||||
|
"backend/internal/latest/sighting_test.go": {
|
||||||
|
"mtime": 1786868815.9760568,
|
||||||
|
"ast_hash": "3b1372bbeeb538b73f86f57da210b4cc",
|
||||||
|
"semantic_hash": ""
|
||||||
|
},
|
||||||
|
"backend/internal/store/migrations/0011_series_sightings.sql": {
|
||||||
|
"mtime": 1786867605.3728056,
|
||||||
|
"ast_hash": "d0ade928a6e951eb179e4d0255cca526",
|
||||||
|
"semantic_hash": ""
|
||||||
|
},
|
||||||
|
"docs/adr/0011-sighting-deferral-trust-model.md": {
|
||||||
|
"mtime": 1786872970.473592,
|
||||||
|
"ast_hash": "0b6ca704f62185c9d629320f2d8475b5",
|
||||||
|
"semantic_hash": ""
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -905,27 +905,37 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Records the newest chapter a site has published. Silent: this fires from
|
// Reports the newest chapter a site has published. Silent: this fires from
|
||||||
// page visits and background checks the user did not ask for, and it never
|
// page visits and background checks the user did not ask for, and it never
|
||||||
// reorders the list — updated_at is a candidate the server discards unless
|
// reorders the list — updated_at is a candidate the server discards unless
|
||||||
// reading progress moved.
|
// reading progress moved.
|
||||||
async function applyLatestChapterIfChanged(existing, latest) {
|
//
|
||||||
|
// An unchanged number is still sent. It is the read that lets the backend
|
||||||
|
// skip its own poll of this series (a Sighting, issue #103), so the common
|
||||||
|
// case — visiting a series with nothing new — is exactly the one worth
|
||||||
|
// reporting. Only the local write and the re-render are skipped.
|
||||||
|
async function reportLatestChapter(existing, latest) {
|
||||||
if (!existing || !latest) return;
|
if (!existing || !latest) return;
|
||||||
if (existing.latest_chapter_num === latest.num) return;
|
const changed = existing.latest_chapter_num !== latest.num;
|
||||||
const bm = Object.assign({}, existing, {
|
let bm = existing;
|
||||||
|
if (changed) {
|
||||||
|
bm = Object.assign({}, existing, {
|
||||||
latest_chapter: latest.label,
|
latest_chapter: latest.label,
|
||||||
latest_chapter_num: latest.num,
|
latest_chapter_num: latest.num,
|
||||||
updated_at: Date.now(),
|
updated_at: Date.now(),
|
||||||
});
|
});
|
||||||
upsertLocal(bm);
|
upsertLocal(bm);
|
||||||
render();
|
render();
|
||||||
|
}
|
||||||
// A queued write owns this row; the drain sends latest_chapter
|
// A queued write owns this row; the drain sends latest_chapter
|
||||||
// with it, carrying the correct bucket.
|
// with it, carrying the correct bucket.
|
||||||
if (queueGet(bm.key)) return;
|
if (queueGet(bm.key)) return;
|
||||||
try {
|
try {
|
||||||
const saved = await apiPut(bm.key, bm);
|
const saved = await apiPut(bm.key, bm);
|
||||||
|
if (changed) {
|
||||||
upsertLocal(saved);
|
upsertLocal(saved);
|
||||||
render();
|
render();
|
||||||
|
}
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
/* offline — the local cache still shows it, retried on a later visit */
|
/* offline — the local cache still shows it, retried on a later visit */
|
||||||
}
|
}
|
||||||
@@ -937,7 +947,7 @@
|
|||||||
if (p.type !== "series") return;
|
if (p.type !== "series") return;
|
||||||
const existing = state.byKey[keyOf(p)];
|
const existing = state.byKey[keyOf(p)];
|
||||||
if (!existing) return;
|
if (!existing) return;
|
||||||
applyLatestChapterIfChanged(
|
reportLatestChapter(
|
||||||
existing,
|
existing,
|
||||||
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
|
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
|
||||||
);
|
);
|
||||||
@@ -977,7 +987,7 @@
|
|||||||
const html = await res.text();
|
const html = await res.text();
|
||||||
latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
|
latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
|
||||||
}
|
}
|
||||||
await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest);
|
await reportLatestChapter(state.byKey[bm.key] || bm, latest);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
/* offline or blocked — try again after the throttle window */
|
/* offline or blocked — try again after the throttle window */
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -809,27 +809,37 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Records the newest chapter a site has published. Silent: this fires from
|
// Reports the newest chapter a site has published. Silent: this fires from
|
||||||
// page visits and background checks the user did not ask for, and it never
|
// page visits and background checks the user did not ask for, and it never
|
||||||
// reorders the list — updated_at is a candidate the server discards unless
|
// reorders the list — updated_at is a candidate the server discards unless
|
||||||
// reading progress moved.
|
// reading progress moved.
|
||||||
async function applyLatestChapterIfChanged(existing, latest) {
|
//
|
||||||
|
// An unchanged number is still sent. It is the read that lets the backend
|
||||||
|
// skip its own poll of this series (a Sighting, issue #103), so the common
|
||||||
|
// case — visiting a series with nothing new — is exactly the one worth
|
||||||
|
// reporting. Only the local write and the re-render are skipped.
|
||||||
|
async function reportLatestChapter(existing, latest) {
|
||||||
if (!existing || !latest) return;
|
if (!existing || !latest) return;
|
||||||
if (existing.latest_chapter_num === latest.num) return;
|
const changed = existing.latest_chapter_num !== latest.num;
|
||||||
const bm = Object.assign({}, existing, {
|
let bm = existing;
|
||||||
|
if (changed) {
|
||||||
|
bm = Object.assign({}, existing, {
|
||||||
latest_chapter: latest.label,
|
latest_chapter: latest.label,
|
||||||
latest_chapter_num: latest.num,
|
latest_chapter_num: latest.num,
|
||||||
updated_at: Date.now(),
|
updated_at: Date.now(),
|
||||||
});
|
});
|
||||||
upsertLocal(bm);
|
upsertLocal(bm);
|
||||||
render();
|
render();
|
||||||
|
}
|
||||||
// A queued write owns this row; the drain sends latest_chapter
|
// A queued write owns this row; the drain sends latest_chapter
|
||||||
// with it, carrying the correct bucket.
|
// with it, carrying the correct bucket.
|
||||||
if (queueGet(bm.key)) return;
|
if (queueGet(bm.key)) return;
|
||||||
try {
|
try {
|
||||||
const saved = await apiPut(bm.key, bm);
|
const saved = await apiPut(bm.key, bm);
|
||||||
|
if (changed) {
|
||||||
upsertLocal(saved);
|
upsertLocal(saved);
|
||||||
render();
|
render();
|
||||||
|
}
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
/* offline — the local cache still shows it, retried on a later visit */
|
/* offline — the local cache still shows it, retried on a later visit */
|
||||||
}
|
}
|
||||||
@@ -841,7 +851,7 @@
|
|||||||
if (p.type !== "series") return;
|
if (p.type !== "series") return;
|
||||||
const existing = state.byKey[keyOf(p)];
|
const existing = state.byKey[keyOf(p)];
|
||||||
if (!existing) return;
|
if (!existing) return;
|
||||||
applyLatestChapterIfChanged(
|
reportLatestChapter(
|
||||||
existing,
|
existing,
|
||||||
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
|
computeLatestChapter(p.site, anchorsFromDocument(document), p.seriesId)
|
||||||
);
|
);
|
||||||
@@ -883,7 +893,7 @@
|
|||||||
if (!res.ok) continue;
|
if (!res.ok) continue;
|
||||||
const html = await res.text();
|
const html = await res.text();
|
||||||
const latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
|
const latest = computeLatestChapter(bm.site, anchorsFromHTML(html), bm.series_id);
|
||||||
await applyLatestChapterIfChanged(state.byKey[bm.key] || bm, latest);
|
await reportLatestChapter(state.byKey[bm.key] || bm, latest);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
/* offline or blocked — try again after the throttle window */
|
/* offline or blocked — try again after the throttle window */
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user