Files
mangaBookmark/plans/2026-07-25-bookmark-list-favorites-design.md
T
sulthan 83c3ffe3ad docs: add background opportunistic refresh to latest-chapter design
Same-origin fetch() from the userscript, throttled per-bookmark, to reduce
the "only fresh when you open the exact series page" gap without server-side
polling (still blocked by Cloudflare). Also documents that no JSON API or
RSS feed exists on either site, ruling out a more stable poll target.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-25 11:40:17 +07:00

12 KiB
Raw Blame History

Bookmark list ordering, latest-chapter display, and favorites

Date: 2026-07-25 Status: Approved by user, pending implementation plan

Context

Three requested additions to the Bromite userscript's bookmark panel (userscript/manga-bookmark.user.js) plus one incidental bug found while verifying feasibility live:

  1. Bookmark list should show the most-recently-read manga first.
  2. Show not just the last chapter read, but the latest chapter available for that manga, if feasible.
  3. A favorites mechanism: a second list/tab showing only favorited manga, without removing favorited manga from the normal list.

1. Reorder by latest read (already implemented)

reindex() in manga-bookmark.user.js has sorted state.list by (b.updated_at || 0) descending since the very first commit (f58d113). updated_at is set whenever progress is recorded (bookmarkCurrent, updateToCurrentChapter, setChapterManual). No code change needed here — confirmed by decision: only an actual progress advance should reorder the list (not simply opening/re-reading an old chapter).

The only risk to this existing behavior is introduced by features 2 and 3 below, since both add new fields synced through the same PUT endpoint that currently (per existing CLAUDE.md) has the server set updated_at unconditionally on every upsert. See section 4.

2. Latest available chapter

Feasibility (verified live via Playwright, 2026-07-25)

  • Chapter reader pages only expose immediate neighbors. On asurascans.com/comics/dungeon-odyssey-f886a8af/chapter/160, the DOM contains only chapters 159–161 (prev/next nav) — not the full list.
  • Series pages list every chapter, newest first by default. On asurascans.com/comics/dungeon-odyssey-f886a8af, the series page's chapter list panel contains one <a href=".../chapter/N">Chapter N ...</a> per chapter (verified: 162 links for a 162-chapter series, first is Chapter 162). This is therefore the only reliable place to learn the true latest chapter number.
  • Demonic's series page (demonicscans.org/manga/Dungeon-Odyssey) similarly lists every chapter via chaptered.php?manga=<id>&chapter=<n> links, interleaved with a "first chapter" quick-jump link. Taking max() of all parsed chapter numbers (rather than assuming list order) is used for robustness on this site.

Conclusion: capturing latest-available-chapter data requires fetching a series page's HTML (no reader-page source, no JSON API — see below). The backend cannot do this itself (Cloudflare blocks server-side fetch, per existing CLAUDE.md constraint), so it must happen from the userscript, running in the user's real browser session. Section "Background opportunistic refresh" below extends this beyond "only when you open that exact series page."

No stable API exists (checked live, 2026-07-25)

  • The series page is server-rendered HTML (Next.js SSR) — network capture during a series-page load shows no _next/data JSON call and no XHR/fetch for chapter data (only unrelated api.asurascans.com calls for announcements/promotions banners). Chapter data is embedded directly in the HTML, not fetched separately.
  • No RSS/feed exists: asurascans.com/feed, /rss, /rss.xml all 404.
  • Conclusion: HTML scraping (DOM when live on the page, raw-text regex when background-fetched — see below) is the only available data source on either site. There is nothing more "API-like" to poll instead.

Behavior

  • New adapter capability: on detect() returning type: "series" for an already-bookmarked series (state.byKey[key] exists), scan the page for chapter links per the site-specific pattern below and compute the max chapter number + its display label.
    • Asura: a[href*="/chapter/"] where the href matches /chapter/([\d.]+)$/ and the link text matches /Chapter\s+[\d.]+/ (excludes the unrelated "First Chapter" quick-jump button, which lacks that text pattern).
    • Demonic: all a[href] matching /chaptered\.php\?manga=\d+&chapter=([\d.]+)/; take the max parsed chapter number across all matches (list order is not assumed reliable).
  • If the computed max differs from the bookmark's stored latest_chapter_num, silently update latest_chapter (label) and latest_chapter_num via the API — but this update must not change updated_at / list order (see section 4).
  • Display: each list item's subtitle line becomes e.g. "Read: Chapter 12 · Latest: Chapter 15" when latest is known and differs from last-read; otherwise unchanged ("Chapter 12 · asura" as today). Plain inline text, no separate badge/count UI.

Background opportunistic refresh (closer to "live")

Live-page capture above only refreshes a series when the user happens to open that exact series page. To reduce that gap without server-side polling (still blocked by Cloudflare — verified: server-side fetch is a datacenter request with no browser session, this is unchanged and not being revisited), the userscript also does same-origin background checks using the user's own real browser session:

  • Same-origin only. fetch() issued from a page on asurascans.com can only safely reach other asurascans.com paths (no CORS trouble, looks like a normal authenticated browser request). It cannot reach demonicscans.org or vice versa. So visiting any page on a site opportunistically refreshes only that site's bookmarks — confirmed live that same-origin fetch() from an already-loaded page succeeds cleanly (tested against asurascans.com/sitemap.xml from a comics/* page).
  • Trigger: on every page load, after init()/refresh(), run backgroundRefreshLatest(). It picks bookmarks belonging to the current site that haven't been checked within a throttle window (LATEST_CHECK_THROTTLE_MS, proposed 4 hours), oldest-checked-first, and checks at most LATEST_CHECK_BATCH of them (proposed 1) per page load — so a normal reading session gradually keeps bookmarks fresh without ever bursting requests.
  • Freshness tracking is local-only, not synced: a small mangabm:lastchecked localStorage map of { [key]: timestampMs }. It's device-local by nature (each device does its own background checks) and keeping it out of the synced Bookmark record avoids polluting the cross-device schema with a per-device value.
  • Fetch + parse: fetch(bookmark.series_url) → res.text() → apply the same regex rule already defined above (Asura: chapter-link href + "Chapter N" text; Demonic: max of all chaptered.php?...chapter= matches) against the raw HTML string instead of the live DOM. No additional parsing logic — this reuses the exact same rule, just fed fetched text instead of document.
  • Update path: identical to live-page capture — PUT the bookmark with new latest_chapter/latest_chapter_num if changed, no updated_at bump (section 4).
  • Failure handling: a failed/blocked background fetch is silently skipped (no toast, no retry loop) — next eligible page load tries again naturally once the throttle window passes.
  • What this buys: instead of "only fresh if you opened that exact series page," bookmarks on a site you're actively reading converge to "checked within the last ~4 hours," which is meaningfully closer to live given frequent reading — without needing server-side scraping that Cloudflare would block anyway. It is still not push/instant; nothing can notify the moment a new chapter is posted without the site itself offering that (it doesn't — no RSS/webhooks, confirmed above).

3. Favorites

  • New field favorite: bool on the bookmark record, synced through the existing PUT endpoint (chosen over local-only storage so favorites persist across devices/reinstalls, consistent with how the rest of the data syncs).
  • Each list item gets a star toggle (☆ / ★) that flips favorite and PUTs the updated bookmark. Toggling must not change updated_at / list order (see section 4).
  • The panel gains two tabs above the bookmark list: All and ★ Favorites. Both apply the same sort (section 1). Switching tabs is local UI state (not persisted) defaulting to "All". A favorited manga continues to appear in "All" — tabs only filter which array is rendered, favoriting never removes the bookmark from state.list.

4. Backend change required: conditional updated_at

Current CLAUDE.md / store.go behavior: PUT /bookmarks/{key} always sets updated_at server-side on every upsert. Once latest-chapter-capture and favorite-toggle both PUT through that same endpoint, this would reorder the list on every series-page visit or star click — contradicting the "reorder only on progress advance" decision from section 1.

Change: Store.Upsert sets updated_at = now() only when:

  • the bookmark is new (no existing row for that key), or
  • last_chapter_num in the incoming payload differs from the currently stored value.

Otherwise the existing stored updated_at is preserved, even though other fields (favorite, latest_chapter, latest_chapter_num, cover, title, etc.) are still updated. This centralizes "what counts as a progress advance" as a single authoritative rule in the backend, applied consistently regardless of which device/browser performed the write.

This is a deliberate deviation from the current CLAUDE.md wording ("server sets updated_at" unconditionally) and needs the doc updated to match.

5. Asura redirect bug (bundled into this work)

Verified live: https://asuracomic.net/comics/dungeon-odyssey-f886a8af returns an HTTP 301 with Location: https://asurascans.com/ (root, no path) — a Cloudflare-edge redirect that discards the path before any JS on asuracomic.net executes. This contradicts the existing code comment ("asuracomic.net currently 301s to asurascans.com", implying path preservation) and means any deep link on asuracomic.net currently lands the user on the asurascans.com homepage with page type "other" — bookmark/progress detection silently does nothing.

There is no client-side fix: the userscript's @match for asuracomic.net/* never gets a chance to run for these URLs, since the redirect happens at the edge before the browser has a document to inject into. Fix is documentation-only: correct the misleading comment in asura.matches()/adapter notes and README to state that asuracomic.net deep links are currently broken, and the user should navigate via asurascans.com links directly. No behavior change to ship.

Data model summary

Bookmark (backend store.go) gains:

field type notes
favorite bool default false
latest_chapter string display label, e.g. "Chapter 162"; empty if never captured
latest_chapter_num *float64 nullable; null if never captured

SQLite migration: additive ALTER TABLE bookmarks ADD COLUMN ... for each, guarded against "duplicate column" errors so it's safe to run against the already-deployed database.

Userscript-local, not synced: mangabm:lastchecked localStorage key, a { [bookmarkKey]: timestampMs } map used only to throttle background refresh (see section 2).

Testing

  • Go: table-driven tests for Store.Upsert's conditional updated_at logic (new bookmark, unchanged progress, changed progress, favorite-only change, latest-chapter-only change).
  • Userscript: manual on-device verification (per existing project convention — no JS test harness in this repo). Verify:
    • list reorders only when a chapter is actually advanced, not on plain re-visit or favorite toggle.
    • latest-chapter capture fires on series-page visits and displays correctly for both sites.
    • background refresh: check one stale same-site bookmark per page load, skips bookmarks checked within the throttle window, updates silently without reordering the list, and doesn't fire cross-site.
    • favorite toggle persists across a panel close/reopen and a refresh() (i.e. round-trips through the backend correctly).
    • favorited manga still appears in "All" tab.