sulthan
6af49e6790
Gives every bookmark a lifecycle bucket — `reading`, `archived`, or `finished` — so on-hold series leave the main list while still being polled for new chapters, completed series get a web-only bucket, and the userscript panel gains quick links to the web UI and both manga sites.
Design: `docs/superpowers/specs/2026-07-27-status-buckets-design.md`
## Data model
One additive column through the existing `addedColumns` migration list:
```sql
ALTER TABLE bookmarks ADD COLUMN status TEXT NOT NULL DEFAULT 'reading'
```
The `DEFAULT` backfills every pre-existing row as `reading`, so there is no separate migration step. `favorite` is unchanged and orthogonal — a series can be an archived favourite.
Rollback is safe: an old binary against the new database omits `status` from its INSERT (it gets the DEFAULT) and never mentions it in the conflict clause, so buckets survive.
## The write rule
`PUT /bookmarks/{key}` decodes a whole `Bookmark` and `Upsert` writes every column it knows about. `latest_checked_at` escaped this by staying out of `bookmarkColumns` entirely — `status` cannot, because the userscript must be able to archive and restore.
So an empty incoming status means **"no opinion"**, not a value, and resolves on the `VALUES` side of the upsert:
```sql
COALESCE(NULLIF(?, ''), (SELECT status FROM bookmarks WHERE key = ?), 'reading')
```
with `DO UPDATE SET status = excluded.status`.
It has to be this way round. `excluded.*` is the row *after* the `VALUES` expressions are evaluated, so applying the default there and then reading `excluded.status` in the conflict clause would see `'reading'` rather than the empty string — and would overwrite an archived row on every progress PUT from a client that knows nothing about the column. One expression, evaluated once, covers insert and update alike. The subquery runs inside the transaction, so it sees the row the statement is about to conflict with.
`TestUpsertEmptyStatusPreservesStored` is the guard on this.
`updated_at` behaviour is unchanged: it moves only when `last_chapter_num` changes, so archiving, finishing, restoring, and favouriting never reorder the list.
## Validation
`PUT /bookmarks/{key}` returns 400 for any status outside `{"", "reading", "archived", "finished"}`, and for `"finished"` specifically. Finishing a series is a web-UI decision, enforced server-side rather than by trusting every client to leave the value alone. The `/ui/*` endpoints have their own session-guarded route and are unaffected.
## Visibility
| Surface | All | Updated | Favourites | Archived | Finished |
|---|---|---|---|---|---|
| Web | reading | reading | reading | archived | finished |
| Userscript | reading | — | reading | archived | not shown |
Archived and finished appear in their own tab and nowhere else — including the web UI's "Continue reading" strip, which is now built from reading-only rows before tab filtering. An archived favourite shows up under Archived only: Favourites means "favourites I am currently reading".
## Backend
- **`store.go`** — `Bookmark.Status`, the column in `schema` / `addedColumns` / `bookmarkColumns` / `scanBookmark` / `Upsert`. `scanBookmark` normalises anything outside the three known buckets to `reading`, so no row can land in no list at all.
- **`store.go`** — `DueForLatestCheck` gains `AND status IS NOT 'finished'`. Archived series keep being polled; that is the whole point of archiving rather than deleting. Finished ones have nothing coming, so polling them only burns fetches and risks a spurious "new chapter" badge. `IS NOT` is null-safe, so a hand-edited NULL still qualifies.
- **`handlers.go`** — status validation on `PUT`, before any write.
- **`web.go`** — `buildListView` filters the new tabs and excludes both buckets from `all` / `new` / `fav` and the recent strip; new `POST /ui/bookmarks/{key}/status`, session-guarded like its siblings, read-modify-writing through `Store.Get` + `Store.Upsert` so the `updated_at` rule stays in one place.
- **`templates/`** — two more tabs; per-card controls (reading → Archive + Finish, archived → Restore + Finish, finished → Restore); empty-state copy for both new tabs.
- **`static/style.css`** — five tabs no longer divide a phone's width legibly, so the row scrolls sideways instead of squeezing.
## Userscript (1.3.0)
- Third tab **Archived** beside All and Favourites. A missing `status` reads as `reading`, so a list cached by the previous version still renders. `finished` matches no tab and is invisible everywhere.
- Per-item **Archive / Unarchive** button on the existing optimistic path: mutate local state and cache, `apiPut`, adopt the server's returned row.
- Header chip row linking the web UI and both manga sites, each `target="_blank" rel="noopener"`.
- `apiPut` now omits `status` unless the caller opts in — see below.
Still free of every `GM_*` API: plain `fetch`, page `localStorage`, on-page UI only.
## One bug worth calling out
The userscript's other mutations (`updateToCurrentChapter`, `setChapterManual`, `toggleFavorite`, `applyLatestChapterIfChanged`) build their payload with `Object.assign({}, existing, …)`, so they echoed the cached `status` back to the server. `GET /bookmarks` has no status filter — finished rows are in `state.list` and only hidden at render time — which made two failures reachable:
1. Reading a chapter of a series marked finished sent `"status":"finished"`, which the API rejects with 400. Progress never synced, behind a misleading "Offline — saved locally, will retry" toast, permanently.
2. Archiving on desktop and then reading on a phone whose cache predated the archive sent `"status":"reading"` and silently un-archived the series — contradicting the README's "reading an archived series leaves it archived".
Fixed at the single choke point: `apiPut(key, obj, { sendStatus = false })` strips `status` from a copy of the body unless the caller opts in, and only `toggleArchive` opts in. Only an explicit archive/restore has an opinion about the bucket; everything else omits the field so the server's keep-on-empty rule applies. Stripping merely the *invalid* values would not have been enough — a stale cached `"reading"` still clobbers a remote archive.
Also: the userscript's `backgroundRefreshLatest` now skips finished series, matching the server poller, instead of spending batch slots fetching pages for a series that has nothing coming.
## Known limitations, deliberate
Both are marked in-code with `ponytail:` comments naming the ceiling and the upgrade path:
- The poller's `Store.Get` + `Store.Upsert` is not wrapped in a transaction, so a client PUT that commits between the two is lost to the stale re-read. Already documented for read progress in `CLAUDE.md`; it now costs a status change too. Accepted for a single-user deployment.
- A card whose new status no longer matches the active tab stays on screen until the next list load. The alternative is an out-of-band swap or a full list refresh per toggle, and the card visibly showing its new state is enough feedback.
## Testing
`go test ./...` passes; `CGO_ENABLED=0 go build ./...` clean.
- **`store_test.go`** — a fresh row defaults to `reading`; a legacy database gains the column with every row `reading`; an `Upsert` carrying `""` preserves the stored bucket while a value replaces it; a status change does not move `updated_at`; `DueForLatestCheck` returns archived and skips finished; the poller's `Get` → `Upsert` round trip preserves `archived`.
- **`main_test.go`** — `PUT` with `finished` or garbage is 400, `""` / `reading` / `archived` round-trip; a PUT that omits the `status` key entirely (what a pre-1.3.0 userscript sends) preserves an archived bucket *and* applies the chapter progress in the same request.
- **`web_test.go`** — each tab returns only its bucket; the recent strip excludes archived and finished; the status endpoint requires a session, rejects unknown values, and does not move `updated_at`; the card renders the right controls per bucket.
Userscript has no automated harness, so it was checked against a live `https://asurascans.com` page: the chips resolve, Archive moves a series out of All and Favourites into Archived, the state survives a full reload (so it came from the server, not local optimism), Unarchive returns it, a series marked finished in the web UI appears in no tab, and — captured on the wire — the archive PUT carries `"status":"archived"` while a favourite toggle on that same archived series carries no `status` key at all.
Reviewed-on: #4
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
132 lines
11 KiB
Markdown
132 lines
11 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Status
|
|
|
|
Greenfield. Only `plans/mangaBookmark.md` exists — no code yet. That plan is the spec; read it before building. Two deliverables: a Go sync backend and a single Bromite-compatible userscript.
|
|
|
|
## What this is
|
|
|
|
A manga read-progress tracker for a user reading on **asurascans.com** (the current domain; asuracomic.net 301s here) and **demonicscans.org** from **Bromite** (mobile Chromium). A userscript injects on-page UI (floating button + slide-in panel) and syncs progress to a self-hosted Go backend so bookmarks unify across both sites and across devices.
|
|
|
|
## Hard constraints (these drive the design — do not violate)
|
|
|
|
Bromite uses Chromium's **native** userscript engine, not Tampermonkey:
|
|
- **No `GM_*` APIs anywhere.** No `GM_setValue`/`GM_getValue` (use page `localStorage`), no `GM_registerMenuCommand` (inject on-page UI), no `GM_xmlhttpRequest` for cross-origin (use plain `fetch()`). Keeping the script GM-free also lets it run in desktop Tampermonkey/Violentmonkey for faster iteration.
|
|
- Cross-origin `fetch()` works **only** against a CORS-enabled backend. Manga sites are `https://`, so backend **must be HTTPS** (mixed-content block otherwise).
|
|
- Asura and Demonic are **separate origins with separate `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync is required, not optional.
|
|
- Userscript runs in an **isolated world**, so the embedded API token is safe from the site's JS.
|
|
- Cloudflare's block on fetching the manga sites is **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both the CGNAT dev machine *and* the deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. This contradicts an earlier, untested assumption that the CGNAT dev IP would be blocked; it was not, at least on this date. Treat "does curl work right now" as a live, time-varying fact to re-check, not a fixed property of a given machine — Cloudflare's bot scoring can flip a previously-clean IP without notice. Any backend fetcher still needs a graceful-degrade path for when it does get challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, or a direct probe) before finalizing, not assumed from a single earlier test.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Bromite userscript (isolated world, per-site adapters, localStorage cache)
|
|
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume)
|
|
```
|
|
|
|
- **Backend** (`backend/`): stdlib `net/http` (a handful of routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). The reverse proxy terminates TLS; the Go service listens plain `:8080`.
|
|
- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`). Sync is **last-write-wins**. Schema and endpoint list are in the plan.
|
|
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
|
|
- **Web UI:** the same binary serves a password-gated browser UI on a second
|
|
hostname — `GET /` (list, or login page when there is no session),
|
|
`POST /login`, `POST /logout`, `GET /static/*`, and htmx fragment endpoints
|
|
under `/ui/*`. Templates and assets are `go:embed`-ed, so `backend/Dockerfile`
|
|
must copy `templates/` and `static/` as well as `*.go`. Sessions are stateless
|
|
HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them and, when empty,
|
|
the web routes are not registered at all. UI mutations read-modify-write
|
|
through `Store.Get` + `Store.Upsert` so the `updated_at` rule stays in one
|
|
place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`.
|
|
- **Latest-chapter poller:** a ticker goroutine in the same binary re-checks
|
|
each bookmarked series' newest published chapter from the backend's own
|
|
network access, so `latest_chapter` stays fresh when the user is not
|
|
browsing. It is a *second, parallel* signal — the userscript keeps its own
|
|
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
|
|
Two independent clocks: a per-bookmark cooldown (`latest_checked_at` column,
|
|
enforced by `Store.DueForLatestCheck`'s WHERE clause) and a wake interval.
|
|
The row is stamped *before* the fetch so a broken series waits out a full
|
|
cooldown instead of retrying every tick, and writes go through
|
|
`Store.Get` + `Store.Upsert` so a new chapter never reorders the list.
|
|
Fetches use `bogdanfinn/tls-client` with a Chrome profile as defence in depth
|
|
against fingerprint-based blocking; any failure logs and skips. See
|
|
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
|
|
The poller's `Store.Get` + `Store.Upsert` is not wrapped in a transaction, so
|
|
a userscript `PUT` that commits between the two can be overwritten by the
|
|
poller's stale re-read — reverting that read progress and, since the stored
|
|
value now differs, moving `updated_at` and reordering the list. This is a
|
|
known, accepted limitation for a single-user deployment, not a bug to fix.
|
|
- **`updated_at` drives list order, so it moves only on real reading progress:** the server applies its timestamp when the row is new or `last_chapter_num` changes, and otherwise keeps the stored value — favouriting a series or recording a newly published chapter must not reorder the list. `PUT` therefore returns the row **as stored**, and clients must adopt that response rather than their own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
|
|
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
|
|
`finished`, orthogonal to `favorite`. Archived and finished appear only in
|
|
their own tab — not in All, Updated, Favourites, or the recent strip. The
|
|
poller keeps checking archived series and skips finished ones. `finished` is
|
|
settable only from the web UI; `PUT /bookmarks/{key}` rejects it with 400.
|
|
**An empty incoming status means "keep the stored one"** — resolved on the
|
|
`VALUES` side of `Store.Upsert`, not in the conflict clause, because
|
|
`excluded.*` is the post-evaluation row and a default applied there would
|
|
wipe the bucket on every PUT from a client that predates the column. See
|
|
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
|
|
- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list), `DB_PATH`
|
|
(default `/data/bookmarks.db`), `PORT` (default `8080`), `WEB_PASSWORD`
|
|
(gates the browser UI; unset disables it),
|
|
`LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER`
|
|
(background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`).
|
|
|
|
### Userscript structure (single IIFE, `manga-bookmark.user.js`)
|
|
|
|
1. **Site adapters** — one per host, `detect(location, document)` returns page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title`/`cover` from **`og:title`/`og:image` meta tags**, not CSS classes.
|
|
2. **API client** — `apiGet/apiPut/apiDelete` with bearer header; `localStorage` key `mangabm:cache` for instant render + offline fallback.
|
|
3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value.
|
|
4. **UI** — rendered inside a **Shadow DOM** root to isolate from site CSS
|
|
(critical on mobile). Three tabs (All / Favourites / Archived) and a row of
|
|
link chips to the web UI and both manga sites; `WEB_BASE` sits in the CONFIG
|
|
block next to `API_BASE`.
|
|
5. **SPA navigation** — Asura is Astro, client-routed on the comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fires without reload. Demonic uses classic reloads (initial `document-idle` run suffices).
|
|
|
|
### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trusting)
|
|
|
|
- **asurascans.com**: series `/comics/<slug>` (slug carries a trailing hash-like suffix, e.g. `-f886a8af`), chapter `/comics/<slug>/chapter/<n>`. Astro-rendered; chapter links are present in raw server HTML (no client-side-only render blocking a server fetch).
|
|
- **demonicscans.org**: series `/manga/<slug>` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title/<slug>/chapter/<n>/<page>` (the older `chaptered.php?manga=<id>&chapter=<n>` form still exists as a redirect and is what series-page chapter-list anchors link through).
|
|
|
|
## Commands (once code exists)
|
|
|
|
Backend (`cd backend`):
|
|
- Test all: `go test ./...`
|
|
- Single test: `go test -run TestName ./...`
|
|
- Build static binary: `CGO_ENABLED=0 go build`
|
|
|
|
Local stack: `docker compose up` (named volume mounted at `/data`, `restart: unless-stopped`).
|
|
|
|
Smoke test: `curl` the endpoints with `Authorization: Bearer <token>`; confirm `OPTIONS` preflight returns CORS headers and `/healthz` returns 200.
|
|
|
|
## Forge: Gitea, not GitHub
|
|
|
|
`origin` is a self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` does not work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones:
|
|
|
|
- Open a PR: `tea pr create --head <branch> --base main --title "..." --description "..."`
|
|
- List / view / check out: `tea pr list`, `tea pr <n>`, `tea pr checkout <n>`
|
|
- Issues: `tea issue create`, `tea issue list`
|
|
- Auth lives in `tea login`, not a `GH_TOKEN` env var.
|
|
|
|
`tea` prints its output as rendered boxes rather than plain text; the PR URL lands on the last line.
|
|
|
|
## Security invariants
|
|
|
|
- Auth on `/bookmarks*`: require `Authorization: Bearer <API_TOKEN>`, **constant-time compare**, 401 otherwise.
|
|
- CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` with `204`.
|
|
|
|
## Relevant skills
|
|
|
|
`multi-stage-dockerfile` and `docker-compose-orchestration` for the container work (referenced in the plan).
|
|
|
|
## graphify
|
|
|
|
This project has a knowledge graph at graphify-out/ with god nodes, community structure, and cross-file relationships.
|
|
|
|
Rules:
|
|
- For codebase questions, first run `graphify query "<question>"` when graphify-out/graph.json exists. Use `graphify path "<A>" "<B>"` for relationships and `graphify explain "<concept>"` for focused concepts. These return a scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output.
|
|
- If graphify-out/wiki/index.md exists, use it for broad navigation instead of raw source browsing.
|
|
- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain do not surface enough context.
|
|
- After modifying code, run `graphify update .` to keep the graph current (AST-only, no API cost).
|