Compare commits

...

59 Commits

Author SHA1 Message Date
sulthan c62c3bb07b chore: drop asuracomic.net from the userscript, CORS allowlist and docs (#97)
Closes #96.

## What

Removes every reference that still invites a Reader onto `asuracomic.net`.
The domain's deep links 301 to the `asurascans.com` **root**, discarding the
path (re-checked 2026-07-25), so a page on it never yields a series document
client-side and a stored address on it never yields a series page server-side.
#95 already pinned each Site to one hostname, so the backend rejects such an
address cleanly; this is the cleanup around that.

| File | Change |
|---|---|
| `userscript/manga-bookmark.user.js` | drops the `@match`, narrows the asura adapter to `/(^\|\.)asurascans\.com$/` |
| `userscript/test/logic.test.js` | new test pinning the narrowed host match |
| `.env.example`, `docker-compose.yml` | origin dropped from the `ALLOWED_ORIGINS` default |
| `DEPLOY.md` | same, and the sample list gains the two novel origins it was missing |
| `backend/api_test.go` | CORS fixtures and round-trip seed move to `asurascans.com` |
| `README.md`, `AGENTS.md` | notes say the host is dropped, not "stays matched" |

## Behaviour

- A Reader landing on `asuracomic.net` gets no userscript UI. Previously the
  script loaded and could do nothing useful — the redirect had already
  discarded the path.
- A request whose `Origin` is `https://asuracomic.net` is no longer reflected
  by a deployment using the shipped defaults.
- No backend logic changed: the CORS rule, the address gate and the poller are
  untouched. `AllowedOrigins` is data, not code.

## Security invariant preserved

CORS still reflects `Origin` only when it appears in `ALLOWED_ORIGINS`, with
`GET,PUT,DELETE,OPTIONS` and a `204` preflight — `TestCORSPreflight` and
`TestCORSDisallowedOrigin` still pin both halves, now against a live origin.
This change only removes a value from the allowlist, which is a narrowing.

## Verification

- `go test ./...` — full backend suite green (real Postgres per package).
- `node --test test/*.test.js` — 66/66 green, up one from the new match test.

## Deploy note (does not happen on merge)

The live allowlist comes from the VPS `.env`, not from these defaults, so the
origin must be dropped there in the same deploy. The one-off row repair for any
stored `asuracomic.net` address is in #96.

Reviewed-on: #97
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-12 05:53:17 +07:00
sulthan 21615be2bd feat: one registry entry per Site, one shared Series-page read (#95)
Closes #94.

## What

Two phases per the spec, in three feature commits plus two review-fix commits:

**Phase one — one registry entry per Site** (`2d134fb`)
The six per-site comparison points that used to live across three files collapse
into one `sites` map in `backend/internal/latest/sites.go`: Latest Chapter parse,
Cover parse, browser-backed list, fetcher route, host pins, and the browser
payload read all become lookups into it. `browserBackedSites()` is derived from
the registry (sorted, deterministic); `fetcherFor` and `fetchableSeriesURL` keep
their signatures and become lookups; `BrowserFetcher.Get` dispatches through the
entries' `Read`/`Done` while the tab lifecycle stays in `BrowserFetcher.run`.

**Phase two — one shared Series-page read** (`f215130`)
`readSeriesPage` (new `read.go`) performs the read the Poll and the Acquisition
have in common: gate, route, fetch, parse Latest Chapter, parse Cover address.
It returns facts only — polling and persistence policies (stamp order, cooldowns,
cover policy) stay with the callers; `acquire.go` gained the comment naming the
deliberate post-fetch stamp order. The poll's legacy cover heal and the
no-chapter byte-count diagnostic were restored after review (`d998f87`) so the
claims "the Poll keeps its own Cover policy" and "pinning is the only
behavioural change" both hold.

## Behaviour

- All six Sites now pin their host exactly; asura/demonic/comix previously
  accepted any https host. For asura this is a strict improvement: its dead old
  domain redirects deep links to the site root and would parse the wrong
  document.
- Everything else is unchanged: existing parse tables, the challenge-body table
  and the gate table pass unmodified except the one deliberate exception — the
  gate table gains the three new pin cases.

## Security invariants preserved

- The address gate is recognisably the same rule, now a single registry lookup:
  `https` + exact hostname match, all callers route through it. No fetch path
  was widened; asura/demonic/comix were narrowed.
- The second host pin inside each browser entry's Read is retained deliberately
  (browser = strong SSRF primitive, `series_url` is client-supplied) and is not
  deduplicated against the shared gate.
- Review hardening: `fetcherFor` now fails closed for unknown site strings
  (previously fell through to the TLS fetcher on an unreachable path), and the
  browser dispatch iterates a sorted list so outcomes cannot depend on map order.
- The security review's log-injection finding was checked against Go's
  `url.Parse` and does not hold: control characters are rejected anywhere in a
  URL, so a client-supplied value in a log line cannot carry a newline.

## Review

Reviewed on three axes (spec, standards, security) by read-only subagents over
`672c16f..f1b26f4`. No blocking findings; all minor/nit findings addressed in
`d998f87` and `700de20`. Verified end to end with `go test ./...` (Docker
Postgres per test package) on every commit.

## Out of scope (tracked separately)

- Dropping asuracomic.net (CORS allowlist, userscript match, API fixtures,
  live env) — separate issue, per spec.

Reviewed-on: #95
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-12 05:43:37 +07:00
sulthan 672c16ffbf Remove the client latest-chapter scan for lightnovelworld (#91) (#93)
Follows the spec published on #91: remove the client-side latest-chapter scan for lightnovelworld rather than porting #87's truncation into a second codebase.

- lightnovelworld.latestChapterFromAnchors deleted, not stubbed: absence is what the background-fetch guard keys off.
- computeLatestChapter tolerates an adapter with no scanner (yields null) and is exported as the test seam.
- backgroundRefreshLatest skips a scanner-less Site before the due filter: no Series page fetched, no freshness timestamp recorded, no batch slot consumed. The on-page path (maybeCaptureLatestOnSeriesPage) routes through the same null-tolerant computation.
- novelfull's scanner, the shared max-chapter helper and all four manga Sites untouched.
- userscript/AGENTS.md records the Poll-only contract for this Site and why.

All seven acceptance criteria from the spec met. node --check clean; novel suite 30/30 (the regression pin fails if a lnw scan is reintroduced, scoped or not); manga suite 35/35, manga userscript byte-for-byte unchanged.

Two-axis code review: no hard standard violations, spec-clean; one follow-up commit matching the sibling adapter guard from the manga script.

Reviewed-on: #93
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-11 20:58:18 +07:00
sulthan e7e22a12a5 lightnovelworld Series identity is read from the chapter page (#80) (#92)
Implements spec #80 / ADR-0008 — Gitea issues #86, #87, #88, #89, #90, all closed.

A Reader bookmarks a novel on lightnovelworld and it never shows a New Chapter, because the Series identity was derived from the chapter address instead of read from the page. One Series can publish under several Chapter Slugs, so the derived key points at a slug that 404s.

- **#89** — the userscript's lnw adapter stops deriving `seriesUrl`/`seriesId` from the path. It reads the page's own pointer (`a[aria-label='All Chapter']`), falling back to the microdata breadcrumb's second crumb, and carries `chapterSlug` on the page object, stored nowhere.
- **#87** — the Poll's lnw chapter scan is unscoped (no stored-slug pattern can cover a Series' whole list) and truncated at the `wpd-threads` comment thread, the one region a visitor can write to. Marker absent means skip and log with the body length, never scan whole. Corrects the `maxBodyBytes` headroom comment to the measured 3.5x.
- **#86** — the scan fixture is now text trimmed from a real, wholly-fetched Series page instead of a hand-written cross-series anchor that no live page carries.
- **#90** — stale stored rows repair themselves on the next chapter visit: a pure transform over cache, queue and last-checked map, silent to the Reader, with progress, favourite and lifecycle bucket preserved when two rows merge.
- **#88** — an env-gated live canary (`SMOKE_LNW_SERIES_URL`) proving the marker still occurs exactly once and still follows the last chapter anchor, asserted against the production symbols themselves.

Verified on the merged branch: `go test ./...` green, `gofmt -l internal/latest/` silent, both userscripts `node --check` clean, 35/35 + 29/29 logic tests. Live canary green (marker once at byte 612,182 of 651,795). #90 verified on device with Playwright.

Open follow-up: **#91** — the userscript's client-side latest-chapter scan is still scoped to the derived slug.

Reviewed-on: #92
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-11 18:21:50 +07:00
sulthan 90d8ab72ad chore: refresh graphify map and tidy AGENTS.md (#84)
graphify update regeneration: semantic hashes now populated in manifest.json, graph rebuilt (1634 nodes, 3179 edges). Track the map (5 curated files + .graphify_root) so a fresh checkout starts with it; cost.json, cache/, dated snapshots and .rebuild.lock stay ignored.

AGENTS.md: drop stale Relevant skills and Notes sections; graphify rule now says to always query the graph first.

Reviewed-on: #84
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-11 11:19:47 +07:00
sulthan c400c91a80 Implement a batch of tickets through per-ticket subagents (#82)
## What this adds

Two files that turn the one-ticket-at-a-time `/implement` loop into an orchestrated batch.

**`.claude/skills/implement-tickets/SKILL.md`** — user-invoked (`disable-model-invocation: true`, so it costs no context until typed). The agent that runs it is an orchestrator, not an implementer:

1. Collect the tickets over `tea`, reading each `Blocked by` line.
2. Plan waves from the blocking edges, three tickets wide, and fix every cross-ticket contract (shared signature, JSON shape, column, token) before anything is dispatched.
3. Present the plan and stop for approval.
4. Per ticket: `git worktree add ../ticket-<n>`, copy the gitignored `.env`, claim the issue, write a brief to `.scratch/`, then dispatch the whole wave as one `task` batch.
5. Land each result — merge `--no-ff`, comment the report, close, remove the worktree. Textual conflicts are the orchestrator's; a semantic clash goes back to whichever ticket owns the contract.
6. Full suite once on the merged base.

**`.omp/agents/ticket-implementer.md`** — the worker. Brief-driven, worktree-bound, and gated on review before it reports: it runs the `code-review` skill over its own diff with `cr-spec` and `cr-standards` on the two axes, fixes Critical and Important findings in at most two rounds, and returns a short status contract (`DONE` / `DONE_WITH_CONCERNS` / `BLOCKED` / `NEEDS_CONTEXT` / `REVIEW_BLOCKED`).

The brief template makes the subagent read `tea issue <n> --comments` for its ticket and for the issue that ticket refers to — the comments carry decisions the body never got updated with — and names the `tdd` skill at each seam where a test comes first. Briefs are written in the ubiquitous language of `CONTEXT.md`; a brief that says "scrape" where the domain says Poll hands the subagent the wrong model of the system.

## Verification

Dispatched a real `ticket-implementer` as a probe. The agent resolved from `.omp/agents`, and it spawned `cr-spec`, which replied. That was the one thing that could have silently killed the design: `task.maxRecursionDepth` defaults to 2, and the chain is session to orchestrator to implementer to reviewer. It clears. If that ever changes, the implementer returns `REVIEW_BLOCKED` and the orchestrator runs the review itself.

Confirmed against the omp binary that `autoloadSkills: code-review, tdd` is split by `parseArrayOrCSV`, not swallowed as one unknown name.

## Notes

- Agents are discovered from `.omp/agents`, never `.claude/agents` — the latter is deliberately skipped by omp because its frontmatter is a different contract.
- No product code changes. `.gitignore` gains `.scratch/`, where briefs and reports live.
- Not included: retry after a failed dispatch, a state file for resuming a crashed wave, a cheap model tier for mechanical tickets. Add them when a real batch needs them.

Reviewed-on: #82
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-11 10:08:09 +07:00
sulthan f1eb7d514c Record the lightnovelworld series-identity decision (#77) (#81)
Docs only. No code, no tests, nothing to run. Implementation is specified in #80.

Outcome of a grilling session on 2026-08-11 against #77, backed by live measurement of lightnovelworld over 2026-08-10/11.

## What changed

**`docs/adr/0008-series-identity-is-discovered-not-derived.md`** (new)

A Series identity is discovered from the Site's own links, never derived from an address.
On lightnovelworld the userscript reads the chapter page's `All Chapter` anchor instead of
building a `/novel/<slug>/` address by string manipulation. A Chapter Slug is not an
identity and is not stored. The backend's chapter scan drops its per-Series scoping and
runs against the body truncated before the visitor comment thread.

Evidence in the ADR: 3 of 41 sampled novels serve chapters under a slug that differs from
their series slug, divergence runs in both directions, one novel serves chapters under two
slugs, and neither slug is computable from the other. The pointer was checked on 8 chapter
pages and agreed every time. Three narrower selectors are recorded as rejected, each with
the measurement that killed it.

Three rejected options are recorded with reasons: correcting the stored address only, which
keeps an identity the Site does not guarantee; scoping the scan to a container, which the
probe refuted; and a SQL migration, which is impossible because the database holds no
source for the correct slug.

**`CONTEXT.md`**

- **Series** - identity is the canonical slug the Site publishes, never the title and never a Chapter Slug.
- **Chapter Slug** - new term. A slug a Site builds its chapter addresses from. Not an identity: one Series may have several, and none is computable from another.
- **Latest Chapter** - now the highest-numbered chapter, explicitly not a date and not the Site's own newest-chapter banner. Settles #79.

**`docs/research/lightnovelworld-chapter-vs-series-slug.md`** (new, committed with its corrections)

The 41-novel survey behind the ADR. Two claims are struck through and corrected in place,
with the date and sample size of the probe that refuted each: the `ul.clstyle` container it
named is the hidden, empty "Latest Reading" template rather than the chapter list, and its
caveat about the comment region understated the risk, because that region is writable by
any visitor while the scan takes an unbounded maximum into a Series row shared by every
Reader (ADR-0003).

## Review notes

Nothing here constrains code that exists today - the ADR describes work not yet written.
The part worth disagreeing with, if any of it is wrong, is the fail-closed rule: a missing
truncation marker means skip the Series and log, never scan the whole page.

Related: #77 (the defect), #80 (the spec), #79 (the numbering anomaly, closed by decision),
#71 (the same size cap seen from the cover side).

Reviewed-on: #81
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-11 09:34:26 +07:00
sulthan 1ee5eb67ea Clear the stale Chrome singleton lock at browser boot (#76)
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 19:11:24 +07:00
sulthan b22ae82897 Restore the novel script's lost module-scope constants (#74) (#75)
> *This was generated by AI during triage.*

Fixes #74.

`novel-bookmark.user.js` was split out of `manga-bookmark.user.js` and lost four module-scope constants. Every use of them is behind a `try/catch` or a fire-and-forget promise, so the `ReferenceError`s were swallowed rather than reported.

| constant | used at | effect while missing |
| --- | --- | --- |
| `LATEST_CHECK_THROTTLE_MS` | `:722` | `backgroundRefreshLatest()` throws before computing `due` — no background latest-check ever runs for novels (the symptom in #74) |
| `LATEST_CHECK_BATCH` | `:724` | same throw |
| `CACHE_KEY` | `:215`, `:224` | `loadCache()` always returns `[]`, `saveCache()` silently no-ops — the local cache never persists |
| `LASTCHECKED_KEY` | `:232`, `:241` | last-checked map never persists, so the throttle would not hold even once the first two are defined |

#74 named only the two throttle constants. The two cache keys are the same lost lines with the same root cause, so they are restored here too — fixing only the pair the issue named would leave `backgroundRefreshLatest()` re-fetching every series on every navigation, because `saveLastChecked()` would still be a no-op.

Values and comments copied verbatim from `manga-bookmark.user.js:37-43`; throttle 4h, batch 1.

## Verification

- `node --check userscript/novel-bookmark.user.js` — clean.
- `node --test userscript/test/logic.test.js userscript/test/novel-logic.test.js` — 47/47 pass.
- New test `every SCREAMING_CASE constant the script uses is declared in it` scans both scripts (comments and string literals stripped first, so prose and SVG path data do not trip it). Confirmed it fails — 1 failing test — when `LATEST_CHECK_BATCH` is deleted again, and passes when restored.

A behavioural test cannot reach this: the storage helpers and the background refresh are exactly the layers the harness does not cover (see the `testing-the-userscript` skill), and the errors are swallowed anyway. A static guard is the only instrument that sees this bug class.

On-device confirmation that the ember now lights for novels is still outstanding — that needs Violentmonkey against a live novelfull/lightnovelworld page.

Reviewed-on: #75
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 18:12:03 +07:00
sulthan 7c7d597019 Delete the kagane-specific cover path (#63) (#73)
Closes #63

Deletes the second way to reach a Cover. Since #62, every Site's cover bytes land in the content-addressed store at creation or on the poll, and the one public route serves them all — nothing needs the kagane proxy anymore.

## What went

- **Template-level rewrite:** `Bookmark.CoverURL()` and both templates' use of it. Cards and chrome now render `.Cover` — the wire value — and nothing else. `Bookmark.CoverSource` was dead once `CoverURL` went, so it and its `bookmarkColumns` entry are gone too.
- **Kagane-only cover route and its identifier validation:** `GET /img/kagane/{id}`, `web.CoverFetcher`, `coverIDRe`, and the whole `internal/web/cover.go`.
- **The proxy's persistence:** `store.KaganeImageID`, `GetKaganeCover`, `PutKaganeCover`, `kaganeCoverSourceURL`, `kaganeCoverRe`.
- **The kagane-shaped branch in the byte-fetch routing:** `fetchCoverBytes` no longer takes a `site` argument and no longer names a Site. The URL shape kagane's API publishes is claimed by the browser module itself — `kaganeImageURLRe` + `browserCoverURL` live in `latest/browser.go` with the rest of the per-Site knowledge — and `BrowserFetcher.Image` is now URL-driven (it validates the URL it will navigate to, same SSRF discipline as before). The no-plain-TLS-fallback rule for a claimed URL is preserved: a claimed address with no browser is an error, never a challenge-page fetch.

## What stayed (deliberately)

- `BrowserFetcher.Image` and the browser-backed acquisition path: kagane genuinely serves cover bytes behind the challenge + `cross-origin-resource-policy: same-origin`, so the sidecar remains the only fetcher for them — it just routes by URL claim now instead of by Site name.
- `fetcherFor`'s per-Site page routing (kagane/novelfull page fetches) — that is the page path, not a cover path.

## Acceptance criteria

- [x] Template-level kagane cover rewrite gone
- [x] Kagane-only cover route and its identifier validation gone
- [x] Tests removed/rewritten against the general route, guarantees kept: unstored + traversal-shaped addresses serve nothing (`TestPublicCoverRejectsUnknownAddress`), non-image content types never echoed (`TestPublicCoverNeverEchoesNonImage` — new; the store-side gate was already pinned by `TestCoverStoreAcceptsAnySourceURL`). Store reopen-persistence and filesystem content-addressing tests rewritten against `PutCover`/`GetCover`, no guarantee lost.
- [x] No Site name in a cover code path outside the acquisition module (`grep kagane backend`: store/web/templates/api are clean; remaining hits are `latest/browser.go` + `latest/sites.go`, tests, docs)
- [x] Web UI and panel render Covers for all six Sites (templates render the wire address; panel renders `b.cover` — untouched, it never had a kagane path)
- [x] `go test ./...` green

## Verification

- `go vet ./...` clean
- `go test ./...` — all packages pass (root 16.9s, latest 12.7s, store 12.7s, web 0.004s)
- `CGO_ENABLED=0 go build` produces the static binary
- Cover-path tests run verbosely: `TestPublicCoverServesStoredBytesUnauthenticated`, `TestPublicCoverRejectsUnknownAddress` (unknown/malformed/traversal/empty), `TestPublicCoverNeverEchoesNonImage`, `TestListRendersAcquiredCover`, `TestAcquireKaganeCoverThroughBrowser`, `TestRunOncePrefetchesKaganeCover`, `TestRunOnceRoutesNonKaganeCoverToPublicFetcher` all pass; the three `SMOKE_*` tests skip without the browser sidecar, as designed

Live browser verification of the "web UI and panel render Covers for all six Sites" criterion is being run separately with Playwright against real Site pages and a locally mocked backend.

Reviewed-on: #73
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 18:02:47 +07:00
sulthan 78234f3c19 Browser-backed Sites join the Cover pipeline (#62) (#72)
Fixes #62

Browser-backed Sites join the Cover pipeline: kagane and novelfull Series now get their Covers at creation, through the same acquisition path as every other Site, instead of waiting for a poll pass.

## What changed

`latest.Acquirer` (creation-time acquisition, fired by the first Bookmark of a Series) previously skipped kagane and novelfull entirely — their pages only yield a Cloudflare challenge to the TLS client, so the request was spent for nothing. It now routes them like the poller does, with the two Sites split exactly as the issue demands:

- **kagane** — page fetched through the browser sidecar, cover URL extracted from the API JSON, bytes fetched through the browser sidecar (the only path that clears the challenge) into the content-addressed store. With no `BROWSER_WS_URL` configured, acquisition is skipped entirely and nothing falls back to a plain fetch.
- **novelfull** — page fetched through the browser sidecar, cover URL extracted from the HTML, bytes fetched over plain TLS through the ordinary gated fetcher (its image paths answer 200 with `access-control-allow-origin: *`, measured 2026-08-09). With no browser configured, the page fetch falls back to the TLS client — novelfull's challenge is a live time-varying fact (AGENTS.md), so when the page body answers, the Cover still lands; when it is challenged, nothing happens.

The byte-routing rule (kagane → browser, every other Site → TLS) is now one shared function (`latest.fetchCoverBytes`) used by both the Poller and the Acquirer, so the two cannot drift apart.

## Acceptance criteria

- [x] kagane cover bytes are fetched through the browser sidecar and stored in the content-addressed store — `TestAcquireKaganeCoverThroughBrowser`
- [x] novelfull cover URLs are extracted from the browser-fetched HTML, and its bytes are fetched over plain TLS — `TestAcquireNovelfullCoverOverPlainTLS`
- [x] With no browser sidecar configured, kagane Covers are absent and nothing falls back to a plain fetch — `TestAcquireKaganeSkippedWithoutBrowser`
- [x] With no browser sidecar configured, novelfull Covers still work if its page body is available — `TestAcquireNovelfullCoverWithoutBrowser`
- [x] Manually verified on-device: a kagane Series shows its Cover in the panel, not a broken-image glyph — being run by a separate manual-verification agent against a mocked scenario (no prod data); not part of this PR
- [x] `go test ./...` is green, with live-network checks gated behind `SMOKE_BROWSER_WS_URL` like the existing kagane image smoke test — new `TestSmokeAcquireKaganeCover` proves the end-to-end acquire path against the real browser when the env var is set

## Verification

- `go test ./...` green across all packages
- New unit tests exercise every routing decision with fakes — no network in the default suite
- Smoke test gated behind `SMOKE_BROWSER_WS_URL`, skipped by default

## Post-review changes (a66491a)

- **One routing rule for pages too** — `fetcherFor` is now a shared function used by both the Poller and the Acquirer; novelfull falls back to the plain-TLS fetcher in *both* when no browser is configured, so pre-existing (client-scraped) novelfull rows get healed by the poll as well, not just Series created after this change (`TestNovelfullUsesTLSWhenNoBrowserFetcher`).
- **Byte-level no-fallback proof** — `TestAcquireKaganeBytesNeverFallBackToPlainTLS` pins that kagane cover bytes never route to the TLS fetcher even when the page came through a browser.
- **Acquirer wired independent of the TLS client** — if `NewTLSFetcher` fails, kagane/novelfull acquisition still works via the sidecar (`main.go`).
- AGENTS.md (root + backend) updated for the novelfull plain-TLS fallback.

Reviewed-on: #72
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 11:06:36 +07:00
sulthan b9220b3dfc The Poll fills blank Covers for every Site and both Libraries (#70)
Closes #61.

## Summary

Permanently-blank Series (the half of #47 that creation-time acquisition cannot reach) heal on the next due poll cycle. The cover path is no longer kagane-only: every Site and both Libraries fill a blank Cover from the series page the chapter poll already fetched, and never replace a Cover that already exists.

## What changed

### `backend/internal/latest/poller.go`

- **`fillBlankCover`** — when `Cover` and `CoverAddress` are both blank, extract a source URL via `coverFrom` from the series-page body and store bytes through `SetSeriesCover`. Skips any Series that already has a source URL (owned by prefetch) or a stored address (never overwrite).
- **`prefetchCover`** — source-URL healing path, now site-uniform. Kagane no longer special-cases into `PutKaganeCover` alone; every Site lands on `SetSeriesCover`, so the wire Cover becomes a content-addressed public URL. Reuses already-stored bytes when present.
- **`storeCover` / `fetchCoverBytes`** — shared fetch+persist. Only kagane routes image bytes through the browser fetcher; every other Site uses plain TLS `CoverBytesFetch`. Failures log with the Series key and never return to the chapter path.
- **`checkOne`** — after a successful series-page fetch, calls `fillBlankCover` once regardless of whether chapter extraction succeeded (cover fill is independent of the chapter signal).

### `backend/internal/latest/poller_test.go`

Extended the existing poller harness (real store, fake fetchers) rather than a new one:

- `TestRunOnceFillsBlankCoverFromSeriesPage` — asura manga, lightnovelworld novel, kagane manga; asserts wire Cover + correct fetcher routing.
- `TestRunOnceDoesNotReplaceExistingCover` — second poll does not refetch.
- `TestRunOnceRetriesFailedBlankCoverOnNextPoll` — failed fill stays blank, next due cycle retries (no separate queue).
- `TestRunOnceBlankCoverFailureDoesNotBlockChapter` — chapter still lands; failure log carries the Series key.
- Kagane prefetch test now also asserts the content-addressed wire Cover.

## Acceptance criteria (#61)

| Criterion | Status |
|---|---|
| Cover prefetch runs for every Site | done |
| Cover prefetch runs for both Libraries | done |
| Poll fills a blank Cover | done |
| Poll never replaces an existing Cover | done |
| Failed cover fetch does not fail/block chapter poll | done |
| Failed cover fetch retried next poll, no separate queue | done |
| Failures logged with the Series | done |
| Existing poller tests extended | done |
| `go test ./...` green | done |
| Manually verified: blank Series gets Cover after a poll cycle | **left for you** |

## Out of scope / not closed

- Does **not** close #47 or #55 (per ticket).
- No migration/backfill script — the Poll walks every Series already.
- No admin refetch (#54).

## Review notes addressed

- Removed the kagane-only `PutKaganeCover` branch from prefetch so source-URL healing also sets `CoverAddress` (wire Cover).
- Guard so `fillBlankCover` does not double-fetch after `prefetchCover` healed the same snapshot.
- Single `fillBlankCover` call site after the series-page fetch.

## Test plan

- [x] `go test ./...` (backend; needs Docker/Postgres via `pgtest`)
- [ ] After deploy: pick a Series that was blank, wait one poll cycle, confirm Cover in web UI and userscript panel

Reviewed-on: #70
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 10:14:27 +07:00
sulthan e2c054e7ce Covers render in the userscript panel, from a public route (#60) (#69)
Closes #60.

Spec: #55. Originating bug: #47. Architecture: `docs/adr/0007-backend-hosts-cover-bytes.md`. Neither #47 nor #55 is closed from here.

## What this branch does

The panel now renders Covers from the deployment's own origin, and both userscripts stop having an opinion about where a Cover lives.

**The public route was already in place.** `GET /covers/{address}` landed with #59 (`92eba07`) and is registered on the bare mux, outside `httpmw.Auth` and outside the web UI's Discord session — `backend/main.go:210-214`, handler `backend/internal/api/handlers.go:142-158`. It reads no cookie and no header, answers `404` for an address that was never stored (and for a row whose file has gone missing — recorded-but-gone is not-found, never a fabricated body), refuses anything that is not `^[0-9a-f]{64}$` *before* the value becomes a path, and sets `Cache-Control: public, max-age=604800, immutable`. Those four properties are asserted by `backend/cover_test.go:231-278`. This branch re-verified them rather than re-implementing them; the only backend line it touches is a comment.

**Both userscripts lose cover scraping entirely.** Every adapter's `cover:` field is gone, along with the two helpers that fed them: the manga script's `coverFromPage()` (the `img[alt]` DOM scan comix needed, because comix publishes no `og:image`) and the novel script's `metaName()` plus the now-callerless module-level `meta()`. Nothing under `userscript/` reads `og:image`, `meta[name=image]`, or `img[alt]` any more.

**Nothing sends a cover either.** `delete body.cover` sits in `apiPut` — `manga-bookmark.user.js:486`, `novel-bookmark.user.js:275` — which is the single chokepoint every write passes through (`pushBookmark`, the retry-queue flush, `toggleFavorite`, `toggleArchive`). It operates on the `Object.assign` copy, so the in-memory row keeps the cover it renders with. This matters beyond tidiness: a Reader upgrading from an older copy has `localStorage` rows carrying third-party scraped URLs, and without the strip those would ride back up on the next write. The handler discards the field regardless (`handlers.go:53-59`) — it is permanently inert, not pending removal.

**Failed loads get the designed empty state, not the broken-image glyph.** `onerror: (e) => e.target.replaceWith(el("div", { class: "cover ph" }))` on the cover `<img>` in both card renderers (`manga:1380-1390`, `novel:1134-1144`). The replacement is byte-identical to the existing no-cover branch on the very next line, so it picks up the `.cover.ph` styling already in the panel CSS — no new tokens, no new rule. `el()` routes any `on*` prop through `addEventListener`, so this is a listener, not an inline attribute string, and the swap is a `createElement` + DOM call with no markup parsing anywhere near it. This is the half of #47 that was visible on kagane.

**The deleted scraping's tests went with it**: the two comix cover cases, the `pageImages` and `namedMetas` fixtures, the `img[alt]` and `meta[name=...]` stub branches, the now-dead `querySelectorAll` stub member, and every stale `og:image` fixture and `p.cover` assertion across both suites. The export lists needed no change and that was checked, not assumed — `coverFromPage` and `metaName` were module-private on `origin/main` and no cover symbol ever appeared in `module.exports`.

Docs that described the deleted behaviour were corrected in the same breath, because leaving them would instruct the next agent to put the scraping back: `userscript/AGENTS.md` (adapter contract + the per-site notes for comix, kagane and novelfull), the README's adapter reference, and the userscript testing skill's stub table.

## Verification

- `go test -count=1 ./...` — green across all nine packages (`backend` 29.8s, `latest`, `store`, `session`, `token`, `userscript`, `web`).
- `node --check` clean on both userscripts; `node --test` on both logic suites — 46 tests, 46 pass.
- `gofmt -l` clean; `go build ./...` clean.
- The `onerror` swap is DOM behaviour and deliberately has no coverage in the Node harness — that harness stubs a browser precisely so it never needs a DOM, and #60 says not to invent coverage for it. It was instead exercised for real: the `el()` helper and the exact render expression were loaded into a headless Chromium with a deliberately unloadable `src`, and the resulting DOM was `<div class="cover ph"></div>`. Ad hoc, not committed.
- **Not done, needs you:** the on-device criterion — a comix Series bookmarked mid-chapter showing its Cover in the panel. That needs a real install against the deployment and is the one box left unticked on #60.

## Reviewed

Both `/code-review` axes ran against `cc0fa92`. Spec found no missed requirement and no scope creep; standards found the diff clean on the four areas it scrutinised (the `delete body.cover` placement, the `onerror` handler's DOM safety, comment quality, dead-code removal). Their combined findings — the dead `querySelectorAll` stub, the stale README and skill text, and the handler comment whose premise this change invalidates — are fixed in `8b58019`.

## Out of scope, deliberately

The kagane-specific cover proxy still exists and still carries its session gate (#63 deletes it). The poll's blank-Cover fill (#61) and browser-backed Sites joining the pipeline (#62) are untouched.

Reviewed-on: #69
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 08:52:57 +07:00
sulthan 92eba07da7 A newly bookmarked Series acquires its Cover at creation (#59) (#68)
Closes #59.

Part of spec #55, and the ticket that fixes the reported bug #47. Architecture: `docs/adr/0007-backend-hosts-cover-bytes.md`. Does not close #47 or #55.

## What changed

A Reader bookmarks a Series nobody holds yet — the exact case in #47 — and within seconds the list shows its artwork instead of a broken image. The first Bookmark to create a Series fires `Store.OnSeriesCreated` after commit, and the new `latest.Acquirer` turns that into **one** series-page fetch that yields both the Latest Chapter and the cover URL. The bytes go through the gated cover fetcher from #57 and are stored content-addressed through #56, so the wire carries an absolute URL on this deployment's own origin — never a third-party address, and never one that 404s.

### Store

- Migration `0009_series_cover_address.sql` adds `series.cover_address`. The two facts are now split: `series.cover` is the third-party source address the bytes came from (the acquisition path's dedupe key), `series.cover_address` is the SHA-256 they are stored under. An empty `cover_address` is precisely what "no Cover yet" means, which is the distinction both the API and the UI depend on.
- `SetSeriesCover` writes the address only after the bytes are on disk, so the wire can never name an object that is not there.
- `CoverWireURL` builds `PUBLIC_BASE_URL + /covers/<sha256>` for every scanned row, and returns `""` for a blank address.
- The cover columns are gone from `Upsert`'s `INSERT` and its `DO UPDATE`. A client-supplied cover cannot reach the shared Series row on any path, not just the creation path.
- `Open` now rejects a base URL that is not an absolute `http(s)` origin: `PUBLIC_BASE_URL=bookmarks.example.com` would otherwise start cleanly and emit addresses no browser can load.

### Acquisition

- `internal/latest/acquire.go`: one fetch, gated by the poller's own `fetchableSeriesURL` (a `series_url` arrives in a client-supplied PUT body, so without the gate a token-holder chooses what the server fetches from its own network position).
- Asynchronous and log-and-drop. The Bookmark, its progress and its Latest Chapter are already committed; a Site that is down or a cover that cannot be produced disturbs none of them.
- Bounded by a two-slot semaphore. A bulk sync creating N Series would otherwise fire N simultaneous requests from one IP — the traffic shape the poller's stagger exists to avoid.
- Cancelled at shutdown (shares the poller's context) and stamps `latest_checked_at`, so the poller does not refetch the same page a tick later.
- Browser-backed Sites (kagane, novelfull) are deliberately skipped: their pages only yield a Cloudflare challenge to the TLS client, so the request would be spent for nothing. They arrive in #62.

### Wire and route

- `GET /covers/{address}` serves the bytes publicly and uncredentialed with `Cache-Control: public, max-age=604800, immutable`. The address is gated by a `^[0-9a-f]{64}$` pattern and cross-checked against a pure function of itself before any filesystem read, so no request shaped like a traversal reaches disk.
- `PUT /bookmarks/{key}` still accepts a `cover` field and discards it, permanently. Rejecting it would break every installed userscript the moment this deploys, and ADR-0004's compatibility argument depends on those scripts continuing to work. The decode site says so in place of a TODO nobody intends to keep.
- `store.CoverContentType` canonicalises comix's non-standard `image/jpg` to `image/jpeg`, so one image cannot land under two spellings. This one was found by the live smoke test, not by reading.

### Config

`PUBLIC_BASE_URL` is new and required (cover URLs must go out absolute — the userscript renders them on third-party origins, where a relative path resolves against the Site). Documented in `.env.example`, `docker-compose.yml` (`:?` so compose fails too), `DEPLOY.md` and `backend/AGENTS.md`.

## Acceptance criteria

All twelve of #59's criteria are met; the checklist on the issue is ticked with the evidence.

## Verification

- `go test ./...` green (Docker-backed Postgres suite).
- Live smoke against a real backend + Postgres: bookmarking `comix:n8we-dungeons-and-crayons` produced `"cover": "http://127.0.0.1:8099/covers/8ce74d80…"` and `"latest_chapter": "Chapter 81"` within seconds of the PUT; `curl` on that address returned `200`, `Content-Type: image/jpeg`, `Cache-Control: public, max-age=604800, immutable`, and a 280x420 JPEG. That run is what surfaced the `image/jpg` content type.
- Mutation-checked the asynchrony test: removing the `go` from `Acquire` turns `TestAcquireDoesNotBlockTheWrite` red.

## Reviewed

Both axes of `/code-review` were run against this diff before commit. Their findings that were actionable here are folded in: the concurrency bound, the shutdown tie, the `PUBLIC_BASE_URL` validation, the missing `latest_checked_at` stamp, and a test that could not fail.

## Known sequencing

A kagane/novelfull Series created between this deploy and #62 has no cover source at all: the acquisition skips those Sites and `Upsert` no longer persists the userscript-scraped address. This is #59's stated boundary rather than a defect, but it is a user-visible gap on two Sites and should order #62 accordingly.

Reviewed-on: #68
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 04:07:53 +07:00
sulthan b6b88bde8a feat(latest): extract per-site covers (#58) (#67)
Closes #58

## Summary

- Add pure per-Site cover extraction beside latest-chapter parsing for all six Sites.
- Read Asura, Demonic, LightNovelWorld, and NovelFull metadata; read the Comix target detail state; read Kagane's browser-fetched `series_covers[].image_id` JSON.
- Preserve published cover URLs, percent-encode Demonic raw spaces, select Comix's smaller published `medium`, and avoid thumbnail rendition URL synthesis.
- Add live-source fixtures plus no-cover and Cloudflare challenge coverage for every Site.

## Correctness

- Scope Comix extraction to the requested series detail key, avoiding recommended posters.
- Parse Kagane's current live API shape and emit its canonical compressed image route from the published image ID; unrelated JSON fields are ignored.
- Validate Kagane image IDs against the existing UUID-shaped route constraint.
- Keep extraction pure; storage, polling, and wire integration remain outside issue #58.

## Acceptance criteria

- [x] Cover extraction exists for all six Sites in the existing latest parser module.
- [x] Each Site has a live-source fixture with source URL and date.
- [x] Comix reads the state blob, not metadata.
- [x] Demonic raw spaces are percent-encoded.
- [x] Comix returns the smaller published rendition.
- [x] No-cover pages return empty.
- [x] Cloudflare challenge pages return empty.
- [x] No thumbnail URL is synthesized by editing a published URL.
- [x] `go test ./...` passes.

## Verification

- `go test ./...`
- `go vet ./...`
- `git diff --check`

Parent issues #47 and #55 remain open as requested.

Reviewed-on: #67
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 01:43:57 +07:00
sulthan 9d6d3bde72 Add gated cover byte fetcher (#66)
## Summary

Adds a plain-TLS cover byte fetcher with a destination-class SSRF gate and wires public cover sources through the content-addressed filesystem store.

## Changes

- Resolve hostnames before connecting; refuse non-HTTPS, loopback, private, link-local, unique-local, CGNAT, credentials, and mixed public/private DNS answers.
- Re-check every redirect and resolve/classify again at dial time to close DNS rebinding.
- Reuse `maxBodyBytes`; reject oversized responses and non-image content types before persistence.
- Add generic `Store.GetCover`/`PutCover` source-URL storage while preserving the browser-backed kagane path.
- Keep cover prefetch failures isolated from chapter polling.
- Add observable tests for TLS, no-connection refusals, all refused address classes, redirect blocking, streaming body caps, non-image rejection, content-addressed persistence, DNS rebinding, and poller routing.

## Verification

- `go test -count=1 ./...`
- `go vet ./...`

Both pass. No test touches the live network.

Closes #57

Reviewed-on: #66
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 00:45:55 +07:00
sulthan e8d1cba6c5 Move cover bytes to content-addressed filesystem storage (#65)
Refs #56

## Summary

Moves Kagane cover bytes out of Postgres bytea storage into an immutable, content-addressed filesystem store. Reader-visible behavior remains unchanged: the existing session-gated route serves stored bytes, missing bytes use the existing browser fetch path, and no browser still returns a missing cover.

## Changes

- Added migration 0008, which drops the legacy `covers` table and recreates it with only `address`, `path`, and `content_type`. Existing byte rows are intentionally dropped.
- Added SHA-256 source-URL addressing with two-level sharding (`ab/cd/<sha256>`). Writes use a temp file plus atomic link; reads validate the stored relative path before opening it.
- Made `COVER_DIR` required in runtime config and Compose. Compose passes it as a Docker build argument and volume target, so custom durable paths keep image ownership, runtime config, and the named `cover-data` volume aligned.
- Updated every `store.Open` caller and documented configuration, deployment, backup, and troubleshooting behavior.
- Added filesystem, restart, migration-drop, no-browser, and content-addressing coverage.

## Verification

- `go test ./...`
- `CGO_ENABLED=0 go build ./...`
- `docker build --build-arg COVER_DIR=/data/covers -t manga-bookmark-cover-check-custom ./backend`
- `docker compose config --format json` confirms custom `COVER_DIR` is the volume target
- `git diff --check origin/main`
- LSP diagnostics clean for touched Go files

Parents #47 and #55 remain open as required by #56.

Reviewed-on: #65
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-10 00:11:20 +07:00
sulthan 30c57bd39c Define Cover and record hosting its bytes (#47) (#64)
Defines **Cover** in the glossary and records ADR-0007, the decision behind #47's fix.

## Why these two files, and why now

`CONTEXT.md` named Cover inside the **Series** entry — "facts true regardless of who is reading — title, cover, Latest Chapter" — but never said *what* one is. That gap is the bug. Nothing in the model distinguished "an address on a Site" from "an image a Reader's browser can display", so both clients were left to work it out independently, and one of them got it wrong. kagane serves covers with `cross-origin-resource-policy: same-origin`, the web UI rewrote them to a proxy in its templates, the JSON API did not, and the panel rendered a broken-image glyph. The new entry closes the ambiguity: *an address no client can load is not a Cover, it is a missing one.*

ADR-0007 records what follows from that — the backend fetches, stores and serves every Site's cover bytes — plus the alternatives that were rejected and, more importantly, the two places this deliberately departs from existing precedent:

- **Destination-class control instead of a host allowlist.** `fetchableSeriesURL` sets the allowlist precedent for `series_url`, and covers do not follow it. Cover hosts are CDNs that move independently of their Site — demonicscans serves its covers from `readermc.org` — so an allowlist would stop producing Covers the day a Site switched CDN, and that failure would look exactly like #47. The resolve-then-classify step is what actually stops the SSRF.
- **A public cover route where the kagane proxy is session-gated.** An `<img>` cannot send a bearer token, and it cannot be given one either: the panel's shadow root is `mode: "open"`, so the host page's JavaScript can read any `src` the script sets.

Both are security-adjacent departures, which is precisely why they are written down rather than left in a commit message.

## Scope

Documentation only — no code, no schema, no behaviour. The implementation is #56–#63.

## Why this should merge promptly rather than sit

All eight implementation tickets cite `docs/adr/0007-backend-hosts-cover-bytes.md` as the authority for decisions they must not relitigate, and they are written in the vocabulary this glossary entry defines. An agent picking up #56 reads both from `main`. Until this lands they get a 404 and either invent a rationale or stall — so this PR gates the tickets, not the other way round.

Related: #47 (bug), #55 (spec), #54 (deferred admin refetch).
Reviewed-on: #64
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-09 23:19:37 +07:00
sulthan 8081a0a5d8 Give the Tailscale ACL step a working policy file (#53)
Follow-up to #52, which merged before this landed. Docs only — no code, no compose changes.

`DEPLOY.md` §7 told the operator to "tag the two machines" and showed a bare `acls` fragment. Following it literally does not work and is actively harmful:

- the fragment references `tag:bookmark-api` / `tag:bookmark-browser` without a `tagOwners` section, so the policy is rejected on save;
- it never says how a tag gets onto a device (`tailscale up --advertise-tags=...`, which re-authenticates);
- replacing the tailnet's default allow-all with only that one rule **removes the operator's own SSH access to the browser machine**.

Replaced with a complete, saveable policy file: `tagOwners`, the CDP rule, a second rule preserving own-device access including `:22`, and a `tests` block so a later edit that widens 9222 is rejected rather than silently applied.

Also records two things that were assumed rather than stated:

- **Why tagging is load-bearing.** Tailscale has no `deny`, so restricting 9222 means removing the blanket accept and enumerating what remains. That is only expressible if the browser machine falls outside a selector that still covers your own devices — which is exactly what a tag does, since a tagged device has no user and stops matching `autogroup:member` / `autogroup:self`. Without that, the whole step reads as arbitrary ceremony.
- **Tagging replaces a device's user identity**, so it suits a dedicated box and disrupts a daily driver. Both paths are now written down.

Finally, separates two checks the old text conflated: the existing `curl` runs on the home machine and proves only the **bind**, because node-local traffic is not filtered. Proving the **ACL** needs a third device, so that check is now its own step.

Verified: `tailscale.com/docs/reference/syntax/policy-file` and `/docs/features/tags` (validated Apr 2026 / Dec 2025) for `tagOwners`, `autogroup:self` semantics vs tagged devices, `--advertise-tags` re-auth and key-expiry behaviour. Markdown fences balanced.
Reviewed-on: #53
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-09 16:12:03 +07:00
sulthan 2a3bb6922d Move the browser off the VPS to its own unit (#46) (#52)
Closes #46 once deployed.

The headless browser leaves the API stack and becomes its own compose unit
(`chrome/docker-compose.yml`) intended for the home machine, reached over the
tailnet. No fallback sidecar is left on the VPS.

The backend needs no code change — `BROWSER_WS_URL` was already the only
coupling. Its default is now empty rather than a pinned Docker IP, so an
unconfigured or unreachable browser degrades exactly as it always has: plain-TLS
libraries unaffected, kagane/novelfull logged and skipped, stored covers still
served.

### What shipped

- `chrome/docker-compose.yml` + `chrome/.env.example` — the browser unit, with
  the CDP port bound to `${BROWSER_BIND_ADDR}` (no default) and the resource
  limits from the epic: 512 MiB / 1 GiB memory+swap, `oom_score_adj 800`,
  halved CPU weight, shm 1 GiB -> 128 MiB.
- API stack drops the service, its `depends_on` and the `browser` network.
- `bookmark-api` gains the `default` network. Dropping `browser` had left it on
  `db` alone, which is `internal: true` — no published port and, worse, no
  egress for the poller at all. Caught by actually bringing the stack up.
- ADR-0006 for the topology; `DEPLOY.md` §7 for first-time setup of the browser
  machine; `REDEPLOY.md` §8 for its independent update cadence; architecture
  diagrams, config tables and troubleshooting rows across README/AGENTS/env.

### Verified locally

- Browser unit builds and runs: Chrome 151, UA carries no `HeadlessChrome`,
  all limits applied as declared.
- **Live smoke passes through the new unit**: `TestSmokeKaganeImage` fetched
  56710 bytes of `image/webp`, `TestSmokeKaganeGet` got a 200 with a real
  chapter list. The challenge cleared under the reduced 128 MiB shm.
- Bind isolation proven: refused on the host's non-loopback address, accepted
  on the configured one.
- 321 MiB peak of the 512 MiB cap after a full solve; 0 restarts, no OOM kill.
- API stack comes up clean, `/healthz` 200; egress confirmed present on
  `default` and absent on `db`.
- `go test ./...`, `go vet`, `gofmt` clean.

### Left to the operator

Provisioning the home machine, the Tailscale ACL, setting `BROWSER_WS_URL` in
production, and observing acceptance criteria 5-7 (covers with the machine off,
several days of zero OOM/restarts, VPS memory improvement). `DEPLOY.md` §7 now
carries the before/after `free -m` reading those need.

Reviewed-on: #52
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-09 15:28:21 +07:00
sulthan d1800d0707 Prefetch Kagane covers during latest polling (#51)
## Summary
- Add an optional browser-backed cover fetcher to the latest-chapter poller.
- Prefetch missing Kagane covers during the existing due-series cycle and persist them before a Reader opens the web UI.
- Keep chapter polling, cooldown stamping, and on-first-view fallback independent from cover failures.

## Behavior and safety
- Stored Kagane covers are detected before browser work, so later poll cycles do not refetch them.
- Nil cover fetchers and non-Kagane series retain the existing behavior.
- Shared Kagane image-id and content-type validation prevents challenge or non-image responses from poisoning persistent cover storage.
- The browser is wired into both the chapter and cover poller paths from the composition root.

## Verification
- `go test ./...`
- Focused latest, store, and web package tests
- Deterministic tests cover missing covers, cached covers, failed fetches, invalid content types, nil fetchers, and non-Kagane series.

Closes #45

Reviewed-on: #51
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-09 14:58:59 +07:00
sulthan 84cfd1b2c1 Make browser sidecar on-demand (#44) (#50)
Closes #44. Chrome now starts on first CDP connection, tracks concurrent helpers, reaps after 300 seconds idle, preserves the named profile, and classifies reap interruptions. Shutdown stops Chrome's process group so cookie batches flush. ADR-0005 records the measured constraints and decisions. Verification: docker build, live CDP wake, graceful stop cleanup, sh -n, and go test ./... (7 packages, 3 no tests).

Reviewed-on: #50
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-09 07:39:07 +07:00
sulthan bfae84c5c3 Persist kagane covers in Postgres (#49)
Closes #43

Persist kagane cover bytes in a dedicated Postgres covers table keyed by image ID. The web handler reads storage before the browser, writes validated fetches through, and no longer keeps an in-process cover cache. Added migration, store persistence tests including reopen, handler coverage for stored/miss/rejected paths, and corrected repository guidance.

Verification:
- go test ./...
- CGO_ENABLED=0 go build ./...

Reviewed-on: #49
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-09 06:52:02 +07:00
sulthan cd3a7e3d01 feat(latest): split browser poll cooldown (#48)
## Summary

Split latest-chapter polling cooldowns by fetch cost. Browser-backed kagane and novelfull series now rest longer without changing the cadence of plain-TLS sites.

## Behavior

- Plain-TLS series keep the 1h default cooldown.
- Browser-backed series use `LATEST_CHAPTER_POLL_BROWSER_COOLDOWN`, defaulting to 6h.
- Both cooldowns share the existing 15m minimum floor; invalid values retain the existing fallback behavior.
- The poller still selects both classes in one due query per cycle.
- Existing ordering and exclusions remain unchanged: reader-count precedence, least-recently-checked ordering, finished exclusion, archived polling, and orphan exclusion.

## Implementation

- Added the browser cooldown to backend configuration and passed it through production poller construction.
- Added the browser-site list as the single routing source used for both due-query cutoff selection and fetcher choice.
- Kept all query values parameterized; the site list is passed as a bound PostgreSQL array parameter.
- Updated startup logging to report interval, plain cooldown, browser cooldown, batch, and stagger.
- Documented the variable, default, and floor in `README.md`, `.env.example`, `backend/AGENTS.md`, and `docker-compose.yml`.

## Review findings addressed

The first review found that configuration parsing was correct but `startLatestPoller` did not pass `BrowserCooldown` into `latest.Poller`; every browser-backed row would therefore have been due immediately. Production construction now goes through `newLatestPoller`, with a regression test covering both cooldown fields.

The review also identified duplicated browser-site knowledge in fetch routing. `slices.Contains(browserBackedSites, site)` now reuses the same list already supplied to the store query.

## Verification

- Focused backend tests pass: `go test ./internal/latest ./internal/store .`.
- Full suite passes: `go test ./...`.
- `graphify update .` completed.
- Issue #42 was updated and closed.

Reviewed-on: #48
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-09 06:25:48 +07:00
sulthan 741b23322b Fix comix titles and covers, kagane volume chapters, and kagane cover rendering (#37)
Fixes five reported symptoms across comix.to and kagane.to. Diagnosing them turned up two latent bugs underneath, both of which had to be fixed for the kagane cover work to function at all.

## Reported symptoms and their causes

| # | Symptom | Cause |
|---|---------|-------|
| 1 | comix bookmark titled `Comix - Read Comics online for free` | comix is an SPA that rewrites `document.title` on client routing but never touches the server-rendered `og:title`. The adapter read `og:title`, so a cold load stored the homepage's title. |
| 2 | next comix bookmark gets the *previous* series' title | Same cause. After an in-page hop, `og:title` still holds whatever page loaded first. |
| 3 | comix cover shows the placeholder | comix serves no `og:image` at all, so `coverFromPage()` had nothing to read. |
| 4 | kagane chapter never appears in the bookmark list | Reader URLs carry no chapter number, so it is parsed out of `og:title`. Volume-numbered series render `"<Series> - Volume <v> Chapter <n>"`, which the suffix regex did not match, so `chapterNum` came back null and nothing was recorded. |
| 5 | kagane title includes the chapter, e.g. `SP Baby - Volume 1 Chapter 1` | Same unmatched regex — the tail was never stripped. One fix covers 4 and 5. |
| 6 | kagane cover blocked in the web UI | kagane serves covers behind its Cloudflare challenge **and** with `cross-origin-resource-policy: same-origin`. No `<img>` on the UI's origin can load one even from a browser holding the clearance cookie. Hot-linking cannot be made to work. |

## What changed

**Userscript.** comix titles now come from `document.title` with the chapter page's `" - Ch.<n>"` tail stripped, and the cover is the `img` whose `alt` matches the cleaned title. comix fills `document.title` a beat *after* the URL changes — later than the nav watcher's 300 ms snapshot — so the watcher also re-detects when the `detect()` signature changes, not only when the URL does. The kagane suffix regex takes an optional `Volume <v> ` segment. All three page shapes were captured live on 2026-08-08 and pinned as regression tests.

**Cover proxy.** `Bookmark.CoverURL()` rewrites a stored kagane `og:image` to `/img/kagane/{id}`; templates render `.CoverURL` instead of `.Cover`. The endpoint is session-gated like every other UI route and fetches through the shared headless browser, which is same-origin with kagane and so satisfies both the challenge and the CORP header. Results are memoised in-process, so a cover costs one navigation per deployment lifetime. With `BROWSER_WS_URL` unset the endpoint answers 404 rather than reaching for a nil fetcher — the same degrade-to-userscript behaviour the poller already has.

The image id is matched against a UUID regex before it reaches the browser. That gate is load-bearing rather than tidiness: the cover is a stored client-supplied string, so an unvalidated one turns this endpoint into an SSRF primitive aimed at the deployment's own network. `ServeMux` path-cleans a traversal into a redirect before the handler runs, but the handler does not depend on that, and a test pins it.

## Two latent bugs found underneath

**`BrowserFetcher.run` never let a challenge solve.** It navigated, waited for `body`, read once, and closed the tab — roughly half a second end to end. The Cloudflare interstitial has a `body` too, so `WaitReady` was satisfied by the challenge page itself. This made the challenge *unclearable* rather than merely slow: an interstitial needs several seconds of a live page to solve itself and write clearance into the browser's shared cookie jar, so tearing the tab down first means every subsequent call is challenged exactly like the one before it. `run` now holds one tab and re-reads until the caller's predicate reports an answer, bounded by `challengeTimeout` and the caller's own deadline. Exhausting the budget maps back to the 403 the poller already expects, keeping a challenged site distinct from a broken transport.

**`chromedp/headless-shell` cannot clear kagane's challenge at all.** It is a stripped Chrome build and the tells are structural rather than a header: `navigator.webdriver` is true, the plugin list is empty, and the client hints are Chromium- rather than Chrome-branded. Overriding `webdriver` through CDP was tried on its own and changed nothing.

All measured 2026-08-08 from one IP against the same cover, so the comparisons are like for like:

| Browser | Result |
|---------|--------|
| `chromedp/headless-shell:stable` | never cleared (90 s) |
| `zenika/alpine-chrome` | never cleared — ships Chrome 124, old enough that Cloudflare refuses it and old enough to break chromedp's CDP structs |
| `google-chrome`, default UA | never cleared (60 s) — `--headless=new` advertises `HeadlessChrome` |
| `google-chrome`, stock UA, `TZ=UTC` | never cleared (90 s) |
| `google-chrome`, stock UA, any non-UTC `TZ` | **cleared in ~4 s** |

Both remaining tells are load-bearing, and each was tested in isolation. `chrome/` is a Debian image with `google-chrome-stable`, a UA whose version is read back out of the binary at startup (a hardcoded one would drift out of step with the `Sec-CH-UA` hints on the next Chrome update and become a fresh tell), and no `--enable-automation`.

### The timezone tell: UTC, not a country mismatch

The first pass concluded the zone had to match the egress IP's country. Re-measuring against the actual deployment case shows that was wrong, and the correction is in `1552dd1`.

The original inference read the host's `/etc/timezone` (`Asia/Bangkok`) and assumed a Thai egress. It isn't — this host egresses from an Indonesian IP. `Asia/Bangkok` cleared not because it matched a country but because it simply isn't UTC, and the two share +07, which hid the distinction. Same container, same Indonesian IP:

| `TZ` | Result |
|------|--------|
| `UTC` | never cleared (60 s, **twice**) |
| `Asia/Jakarta` | cleared in 4 s |
| `America/New_York` | cleared in 4 s |

`America/New_York` matches neither the country nor the offset nor the hemisphere and clears just as fast. A UTC clock is itself the bot signal — Cloudflare scores it as the datacenter default — and any real zone satisfies the check. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one, and a deployment that changes region need not keep it in sync.

One sharp edge remains: the usual `-v /etc/localtime:/etc/localtime:ro` does **not** work. Chrome resolves the zone through ICU, which takes the name from that path's symlink target and ignores the file's contents, so glibc reports the host zone while Chrome still reports UTC. `/etc/timezone` carries the name and is mounted instead.

Chrome also binds its DevTools port to loopback and silently ignores `--remote-debugging-address`, which is why headless-shell fronted it with socat. This image does the same, so it stays a drop-in: the compose service keeps the `headless-shell` name and its pinned address, and `BROWSER_WS_URL` is unchanged.

## Verification

```
go test ./...        all packages ok
node --test          37 + 12 pass, 0 fail

SMOKE_BROWSER_WS_URL=... go test -run TestSmokeKagane ./internal/latest
  TestSmokeKaganeImage  PASS (5.29s)  fetched 56710 bytes of image/webp
  TestSmokeKaganeGet    PASS (1.17s)  status=200, real chapter-list JSON
```

The smoke test ran against the exact compose configuration — built image, empty `BROWSER_TZ`, `/etc/timezone` mounted, cold profile — hitting real kagane.to. It skips unless `SMOKE_BROWSER_WS_URL` names a sidecar, so `go test ./...` stays hermetic and Docker-only.

A red smoke run means the challenge is not clearing from that IP, which is a live, time-varying fact to re-check rather than necessarily a defect.

## Security invariants

- Auth unchanged. `/img/kagane/{id}` is session-gated by `requireSession`, the same guard as every other UI route.
- Outbound fetch gated: the id is UUID-validated before it reaches the browser, keeping the existing rule that a client-supplied string never selects a fetch target unchecked.
- No new secrets, no new logging of credentials, no change to CORS, sessions, or crypto.
- Templates still escape everything; `.CoverURL` returns a plain string and is not wrapped in `template.HTML`/`URL`.
- One new dependency-free image (`chrome/`) built from Debian plus Google's own apt repo; no new Go modules.

## Deploying

Needs `docker compose build headless-shell`.

**A UTC host must set `BROWSER_TZ`, or kagane silently stops working.** With it unset the sidecar falls back to the host's `/etc/timezone`; on a UTC server that yields UTC, which is the one value that never clears. Any real zone works — `BROWSER_TZ=Asia/Jakarta` for the current deployment. `.env.example` now documents this; it previously did not mention the knob at all.

Only the browser sidecar reads `BROWSER_TZ`. The backend keeps its UTC clock, and stored timestamps are unix ms, so nothing else shifts.

## Deliberately not done

Retry/backoff around the cover proxy, and a panel-side cover fix. The panel renders no covers, and covers cache in-process after the first fetch. Worth adding if kagane starts rate-limiting.

## Correction after review of the deployment case

`1552dd1` was added after the branch was first pushed: the deployment host runs UTC with an Indonesian egress IP, which prompted re-measuring the timezone claim and falsifying it. The earlier commits' reasoning is left intact rather than rebased away, so the diagnostic trail — including the wrong turn and what disproved it — stays readable.

Reviewed-on: #37
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 23:27:32 +07:00
sulthan 2ef769d421 Open registration to guild members (#27) (#36)
Closes #27.

Guild membership is now the whole gate. `discordCallback` checks membership
(and `DISCORD_REQUIRED_ROLE` when set), then `Store.EnsureReader` creates the
Reader on first sight and returns the same row on every later login. The
refusal returns before `EnsureReader`, so a turned-away sign-in leaves no row
behind. `OWNER_DISCORD_ID` still seeds the owner, but only as the
administrator — it no longer gates login.

The cutover grace path goes with it: `API_TOKEN`, `API_TOKEN_GRACE_UNTIL` and
the legacy branch in `httpmw.ResolveReader` are deleted, so a credential
authenticates exactly one Reader or nothing. `userscript.Handler` drops its
re-derivation too — the resolved path segment is already the credential.

New surfaces: an empty library offers both install links (behind the
tab-specific empty states, so "No favourites yet" still wins), and the owner
alone gets a Readers panel with `POST /readers/{id}/revoke`. The owner's own
row is not revocable — 404, not a self-logout.

Isolation is asserted from both directions for read, modify and delete, and
the shared-series invariant is pinned: two Readers on one series produce one
series row, two independent progresses, one poll per due cycle, and one
Reader's delete leaves the other's bookmark and the poll intact.

Verified: `go test ./...` green; live smoke against a throwaway Postgres —
empty-library state in both colour branches, roster rendering, a real revoke
through the panel (target 401s next request, owner untouched), owner
self-revoke refused 404, per-Reader `/u/<cred>` and bearer auth both 200 with
404 for an unknown credential.

Reviewed-on: #36
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 20:23:17 +07:00
sulthan c2b47eb05b Offer the userscripts as a download for mobile Violentmonkey (#26) (#35)
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 16:39:30 +07:00
sulthan 1b1820d85a Cut production over: runbook corrections, env contract, Discord OAuth endpoint fix (#26) (#34)
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 16:06:47 +07:00
sulthan 2cc1e69f5d Prove the import against a copy of the real library (#25) (#33)
Closes #25.

Retires the biggest risk in #18 — losing the owner's reading history — on a copy, before production is anywhere near it.

## What was run

A throwaway generator (python3 stdlib `sqlite3`, ~20 lines, **not committed**) read a copy of `bookmarks-20260807-213515.db` and emitted plain SQL: 29 distinct Series first, then 29 Bookmarks referencing them, each `INSERT ... SELECT id FROM owner` so the reader id is resolved rather than hardcoded. The target was a scratch Postgres whose schema and owner Reader were built by the real binary (`go run .` against a throwaway container), not by hand-written DDL. Production was not touched.

## Verified

| check | result |
|---|---|
| Bookmarks total | 29 |
| reading / archived / other | 18 / 11 / 0 |
| Series | 29, equal to the distinct `(site, series_id)` count in the source |
| Readers | 1; Bookmarks not owned by the owner: 0 |
| Field-by-field diff, all 29 rows x 15 columns | 0 differences |
| `GET /bookmarks` over the real read path | 29 rows, values match source |
| `TRUNCATE bookmarks, series;` then re-apply | clean, 29 again |

The spot-check the ticket asked for was widened to a full row-by-row comparison — 29 rows is small enough that sampling was the more expensive option.

## What is committed

`CUTOVER.md` only, plus two cross-links from `REDEPLOY.md`. The generator stays out of the repository: its output is the owner's reading history, and it reads SQLite, which the backend module dropped in ADR-0001. So the runbook specifies the transformation — column mapping, ordering, nullability, quoting, the temp-table ownership trick — rather than shipping a script. `backend/go.mod` gains nothing.

## Review

Two-axis review ran on the diff; six findings applied, all in the runbook:

- Six source columns (`title`, `series_url`, `cover`, `last_chapter`, `last_chapter_url`, `last_chapter_num`) are nullable in SQLite but `NOT NULL` in Postgres and must be coalesced — the opposite of `latest_chapter_num`, the one column where `NULL` is meaningful. The 2026-08-07 export had none; a fresh one is not promised the same.
- `CREATE TEMP TABLE ... ON COMMIT DROP` must sit *inside* the transaction, or psql's autocommit drops it instantly.
- The ownership check now resolves the Reader by Discord id; comparing against `ORDER BY id LIMIT 1` was true by construction and could never fail.
- The spot-check now samples archived and favourite rows explicitly instead of hoping they fall inside `ORDER BY updated_at DESC LIMIT 5`.
- `git pull --ff-only` before `up -d --build`, or a pre-cutover server rebuilds the SQLite image.
- `python3` and `jq` named as prerequisites; column count corrected to sixteen.

Every query in the runbook was executed against the scratch database as written.

`go vet`, `CGO_ENABLED=0 go build ./...` and `go test ./...` all pass — no Go code changed.

Reviewed-on: #33
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 15:15:33 +07:00
sulthan 27cf0955de Per-Reader userscript credential with UI install and rotation (#24) (#32)
Closes #24. Child of #18; based on current main (includes Postgres, Reader table, Discord OAuth).

## What

Each Reader's userscript credential is derived from `TOKEN_KEY`, their Discord id and a token epoch (HMAC-SHA256, hex); only its SHA-256 sits in `readers.token_sha256` (new `token_epoch` column, migration 0006). One credential authenticates the script download path and the API bearer header.

- `internal/token`: derivation + hashing; the seed refreshes the owner's epoch-0 hash only before first rotation, so a restart can never resurrect a rotated-away credential
- `httpmw.Auth`/`ResolveReader`: acting Reader resolved from the credential hash, stashed in request context; the retired global `API_TOKEN` resolves to the owner until `API_TOKEN_GRACE_UNTIL` (enforced in code, logged per use) on both the bearer and script-download paths
- Userscript handler renders the bindmounted file with the resolved Reader's credential substituted for `__API_TOKEN__`; a legacy-path request during grace serves the derived credential, so installed devices self-migrate on their next update poll
- Web UI: "Userscripts" panel — session-gated install endpoints render the script directly (credential never in markup, address bar, or a redirect), confirm-gated rotation with an atomic epoch bump + hash rewrite and a reinstall warning
- Both userscripts carry `__API_TOKEN__` placeholders; the committed global-token literal is removed

## Design note

Credentials are derived rather than stored-random because the server must rebuild install URLs after restarts while the DB holds only hashes. HMAC output is high-entropy and unbrute-forceable; the AC's intent (unguessable, DB-leak-proof) is met.

## Deploy (also in DEPLOY.md)

1. Add `TOKEN_KEY` (`openssl rand -hex 32`) — required; changing it later invalidates every credential.
2. Keep `API_TOKEN` + set `API_TOKEN_GRACE_UNTIL` for the 14-day window.
3. After deploy, sign in → Userscripts → reinstall both scripts on every device. This also retires the old global credential for real — its literal survives in git history (present since 0ef5286), so rotation is what kills it.

## Verification

- Full Go suite green against real Postgres per test; userscript JS suite 45/45
- New router-level tests: per-Reader isolation (read/write/delete), grace expiry on bearer + script path, self-migrating legacy path, install serving, rotation (old cred 401/404, new cred works, install renders new credential), app page leaks no credential
- Store tests: hash lookup, token info, atomic rotation with stale-epoch rejection, rotation survives restart
- Live smoke of the built binary: grace acceptance logged, derived auth, substitution, restart resilience, stored hash = SHA-256 of derived credential

Reviewed-on: #32
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 14:54:03 +07:00
sulthan bcc6b45515 feat(backend): Discord OAuth login with DB-backed sessions (#23) (#31)
Implements #23 per ADR-0002.

- Discord authorization code grant (identify + guilds.members.read), form-encoded token exchange
- Guild membership gate via the single-guild endpoint; optional DISCORD_REQUIRED_ROLE (empty default)
- Owner Discord ID is the only identity allowed to sign in
- Sessions are DB rows with opaque random ids; cookie carries only the id; expiry enforced; delete = revoke
- HMAC session signing, derived key, and WEB_PASSWORD removed; no replacement signing secret
- Login rate limiting preserved on the callback
- Full flow tested through the real router against a local Discord stub (DISCORD_API_BASE)
- Env: DISCORD_CLIENT_ID/_CLIENT_SECRET/_GUILD_ID/_REQUIRED_ROLE/_API_BASE/_REDIRECT_URI; docs updated

go test ./... passes.

Reviewed-on: #31
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 08:51:22 +07:00
sulthan 8cebb94b92 Give every Bookmark an owner (Reader table) (#30)
Closes #22

## What

A `readers` table appears; every Bookmark belongs to one. The owner is seeded as the first and only Reader, and all existing rows are attached to them.

- **Migration 0003**: `readers` (discord_id UNIQUE, token_sha256 UNIQUE, created_at).
- **Migration 0004** (run-once, version-table-gated): attaches existing bookmarks to the seeded owner, drops the surrogate `key` column, composite PK `(reader_id, site, series_id)`, FK to readers `ON DELETE CASCADE` — a duplicate Bookmark for one Reader and Series is impossible at the database level.
- **Seed**: `Store.Open` runs schema to 0003, seeds exactly one owner row from `OWNER_DISCORD_ID` (hash = SHA-256 of `API_TOKEN`, refreshed on every start so rotation stays current), then migrates the rest.
- **Scoping**: `List/Get/Upsert/Delete` take `readerID`; the wire `key` is derived as `site:series_id` on read. Handlers act as `Store.OwnerID()` while the global token remains the only credential.
- **Unchanged**: authentication and the flat wire format — nothing observable changes from outside.
- **New env** `OWNER_DISCORD_ID` (required): compose, .env.example, DEPLOY.md, README.md, backend/AGENTS.md updated.

Series-level methods (due queue, mark-checked, set-latest-chapter) stay unscoped deliberately: series are shared rows polled once per due cycle, and the reader_count ordering requires cross-reader visibility (ADR-0003).

## Verification

- `go test ./...` green, including new tests: seed idempotency + hash refresh, 0004 attach migration, DB-level duplicate impossibility, per-reader scoping, reader-delete cascade.
- Live smoke test on fresh Postgres: seed → PUT/GET (flat wire intact) → restart idempotent; stored hash matches SHA-256 of the token.

## Deploy note

`OWNER_DISCORD_ID` is required after this lands — the backend refuses to start without it. Set it to the owner's Discord snowflake (Settings → Advanced → Developer Mode → right-click name → Copy User ID).

Reviewed-on: #30
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 08:05:17 +07:00
sulthan 984965ed9f Split Series from Bookmark, keeping the wire format flat (#21) (#29)
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 07:19:54 +07:00
sulthan 08749df050 feat(backend)!: run on Postgres with a migration-owned schema (#28)
Swap modernc.org/sqlite for jackc/pgx/v5 with no observable change:
same endpoints, same wire format, same updated_at ordering rule.

The schema now comes from numbered SQL embedded in the binary and
applied on startup, one transaction each, recorded in
schema_migrations. That replaces two pieces of SQLite-era machinery,
both deleted rather than ported: the column probing (Postgres has ADD
COLUMN IF NOT EXISTS, and there is no legacy database left to probe)
and the Asura key rewrite, which has run clean on every start for
months now that the userscripts strip build hashes before writing. Its
regexp survives as latest.asuraBuildHash, where the poller still needs
it to scope chapter links to a series whose slug carries a rotating
hash.

Types get real: favorite is a boolean, chapter numbers double
precision, timestamps stay unix-ms bigint. SQLite's null-safe IS NOT
becomes IS DISTINCT FROM, which is what implements the rule that only
reading progress reorders a list. Inside COALESCE/NULLIF the status
and kind parameters need an explicit ::text -- there is no target
column to infer from and Postgres refuses to guess.

Tests lose their free t.TempDir() database, so Docker is now a hard
prerequisite for `go test ./...`: internal/pgtest starts one
postgres:17-alpine per test binary and hands each test a database of
its own.

Also lands CONTEXT.md and the four ADRs written while scoping #18.

BREAKING CHANGE: DB_PATH is retired for DATABASE_URL, which is
required and has no default. Compose gains a postgres service on an
internal network with its own volume; POSTGRES_PASSWORD joins .env.
The old bookmarks-data volume is deliberately left undeclared so
`docker compose down -v` cannot take the pre-migration database with
it. main is not deployable until #25 and #26 land.

Closes #20

Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-08 06:52:20 +07:00
sulthan b9f9aea82c docs: secure-coding rules for agents, and refresh stale AGENTS.md top-matter (#16)
Two doc commits: a new secure-coding rules section, plus a fix for top-matter the rebrand left stale.

## `9156525` — secure-coding rules

`AGENTS.md` carried two security invariants (bearer auth, CORS) but nothing about the code an agent actually writes here. That is the gap worth closing: measured rates for AI-generated web/backend code are ~40% vulnerable (Pearce et al.), 45% failing security tests (Veracode 2025), and users *with* assistants shipped SQLi at 36% vs 7% for the control group (Perry et al., Stanford). The failure classes cluster on broken access control, injection, session/error handling and invented dependencies — all live surfaces in this repo.

Rules were **extracted, not pasted**. Every one names a guard that already exists in-tree, so the instruction is *match this*, not *invent something*:

| Rule | Existing anchor |
| --- | --- |
| parameterized SQL only; constants may concatenate | `store.go` — all queries use `?` |
| `html/template` only; no `template.HTML` on stored data | `web.go:85` |
| client-supplied URLs pass the fetch gate | `poller.go:144 fetchableSeriesURL` |
| cap remote bodies | `fetch.go:16 maxBodyBytes` |
| `subtle.ConstantTimeCompare`, never `==` | `middleware.go:22`, `session.go:61` |
| generic error out, detail to log, never log the token | `web.go:151` |
| `X-Forwarded-Proto` for Secure; **rightmost** XFF for IP | `session.go:68,101` |
| cookie flags; expiry checked before signature | `session.go:72-94`, `Verify` |
| validate at handler boundary | `handlers.go:48` `MaxBytesReader` 64 KB, 400 on bad key/status/kind |
| site strings via `el({text})`, never `{html}` | `el()` in both userscripts |
| `fetch()`/`authHeaders()` → `API_BASE` only | existing `authHeaders` |
| `localStorage` = cache/queue, never credentials | shared with site JS |

Plus a dependency rule (stdlib first; verify a package exists before adding — ~20% of LLM-proposed packages don't resolve, which is the slopsquatting vector) and a review gate marking auth/CORS/session/crypto/fetch-gate as security-critical.

Deliberately **excluded**: container signing, k8s admission control, IaC scanning, PII/HIPAA/PCI, C/C++ memory safety. Per OpenSSF's guide for AI assistant instructions, irrelevant rules make a model generate code compensating for attacks that cannot happen. None of those apply to a single-user Go + SQLite + userscript stack.

Sources: OWASP AISVS 1.0 Appendix C, OWASP Top 10 / ASVS v5, OpenSSF *Security-Focused Guide for AI Code Assistant Instructions* (2025-08-01).

## `5d4d890` — stale top-matter

The rebrand rewrote root `AGENTS.md` as a compression pass and switched Bromite -> Violentmonkey, but left the project described as a manga-only tracker over two sites. Six sites, two libraries and two userscripts now exist.

Root `AGENTS.md`:

- *What this is* names both scripts with their site lists, the `kind` column, and the `<site>:<series_id>` key shape.
- Origins constraint generalised past Asura/Demonic.
- Records that **kagane and novelfull are reliably Cloudflare-challenged** and browser-polled over CDP. Without it that bullet list reads as contradicting the code, since the paragraph above asserts blocking is "not universal — and not reliably reproducible".
- Diagram says two userscripts.

`backend/AGENTS.md` — two instances of the same defect, found while verifying the above:

- Store key list gained `novelfull|lightnovelworld` and the `kind` column.
- **`NOVEL_USERSCRIPT_PATH` documented** — it shipped in `main.go:150` undocumented.

The dated Cloudflare paragraph is left verbatim: it is a timestamped observation ("Verified 2026-07-26"), so rewriting it would falsify a record rather than update it. `userscript/AGENTS.md` is untouched; it already documents both novel adapters and the `LIBRARY`/`STORE_PREFIX` split.

## Verification

Docs-only, no code touched. Every code reference above was read at `4229c17` before being cited — the fetch gate, body cap, constant-time compares, cookie flags, handler validation, `el()` helper and both route registrations. No invented line numbers.

## Not addressed here

The `API_TOKEN` literal is committed in plaintext in both userscripts and in their `@downloadURL`/`@updateURL` lines. The new rules say not to propagate it, but the actual remedy is rotation plus build-time substitution, since the value is already in git history. Separate change; flagging it so it does not get lost.

Reviewed-on: #16
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-06 20:07:34 +07:00
sulthan 4229c179b0 rebrand: MangaBM → BookmarkManager, add novel library support (#15)
Two intertwined changes — the rebrand and the novel library were developed on
the same branch because the novel UI plumbing is part of the new "Bookmark
Manager" wordmark in the web shell.

## What it does

- **Rebrand**: MangaBM → BookmarkManager across the Go module, compose stack,
  env vars, Traefik hostnames, container/image names, userscript storage
  prefixes (`mangabm:cache` → `bmgr:manga:cache`, `mangabm:queue` → `bmgr:manga:queue`),
  and docs.
- **Novel library**: same backend, two libraries. New `kind` column splits
  bookmarks into `manga` / `novel`; PUT validates it. Two userscripts:
  - `manga-bookmark.user.js` — unchanged behaviour, just stamps its own `kind`.
  - `novel-bookmark.user.js` — separate Violentmonkey install with adapters
    for **novelfull.com** (polled via headless browser — Cloudflare JS
    challenge) and **lightnovelworld.net** (polled via plain TLS).
- **Web UI**: library switch on the app shell. Login art, libswitch, and
  novel-site colours from the Cinder design snapshot.

## Plumbing

- `addedColumns` ALTER for `kind` runs on first start after upgrade; every
  pre-existing row is backfilled to `'manga'`. No manual SQL, no down-time.
- `ALLOWED_ORIGINS` gains the two novel sites.
- New `NOVEL_USERSCRIPT_PATH` env (default `/userscript/novel-bookmark.user.js`),
  bindmounted alongside the manga script.
- Traefik router names `mangabm*` → `bmapi*` / `bmweb*`.

## Test status

- `go test ./...` — green
- `node --test userscript/test/logic.test.js` — 34 pass
- `node --test userscript/test/novel-logic.test.js` — 11 pass
- `node --check` on both userscripts — clean

## Notes for the redeploy

.env keys were renamed (`MANGA_API_HOST` → `BOOKMARK_API_HOST`,
`MANGA_WEB_HOST` → `BOOKMARK_WEB_HOST`). Update DNS / Traefik labels on the
prod override before pulling, otherwise the public hostnames go dark.
See the redeploy instructions I'll post next to this PR.

Reviewed-on: #15
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-06 03:58:23 +07:00
sulthan c445762244 Merge pull request 'docs: split CLAUDE.md into per-directory guidance' (#14) from docs/split-claude-md into main
Reviewed-on: #14
2026-08-04 20:35:16 +07:00
sulthan 7a4afcc73e Merge remote-tracking branch 'origin/main' into docs/split-claude-md
# Conflicts:
#	CLAUDE.md
2026-08-04 19:55:19 +07:00
sulthan 4ee0b0f092 docs: split CLAUDE.md into per-directory guidance, add opencode agents
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-04 19:51:33 +07:00
sulthan 180ee78b1f Add comix.to and kagane.to support (#13)
Tracks read progress on comix.to and kagane.to alongside asura and demonic, in both the userscript and the backend.

Implements `docs/superpowers/plans/2026-08-03-comix-kagane-support.md`.

## Userscript

- `comix` adapter — `/title/<id>-<slug>`; only the id prefix is identity (the slug follows the title). No `og:image`, so the cover is matched by `alt`.
- `kagane` adapter — reader URLs are uuids with no chapter number, so it comes out of `og:title`; anchor scanning is structurally impossible, replaced by `latestChapterFromApi` against kagane's same-origin JSON API.
- `seriesId` threaded through `latestChapterFromAnchors` so comix can scope its scan to its own series and a recommendation strip cannot win the maximum.
- `@match` for both hosts, panel chips, v1.6.0.

## Backend

- `latestChapterFrom` cases: comix parses the SSR JSON state blob (`latestChapterUrl`, scoped to the series id); kagane parses API JSON (`chapter_no`).
- Poller allowlist extended; `Poller.BrowserFetch` with `fetcherFor(site)` routes kagane to a browser fetcher. Nil means kagane is not polled at all — never a fallback to the TLS fetcher, which would only ever retrieve a challenge page.
- `BrowserFetcher`: chromedp against a `headless-shell` sidecar. kagane sits behind a Cloudflare JS challenge that no TLS fingerprint clears, and the request is made inside the page rather than by replaying `cf_clearance`.
- `BROWSER_WS_URL` wiring, sidecar in both compose files (no `ports:`, dedicated non-external network), Dockerfile on `golang:1.26-alpine` — chromedp requires go 1.26.
- Web UI `--comix` / `--kagane` tokens in both colour branches.

## Notes for review

- `series_url` is client-supplied and a headless browser is a strong SSRF primitive, so kagane's host is pinned twice: in `fetchableSeriesURL` and again in `kaganeAPIURL`.
- Three chained defects found during verification made the browser path dead under Compose (sidecar flag collision, Chrome's Host-header DNS-rebinding check, the wrong chromedp option). Fixed; the compose comments record the wrong configurations too, so they don't get "simplified" back.
- `ALLOWED_ORIGINS` now includes both new origins. Without it every write from comix/kagane silently fails CORS preflight, parks in the retry queue, and drops at the cap.

## Verification

221 backend tests, 32 userscript tests, static `CGO_ENABLED=0` build, both compose configs.

Two gaps, both real:

1. The userscript on live pages via Violentmonkey needs a human browser profile — not run. Check: comix series page (title/cover, no chapter), comix chapter page (records the number; an *older* chapter must not regress it), comix SPA navigation without reload, kagane series page (og:image cover), kagane reader (number from `og:title`), both chips opening the right sites.
2. The kagane browser path has not completed end-to-end anywhere. Dial/navigate/fetch is confirmed, but Cloudflare 403'd headless-shell's Chrome on every attempt from the dev sandbox, and comix's poll-through-Docker was blocked by that environment's TLS interception. Both environment-dependent rather than branch defects — the first real deploy is the actual verification.

Reviewed-on: #13
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-08-03 19:53:45 +07:00
sulthan 0725b11275 docs: sync AGENTS.md to current architecture (internal/, Cinder, confirm-gated, edge-tab)
Captures what shipped on the branch:
- backend split into internal/ packages; composition root = main.go
- web UI go:embed now lives under internal/web/; Dockerfile must copy tree
- impeccable detector caveat (root-absolute /static/ paths) and false-clean
- confirm-row pattern for archive/finish/remove; --ember reserved
- edge-tab hitbox design (7x44 visible, 28x72 hit, touch-action + arm hold)
- Cinder design system section + ember-law reference
2026-08-03 19:52:40 +07:00
sulthan d8c6074559 fix: extend CORS origins to comix and kagane, preserve progress on unparseable chapters
- .env.example, DEPLOY.md, docker-compose.yml: add comix.to/kagane.to to
  ALLOWED_ORIGINS so the userscript isn't CORS-blocked on either new site
- .env.example: comment out BROWSER_WS_URL's DNS-name default, which
  overrides the working compose default and 500s Chrome's DevTools handler
- userscript: updateToCurrentChapter() now falls back to the stored
  chapter/label when chapterNum is unparseable, instead of wiping progress
  (kagane's og:title lacks a number when a chapter has no episode suffix)
- README.md: document comix/kagane in the config table, adapter reference,
  and key examples
2026-08-03 18:32:49 +07:00
sulthan ba23411a74 fix: address issues found in end-to-end verification
Chained defects made the kagane browser-fetch path completely non-functional
in Docker Compose: headless-shell's compose command re-declared
--remote-debugging-port, colliding with the image's own entrypoint/socat
proxy (EOF on every dial); the sidecar was then only reachable by Docker DNS
name, which Chrome's DevTools HTTP handler rejects with a 500
(Host-header/DNS-rebinding check); and NewBrowserFetcher's NoModifyURL option
skipped /json/version discovery entirely, dialing a bare host:port that
Chrome 404s since /devtools/browser/<uuid> is minted fresh per Chrome start.
Fixed by trimming the redundant command flags, pinning headless-shell to a
static IP so BROWSER_WS_URL can name it directly, and removing NoModifyURL so
chromedp's discovery (which echoes the request's Host back into
webSocketDebuggerUrl) does the right thing on its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 18:21:25 +07:00
sulthan 877d3df010 feat(web): add comix and kagane site colours 2026-08-03 17:55:50 +07:00
sulthan 3adbfb7ad9 fix: scope headless-shell to a dedicated network, off the Traefik proxy network
Prod override put headless-shell on the externally-managed `proxy`
network so manga-api (confined there for Traefik routing) could still
resolve it. That reopened CDP (port 9222, raw remote code execution)
to every other container on that shared network, not just manga-api.

Give both services a project-private `browser` network (defined in
the base compose file, not `internal: true` since headless Chrome
needs outbound access to kagane.to). manga-api joins both `proxy` and
`browser` in the prod override; headless-shell never touches `proxy`.
2026-08-03 17:52:34 +07:00
sulthan 2c7b4952f3 feat: wire headless-shell sidecar for challenge-gated polling
Also bumps backend/Dockerfile's build stage to golang:1.26-alpine —
chromedp v0.16.0 and cdproto both require go 1.26, and the pinned
1.24-alpine base no longer builds the module.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 17:46:29 +07:00
sulthan 1d9b1200bb feat(latest): add chromedp browser fetcher for challenge-gated sites 2026-08-03 17:29:21 +07:00
sulthan 89eaef70d4 feat(latest): allow comix and kagane, route kagane to a browser fetcher 2026-08-03 17:23:10 +07:00
sulthan 2fcf882c49 feat(latest): parse comix SSR state and kagane API for latest chapter 2026-08-03 17:18:38 +07:00
sulthan 48c2d57d7e feat(userscript): match comix and kagane, add panel chips 2026-08-03 17:15:41 +07:00
sulthan 076e8bb5d8 feat(userscript): refresh kagane latest chapters through its API 2026-08-03 17:12:22 +07:00
sulthan a5c5e11c21 feat(userscript): add kagane.to site adapter 2026-08-03 17:08:33 +07:00
sulthan b96caf13e6 refactor(userscript): thread seriesId through latest-chapter scan 2026-08-03 17:04:19 +07:00
sulthan 78eaecb119 Add test coverage for comix coverFromPage()
Extend the test harness's document stub with a querySelectorAll("img[alt]")
fake (module-level pageImages fixture, mirroring metaTags), then assert on
p.cover for a matching alt and for no match. Previously the guard in
coverFromPage() always short-circuited under test since querySelectorAll
didn't exist on the stub, so the alt-matching loop had zero coverage.
2026-08-03 17:00:54 +07:00
sulthan e392ec3de0 feat(userscript): add comix.to site adapter 2026-08-03 16:57:14 +07:00
sulthan eeb601cbe2 Split backend into internal packages by responsibility
All Go files lived flat in backend/ as one package main. Move store,
latest-chapter polling, sessions, HTTP middleware, the JSON API, the
userscript handler, and the web UI (with its templates/static assets)
into backend/internal/{store,latest,session,httpmw,api,userscript,web},
each with an exported API. main.go becomes the composition root wiring
them into newRouter; root-level tests cover the assembled router while
package-local tests cover unit behavior. Update Dockerfile/.dockerignore
for the new internal/ tree and CLAUDE.md to describe the layout.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-02 19:41:38 +07:00
sulthan e250762ea6 Move userscript to Violentmonkey, sync docs to shipped Cinder design
CLAUDE.md: swap Bromite for Violentmonkey throughout, add installed
golang skills to relevant skills, add comment-writing rules, add a
design-system pointer rule. docs/design-system.md: rewrite against
the current Claude Design project and the tokens/components already
shipped in backend/static/style.css (danger/slate/moss/clay/trash,
action key, brand mark).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-02 19:18:56 +07:00
sulthan 3c935ba7c3 Action key under the tabs, plus the brand mark (#12)
The card action strip is icon-only, so nothing on screen said what the six
glyphs do. This adds a permanent key line under the tabs naming each one.

The key is tab-shaped, not row-shaped: archived and finished swap Archive
for Restore, and finished drops Done — the same conditions card.html already
uses for the buttons. It rides along in writeChromeOOB, otherwise an htmx
tab switch would leave the previous bucket's key behind.

Also from the Claude Design pass:

• per-action accent on hover/press — slate archive, moss finished, clay
chapter — so a press says which lane it belongs to; ember stays reserved for
new-chapter heat
• cover 74→80px desktop / 86→93px phone, title 19→21px, meta 10→11px
• favourite mark pinned to the right edge of the measure instead of trailing
the title, which drifted whenever a new-chapter title shrink-wrapped
• recent strip sizes now derive from --cover-w instead of repeating magic
numbers

### Brand mark

The design project grew a logo, so it lands in three places with three
different colour sources:

• `backend/static/logo.svg` — fixed palette, because a favicon has no page to
inherit from. Linked as `rel="icon"` from both the app and the login page.
• `{{define "mark"}}` in chrome.html — takes `--ink`/`--ember`/currentColor, so
the header mark flips with the light/dark theme. Used by app.html and
login.html.
• the userscript panel header — same drawing inside the shadow root, next to
the tokens it uses.

`.brand` becomes a flex row in both stylesheets and the topbar aligns centre
rather than baseline. The 5px stroke on the 200-unit grid thins out at brand
size, so it is nudged to 6.5 instead of scaled blindly. No mark on the edge
tab: a 7px sliver has nowhere to put one.

go test ./... passes (static-asset test now covers logo.svg), node --test on
the userscript logic suite passes 14/14. Verified live at 412px and desktop:
key renders, per-tab variant correct on /?tab=finished (play/star/pencil/undo/
trash, no Done), and the OOB swap fires on /ui/list.

Reviewed-on: #12
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-07-31 17:14:15 +07:00
sulthan f3b55fd883 Work the design critique down: chapter format, colour law, search, strip, a11y (#11)
Two rounds of design-critique fixes on the web UI. Every visual change was verified at 390x844 and 1280x900 in both dark and light with screenshots; `go test ./...` is green throughout; no new dependencies.

## Earlier commits on this branch

The two oldest commits predate this session and were never opened as their own PR, so they are under review here too:

- Confirm-gate the lifecycle actions, cluster the action strip by consequence.
- Fix the accessibility findings from the audit: contrast, focus, reduced motion.

## The rest

**Chapter format.** The userscript and the poller both write `"Chapter N"`, and the templates prefixed `Ch ` again, so every real Asura row read `Ch Chapter 250` — while a manual edit stored a bare `250`, leaving two formats in one list. `DisplayChapter`/`DisplayLatest` on `Bookmark` now strip the lead-in and re-add exactly one `Ch `.

**Zero-result search.** The client filter only toggled `card.hidden`, so a query matching nothing left a blank list under a fully populated, unfiltered "Continue reading" strip. There is now a no-match state with a Clear-search button, and the strip goes down while a filter is active.

**The colour law.** `--ember` is documented as meaning "new chapter" and was spent on eight things, including setting "Nothing new." in the colour reserved for new chapters. Destruction moves to a new `--danger` token; text-input focus follows the searchbar idiom and turns `--paper`. Contrast, both themes: `--danger` on the page 4.82 / 6.65, the solid Remove button 4.94 / 7.30, the confirm question 9.00 / 7.98.

**The remove confirm.** Buttons 40px 8px apart became 46px 12px apart, and the question names the series and the loss instead of asking "Remove this?". It opens with **Cancel** focused, not Remove — the two reversible rows still open on their affirmative.

**The recent strip.** It was the head of the same `updated_at DESC` list rendered directly below it, on every tab, costing ~240px of the first phone screen. It is now scoped to series with a chapter waiting, and only on All. With nothing new anywhere it does not render — deliberate.

**Stale chrome.** The strip and the Updated badge describe the whole library but live outside the swapped `#list`, so archiving a series left it under "Continue reading" with the badge still counting it, and `/?tab=all` reached by htmx differed from the same URL reloaded. Both regions move into `chrome.html` and refresh out of band on every mutation and every tab switch.

**Accessibility and touch.** Esc closes any open panel and returns focus to the cell that owns it; opening a confirm moves focus into it; the inline error scrolls into view and no longer self-destructs after 5s; every tab and desktop action cell clears 44px; `role="alert"` on the login error; the card monogram is no longer announced; the busy bar is clipped by its own travel rather than by `overflow: hidden` on the card.

**Chapter form label.** The panel's only visible text named the published chapter while the field held your progress. The field gets a real label; "Latest known" moves below it.

**gzip.** Nothing was compressed. A stdlib middleware handles the four text types and leaves woff2 alone: style.css 21.8 -> 6.5 KB, htmx 50.9 -> 16.4, filter.js 7.6 -> 2.9.

## Not addressed

The `role="status"` error slot is still mutated while hidden and then revealed, which is the non-announcing pattern the confirm rows were fixed for. Delete is still a silent vanish. Both are flagged in the critique snapshot under `.impeccable/critique/`.

Design health went 24/36 (66.7%) to 29/40 (72.5%) between snapshots; the two P1s that survived were found and fixed after that run.

Reviewed-on: #11
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
2026-07-30 22:34:23 +07:00
131 changed files with 72339 additions and 4448 deletions
+134
View File
@@ -0,0 +1,134 @@
---
name: implement-tickets
description: "Orchestrate a batch of tickets: plan the briefs, then hand each ticket to its own implementer subagent in its own worktree."
disable-model-invocation: true
---
# Implement tickets
You are the **orchestrator**. You write briefs, dispatch, land results, and talk
to the tracker. You do not write the implementation — every line of ticket code
is written by a `ticket-implementer` subagent in its own git worktree. Reach for
the editor yourself only for a merge conflict resolution.
Ticket source and `tea` usage: `docs/agents/issue-tracker.md`. Codebase
questions: `graphify query "<question>"` before grepping.
## 1. Collect the tickets
The user's argument is the selector: issue numbers, a label, a parent issue, or
nothing. With nothing, take the open issues labelled `ready-for-agent`.
Fetch each with `tea issue <n> --comments`, and read the **whole** body —
acceptance criteria and the `Blocked by` line are what the rest of this skill
runs on. A ticket whose blockers are still open is out of this batch unless a
blocker is also in it.
## 2. Plan the batch
Explore enough of the codebase to write briefs a fresh context can act on: the
files each ticket lands in, the patterns it must follow, the `AGENTS.md`
invariants it touches.
Then decide three things:
- **Waves.** Blocking edges set the order; tickets with no open blocker inside
the batch share a wave. Cap each wave at **3** concurrent tickets unless the
user set another width.
- **Contracts.** Two tickets in one wave that meet at a function signature, a
JSON shape, a table column, or a token name: you decide the shape now and
write the identical wording into both briefs. A contract left for the
subagents to negotiate is a merge conflict you scheduled.
- **Splits.** A ticket too big for one fresh context window goes into the wave
as two briefs, or back to the user.
## 3. Get the plan approved
Present, and stop:
- the wave list, and for each ticket: number, title, one-line brief summary,
the files or areas it will touch, its verification commands
- every cross-ticket contract, verbatim as it will appear in the briefs
- anything you had to assume
Wait for approval. Apply the user's edits to the plan, do not relitigate them.
## 4. Run a wave
Per ticket, before dispatch:
```bash
git worktree add ../ticket-<n> -b ticket/<n>-<slug> <base> # base = the branch you are on
cp .env ../ticket-<n>/ 2>/dev/null # gitignored, worktrees do not get it
tea issue edit <n> --add-assignees <your gitea username> # tea login list has it
```
Write the brief to `.scratch/<batch-slug>/t<n>-brief.md` using the template
below, in the ubiquitous language of `CONTEXT.md` — a brief that says "scrape"
where the domain says Poll hands the subagent the wrong model of the system.
Then dispatch the whole wave in **one** `task` batch, every item on the
`ticket-implementer` agent. Each dispatch names: the absolute brief path, the
worktree path, the branch, the base ref, and the report path
`.scratch/<batch-slug>/t<n>-report.md`.
<brief-template>
# Ticket #<n> — <title>
**Read first.** `tea issue <n> --comments` for this ticket, then the issue it
refers to — the parent or spec — the same way. The comments carry decisions the
body never got updated with. This brief stays the requirements; those two reads
are the intent behind them.
**Goal.** The end-to-end behaviour this ticket makes work, from the user's side.
**Acceptance criteria.** Verbatim from the ticket.
**Contract.** The exact shared signatures / shapes / names this ticket must
implement or consume, and which sibling ticket is on the other end. Omit when
the ticket touches nothing shared.
**Where it lands.** The files and packages, and the existing pattern to follow
in each.
**Binding invariants.** The `AGENTS.md` rules this change can break — name them.
**TDD seams.** Where a test comes first — run the `tdd` skill at each one and
follow its red → green loop. Or "none — verify after".
**Verify.** The exact commands, e.g. `cd backend && go test ./...`,
`node --test userscript/test/logic.test.js`.
**Out of scope.** What not to touch, especially a sibling ticket's files.
</brief-template>
## 5. Land the wave
The wave is landed when every ticket in it is closed, reverted, or handed back
to the user. Per returned ticket:
| Status | What you do |
| --- | --- |
| `DONE` | merge, comment, close |
| `DONE_WITH_CONCERNS` | merge, comment the concerns, close only if you judge them non-blocking — otherwise leave open and tell the user |
| `BLOCKED` / `NEEDS_CONTEXT` | supply what is missing and re-dispatch, or hand back to the user with the specifics. Never implement it yourself |
| `REVIEW_BLOCKED` | run `code-review` over the branch yourself (`cr-spec` + `cr-standards`), then treat the outcome as the statuses above |
Merge from your own checkout: `git merge --no-ff ticket/<n>-<slug>`. A textual
conflict is yours to resolve (`resolving-merge-conflicts`). A **semantic**
clash — both sides green apart, wrong together — goes back to whichever ticket
owns the contract, as a re-dispatch with the collision described.
Then `tea comment <n> "<the report summary>"`, `tea issue close <n>`, and
`git worktree remove ../ticket-<n>`. Keep the report file.
Only once the whole wave is landed does the next wave start — its briefs may
need what this one changed.
## 6. Close the batch
Run the full suite once on the merged base, and report: a line per ticket with
its status, commits, and open concerns, plus anything still assigned or open on
the tracker. A red suite after every ticket went green is an interaction bug —
diagnose it, name the two tickets, and fix it or hand it back with both named.
@@ -14,7 +14,7 @@ parsers, helpers. UI, network, and storage behaviour are verified on-device.
```bash
node --check userscript/manga-bookmark.user.js # parse check, silent on success
node --test userscript/test/logic.test.js # 14 tests as of 2026-07-28
node --test userscript/test/logic.test.js # 35 tests as of 2026-08-10
```
Run both before every commit that touches the userscript.
@@ -31,7 +31,7 @@ The test file installs four globals **before** requiring the userscript:
|---|---|---|
| `localStorage` | `Map`-backed stub | `loadCache`, `loadQueue`, and the key-migration IIFE touch it at module scope |
| `location` | `{href, hostname, pathname, origin}` | read during boot |
| `document` | `querySelector` for `meta[property="…"]` only, plus a no-op `addEventListener` | adapters read `og:title`/`og:image` |
| `document` | `querySelector` for `meta[property="…"]` only, plus a no-op `addEventListener` | adapters read `og:title` (covers are the backend's, never scraped) |
| `document.body` | **left `undefined`** | this is the whole trick |
`document.body === undefined` sends the userscript's boot block down its `else`
+90 -23
View File
@@ -1,34 +1,73 @@
# Copy to .env and fill in. Never commit the real .env.
# Long random secret shared with the userscript's API_TOKEN. Generate one:
# Secret every Reader's userscript credential is derived from (issue #24):
# the backend rebuilds install URLs from it, and only SHA-256 hashes of the
# credentials ever touch the database. Generate one:
# openssl rand -hex 32
API_TOKEN=changeme-generate-a-long-random-token
TOKEN_KEY=changeme-generate-a-long-random-token
# The owner's Discord user ID — seeded at startup as the first Reader, the
# administrator (the only one who can revoke another Reader's sessions), and
# the owner of every bookmark that predates registration. Discord snowflake,
# e.g. 1046923170000000000.
OWNER_DISCORD_ID=changeme-your-discord-user-id
# Comma-separated origins allowed to call the API (CORS). Both Asura domains
# plus Demonic. Add/remove as the sites' hostnames change.
ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org
# plus Demonic, Comix, Kagane, and the two novel sites. Add/remove as the
# sites' hostnames change.
ALLOWED_ORIGINS=https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to,https://novelfull.com,https://lightnovelworld.net
# Password for the bundled Postgres container, and therefore half of the
# DATABASE_URL compose builds for the backend. Generate one:
# openssl rand -hex 24
POSTGRES_PASSWORD=changeme-generate-a-long-random-password
# Override only to point the backend at a Postgres compose does not run.
# DATABASE_URL=postgres://user:pass@host:5432/bookmarks?sslmode=require
# Directory inside bookmark-api for immutable, content-addressed Cover bytes.
# Compose builds the image and mounts its named volume at this path.
COVER_DIR=/covers
# Public origin this deployment answers on, no trailing slash. Required: Cover
# URLs go out absolute, because the userscript renders them on a Site's own
# origin where a relative path would resolve against the Site (ADR-0007).
PUBLIC_BASE_URL=https://bookmark-api.example.com
# --- Prod override (Traefik) only ---
# Subdomain Traefik routes to this service (required by the prod override).
# MANGA_API_HOST=manga-api.example.com
# BOOKMARK_API_HOST=bookmark-api.example.com
# Traefik's docker network name, if not "proxy".
# PROXY_NETWORK=proxy
# Traefik HTTPS entrypoint + cert resolver names, if yours differ from these.
# TRAEFIK_ENTRYPOINT=websecure
# TRAEFIK_CERTRESOLVER=le
# --- Web UI ---
# Password for the browser UI at https://$MANGA_WEB_HOST. Leave unset to
# disable the web UI entirely (the routes are not registered at all).
# Generate one: openssl rand -base64 18
WEB_PASSWORD=
# --- Web UI (Discord OAuth) ---
# Sign-in is a Discord authorization code grant (ADR-0002), and it is also
# registration: any member of the configured guild becomes a Reader on their
# first successful login, with their own empty library. Create the application
# at https://discord.com/developers/applications and register the exact
# callback URL ($BOOKMARK_WEB_HOST/auth/discord/callback) as an OAuth2
# redirect.
DISCORD_CLIENT_ID=
DISCORD_CLIENT_SECRET=
# The guild whose membership gates sign-in (Developer Mode -> right-click the
# server -> Copy Server ID).
DISCORD_GUILD_ID=
# Exact callback URL, e.g. https://bookmark.example.com/auth/discord/callback.
# Discord matches it verbatim, so it must equal the registered redirect.
DISCORD_REDIRECT_URI=
# Optional: a role snowflake members must hold on top of guild membership.
# Empty (the default) means membership alone suffices.
# DISCORD_REQUIRED_ROLE=
# Subdomain Traefik routes to the browser UI (required by the prod override,
# whether or not WEB_PASSWORD is set). Left commented on purpose: an example
# value here would be a silent wrong-hostname fallback, and Traefik would
# publish the UI router on a domain you do not own. The same container also
# answers on MANGA_API_HOST for the userscript's API.
# MANGA_WEB_HOST=manga.example.com
# Subdomain Traefik routes to the browser UI (required by the prod override).
# Left commented on purpose: an example value here would be a silent
# wrong-hostname fallback, and Traefik would publish the UI router on a domain
# you do not own. The same container also answers on BOOKMARK_API_HOST for the
# userscript's API.
# BOOKMARK_WEB_HOST=bookmark.example.com
# --- Latest-chapter poller ---
# The backend re-checks each bookmarked series' newest published chapter on its
@@ -37,13 +76,15 @@ WEB_PASSWORD=
# Set to 0 to turn it off entirely.
# LATEST_CHAPTER_POLL_ENABLED=1
#
# Two independent clocks. COOLDOWN is how long one series rests between checks;
# INTERVAL is how often the poller wakes up and looks for series past that
# cooldown. Shortening INTERVAL cannot shorten a COOLDOWN.
# LATEST_CHAPTER_POLL_COOLDOWN=1h # per series, floor 15m
# LATEST_CHAPTER_POLL_INTERVAL=10m # how often to wake
# LATEST_CHAPTER_POLL_BATCH=14 # series per wake
# LATEST_CHAPTER_POLL_STAGGER=20s # delay between fetches in a batch
# Two independent clocks. COOLDOWN is how long a plain-TLS series rests between
# checks; BROWSER_COOLDOWN is the longer rest for kagane and novelfull. INTERVAL
# is how often the poller wakes up and looks for series past their cooldowns.
# Shortening INTERVAL cannot shorten either cooldown.
LATEST_CHAPTER_POLL_COOLDOWN=1h # plain-TLS per series, floor 15m
LATEST_CHAPTER_POLL_BROWSER_COOLDOWN=6h # browser-backed per series, floor 15m
LATEST_CHAPTER_POLL_INTERVAL=10m # how often to wake
LATEST_CHAPTER_POLL_BATCH=14 # series per wake
LATEST_CHAPTER_POLL_STAGGER=20s # delay between fetches in a batch
#
# Uses a ticker, not an immediate first run: the first poll happens one
# INTERVAL after startup, not at startup. A container restarting more often
@@ -52,3 +93,29 @@ WEB_PASSWORD=
# BATCH x (COOLDOWN / INTERVAL) series hold the cooldown cadence — 84 with these
# defaults. Beyond that the cadence stretches uniformly rather than breaking;
# raise BATCH or lower INTERVAL. Keep BATCH x STAGGER under INTERVAL.
# CDP endpoint of the browser, used for the two sites behind a Cloudflare
# JavaScript challenge (kagane, novelfull) and by the web UI's kagane cover
# proxy. Unset disables browser polling and serves 404 for covers not already
# stored; those sites then rely on the userscript alone. That is also exactly
# how an unreachable browser degrades, so a home machine that is off costs
# chapter freshness and nothing else.
#
# The browser does NOT run in this stack. It is its own compose unit on the
# home machine (chrome/docker-compose.yml, chrome/.env.example) and is reached
# over the tailnet, so set this to that machine's tailnet address:
#
# BROWSER_WS_URL=ws://100.x.y.z:9222
#
# It must be the tailnet **IP**, never a MagicDNS hostname and never the old
# Docker service name: Chrome's DevTools HTTP handler 500s any /json/version
# request whose Host header isn't an IP or "localhost", which silently breaks
# every kagane poll. Left unset here on purpose — a wrong default would poll a
# stranger's address, and "no browser" is a safe, self-announcing state.
# BROWSER_WS_URL=ws://100.x.y.z:9222
# Zone the backend stamps its log lines in. Cosmetic only. Nothing else in
# the service has a zone: bookmark timestamps are unix ms, and the two real
# time columns are timestamptz. Defaults to Asia/Jakarta; set to UTC for the
# conventional server default.
# API_TZ=Asia/Jakarta
+9 -1
View File
@@ -5,9 +5,17 @@
backend/server
backend/backend
.playwright-mcp/
graphify-out/
# graphify map is committed; only regenerable/local parts are ignored
graphify-out/cost.json
graphify-out/cache/
graphify-out/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]/
graphify-out/.rebuild.lock
plans/
.scratch/
docs/superpowers/
.superpowers/
go.work
go.work.sum
# impeccable-ignore-start
# Ephemeral output, runtime state, and per-dev overrides.
+90
View File
@@ -0,0 +1,90 @@
---
name: ticket-implementer
description: Implements one ticket end to end inside its own git worktree - reads a brief file, implements, tests, commits, runs the two-axis code review through cr-spec and cr-standards, fixes findings, writes a report file, returns a short status contract. Dispatched by the implement-tickets skill.
model: opencode-go/minimax-m3
thinking-level: high
tools: read, write, edit, bash, grep, glob, lsp, todo, ast_edit, task
spawns: cr-spec,cr-standards
autoloadSkills: code-review, tdd
---
You implement **one ticket** dispatched by an orchestrator. Your dispatch names:
a **brief file**, a **worktree path**, a **branch**, a **base ref**, and a
**report file** path.
## The worktree is your whole world
Every command runs with `cwd` set to the worktree path, and every file path you
read or write is under it. The orchestrator's checkout is a different directory
on the same repo — editing it corrupts a sibling agent's run. If a command must
run elsewhere, say so in the report instead of doing it.
Your branch is already checked out there. Never `git checkout`, `git switch`,
`git rebase`, or `git worktree` anything.
## Order of work
1. Read the brief file. It is the single source of requirements — use its exact
values verbatim.
2. Read the ticket and the issue it refers to, as the brief's **Read first**
section names them: `tea issue <n> --comments` for each. The ticket's
comments and its parent carry the intent and the decisions behind the brief.
Read no other ticket and no other brief.
3. Read `AGENTS.md` in the worktree, plus the nested `AGENTS.md` for the area
you touch. Its invariants bind you: security rules, design system, comment
policy.
4. Ask before writing code if requirements, acceptance criteria, approach, or
dependencies are unclear. Asking is free; guessing is not.
5. Implement exactly what the brief specifies. At each TDD seam the brief names,
run the `tdd` skill and follow its red → green loop.
Follow the patterns already in the codebase; improve what you touch,
restructure nothing outside the ticket.
6. Verify. Focused tests while iterating, the brief's full verification commands
once at the end. Test output must be pristine.
7. Commit to your branch. Reference the ticket number in the subject.
8. Review (below), fix, re-verify, commit the fixes.
9. Write the report file, then return the status contract.
## Review
After your first green commit, run the **`code-review`** skill over
`<base ref>...HEAD` in the worktree, with two changes to how it dispatches:
use the **`cr-spec`** agent for the Spec axis and **`cr-standards`** for the
Standards axis, both in one batch, and give the Spec axis your brief file plus
the ticket body as the spec.
Fix every Critical and Important finding, then re-run the tests that cover the
amended code. Two fix rounds maximum: anything still open after that goes in the
report and downgrades your status to `DONE_WITH_CONCERNS`. Judgement-call smells
you deliberately reject are a report line, not a silent drop.
If the review spawn is refused (recursion depth, unknown agent), do not skip the
gate — return `REVIEW_BLOCKED` with the diff range so the orchestrator runs it.
## Escalate rather than guess
Bad work is worse than no work, and escalating is never penalised. Return
`BLOCKED` or `NEEDS_CONTEXT` — with what you tried and what you need — when the
ticket needs an architectural decision with several valid answers, when it
collides with another ticket's changes, when it means restructuring the plan did
not anticipate, or when you have read file after file without progress.
## Report
Write to the report file: what you implemented, what you tested with the
commands and their output, TDD evidence (RED command + failing output + why that
failure was expected; GREEN command + passing output) where the brief required
TDD, files changed, the review's findings and what you did about each, and any
remaining concerns.
Then return **only** this, under 15 lines:
- **Status:** DONE | DONE_WITH_CONCERNS | BLOCKED | NEEDS_CONTEXT | REVIEW_BLOCKED
- branch name and commits created (short SHA + subject)
- one-line test summary ("14/14 passing, output pristine")
- one-line review summary ("spec clean; 2 Important fixed, 1 Minor declined")
- concerns, if any
- the report file path
Put the specifics of a BLOCKED / NEEDS_CONTEXT / REVIEW_BLOCKED in the returned
message itself — the orchestrator acts on it directly.
+47
View File
@@ -0,0 +1,47 @@
---
description: Code-writer subagent for subagent-driven development. Fast model (ocg/deepseek-v4-flash) for mechanical, well-specified implementation tasks. Escalates complicated tasks so the controller can re-dispatch on minimax-m3.
mode: subagent
model: 9router/ocg/deepseek-v4-flash
---
You are the implementer subagent for Subagent-Driven Development. You implement one task, exactly as specified, and report back with evidence.
## Before You Begin
If you have questions about requirements, acceptance criteria, approach, dependencies, or anything unclear in the task description — ask now. Raise concerns before starting work. Don't guess or make assumptions.
## Your Job
1. Implement exactly what the task specifies
2. Write tests (follow TDD when the task says to)
3. Verify the implementation works (run the focused test while iterating; run the full suite once before committing)
4. Commit your work
5. Self-review (below)
6. Report back
Follow existing patterns in the codebase. Don't restructure code outside your task. Don't overbuild — only what was requested (YAGNI).
## When You're in Over Your Head
It is always OK to stop and say "this is too hard for me." Bad work is worse than no work. STOP and escalate when the task requires architectural judgment, multi-file integration you can't see clearly through, or you're reading file after file without progress.
**Report BLOCKED or NEEDS_CONTEXT** with specifics: what you're stuck on, what you tried, what help you need. If the task turns out more complicated than mechanical (design judgment, broad codebase understanding), escalate so the controller can re-dispatch you on the more capable minimax-m3 agent.
## Self-Review Before Reporting
- **Completeness:** everything in the spec implemented? edge cases handled?
- **Quality:** names accurate? code clean and maintainable?
- **Discipline:** avoided overbuilding? only what was requested?
- **Testing:** do tests verify real behavior? output pristine (no stray warnings)?
Fix what you find before reporting.
## Report Format
Report back with ONLY (under 15 lines):
- **Status:** DONE | DONE_WITH_CONCERNS | BLOCKED | NEEDS_CONTEXT
- Commits created (short SHA + subject)
- One-line test summary (e.g. "14/14 passing, output pristine")
- Concerns, if any
- Report file path (if the controller gave you one)
If BLOCKED or NEEDS_CONTEXT, put the specifics in the final message itself — the controller acts on it directly. Use DONE_WITH_CONCERNS if you completed the work but have doubts. Never silently produce work you're unsure about.
+69
View File
@@ -0,0 +1,69 @@
---
description: Reviewer subagent for subagent-driven development. Capable model (ocg/minimax-m3) for task-scoped and whole-branch code review; also the re-dispatch target when implementation tasks are complicated.
mode: subagent
model: 9router/ocg/minimax-m3
---
You are the reviewer subagent for Subagent-Driven Development. You verify one task's implementation matches its requirements (spec compliance) and is well-built (code quality). You may also be dispatched for whole-branch review.
## Inputs
- Task brief file (requirements — use exact values verbatim)
- Implementer's report file
- Diff file (commit list, stat summary, full diff with context)
## Method
Read the diff file once — it is your view of the change. The context lines ARE the changed files: do not read a changed file separately unless a hunk you must judge is cut off mid-function (say so in your report). Do not re-run git commands. Inspect code outside the diff only to evaluate a concrete risk you can name — one focused check per named risk, and name both the risk and what you checked.
Your review is read-only. Do not mutate the working tree, index, HEAD, or branch state.
## Do Not Trust the Report
Treat the implementer's report as unverified claims. It may be incomplete, inaccurate, or optimistic. Verify against the diff. Design rationales in the report ("kept it per YAGNI") are the implementer grading their own work — a stated rationale never downgrades a finding's severity.
## Tests
The implementer already ran the tests and reported results. Do not re-run the suite to confirm. Run a test only when reading the code raises a specific doubt no existing run answers — a focused test, never a package-wide suite. If heavy validation seems warranted, recommend it in your report instead. Warnings or noise in the reported test output are findings — output should be pristine.
## Part 1: Spec Compliance
Compare the diff against the brief:
- **Missing:** requirements skipped, missed, or claimed without implementing
- **Extra:** features not requested, over-engineering, nice-to-haves
- **Misunderstood:** right feature built the wrong way, wrong problem solved
If a requirement can't be verified from this diff alone (lives in unchanged code or spans tasks), report it as a ⚠️ item instead of broadening your search.
## Part 2: Code Quality
- Clean separation of concerns? proper error handling? DRY without premature abstraction? edge cases?
- Do new/changed tests verify real behavior, not mocks? edge cases covered?
- Does each file have one clear responsibility? units independently testable? did this change create/significantly grow large files?
Point at evidence: file:line references for every finding. A tight report that cites lines gives the controller everything it needs.
## Calibration
Not everything is Critical. Important = this task can't be trusted until fixed: incorrect or fragile behavior, a missed requirement, maintainability damage you'd block a merge over (verbatim duplication of a logic block, swallowed errors, tests that assert nothing). "Coverage could be broader" and polish suggestions are Minor. If the plan explicitly mandates something this rubric calls a defect, that IS a finding — report Important, labeled plan-mandated. Acknowledge what was done well before listing issues.
## Output Format
### Spec Compliance
- ✅ Spec compliant | ❌ Issues found: [what's missing/extra/misunderstood, with file:line]
- ⚠️ Cannot verify from diff: [requirements you couldn't verify, what the controller should check]
### Strengths
[What's well done? Be specific.]
### Issues
#### Critical (Must Fix)
#### Important (Should Fix)
#### Minor (Nice to Have)
For each: file:line, what's wrong, why it matters, how to fix (if not obvious).
### Assessment
**Task quality:** [Approved | Needs fixes]
**Reasoning:** [1-2 sentence technical assessment]
Your final message is the report itself: begin directly with the spec-compliance verdict. Every line is a verdict, a finding with file:line, or a check you ran — no preamble, no process narration, no closing summary.
+123 -116
View File
@@ -2,159 +2,166 @@
Guidance for OpenCode (and Claude Code) working in this repo.
## Status
Active. Backend (`backend/`) and userscript (`userscript/manga-bookmark.user.js`) built. Plan `plans/mangaBookmark.md` = original spec, may drift; trust code + design docs in `docs/superpowers/specs/` over plan.
## What this is
Manga read-progress tracker for user reading on **asurascans.com** (current domain; asuracomic.net 301s here) and **demonicscans.org** from **Bromite** (mobile Chromium). Userscript injects on-page UI (floating button + slide-in panel), syncs progress to self-hosted Go backend so bookmarks unify across both sites and devices.
Read-progress tracker for two libraries — manga and novels — behind one self-hosted Go backend. Two separate Violentmonkey userscripts inject on-page UI (floating button + slide-in panel) and sync progress, so bookmarks unify across sites and devices:
## Hard constraints (drive design — do not violate)
- `manga-bookmark.user.js` — **asurascans.com** (asuracomic.net is dropped: its deep links 301 to the asurascans.com root, discarding the path), **demonicscans.org**, **comix.to**, **kagane.to**.
- `novel-bookmark.user.js` — **novelfull.com**, **lightnovelworld.net**.
Bromite uses Chromium's **native** userscript engine, not Tampermonkey:
- **No `GM_*` APIs anywhere.** No `GM_setValue`/`GM_getValue` (use page `localStorage`), no `GM_registerMenuCommand` (inject on-page UI), no `GM_xmlhttpRequest` for cross-origin (use plain `fetch()`). GM-free script also runs in desktop Tampermonkey/Violentmonkey for faster iteration.
- Cross-origin `fetch()` works **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block).
- Asura and Demonic = **separate origins, separate `localStorage`** — shared remote store only way to unify bookmarks. Cloud sync required, not optional.
- Userscript runs in **isolated world**, so embedded API token safe from site's JS.
- Cloudflare's block on manga sites is **IP-reputation-based, not universal — not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s w/ real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier, untested assumption CGNAT dev IP would be blocked; wasn't, at least this date. Treat "does curl work now" as live, time-varying fact to re-check, not fixed property of machine — Cloudflare bot scoring can flip clean IP without notice. Any backend fetcher still needs graceful-degrade path for when challenged; adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test.
One backend, one `bookmarks` table: a `kind` column (`manga`|`novel`) splits the libraries and the web UI switches between them. Rows are keyed `<site>:<series_id>`.
## Hard constraints (drive design — don't violate)
Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free where plain web APIs suffice — keeps portability across engines:
- **Avoid `GM_*` unless needed.** Prefer page `localStorage` over `GM_setValue`/`GM_getValue`, on-page UI over `GM_registerMenuCommand`, plain `fetch()` over `GM_xmlhttpRequest` for cross-origin.
- Cross-origin `fetch()` work **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block).
- Every site is its **own origin with its own `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync required, not optional.
- Userscript run in **isolated world**, so embedded API token safe from site's JS.
- Cloudflare's block on manga sites **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — Cloudflare's bot scoring can flip previously-clean IP without notice. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test.
- **kagane.to and novelfull.com are the exception to the above** — both sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`). When that's unset, kagane is skipped entirely (a plain fetch would only retrieve a challenge page) while novelfull pages are still attempted over plain TLS — its challenge is a live time-varying fact and its cover bytes never need the browser. The four other sites poll fine over plain TLS.
- **The CDP browser must look like a real browser, and stock headless images don't.** Measured 2026-08-08 against kagane.to, all from the same IP: `chromedp/headless-shell:stable` never cleared the challenge in 90s (`navigator.webdriver` true, empty plugin list, Chromium-branded client hints — suppressing `webdriver` alone changed nothing); `zenika/alpine-chrome` ships Chrome 124, refused outright; real Chrome with the default `--headless=new` UA never cleared, because the UA says `HeadlessChrome`; real Chrome with a stock UA **and** a non-UTC clock zone cleared in ~4s. Hence `chrome/` — a Debian image with `google-chrome-stable`, a version-derived UA, and `TZ`/`BROWSER_TZ`. Chrome reads the zone *name* through ICU from `/etc/localtime`'s symlink target, ignoring the file's contents, so mounting the host's `/etc/localtime` does **not** work; `/etc/timezone` is mounted instead.
- **The browser is not in the API stack and must not be put back.** It's its own compose unit (`chrome/docker-compose.yml`) on a second machine, reached over the tailnet — it held 471 MiB on a 1974 MiB swapless VPS, and a residential egress scores better with Cloudflare anyway (ADR-0006). Consequences that constrain code: `BROWSER_WS_URL` must be a tailnet **IP** (a MagicDNS name 500s at `/json/version`, same trap as the old Docker service name); the CDP port binds to the tailnet address only, since CDP authenticates nothing and that host has a real LAN; and the browser is on-demand (ADR-0005), so an unreachable or asleep one must degrade exactly as an unset `BROWSER_WS_URL` — plain-TLS libraries unaffected, kagane/novelfull logged and skipped, stored covers still served. Never add `chromedp.NoModifyURL`: discovery per fetch is what makes a restarted Chrome invisible.
- **UTC is the tell, not a country mismatch.** A UTC clock is the datacenter default, so Cloudflare scores it as one; any real zone clears. Measured 2026-08-08, identical container, one Indonesian egress IP: UTC never cleared in 60s (twice), while `Asia/Jakarta` **and** `America/New_York` both cleared in 4s. An earlier note here claimed the zone had to match the egress IP's country — that was wrong, inferred from the host clock (`Asia/Bangkok`) rather than the measured egress. `BROWSER_TZ` therefore needs a plausible zone, not a geolocated one.
- **A challenged page needs the tab kept open.** The interstitial takes seconds to solve and only then writes clearance into the browser's shared cookie jar. Navigate-read-close never clears anything; `BrowserFetcher.run` holds one tab and re-reads until the payload arrives.
## Architecture
```
Bromite userscript (isolated world, per-site adapters, localStorage cache)
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume)
Two Violentmonkey userscripts (isolated world, per-site adapters, localStorage cache)
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> Postgres (volume)
|
| CDP over tailnet
v
on-demand Chrome, separate machine (chrome/)
```
- **Backend** (`backend/`): stdlib `net/http` (handful of routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`.
- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`). Sync **last-write-wins**. Schema + endpoint list in plan.
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- **Web UI:** same binary serves password-gated browser UI on second
hostname — `GET /` (list, or login page when no session),
`POST /login`, `POST /logout`, `GET /static/*`, htmx fragment endpoints
under `/ui/*`. Templates/assets `go:embed`-ed, so `backend/Dockerfile`
must copy `templates/` and `static/` plus `*.go`. Sessions = stateless
HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them, when empty
web routes not registered at all. UI mutations read-modify-write
through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one
place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`.
- **Latest-chapter poller:** ticker goroutine in same binary re-checks
each bookmarked series' newest published chapter from backend's own
network access, so `latest_chapter` stays fresh when user not
browsing. Second, parallel signal — userscript keeps own
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
Two independent clocks: per-bookmark cooldown (`latest_checked_at` column,
enforced by `Store.DueForLatestCheck`'s WHERE clause) and wake interval.
Row stamped *before* fetch so broken series waits full
cooldown instead of retrying every tick; writes go through
`Store.Get` + `Store.Upsert` so new chapter never reorders list.
Fetches use `bogdanfinn/tls-client` w/ Chrome profile as defence in depth
against fingerprint-based blocking; any failure logs and skips. See
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
Poller's `Store.Get` + `Store.Upsert` not wrapped in transaction, so
userscript `PUT` committing between the two can be overwritten by
poller's stale re-read — reverting read progress and, since stored
value now differs, moving `updated_at` and reordering list. Known,
accepted limitation for single-user deployment, not bug to fix.
- **`updated_at` drives list order, moves only on real reading progress:** server applies timestamp when row new or `last_chapter_num` changes, else keeps stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**; clients must adopt that response over own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
`finished`, orthogonal to `favorite`. Archived and finished appear only in
own tab — not All, Updated, Favourites, or recent strip. Poller keeps
checking archived series, skips finished ones. `finished` settable only
from web UI; `PUT /bookmarks/{key}` rejects it w/ 400.
**Empty incoming status means "keep stored one"** — resolved on
`VALUES` side of `Store.Upsert`, not conflict clause, since
`excluded.*` = post-evaluation row and default applied there'd
wipe bucket on every PUT from client predating column. See
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list), `DB_PATH`
(default `/data/bookmarks.db`), `PORT` (default `8080`), `WEB_PASSWORD`
(gates browser UI; unset disables it),
`LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER`
(background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`).
`USERSCRIPT_PATH` (file served at `/u/{token}/manga-bookmark.user.js`,
default `/userscript/manga-bookmark.user.js`, supplied by a bindmount).
### Userscript structure (single IIFE, `manga-bookmark.user.js`)
1. **Site adapters** — one per host, `detect(location, document)` returns page `type` + IDs. ID type/IDs from **URL regex** (most stable); pull `title`/`cover` from **`og:title`/`og:image` meta tags**, not CSS classes.
2. **API client** — `apiGet/apiPut/apiDelete` w/ bearer header; `localStorage` key `mangabm:cache` for instant render + offline fallback.
3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value.
4. **Retry queue** — every write goes through `pushBookmark`/`pushDelete`, so
failed mutation parked in `localStorage` (`mangabm:queue`) and replayed on
next navigation, reconnect, or `refresh()`. Entries are markers
(`{key, op, sendStatus, attempts}`), never payloads — body read from
cache at send time, so one entry per key gives ordering + coalescing for
free. `sendStatus` **sticky**: while archive pending, later writes to
that key keep carrying bucket, stops successful
in-between write from silently un-archiving series. `refresh()` drains
before fetching, overlays anything still pending, so list never
flaps. 400 drops entry, 401 aborts pass and keeps queue,
transient failures retry to cap of 10. Latest-chapter writes deliberately
stay out of queue. See
`docs/superpowers/specs/2026-07-27-offline-retry-queue-design.md`.
5. **UI** — rendered inside **Shadow DOM** root to isolate from site CSS
(critical on mobile). Three tabs (All / Favourites / Archived) + row of
link chips to web UI and both manga sites; `WEB_BASE` sits in CONFIG
block next to `API_BASE`.
6. **SPA navigation** — Asura is Astro, client-routed on comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fires w/o reload. Demonic uses classic reloads (initial `document-idle` run suffices).
### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trusting)
- **asurascans.com**: series `/comics/<slug>` (slug carries a trailing
site-wide build-hash suffix, e.g. `-059befe1`, that **rotates on every
redeploy**), chapter `/comics/<slug>/chapter/<n>`. `seriesId` must strip
the hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in the userscript,
`asuraBuildHash` in the backend); URLs keep the full slug — stale-hash
URLs 302 to current ones. Astro-rendered; chapter links present in raw
server HTML.
- **demonicscans.org**: series `/manga/<slug>` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title/<slug>/chapter/<n>/<page>` (older `chaptered.php?manga=<id>&chapter=<n>` form still exists as redirect, what series-page chapter-list anchors link through).
Encodings (incl. triple-encoded punctuation like `%25252D`) are identical
on /manga/ and /title/ pages, so decode-once seriesIds match — verified
2026-07-28.
Two deployable units on two machines: the API stack (`docker-compose.yml` + `docker-compose.prod.yml`, on the VPS) and the browser (`chrome/docker-compose.yml`, on the home machine). They share nothing but `BROWSER_WS_URL` and update independently. Backend-specific architecture (packages, endpoints, poller, config env vars) lives in `backend/AGENTS.md`. Userscript-specific structure (adapters, retry queue, UI, live URL shapes) lives in `userscript/AGENTS.md`. Deploy order `DEPLOY.md` (§7 for the browser), redeploy `REDEPLOY.md` (§8 for the browser).
## Commands
Backend (`cd backend`):
- Test all: `go test ./...`
- Test all: `go test ./...` — **needs Docker.** Each test package starts a throwaway `postgres:17-alpine` container (`internal/pgtest`).
- Single test: `go test -run TestName ./...`
- Build static binary: `CGO_ENABLED=0 go build`
Local stack: `docker compose up` (named volume mounted at `/data`, `restart: unless-stopped`).
Local stack: `docker compose up` (bookmark-api + postgres only; `postgres-data` named volume, `restart: unless-stopped`). No browser — without `BROWSER_WS_URL` the poller logs and skips kagane and novelfull. To run one: `cd chrome && BROWSER_BIND_ADDR=172.17.0.1 docker compose up -d --build`, then `BROWSER_WS_URL=ws://172.17.0.1:9222` in the root `.env` (bridge gateway, so the API container can name it by IP).
Smoke test: `curl` endpoints w/ `Authorization: Bearer <token>`; confirm `OPTIONS` preflight returns CORS headers and `/healthz` returns 200.
Live CDP proof (needs that browser and network, skipped otherwise):
`SMOKE_BROWSER_WS_URL=ws://<ip>:<port> go test -run TestSmokeKagane ./internal/latest`
— fetches a real kagane cover and chapter list. A red run means the challenge is
not clearing from this IP, which is a live fact to re-check, not necessarily a defect.
Smoke test: `curl` endpoints with `Authorization: Bearer <token>`; confirm `OPTIONS` preflight return CORS headers and `/healthz` return 200.
## Forge: Gitea, not GitHub
`origin` = self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` doesn't work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones:
`origin` is self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` don't work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones:
- Open PR: `tea pr create --head <branch> --base main --title "..." --description "..."`
- List / view / check out: `tea pr list`, `tea pr <n>`, `tea pr checkout <n>`
- Issues: `tea issue create`, `tea issue list`
- Auth lives in `tea login`, not `GH_TOKEN` env var.
`tea` prints output as rendered boxes not plain text; PR URL lands on last line.
`tea` print output as rendered boxes rather than plain text; PR URL lands on last line.
## Design system
Web UI + userscript panel follow **Cinder**, rules in `docs/design-system.md`
— source of truth Claude Design project `BookmarkManager Web UI`
(`969ac210-fe02-4c01-ae1b-9a271dcc779a`). Read it before touching
`backend/internal/web/static/style.css`, `backend/internal/web/templates/*`, or userscript
`TEMPLATE`/`CSS`. Core law: **ember means new chapter only** — no other
state (busy, error, destruction) may use `--ember`; destruction gets
`--danger`. No cards/corners/shadows, one `--measure: 760px` column, tokens
only (never hardcode hex outside `:root`), both colour branches touched
together. Any move that pulls series out of list (archive/finish/remove)
must be confirm-gated via its own `.confirm-row`; only restore fires
instantly.
## Security invariants
- Auth on `/bookmarks*`: require `Authorization: Bearer <API_TOKEN>`, **constant-time compare**, 401 otherwise.
- CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` w/ `204`.
Existing guarantees — don't regress:
## Relevant skills
- Auth on `/bookmarks*`: require `Authorization: Bearer <credential>` — the acting Reader's credential, matched by SHA-256 against `readers.token_sha256` — **constant-time compare** (via the hash, never the secret itself), 401 otherwise.
- CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` with `204`.
`multi-stage-dockerfile` and `docker-compose-orchestration` for container work (referenced in plan).
## Secure coding rules (code you write here)
Anchored to OWASP Top 10 / ASVS. Every rule below already has a working example in-tree — match it, don't start a second convention. AI-written backends fail on exactly these: broken access control, injection, weak session/error handling, invented dependencies.
Go backend:
- SQL always parameterized (`$N`). Only compile-time constants (`bookmarkColumns`) may be concatenated into query text — never a request value, not even a validated one.
- `html/template` only for anything a browser parses, never `text/template`. Never wrap stored or fetched strings in `template.HTML`/`JS`/`URL`; that switches off the escaping every template depends on.
- Any outbound fetch of a client-supplied URL passes `fetchableSeriesURL` (site + `https` + host check) first. `series_url` arrives in a PUT body, so without the gate the poller will probe arbitrary hosts from the server's own network position. New fetch path reuses the gate rather than re-deriving one.
- Cap every remote body with `io.LimitReader` (`maxBodyBytes`). An unbounded read is an OOM handed to whatever is on the other end.
- Compare secrets with `hmac.Equal` / `subtle.ConstantTimeCompare`, never `==`. A credential is matched by the SHA-256 the `readers` table holds, which is already a fixed-width equality — a new secret comparison must not regress to `==`.
- Errors: generic text to the client (`http.Error(w, "internal error", 500)`), detail to `log.Printf`. Never log `TOKEN_KEY`, a Reader's credential, `DISCORD_CLIENT_SECRET`, a session id, or a whole `Authorization` header.
- Proxy headers are trusted only where they already are: `X-Forwarded-Proto` for the Secure cookie flag, **rightmost** `X-Forwarded-For` for client IP (leftmost is attacker-supplied). Don't read either anywhere else.
- Session cookies keep `HttpOnly`, `SameSite`, `Secure`-when-HTTPS; expiry is enforced by the `sessions` table lookup, not a signature.
- Stdlib crypto only. No hand-rolled hashing, no MD5/SHA-1 anywhere security-bearing.
- Validate at the handler boundary before storing: body capped by `http.MaxBytesReader` (64 KB), empty `key` and unknown `status`/`kind` rejected with `400`. A bad value that reaches the store becomes every later reader's problem.
Userscript:
- Site-derived and stored strings render via `el(..., {text})` / `textContent`. `{html}` and `innerHTML` are for author-written literal markup only (`TEMPLATE`, `CSS`) — never a title, chapter label, or API response field. The page DOM belongs to a third-party site; treat it as attacker-controlled.
- Isolated world protects the credential from the site's JS. It does not protect anything from an `innerHTML` sink you add yourself.
- The userscripts carry `__API_TOKEN__` placeholders, substituted at serve time with the requesting Reader's credential (`internal/userscript`). Never put a real credential in the repo, docs, commit messages, or issues. Rotation is a web-UI action (epoch bump, `internal/token`); `TOKEN_KEY` in backend env is what derives every credential — never log it.
- `fetch()` targets `API_BASE` only — no dynamic origin, no site-supplied URL. `authHeaders()` goes nowhere but the backend.
- `localStorage` is shared with the site's own JS: cache and queue live there, credentials never do.
- Wrap every `localStorage` read/write and `JSON.parse` in try/catch (quota, private mode, corrupt entry), as the existing helpers do.
Dependencies: stdlib first; a new module needs a stated reason. Confirm a package actually exists before adding it — a plausible name may be fiction (~20% of LLM-proposed packages don't resolve, which is how slopsquatting lands). Pin exact versions.
Review gate: auth, CORS, session, crypto, and the fetch gate are security-critical. Editing one is not a drive-by change — say which invariant you preserved and run `go test ./...` before calling it done.
## Comments
Comment only if code alone can't carry info. Cost per read — must earn spot.
Write for:
- Why not what. Tradeoffs, non-obvious decisions.
- Load-bearing detail looking incidental — say so if "simplify" breaks it.
- Non-local consequence, invisible from function alone.
- Wire format / encoding / interface contract — save callers re-deriving.
- Gotcha/workaround, with ref if exists.
- Domain/business rule not derivable from code.
Skip:
- Restating code (no `// increment i` above `i++`).
- Trivial getter/setter/pass-through.
- Banners, dividers, `// helpers`.
- Change narration (`// fix bug`, `// as requested`, `// new impl`) — git's job.
- Commented-out code — delete.
- TODO without concrete action.
Style: one dense comment over function beats one per line inside. Tight, no worked example unless bug subtle. Wrong comment worse than none — update/delete on change. Default fewer — sparse+high-signal beats comprehensive.
Test: "competent reader get this from code in few sec?" Yes → skip. Needs detour through another file/spec/git-blame → write it.
## Agent skills
`AGENTS.md` is the single source of truth for agent guidance; every `CLAUDE.md` in this repo is a symlink to the `AGENTS.md` beside it. Edit `AGENTS.md`.
### Issue tracker
Issues live as Gitea issues on `gitea.violetcrown.my.id` (`sulthan/mangaBookmark`), driven by the `tea` CLI — not `gh`. See `docs/agents/issue-tracker.md`.
### Triage labels
Default five-role vocabulary, label strings unchanged (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`). See `docs/agents/triage-labels.md`.
### Domain docs
Single-context: one root `CONTEXT.md` plus `docs/adr/`, both created lazily. See `docs/agents/domain.md`.
## graphify
Project has knowledge graph at graphify-out/ w/ god nodes, community structure, cross-file relationships.
Project has knowledge graph at graphify-out/ with god nodes, community structure, cross-file relationships.
Rules:
- For codebase questions, first run `graphify query "<question>"` when graphify-out/graph.json exists. Use `graphify path "<A>" "<B>"` for relationships, `graphify explain "<concept>"` for focused concepts. Return scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output.
- For codebase questions and exploration, always first run `graphify query "<question>"` when graphify-out/graph.json exists. Use `graphify path "<A>" "<B>"` for relationships and `graphify explain "<concept>"` for focused concepts. Return scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output.
- If graphify-out/wiki/index.md exists, use for broad navigation instead of raw source browsing.
- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain don't surface enough context.
- After modifying code, run `graphify update .` to keep graph current (AST-only, no API cost).
## OpenCode-specific
- Caveman mode active by default (`/home/tan/.config/opencode/AGENTS.md`). Keep comms terse — drop articles, fluff, pleasantries. Code/commits/security written normal.
- `.superpowers/` and `.agents/` dirs hold skill definitions. Gitea at `gitea.violetcrown.my.id`.
-162
View File
@@ -1,162 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Status
Greenfield. Only `plans/mangaBookmark.md` exists — no code yet. That plan is the spec; read it before building. Two deliverables: a Go sync backend and a single Bromite-compatible userscript.
## What this is
A manga read-progress tracker for a user reading on **asurascans.com** (the current domain; asuracomic.net 301s here) and **demonicscans.org** from **Bromite** (mobile Chromium). A userscript injects on-page UI (floating button + slide-in panel) and syncs progress to a self-hosted Go backend so bookmarks unify across both sites and across devices.
## Hard constraints (these drive the design — do not violate)
Bromite uses Chromium's **native** userscript engine, not Tampermonkey:
- **No `GM_*` APIs anywhere.** No `GM_setValue`/`GM_getValue` (use page `localStorage`), no `GM_registerMenuCommand` (inject on-page UI), no `GM_xmlhttpRequest` for cross-origin (use plain `fetch()`). Keeping the script GM-free also lets it run in desktop Tampermonkey/Violentmonkey for faster iteration.
- Cross-origin `fetch()` works **only** against a CORS-enabled backend. Manga sites are `https://`, so backend **must be HTTPS** (mixed-content block otherwise).
- Asura and Demonic are **separate origins with separate `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync is required, not optional.
- Userscript runs in an **isolated world**, so the embedded API token is safe from the site's JS.
- Cloudflare's block on fetching the manga sites is **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both the CGNAT dev machine *and* the deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. This contradicts an earlier, untested assumption that the CGNAT dev IP would be blocked; it was not, at least on this date. Treat "does curl work right now" as a live, time-varying fact to re-check, not a fixed property of a given machine — Cloudflare's bot scoring can flip a previously-clean IP without notice. Any backend fetcher still needs a graceful-degrade path for when it does get challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, or a direct probe) before finalizing, not assumed from a single earlier test.
## Architecture
```
Bromite userscript (isolated world, per-site adapters, localStorage cache)
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume)
```
- **Backend** (`backend/`): stdlib `net/http` (a handful of routes, no framework) + `modernc.org/sqlite` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). The reverse proxy terminates TLS; the Go service listens plain `:8080`.
- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`). Sync is **last-write-wins**. Schema and endpoint list are in the plan.
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- **Web UI:** the same binary serves a password-gated browser UI on a second
hostname — `GET /` (list, or login page when there is no session),
`POST /login`, `POST /logout`, `GET /static/*`, and htmx fragment endpoints
under `/ui/*`. Templates and assets are `go:embed`-ed, so `backend/Dockerfile`
must copy `templates/` and `static/` as well as `*.go`. Sessions are stateless
HMAC cookies keyed off `API_TOKEN`; `WEB_PASSWORD` gates them and, when empty,
the web routes are not registered at all. UI mutations read-modify-write
through `Store.Get` + `Store.Upsert` so the `updated_at` rule stays in one
place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`.
- **Latest-chapter poller:** a ticker goroutine in the same binary re-checks
each bookmarked series' newest published chapter from the backend's own
network access, so `latest_chapter` stays fresh when the user is not
browsing. It is a *second, parallel* signal — the userscript keeps its own
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
Two independent clocks: a per-bookmark cooldown (`latest_checked_at` column,
enforced by `Store.DueForLatestCheck`'s WHERE clause) and a wake interval.
The row is stamped *before* the fetch so a broken series waits out a full
cooldown instead of retrying every tick, and writes go through
`Store.Get` + `Store.Upsert` so a new chapter never reorders the list.
Fetches use `bogdanfinn/tls-client` with a Chrome profile as defence in depth
against fingerprint-based blocking; any failure logs and skips. See
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
The poller's `Store.Get` + `Store.Upsert` is not wrapped in a transaction, so
a userscript `PUT` that commits between the two can be overwritten by the
poller's stale re-read — reverting that read progress and, since the stored
value now differs, moving `updated_at` and reordering the list. This is a
known, accepted limitation for a single-user deployment, not a bug to fix.
- **`updated_at` drives list order, so it moves only on real reading progress:** the server applies its timestamp when the row is new or `last_chapter_num` changes, and otherwise keeps the stored value — favouriting a series or recording a newly published chapter must not reorder the list. `PUT` therefore returns the row **as stored**, and clients must adopt that response rather than their own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
`finished`, orthogonal to `favorite`. Archived and finished appear only in
their own tab — not in All, Updated, Favourites, or the recent strip. The
poller keeps checking archived series and skips finished ones. `finished` is
settable only from the web UI; `PUT /bookmarks/{key}` rejects it with 400.
**An empty incoming status means "keep the stored one"** — resolved on the
`VALUES` side of `Store.Upsert`, not in the conflict clause, because
`excluded.*` is the post-evaluation row and a default applied there would
wipe the bucket on every PUT from a client that predates the column. See
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
- **Config via env:** `API_TOKEN`, `ALLOWED_ORIGINS` (comma list), `DB_PATH`
(default `/data/bookmarks.db`), `PORT` (default `8080`), `WEB_PASSWORD`
(gates the browser UI; unset disables it),
`LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER`
(background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`).
`USERSCRIPT_PATH` (file served at `/u/{token}/manga-bookmark.user.js`,
default `/userscript/manga-bookmark.user.js`, supplied by a bindmount).
### Userscript structure (single IIFE, `manga-bookmark.user.js`)
1. **Site adapters** — one per host, `detect(location, document)` returns page `type` + IDs. Identify type/IDs from **URL regex** (most stable); pull `title`/`cover` from **`og:title`/`og:image` meta tags**, not CSS classes.
2. **API client** — `apiGet/apiPut/apiDelete` with bearer header; `localStorage` key `mangabm:cache` for instant render + offline fallback.
3. **Progress logic** — auto-upsert `last_chapter` only when `chapterNum >= stored last_chapter_num` (re-reading old chapters must not regress progress; unparseable -> set current). Manual panel override forces any value.
4. **Retry queue** — every write goes through `pushBookmark`/`pushDelete`, so a
failed mutation is parked in `localStorage` (`mangabm:queue`) and replayed on
the next navigation, reconnect, or `refresh()`. Entries are markers
(`{key, op, sendStatus, attempts}`), never payloads — the body is read from
the cache at send time, so one entry per key gives ordering and coalescing for
free. `sendStatus` is **sticky**: while an archive is pending, later writes to
that key keep carrying the bucket, which is what stops a successful
in-between write from silently un-archiving the series. `refresh()` drains
before it fetches and overlays anything still pending, so the list never
flaps. A 400 drops the entry, a 401 aborts the pass and keeps the queue, and
transient failures retry to a cap of 10. Latest-chapter writes deliberately
stay out of the queue. See
`docs/superpowers/specs/2026-07-27-offline-retry-queue-design.md`.
5. **UI** — rendered inside a **Shadow DOM** root to isolate from site CSS
(critical on mobile). Three tabs (All / Favourites / Archived) and a row of
link chips to the web UI and both manga sites; `WEB_BASE` sits in the CONFIG
block next to `API_BASE`. The FAB is a `7 × 44` edge tab whose *hit* area is
widened to `28 × 72` by an invisible `#hit` child; `#fab` must keep
`touch-action: none` and must **not** regain `overflow: hidden`. Because
`touch-action` is resolved at gesture start, the strip cannot be both
browser-scrolled and script-dragged, so `makeDraggable` splits by intent: a
swipe from `#hit` scrolls via `window.scrollBy`, a hold of `ARM_MS` arms a
reposition drag, and the visible sliver drags with no hold. See
`docs/superpowers/specs/2026-07-28-edge-tab-hitbox-design.md`.
6. **SPA navigation** — Asura is Astro, client-routed on the comic/chapter pages: patch `history.pushState`/`replaceState` + listen `popstate`, re-run `detect()` on URL change so auto-update fires without reload. Demonic uses classic reloads (initial `document-idle` run suffices).
### Live URL shapes (verified 2026-07-26, may drift — re-check against live pages before trusting)
- **asurascans.com**: series `/comics/<slug>` (slug carries a trailing
site-wide build-hash suffix, e.g. `-059befe1`, that **rotates on every
redeploy**), chapter `/comics/<slug>/chapter/<n>`. `seriesId` must strip
the hash (`/-[0-9a-f]{8}$/`, `stripBuildHash` in the userscript,
`asuraBuildHash` in the backend); URLs keep the full slug — stale-hash
URLs 302 to current ones. Astro-rendered; chapter links present in raw
server HTML.
- **demonicscans.org**: series `/manga/<slug>` (slug may URL-encode punctuation, e.g. `%2527` for `'`), chapter `/title/<slug>/chapter/<n>/<page>` (older `chaptered.php?manga=<id>&chapter=<n>` form still exists as redirect, what series-page chapter-list anchors link through).
Encodings (incl. triple-encoded punctuation like `%25252D`) are identical
on /manga/ and /title/ pages, so decode-once seriesIds match — verified
2026-07-28.
## Commands (once code exists)
Backend (`cd backend`):
- Test all: `go test ./...`
- Single test: `go test -run TestName ./...`
- Build static binary: `CGO_ENABLED=0 go build`
Local stack: `docker compose up` (named volume mounted at `/data`, `restart: unless-stopped`).
Smoke test: `curl` the endpoints with `Authorization: Bearer <token>`; confirm `OPTIONS` preflight returns CORS headers and `/healthz` returns 200.
## Forge: Gitea, not GitHub
`origin` is a self-hosted Gitea instance (`gitea.violetcrown.my.id`), so **`gh` does not work here — use `tea` (Gitea CLI) for anything past plain git.** Common ones:
- Open a PR: `tea pr create --head <branch> --base main --title "..." --description "..."`
- List / view / check out: `tea pr list`, `tea pr <n>`, `tea pr checkout <n>`
- Issues: `tea issue create`, `tea issue list`
- Auth lives in `tea login`, not a `GH_TOKEN` env var.
`tea` prints its output as rendered boxes rather than plain text; the PR URL lands on the last line.
## Security invariants
- Auth on `/bookmarks*`: require `Authorization: Bearer <API_TOKEN>`, **constant-time compare**, 401 otherwise.
- CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` with `204`.
## Relevant skills
`multi-stage-dockerfile` and `docker-compose-orchestration` for the container work (referenced in the plan).
## graphify
This project has a knowledge graph at graphify-out/ with god nodes, community structure, and cross-file relationships.
Rules:
- For codebase questions, first run `graphify query "<question>"` when graphify-out/graph.json exists. Use `graphify path "<A>" "<B>"` for relationships and `graphify explain "<concept>"` for focused concepts. These return a scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output.
- If graphify-out/wiki/index.md exists, use it for broad navigation instead of raw source browsing.
- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain do not surface enough context.
- After modifying code, run `graphify update .` to keep the graph current (AST-only, no API cost).
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+95
View File
@@ -0,0 +1,95 @@
# Bookmark Manager
Read-progress tracker for serialised fiction. A reader browses third-party manga and
novel sites; userscripts capture where they got to and sync it to a self-hosted backend,
so progress survives across sites and devices.
## Language
**Series**:
One ongoing work — a manga or a novel — as published by a Site. Identified by the canonical
slug the Site itself publishes for it, never by its title and never by a Chapter Slug. A
Series exists once and is shared by every Reader who bookmarks it; it owns the facts that
are true regardless of who is reading — title, cover, Latest Chapter. A Reader cannot
change them; they describe the Series, not anyone's relationship to it.
_Avoid_: manga, title, book, comic
**Site**:
One third-party source a Series is published on. A Series on two Sites is two Series.
_Avoid_: source, host, provider, domain
**Chapter Slug**:
A slug a Site builds its chapter addresses from. Not an identity: one Series may have
several, any of them may differ from the slug that identifies the Series, and none is
computable from another. Only the Site's own links say which ones a Series uses, so a
Chapter Slug is always discovered, never derived.
_Avoid_: series slug, url slug, permalink, chapter path
**Cover**:
The image that stands for a Series wherever it is listed. A fact about the Series like
its title — one Cover per Series, shared by every Reader, never per-Reader. Defined by
what a Reader's browser can display, not by where the Site keeps the picture: an address
no client can load is not a Cover, it is a missing one.
_Avoid_: thumbnail, poster, image URL, artwork
**Reader**:
A person with their own Progress. Exactly one per set of credentials, so there is no
separate "account" concept to model — the credential belongs to the Reader.
_Avoid_: user, account, member, subscriber
**Bookmark**:
One Reader's tracked relationship with one Series, holding only what differs between
Readers: Progress, Favourite, Lifecycle bucket. Facts about the Series itself belong
to the Series, not here.
_Avoid_: entry, item, record, subscription
**Library**:
One of the two halves of the collection — manga or novel — selected by a Bookmark's
`kind`. The web UI and the userscripts each address exactly one Library at a time.
Not a per-person concept: "everything one person has bookmarked" is a different idea
and must not be called a Library.
_Avoid_: section, tab, category
**Progress**:
The furthest chapter a reader has actually read in a Series. Only a change in Progress
is real activity, so only Progress reorders the list.
_Avoid_: position, bookmark (the noun is taken), last read
**Latest Chapter**:
The highest-numbered chapter a Site has published for a Series, discovered without the
reader present. The number is what ranks it, never a date and never the Site's own
"newest chapter" banner — where a Site disagrees with itself, its list of chapters is
the record and its summary of that list is not. Distinct from Progress in every way
that matters: it is a fact about the Site, not about the reader, and it must never
reorder the list.
_Avoid_: newest, current chapter, update
**Poll**:
The backend's own check of a Site for a Series's Latest Chapter, made without the
Reader present. Performed once per Series no matter how many Readers bookmarked it —
a Poll is work done on behalf of the Series, never on behalf of a Reader.
_Avoid_: scrape, refresh, check, sync
**Acquisition**:
The single read of a Series page made the moment the Series first exists, giving it
both its Latest Chapter and its Cover without waiting out the Poll queue. Distinct
from a Poll in the two ways that matter: a Reader is present — it is triggered by
their first Bookmark of that Series — and it is the only read that establishes a
Cover rather than refreshing facts. It happens once in a Series's life; every later
read of the same page is a Poll.
_Avoid_: initial poll, first fetch, prefetch, warm-up
**New Chapter**:
The state where Latest Chapter is ahead of Progress. The single condition the ember
accent is permitted to signal.
_Avoid_: unread, update available
**Lifecycle bucket**:
Which of three mutually exclusive states a Bookmark sits in — reading, archived, or
finished. A Bookmark is in exactly one. Orthogonal to being a favourite.
_Avoid_: state, status (as a domain word), list
**Favourite**:
A reader's manual pin on a Bookmark. Orthogonal to the Lifecycle bucket, and never a
reason to reorder the list.
_Avoid_: starred, pinned, priority
+299
View File
@@ -0,0 +1,299 @@
# SQLite → Postgres cutover runbook
One-way, one-time. Moves the owner's reading history out of the retired SQLite
volume (`<compose project>_bookmarks-data`, holding `/data/bookmarks.db`) and into
the Postgres schema the migration runner builds. There is no dual-write period:
the old database is read once, at cutover, from a **fresh export** — anything
written to SQLite after the export is lost, so the old API must already be down.
Routine deploys are `REDEPLOY.md`; first-time setup is `DEPLOY.md`. This file is
run once and then only ever read for reference.
Proven end to end on 2026-08-08 against a copy of `bookmarks-20260807-213515.db`
into a scratch Postgres: 29 Bookmarks (18 reading, 11 archived, 7 favourites),
29 Series, all owned by the seeded Reader, and every field of every row matching
the source exactly. Production was not touched.
---
## 0. The generator is throwaway
It is written at cutover, run once, and deleted. It is deliberately **not** in
this repository and never will be:
- Its output is the owner's personal reading history. That does not enter
version control.
- It reads SQLite. The backend module dropped `modernc.org/sqlite` (ADR-0001);
a committed generator would drag the dependency back in through the side door.
So §3 specifies the transformation rather than shipping a script. It is a
twenty-line program against a sixteen-column table (fifteen after `key`, which
is dropped) — writing it from the spec below costs less than maintaining it
would.
Beyond `DEPLOY.md`'s prerequisites (Docker and Compose), this runbook needs
`python3`: its stdlib `sqlite3` module is the whole SQLite dependency, and §5's
read-path check uses it in place of `jq`, which the server does not have. It
does not have to run on the server — §3 only reads the snapshot copy, so it can
run on a laptop and the resulting `import.sql` be copied over.
---
## 1. Stop the old API and take a fresh export
**Order matters.** Export after the API stops, or you migrate a snapshot that is
already stale.
```bash
cd ~/mangaBookmark # wherever the checkout lives
COMPOSE="docker compose -f docker-compose.yml -f docker-compose.prod.yml"
BACKUP_DIR="$(cd .. && pwd)/$(basename "$PWD")-backups"; mkdir -p "$BACKUP_DIR"
STAMP=$(date -u +%Y%m%d-%H%M%S)
# The volume is <compose project>_bookmarks-data, and the project name defaults
# to the lowercased *directory* name, not the repo name — on this host the
# checkout is ~/mangaBookmark, so the volume is mangabookmark_bookmarks-data.
# Derive it exactly rather than with a `--filter name=` substring match, which
# would return every volume whose name merely contains the string.
VOL="$(basename "$PWD" | tr '[:upper:]' '[:lower:]')_bookmarks-data"
docker volume inspect "$VOL" >/dev/null && echo "$VOL"
$COMPOSE stop bookmark-api
# A clean SIGTERM closes the store, which checkpoints and unlinks the -wal, so
# bookmarks.db alone is then the whole database. But `compose stop` SIGKILLs
# after 10s, and a surviving -wal holds writes the main file does not — assert
# it is gone rather than assuming the shutdown was clean.
docker run --rm -v "$VOL":/d:ro alpine ls -l /d # -> bookmarks.db, alone
docker run --rm -v "$VOL":/from:ro -v "$BACKUP_DIR":/to \
alpine cp /from/bookmarks.db "/to/bookmarks-$STAMP.db"
ls -lh "$BACKUP_DIR/bookmarks-$STAMP.db"
```
If `-wal` and `-shm` are still there, the container was killed mid-write. Copy
all three under the same basename and let SQLite replay the log when §3 opens
it — copying only `bookmarks.db` silently drops whatever the log still holds.
Work on a **copy** of that file for the rest of this runbook. The export is the
last line of retreat; nothing below should be able to write to it.
```bash
mkdir -p /tmp/cutover && cp "$BACKUP_DIR/bookmarks-$STAMP.db" /tmp/cutover/snapshot.db
chmod 444 /tmp/cutover/snapshot.db
```
---
## 2. Bring up Postgres with the schema and the owner Reader
The new stack builds its own schema and seeds exactly one Reader from
`OWNER_DISCORD_ID` — do not hand-write either. Pull the Postgres-era commit
first: on a server that has only ever run the SQLite build, `--build` without a
pull silently rebuilds the old image and the checks below fail with
"relation readers does not exist".
```bash
git pull --ff-only
git log --oneline -1
# .env needs the new required vars (DATABASE_URL is built from
# POSTGRES_PASSWORD; TOKEN_KEY, OWNER_DISCORD_ID and the DISCORD_* set are
# required). Compose fails at start for a missing one.
git diff HEAD@{1} HEAD -- .env.example docker-compose.yml docker-compose.prod.yml
$COMPOSE up -d --build
docker logs bookmark-api --tail 20 # -> "listening on :8080"
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -c '\dt'
# -> bookmarks, readers, schema_migrations, series, sessions
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks \
-c 'select id, discord_id from readers'
# -> exactly one row, and discord_id is the owner's
```
Two rows in `readers`, or zero, means `OWNER_DISCORD_ID` is wrong or the seed
failed. Stop here — the import attaches history to "the oldest reader row", and
that is only unambiguous while there is one.
`bookmarks` and `series` are empty at this point. That is what makes the import
a plain sequence of `INSERT`s with no conflict handling.
---
## 3. Generate the import SQL
Read `/tmp/cutover/snapshot.db` and emit plain SQL on stdout. The old table is
flat and its columns map one-for-one onto the split schema — no transformation
beyond the split itself:
| SQLite `bookmarks` column | lands in | notes |
|---|---|---|
| `site`, `series_id` | both tables | the Series key; the wire `key` column is dropped, it is re-derived as `site:series_id` on read |
| `title`, `series_url`, `cover`, `kind` | `series` | shared facts (ADR-0003) |
| `latest_chapter`, `latest_chapter_num`, `latest_checked_at` | `series` | `latest_chapter_num` is nullable on **both** sides and `NULL` is meaningful — never coerce it to `0` |
| `last_chapter`, `last_chapter_num`, `last_chapter_url` | `bookmarks` | Progress |
| `favorite`, `status`, `updated_at` | `bookmarks` | `favorite` is `0`/`1` in SQLite and a real `boolean` in Postgres — emit `true`/`false` |
| — | `bookmarks.reader_id` | the seeded owner |
**`latest_chapter_num` is the only column where `NULL` survives.** The SQLite
table declares `title`, `series_url`, `cover`, `last_chapter`,
`last_chapter_url` as bare `TEXT` and `last_chapter_num` as bare `REAL` — all
six nullable — while their Postgres targets are `NOT NULL DEFAULT ''` /
`NOT NULL DEFAULT 0`. One `NULL` in any of them aborts the whole import on a
not-null violation. Coalesce them in the `SELECT` (`ifnull(title,'')`,
`ifnull(last_chapter_num,0)`, …) rather than discovering it at §5. The
2026-08-07 export happened to have none; a fresh export is not promised the
same.
Rules the generator must follow:
- **Series first, Bookmarks second.** `bookmarks` has a foreign key onto
`series (site, series_id)`; the reverse order fails on the first row.
- **`SELECT DISTINCT` the Series.** The old key's uniqueness already makes
`(site, series_id)` unique, so this is belt and braces — but if it ever
collapses two rows, the count check in §5 catches it.
- **Never hardcode the reader id.** Emit
`INSERT INTO bookmarks (reader_id, …) SELECT id, … FROM owner`, where `owner`
is a temp table built once at the top:
`CREATE TEMP TABLE owner ON COMMIT DROP AS SELECT id FROM readers ORDER BY id LIMIT 1;`
A literal id is a number nobody verifies; this one cannot be wrong.
- **Wrap the whole file in `BEGIN; … COMMIT;`, temp table included.** Postgres
has transactional DDL and DML: a failure half way leaves an empty database
rather than half a library. The ordering is load-bearing —
`ON COMMIT DROP` outside the transaction means the temp table drops itself
the instant it is created (psql autocommits) and every
`SELECT … FROM owner` then fails.
- **Quote strings by doubling `'`.** Titles contain apostrophes and the URLs
contain `%5C%27` escapes. Emit standard SQL literals only — no `E''` strings,
no backslash escaping (`standard_conforming_strings` is on, so a backslash is
a literal backslash and the URLs survive verbatim).
```bash
python3 gen_import.py /tmp/cutover/snapshot.db > /tmp/cutover/import.sql
wc -l /tmp/cutover/import.sql # -> 2 header + 29 series + 29 bookmarks + framing
```
---
## 4. Review it by eye
29 rows is small enough to actually read, and this is the last point at which a
mistake is free:
```bash
less /tmp/cutover/import.sql
grep -c '^INSERT INTO series' /tmp/cutover/import.sql # -> 29
grep -c '^INSERT INTO bookmarks' /tmp/cutover/import.sql # -> 29
```
Look for: a title whose apostrophe is not doubled, a `favorite` that is still
`0`/`1`, a `latest_chapter_num` that turned into `0`, and any `reader_id`
written as a bare number.
---
## 5. Apply it
```bash
docker cp /tmp/cutover/import.sql "$($COMPOSE ps -q postgres)":/tmp/import.sql
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -v ON_ERROR_STOP=1 \
-f /tmp/import.sql
```
`ON_ERROR_STOP=1` is not optional: without it `psql` reports the error, keeps
going, and exits `0` on a half-imported database.
Then the checklist. Every number here is asserted, not eyeballed:
```bash
OWNER=$(grep -E '^OWNER_DISCORD_ID=' .env | cut -d= -f2)
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -x -c "
SELECT (SELECT count(*) FROM bookmarks) AS bookmarks_total,
(SELECT count(*) FROM bookmarks WHERE status='reading') AS reading,
(SELECT count(*) FROM bookmarks WHERE status='archived')AS archived,
(SELECT count(*) FROM series) AS series_total,
(SELECT count(*) FROM readers) AS readers_total,
(SELECT count(*) FROM bookmarks
WHERE reader_id <> (SELECT id FROM readers WHERE discord_id='$OWNER'))
AS not_owned_by_owner;"
```
`not_owned_by_owner` resolves the Reader by **Discord id**, not by
`ORDER BY id LIMIT 1`. The second form is the expression §3 tells the generator
to import with, so comparing against it is true by construction and could never
fail; resolving by Discord id is an independent check that the rows landed on
the identity the owner will actually log in as. If that subquery returns NULL
the whole count comes back `0` for the wrong reason — hence `readers_total`
beside it.
Expected, for the 2026-08-07 export: `29`, `18`, `11`, `29`, `1`, `0`. Against a
different export, the invariants rather than the literals are what hold:
- `bookmarks_total` equals the SQLite row count.
- `reading + archived` equals `bookmarks_total` (nothing was `finished`).
- `series_total` equals `SELECT count(*) FROM (SELECT DISTINCT site, series_id FROM bookmarks)`
in the source.
- `readers_total` is `1` and `not_owned_by_owner` is `0`.
Then spot-check the values themselves against the source — read position,
favourite flag and latest chapter. Take the sample from each bucket explicitly:
`ORDER BY updated_at DESC LIMIT 5` alone returns the most recently *progressed*
rows, which are the ones least likely to be archived.
```bash
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -c "
SELECT s.title, b.last_chapter, b.last_chapter_num, b.favorite,
s.latest_chapter, b.status
FROM bookmarks b JOIN series s USING (site, series_id)
WHERE b.status='reading' ORDER BY b.updated_at DESC LIMIT 3;"
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -c "
SELECT s.title, b.last_chapter, b.last_chapter_num, b.favorite,
s.latest_chapter, b.status
FROM bookmarks b JOIN series s USING (site, series_id)
WHERE b.status='archived' ORDER BY b.updated_at DESC LIMIT 2;"
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -c "
SELECT s.title, b.last_chapter, b.last_chapter_num, b.favorite,
s.latest_chapter, b.status
FROM bookmarks b JOIN series s USING (site, series_id)
WHERE b.favorite ORDER BY b.updated_at DESC LIMIT 2;"
```
Compare each against the same row in the snapshot — the generator's own source
is the reference, so read it back with the same `python3` you used in §3.
Finally, prove the **read path**, not just the tables — this is the check that
would catch a correct import behind a broken join:
```bash
API=https://bookmark-api.violetcrown.my.id
# Your own Reader credential: sign in to the web UI and take it from the
# Userscripts panel's install link, or read the API_TOKEN constant out of an
# already-installed script. There is no credential in .env to grep.
TOKEN=<your Reader credential>
curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks |
python3 -c 'import json,sys; print(len(json.load(sys.stdin)))' # -> 29
```
---
## 6. Afterwards
- **Keep the old SQLite volume for a month.** It is already undeclared in
compose, so `docker compose down -v` cannot take it. Remove it by hand once
the Postgres data has been trusted for a while. That happens in a shell where
`$VOL` from §1 is long gone, so re-derive it:
`docker volume rm "$(basename ~/mangaBookmark | tr '[:upper:]' '[:lower:]')_bookmarks-data"`
(see `REDEPLOY.md` §1).
- **Delete the generator and the working copies:** `rm -rf /tmp/cutover`. The
timestamped export in `$BACKUP_DIR` is the copy that is kept.
- **Take the first Postgres dump immediately** — `REDEPLOY.md` §1. Until that
exists, the only backup of the migrated data is the SQLite file it came from.
If the import is wrong, there is nothing to unpick: drop the rows and start
again from §3 — `TRUNCATE bookmarks, series;` leaves the seeded Reader and the
schema in place.
+360 -71
View File
@@ -10,8 +10,8 @@ ACME/cert resolver, and control a domain.
- Docker + Docker Compose on the server.
- A Traefik instance watching a Docker network (default name assumed: `proxy`).
- DNS: an `A`/`AAAA` record for `manga-api.<yourdomain>` pointing at the server.
- The repo copied to the server, e.g. `/opt/mangabm/` (needs `backend/`,
- DNS: an `A`/`AAAA` record for `bookmark-api.<yourdomain>` pointing at the server.
- The repo copied to the server, e.g. `~/mangaBookmark/` (needs `backend/`,
`docker-compose.yml`, `docker-compose.prod.yml`, `.env.example`).
Confirm the Traefik network exists (create if not):
@@ -25,24 +25,48 @@ docker network ls | grep proxy || docker network create proxy
## 1. Configure `.env`
```bash
cd /opt/mangabm
cd ~/mangaBookmark
cp .env.example .env
```
Edit `.env`:
```ini
# Required — long random secret, also goes in the userscript.
API_TOKEN=<paste output of: openssl rand -hex 32>
# Required — secret every Reader's userscript credential is derived from.
# Only SHA-256 hashes of credentials are stored.
TOKEN_KEY=<paste output of: openssl rand -hex 32>
# Required — the owner's Discord user ID. Seeds the first Reader: the
# administrator, and the owner of every bookmark that predates registration.
# The value is the snowflake in your Discord profile (Settings →
# Advanced → Developer Mode → right-click your name → Copy User ID).
OWNER_DISCORD_ID=<discord user id>
# CORS allowlist — leave as-is unless a site changes hostname.
ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org
ALLOWED_ORIGINS=https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to,https://novelfull.com,https://lightnovelworld.net
# Required — password for the bundled Postgres container. Compose builds the
# backend's DATABASE_URL out of it and has no fallback for either.
POSTGRES_PASSWORD=<paste output of: openssl rand -hex 24>
# Leave unset. Only set this to point the backend at a Postgres compose does
# not run; it then replaces the URL built from POSTGRES_PASSWORD above.
# DATABASE_URL=postgres://user:pass@host:5432/bookmarks?sslmode=require
# Required path inside bookmark-api. Compose builds the image and mounts the
# named cover-data volume at this path.
COVER_DIR=/covers
# Required — the origin this deployment answers on, no trailing slash. Cover
# URLs on the wire are absolute, because the userscript renders them on a
# Site's own origin (ADR-0007). Same host as BOOKMARK_API_HOST below.
PUBLIC_BASE_URL=https://bookmark-api.violetcrown.my.id
# Required for the Traefik override. Both have no fallback — compose refuses
# to start without them. MANGA_WEB_HOST is required even if you never set
# WEB_PASSWORD; see 1b.
MANGA_API_HOST=manga-api.violetcrown.my.id
MANGA_WEB_HOST=manga.violetcrown.my.id
# to start without them. BOOKMARK_WEB_HOST is required even if the web UI
# were unused; see 1b.
BOOKMARK_API_HOST=bookmark-api.violetcrown.my.id
BOOKMARK_WEB_HOST=bookmark.violetcrown.my.id
# Only if your Traefik setup differs from these defaults:
# PROXY_NETWORK=proxy
@@ -50,13 +74,25 @@ MANGA_WEB_HOST=manga.violetcrown.my.id
# TRAEFIK_CERTRESOLVER=le
```
Generate + insert the token in one line:
Generate + insert the two secrets in three lines:
```bash
sed -i "s|^API_TOKEN=.*|API_TOKEN=$(openssl rand -hex 32)|" .env
grep -E '^API_TOKEN=' .env # copy this — the userscript needs the same value
sed -i "s|^TOKEN_KEY=.*|TOKEN_KEY=$(openssl rand -hex 32)|" .env
sed -i "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=$(openssl rand -hex 24)|" .env
grep -E '^TOKEN_KEY=' .env
```
`TOKEN_KEY` derives every Reader's userscript credential (issue #24); only
SHA-256 hashes of the credentials are stored, so this secret is what a
database leak alone cannot recover. Changing it invalidates every installed
script at once.
`POSTGRES_PASSWORD` is read **only while the `postgres-data` volume is empty**,
which in practice means at first boot. Changing it afterwards changes the URL
the backend dials but not the password the database expects, and `bookmark-api`
crash-loops on `password authentication failed`. Set it before §2 and leave it
alone.
> Match `TRAEFIK_ENTRYPOINT` / `TRAEFIK_CERTRESOLVER` to your Traefik's actual
> names (check your Traefik static config — common alternatives: `https`,
> `myresolver`, `cloudflare`). Wrong names = no certificate issued.
@@ -65,44 +101,58 @@ grep -E '^API_TOKEN=' .env # copy this — the userscript needs the same value
## 1b. Web UI
The browser UI is served by the same container on a second hostname.
The browser UI is served by the same container on a second hostname. Sign-in
is a Discord authorization code grant (ADR-0002): the owner's Discord account,
gated by membership in one configured guild.
1. Add a DNS `A`/`AAAA` record for `manga.<yourdomain>` pointing at the server —
the same address as `manga-api.<yourdomain>`.
1. Add a DNS `A`/`AAAA` record for `bookmark.<yourdomain>` pointing at the
server — the same address as `bookmark-api.<yourdomain>`.
2. Set both variables in `.env`:
2. Create the Discord application at <https://discord.com/developers/applications>:
- **OAuth2 → Redirects:** add the exact callback URL
`https://bookmark.violetcrown.my.id/auth/discord/callback`. Discord
matches it verbatim — a trailing slash or different hostname breaks
sign-in.
- **OAuth2 → General:** note the Client ID, and generate a Client Secret.
- No scopes or bot setup are needed in the dashboard; the service requests
`identify` and `guilds.members.read` itself, and checks the *user's*
membership of the guild, not the application's.
3. Set the variables in `.env`:
```ini
MANGA_WEB_HOST=manga.violetcrown.my.id
WEB_PASSWORD=<paste output of: openssl rand -base64 18>
BOOKMARK_WEB_HOST=bookmark.violetcrown.my.id
DISCORD_CLIENT_ID=<client id>
DISCORD_CLIENT_SECRET=<client secret>
DISCORD_GUILD_ID=<guild snowflake>
DISCORD_REDIRECT_URI=https://bookmark.violetcrown.my.id/auth/discord/callback
# Optional: only members holding this role may sign in.
# DISCORD_REQUIRED_ROLE=<role snowflake>
```
Generate and insert in one line:
The guild id is in Discord's client with Developer Mode on: right-click the
server name → Copy Server ID. The four uncommented variables are required —
the backend refuses to start without them. Guild membership *is*
registration: any member of `DISCORD_GUILD_ID` becomes a Reader with their
own library on their first sign-in. `OWNER_DISCORD_ID` from §1 is only the
administrator — the Reader who can revoke another Reader's sessions.
```bash
sed -i "s|^WEB_PASSWORD=.*|WEB_PASSWORD=$(openssl rand -base64 18)|" .env
grep -E '^WEB_PASSWORD=' .env # this is what you type into the site
```
3. Redeploy and check:
4. Redeploy and check:
```bash
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
curl -s -o /dev/null -w '%{http_code}\n' https://manga.violetcrown.my.id/
curl -s -o /dev/null -w '%{http_code}\n' https://bookmark.violetcrown.my.id/
```
Expected `200`, serving the login page.
Expected `200`, serving the login page with the Discord button. Signing in
lands on the library; an account outside the guild is refused with a message
that names neither the guild nor its id.
Leaving `WEB_PASSWORD` unset is safe: the web routes are not registered and `/`
returns 404. The userscript's API on `MANGA_API_HOST` is unaffected either way.
`MANGA_WEB_HOST` itself is required by the prod override regardless — like
`MANGA_API_HOST`, its Traefik label has no fallback, so `docker compose up`
refuses to start without it even if `WEB_PASSWORD` is unset and the web UI is
otherwise dormant.
Sessions are signed with a key derived from `API_TOKEN` and `WEB_PASSWORD`, so
rotating either one logs every browser out. The session cookie lasts 60 days.
Sessions are rows in the database: the cookie carries only an opaque id, and
every request looks the row up and checks its expiry. Deleting a session row —
or the whole `sessions` table — logs the browser out immediately; nothing is
signed, so rotating a credential does not affect browser sessions. Sessions
last 60 days.
---
@@ -116,11 +166,24 @@ This merges the base file (build/image/env/volume) with the prod override
(no host port, Traefik network + router labels). Always pass **both** `-f`
flags — the prod file is not standalone.
Two services come up: `bookmark-api` (the backend) and `postgres` (its
database, `postgres:17-alpine`). Postgres publishes no port — it sits alone
with `bookmark-api` on an `internal: true` network — and stops everything if it
is missing: `bookmark-api` waits for `pg_isready` to pass, then applies its
embedded migrations, and only then listens. The schema is created that way;
there is nothing to import by hand.
There is deliberately no browser here. Kagane and novelfull need one, and it
runs on a **separate machine** over the tailnet — §7. Until you do that step,
`BROWSER_WS_URL` is unset, the poller logs and skips those two sites, and
everything else works normally.
Check it's up and healthy:
```bash
docker compose -f docker-compose.yml -f docker-compose.prod.yml ps
docker logs manga-api --tail 20 # expect: "listening on :8080 ..."
# bookmark-api Up; postgres Up (healthy)
docker logs bookmark-api --tail 20 # expect: "listening on :8080 ..."
```
---
@@ -131,21 +194,24 @@ Give Traefik a few seconds to issue the cert, then:
```bash
# Health (no auth) — must be valid TLS, no cert warning.
curl -s https://manga-api.violetcrown.my.id/healthz # -> ok
curl -s https://bookmark-api.violetcrown.my.id/healthz # -> ok
# Auth enforced.
curl -s -o /dev/null -w '%{http_code}\n' \
https://manga-api.violetcrown.my.id/bookmarks # -> 401
https://bookmark-api.violetcrown.my.id/bookmarks # -> 401
TOKEN=$(grep -E '^API_TOKEN=' .env | cut -d= -f2)
# A Reader's own credential. It is derived, never stored in .env — take it from
# the Userscripts panel's install link after signing in, or from an installed
# script's API_TOKEN constant.
TOKEN=<your Reader credential>
curl -s -H "Authorization: Bearer $TOKEN" \
https://manga-api.violetcrown.my.id/bookmarks # -> []
https://bookmark-api.violetcrown.my.id/bookmarks # -> []
# CORS preflight from a real site origin.
curl -s -i -X OPTIONS \
-H 'Origin: https://asurascans.com' \
-H 'Access-Control-Request-Method: PUT' \
https://manga-api.violetcrown.my.id/bookmarks/x | grep -i access-control
https://bookmark-api.violetcrown.my.id/bookmarks/x | grep -i access-control
# -> Access-Control-Allow-Origin: https://asurascans.com (+ Methods/Headers)
```
@@ -156,29 +222,30 @@ a bad cert makes the browser block the userscript's `fetch()` (mixed content).
## 4. Configure the userscript
Edit the config block at the top of `userscript/manga-bookmark.user.js`:
The bindmounted `userscript/*.user.js` files carry `__API_TOKEN__` placeholders
and the deployment's `@downloadURL`/`@updateURL` lines. Check the metadata
block — it ships hardcoded to this deployment's domain, so a deployer who
copies the repo to another domain must edit the two lines or the script
auto-updates from someone else's backend:
```js
const API_BASE = "https://manga-api.yourdomain.com"; // no trailing slash
const API_TOKEN = "<same token as .env>";
// @downloadURL https://bookmark-api.yourdomain.com/u/__API_TOKEN__/manga-bookmark.user.js
// @updateURL https://bookmark-api.yourdomain.com/u/__API_TOKEN__/manga-bookmark.user.js
```
The token sits in the userscript's isolated world — the manga sites' JS can't
read it.
Also edit the `@downloadURL`/`@updateURL` metadata lines near the top of the
file — they ship hardcoded to this deployment's domain and token, so a
deployer who skips them ends up auto-updating from someone else's backend.
See "Installing / updating the userscript" below for how those two lines are
used.
The backend substitutes `__API_TOKEN__` with the requesting Reader's derived
credential at serve time (issue #24), so no real credential ever sits in the
file. Only the `API_BASE` constant and the metadata hostname are deployer
edits; do not put a credential in this file.
---
## 5. Install on Bromite
1. Bromite → **Settings → User scripts** → enable (accept the permission prompt).
2. Put the edited `manga-bookmark.user.js` on the device (save the file, or open
its raw URL). Bromite detects `.user.js` and offers to install.
2. Sign in to the web UI, open the **Userscripts** panel, and open the install
link — Bromite detects `.user.js` and offers to install. The script already
carries your credential; you never see or type one.
3. Confirm install — the `@match` list covers both sites.
4. Open a series on asurascans.com or demonicscans.org → a 📑 button appears
bottom-right → tap → **+ Bookmark this**.
@@ -186,12 +253,15 @@ used.
Optional desktop test: the script is `GM_*`-free, so the same file installs in
Tampermonkey/Violentmonkey for quick checks before going mobile.
Rotating the credential in the same web-UI panel invalidates every installed
copy immediately — reinstall on all devices, or they silently stop syncing.
---
## 6. Smoke-test the full loop
1. Bookmark a series on Asura.
2. `curl -s -H "Authorization: Bearer $TOKEN" https://manga-api.yourdomain.com/bookmarks`
2. `curl -s -H "Authorization: Bearer $TOKEN" https://bookmark-api.yourdomain.com/bookmarks`
on the server — the series should appear.
3. Open a chapter of that series — reopen the panel; last-read updates to that
chapter (auto, never regresses on older chapters).
@@ -200,6 +270,204 @@ Tampermonkey/Violentmonkey for quick checks before going mobile.
---
## 7. The browser, on the home machine
Kagane and novelfull sit behind a Cloudflare JavaScript challenge no TLS
fingerprint clears, so the poller reaches them through a real Chrome over CDP.
That browser does **not** run on the VPS: it held 471 MiB of a 1974 MiB box
with no swap, and it scores better from a residential IP anyway (ADR-0006). It
is its own compose unit, deployed and updated independently of everything
above.
Do this after §2, on the second machine. Both machines must already be on the
same tailnet.
First, on the VPS, record what you are reclaiming — this is the whole point of
the move and there is no way to measure it afterwards:
```bash
free -m | awk '/^Mem:/ {print "available before:", $NF, "MiB"}'
```
Take it again after §7 is finished and the old sidecar is gone. Expect roughly
the sidecar's former footprint back (measured at 471 MiB working set, 595 MiB
cgroup).
**On the home machine:**
```bash
git clone <this repo> ~/mangaBookmark && cd ~/mangaBookmark/chrome
tailscale ip -4 # -> 100.x.y.z, this machine's tailnet IP
cp .env.example .env
echo "BROWSER_BIND_ADDR=$(tailscale ip -4)" >> .env
docker compose up -d --build
```
The clone is only for `chrome/`; nothing else on this machine reads the rest of
the repo. The unit is its own compose project (`bookmark-browser`), so it shares
no volume, network or lifecycle with an API stack that happens to sit beside it.
`BROWSER_BIND_ADDR` has no default on purpose. CDP authenticates nothing —
whatever reaches port 9222 drives the browser and, through it, this host — so
the bind address *is* the access control, backed by Tailscale device identity.
On the VPS that job was done by Docker network membership; this machine has a
real LAN, so `0.0.0.0` would be a hole punched into your home network. Compose
refuses to start rather than guess.
**Narrow it to the one device that needs it.** The bind address keeps CDP off
your LAN; it still leaves port 9222 open to every device on the tailnet, and
CDP has no login — a compromised phone is enough to drive this host. A new
tailnet's policy is allow-all, so this is the step that makes "Tailscale
identity is the access control" true rather than aspirational.
Tailscale has no `deny`, so a restriction is expressed by removing the blanket
grant and enumerating what is left. That only works if the browser machine can
be *excluded* from a selector that still covers your own devices — which is
what tagging buys: a tagged device has no user, so `autogroup:member` and
`autogroup:self` stop matching it. Tagging is the mechanism, not decoration.
In the admin console, under **Access controls**, the shipped policy grants
`{"src": ["*"], "dst": ["*"], "ip": ["*"]}`. Replace it:
```jsonc
{
"tagOwners": {
// Empty list: implicitly owned by the tailnet Owner/Admins, which is you.
"tag:bookmark-api": [],
"tag:bookmark-browser": [],
},
"grants": [
// The only thing on the tailnet that may drive the browser.
{
"src": ["tag:bookmark-api"],
"dst": ["tag:bookmark-browser"],
"ip": ["tcp:9222"],
},
// Your own devices reach your own devices, and the VPS, in full.
{
"src": ["autogroup:member"],
"dst": ["autogroup:self", "tag:bookmark-api"],
"ip": ["*"],
},
// On the browser machine you get SSH and nothing else. Widen this to `*`
// and the restriction above is void; delete it and you are locked out.
{
"src": ["autogroup:member"],
"dst": ["tag:bookmark-browser"],
"ip": ["tcp:22"],
},
// Uncomment if you route traffic through an exit node — dropping the
// blanket grant takes exit-node access with it.
// {"src": ["autogroup:member"], "dst": ["autogroup:internet"], "ip": ["*"]},
],
// Tagged devices left `autogroup:self`, so Tailscale SSH needs them named.
// Irrelevant if you reach these boxes with ordinary sshd over the tailnet —
// that is the `tcp:22` grant above.
"ssh": [
{
"action": "check",
"src": ["autogroup:member"],
"dst": ["autogroup:self", "tag:bookmark-api", "tag:bookmark-browser"],
"users": ["autogroup:nonroot", "root"],
},
],
// Run on every save, so a later edit that reopens 9222 is rejected outright.
"tests": [
{ "src": "tag:bookmark-api", "accept": ["tag:bookmark-browser:9222"] },
{
"src": "you@example.com",
"accept": ["tag:bookmark-browser:22"],
"deny": ["tag:bookmark-browser:9222"],
},
],
}
```
Then apply the tags — on the VPS and the home machine respectively:
```bash
sudo tailscale up --advertise-tags=tag:bookmark-api
sudo tailscale up --advertise-tags=tag:bookmark-browser
```
Each re-authenticates in a browser and issues a new node key; the tailnet IP is
unchanged, so `BROWSER_WS_URL` and `BROWSER_BIND_ADDR` still hold. Key expiry is
disabled once a device is tagged, which is what you want for a server — an
expired key would otherwise take the poller down every few months.
**Tagging replaces the device's user identity**, so do this only to machines
that exist to run these services. If your "home machine" is also your daily
driver, tag it anyway and reach it through the `:22` rule above, or skip the
tag and accept that any device of yours can reach CDP.
Enforcement is by the destination's packet filter, so the check below is real,
not advisory.
Prove the bind is tight, from the home machine itself:
```bash
curl -s -m 3 http://$(tailscale ip -4):9222/json/version # -> JSON
curl -s -m 3 http://<this machine's LAN IP>:9222/json/version
# -> curl: (7) Failed to connect ... Connection refused
```
The first call is also what wakes Chrome: it is not running until something
connects, and it is reaped again after five idle minutes. A cold first response
takes a few seconds; that is the browser starting, not a fault.
That check proves the *bind*, not the ACL — traffic that starts on the node is
not filtered. Prove the ACL from somewhere else: on your laptop or phone the
same URL must now time out, and from the VPS it must answer.
```bash
# on any other device of yours -> hangs until timeout
curl -s -m 5 http://<home machine tailnet IP>:9222/json/version
# on the VPS -> JSON
curl -s -m 20 http://<home machine tailnet IP>:9222/json/version
```
**On the VPS:**
```bash
cd ~/mangaBookmark
echo 'BROWSER_WS_URL=ws://100.x.y.z:9222' >> .env # the home machine's tailnet IP
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
```
It must be the tailnet **IP**. A MagicDNS hostname fails: Chrome's DevTools HTTP
handler answers `/json/version` with a 500 for any `Host` header that is not an
IP or `localhost`, and the failure looks like a broken site rather than a broken
hostname.
**Prove it end to end.** This is the only check that says the challenge actually
clears from that machine's egress — it fetches a real kagane cover and a real
chapter list:
```bash
cd backend
SMOKE_BROWSER_WS_URL=ws://100.x.y.z:9222 go test -run TestSmokeKagane ./internal/latest
```
A red run means "not clearing from this address right now", which is a live
fact to re-check before it is a defect — Cloudflare's scoring moves. Then, from
the web UI, open a bookmarked kagane series and confirm the cover renders. Once
a cover is stored it is served from Postgres forever after, so the browser being
asleep, unreachable, or mid-power-outage costs chapter freshness and nothing
visible.
Finally, take the VPS `free -m` reading again and compare it against the one
from the top of this section.
**Updating the browser** is independent of the API stack and has its own
runbook — `REDEPLOY.md` §8.
---
## Updating
Pull new code, then rebuild:
@@ -208,7 +476,14 @@ Pull new code, then rebuild:
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
```
SQLite data persists in the named volume `bookmarks-data` across rebuilds.
Data persists in the named volume `postgres-data` across rebuilds. (If this
server predates the Postgres migration, the old SQLite volume `bookmarks-data`
is still on disk and deliberately undeclared in compose so `down -v` cannot take
it; see `REDEPLOY.md` §1 for when to remove it.)
The browser is a separate unit on a separate machine with its own update
command — §7. Nothing above touches it, and it needs no coordination: the API
picks up a restarted Chrome's new debugger UUID by itself.
---
@@ -217,11 +492,20 @@ SQLite data persists in the named volume `bookmarks-data` across rebuilds.
| Symptom | Likely cause / fix |
|---------|--------------------|
| No cert / TLS error at the domain | `TRAEFIK_ENTRYPOINT` or `TRAEFIK_CERTRESOLVER` name wrong; or DNS not resolving yet. Check `docker logs <traefik>`. |
| 404 from Traefik | Service not on the `proxy` network, or `MANGA_API_HOST` mismatch. Confirm `docker network inspect proxy` lists `manga-api`. |
| 404 from Traefik | Service not on the `proxy` network, or `BOOKMARK_API_HOST` mismatch. Confirm `docker network inspect proxy` lists `bookmark-api`. |
| `fetch` fails in the userscript, `curl` works | Origin missing from `ALLOWED_ORIGINS`, or mixed content (backend not HTTPS). |
| 401 with the right token | Trailing space/newline in `API_TOKEN`; regenerate and restart. |
| 401 with the right credential | The script's credential no longer matches the stored hash — most likely a rotation happened and the device was not reinstalled. Reinstall from the web UI. |
| 401 after rotation, even right after reinstalling | `TOKEN_KEY` changed between the rotation and the reinstall; credentials are derived from it, so changing it invalidates every credential. Keep it stable. |
| Panel button absent | URL didn't match an adapter, or user scripts disabled in Bromite. |
| `compose ... config` errors about `API_TOKEN` | Run compose from the dir with `.env`, or export the vars. |
| `compose ... config` errors about `TOKEN_KEY`, `OWNER_DISCORD_ID` or `POSTGRES_PASSWORD` | Run compose from the dir with `.env`, or export the vars. All three are required and none has a fallback. |
| `bookmark-api` restarts in a loop, `password authentication failed for user "bookmarks"` | `POSTGRES_PASSWORD` was changed after first boot; Postgres only applies it to an empty `postgres-data`. Restore the old value, or reset the role (`REDEPLOY.md` troubleshooting). |
| `bookmark-api` never logs `listening on :8080` | It is blocked on `postgres` passing `pg_isready`, or a migration failed. `docker compose -f docker-compose.yml -f docker-compose.prod.yml logs postgres`. |
| kagane rows never get a `latest_chapter`; log says `browser fetcher disabled` or nothing at all | `BROWSER_WS_URL` unset. Expected before §7 is done. |
| kagane polls all fail; log shows a 500 from `/json/version` | `BROWSER_WS_URL` names a MagicDNS hostname (or any name). Chrome's DevTools handler only accepts an IP or `localhost` — use the tailnet IP. |
| kagane polls fail with a connection error | Home machine off, off the tailnet, or the unit is down. `tailscale ping <machine>`, then `docker compose ps` in its `chrome/`. Costs freshness only; stored covers keep serving. |
| kagane cover is a placeholder for a newly bookmarked series | Its cover has never been fetched and the browser is unreachable. It fills in on the next successful poll of that series (up to `LATEST_CHAPTER_POLL_BROWSER_COOLDOWN`, default 6h). |
| `compose` in `chrome/` errors `set BROWSER_BIND_ADDR to this machine's tailnet IP` | No `chrome/.env`, or the variable is empty. Deliberate — it has no default so an unset value cannot publish CDP to the LAN. |
| browser container restarts, or is OOM-killed | `docker inspect bookmark-browser --format '{{.RestartCount}} {{.State.OOMKilled}}'`. The 512 MiB cap is sized against a measured 645 MiB untuned peak; a real breach is a Chrome regression worth reading `docker logs` for, not a number to raise reflexively. |
Backend config reference and endpoint list: see `README.md`.
@@ -230,20 +514,25 @@ Backend config reference and endpoint list: see `README.md`.
## Installing / updating the userscript
The backend serves the script itself, so Violentmonkey can auto-update it.
Complements §4 above — that step points `API_BASE`/`API_TOKEN` at your
backend; this one points `@downloadURL`/`@updateURL` at the same place so
auto-updates come from it too.
Complements §4 above — the `@downloadURL`/`@updateURL` lines point at the
credential-bearing path, so auto-updates come from the same place as the
install.
Install once, on the phone (Cromite + Violentmonkey):
Install once, on the phone (Cromite + Violentmonkey): sign in to the web UI,
open the **Userscripts** panel, and open the install link for the library —
the script is served with your credential already inside it. Its
`@downloadURL`/`@updateURL` point at the same credential-bearing path for
updates:
```
https://manga-api.<your-domain>/u/<API_TOKEN>/manga-bookmark.user.js
https://bookmark-api.<your-domain>/u/<your credential>/manga-bookmark.user.js
```
Open that URL in Cromite; Violentmonkey offers to install it. The token is in
the path because Violentmonkey's update poll sends no `Authorization` header,
and the script embeds `API_TOKEN` in plain text — an open URL would leak it. A
wrong token answers 404.
Violentmonkey offers to install it. The credential is in the path because
Violentmonkey's update poll sends no `Authorization` header, and the script
embeds the credential in plain text — an open URL would leak it. A wrong
credential answers 404. The credential is derived from `TOKEN_KEY` and never
appears anywhere but this URL and the rendered script.
Updating, without a redeploy:
+21 -15
View File
@@ -8,47 +8,53 @@ web
## Users
Single user (self-hosted, no accounts, no multi-user planned). Reads manga on **asurascans.com** and **demonicscans.org** primarily via Bromite on mobile, also checks/updates from a desktop browser. The web UI is the cross-device view into progress captured by the userscript while reading.
Members of one private Discord guild, each with their own library. Accounts exist and are created by signing in — there is no signup form, no invite code and no approval step: any member of the configured guild becomes a Reader on their first Discord login. The person running the deployment is the owner, seeded at startup, and the only Reader with an administrative capability (revoking another Reader's sessions).
Reading happens on **asurascans.com**, **demonicscans.org**, **comix.to** and **kagane.to** for manga and **novelfull.com** and **lightnovelworld.net** for novels, primarily via Bromite on mobile, with checks and corrections from a desktop browser. The web UI is the cross-device view into progress the userscripts capture while reading.
## Product Purpose
Tracks read-progress ("last chapter read") per manga series across two otherwise-unrelated manga sites that each have their own separate `localStorage`. A Go backend unifies bookmarks into one store; the web UI is a password-gated browser view of that store for reviewing, favouriting, correcting, or removing bookmarks, and jumping back into a series to continue reading. A background poller also refreshes each series' latest-published-chapter so the list can flag "NEW" without the user visiting the site.
Tracks read-progress ("last chapter read") per series across sites that each have their own separate `localStorage`. A Go backend unifies bookmarks into one store; the web UI is a Discord-gated browser view of one Reader's own bookmarks, for reviewing, favouriting, correcting, shelving or removing them, and jumping back into a series to continue reading. A background poller refreshes each series' latest-published-chapter so the list can flag "NEW" without the Reader visiting the site.
## Positioning
Not a public reading tracker or social app — a private, self-hosted sync layer purpose-built for two specific scraped sites, with no server-side account system (single bearer token + one password-gated session).
Not a public reading tracker or social app — a private, self-hosted sync layer for one Discord community, purpose-built for a fixed set of scraped sites. Multi-Reader, not multi-tenant: libraries are isolated, but the deployment belongs to one group and its membership is the whole access model.
## Operating Context
- Primary reading device: Bromite (mobile Chromium), where a userscript captures progress automatically.
- Primary reading device: Bromite (mobile Chromium), where a userscript captures progress automatically. Each Reader installs their own copy, rendered with their own credential.
- Web UI is a secondary surface: checking list state, correcting a wrong chapter number, removing dead bookmarks, jumping to "continue reading."
- Manga cover art and titles come from the source sites' `og:image`/`og:title` — real content, not placeholders.
- List order is driven by `updated_at`, which moves only on real reading progress (not favouriting, not a newly detected chapter) — a UI constraint the redesign must not break.
- Cover art and titles come from the source sites' `og:image`/`og:title` — real content, not placeholders. They are facts about the series, so they are shared between Readers who track it; progress is not.
- List order is driven by `updated_at`, which moves only on real reading progress (not favouriting, not a newly detected chapter) — a UI constraint the design must not break.
## Capabilities and Constraints
- Two tabs: All / Favourites. Search-filter by title (client-side, `filter.js`).
- Card actions: continue (opens source site), toggle favourite, manual chapter override, delete (with confirm).
- "Continue reading" horizontal strip for recently-progressed series.
- htmx-driven partial updates (card re-render on favourite/chapter/delete), no client-side framework/build step — templates are Go `html/template`, `go:embed`-ed.
- Two libraries (manga, novels) with lifecycle tabs: All / Updated / Favourites / Archived / Finished. Search-filter by title (client-side, `filter.js`).
- Card actions: continue (opens source site), toggle favourite, manual chapter override, archive, finish, remove — each move out of the list confirm-gated.
- "Continue reading" horizontal strip for series with an unread chapter.
- A Reader with no bookmarks at all sees a deliberate empty library offering both userscript install links, not an error and not a blank page.
- Isolation is the load-bearing invariant: two Readers cannot see or change each other's bookmarks. A series both track is one shared row polled once, with independent progress on each side.
- The owner can revoke a specific Reader's sessions; nothing else in the UI differs by Reader.
- htmx-driven partial updates, no client-side framework or build step — templates are Go `html/template`, `go:embed`-ed.
- Mobile-first is a hard functional constraint (primary device is a phone), not just a starting breakpoint.
## Brand Commitments
- Name: **mangaBookmark**.
- **Dark-first is binding**: current dark-by-default / light-follows-system-preference behavior must be preserved as a design constraint, not just a starting default, because reading happens at night.
- Name: **BookmarkManager**.
- **Dark-first is binding**: dark-by-default / light-follows-system-preference must be preserved as a design constraint, not just a starting default, because reading happens at night.
## Evidence on Hand
- Live templates/CSS at `backend/templates/*.html`, `backend/static/style.css` — current implemented UI, functional but not yet treated as an intentional design system.
- No logo, screenshots, or marketing copy exist; none should be fabricated.
- Live templates/CSS at `backend/internal/web/templates/*.html`, `backend/internal/web/static/style.css`, governed by the Cinder design system (`docs/design-system.md`).
- No logo beyond the wordmark, no screenshots, no marketing copy; none should be fabricated.
## Product Principles
- Dark-first, night-reading-optimized — never regress to a light-default or high-glare surface.
- Mobile is the primary target; desktop is an enhancement, not the design center.
- Progress data integrity over visual flourish: `updated_at`/list-ordering behavior is a correctness constraint the UI must respect, not decorate over.
- No accounts, no multi-tenant chrome — the whole product is for one reader.
- A leak between Readers fails silently and looks like working software — isolation is asserted from both directions, never inferred from counting one Reader's rows.
- No roles, no org chrome: the owner's Readers panel is one list with one button (revoke someone's sessions), not an admin console, and otherwise every Reader's view is the same.
- Prefer native platform affordances (system dark/light, native touch targets) over custom widgetry — this is a lean self-hosted tool, not a product to demo.
## Accessibility & Inclusion
+109 -35
View File
@@ -1,21 +1,35 @@
# Manga Bookmark
Track manga read-progress on **asurascans.com** (a.k.a. asuracomic.net) and
**demonicscans.org** from a phone (Bromite / mobile Chromium), synced to a
self-hosted Go backend so bookmarks unify across both sites and all devices.
Track manga read-progress on **asurascans.com**,
**demonicscans.org**, **comix.to**, and **kagane.to** from a phone (Bromite /
mobile Chromium), synced to a self-hosted Go backend so bookmarks unify across
all four sites and all devices.
Two parts:
- **`backend/`** — tiny Go (`net/http` + pure-Go SQLite) sync service. 4 routes,
- **`backend/`** — tiny Go (`net/http` + Postgres via pure-Go `pgx`) sync service. 4 routes,
static binary, distroless container.
- **`userscript/manga-bookmark.user.js`** — single Bromite-compatible userscript
(no `GM_*` APIs) that injects an on-page bookmark UI and syncs via `fetch()`.
```
Bromite userscript (isolated world, Shadow DOM UI, localStorage cache)
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume)
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> Postgres (volume)
|
| CDP over the tailnet
v
headless Chrome, on-demand,
on a separate machine
(chrome/, ADR-0006)
```
Kagane and novelfull sit behind a Cloudflare JavaScript challenge no TLS
fingerprint clears, so the poller reaches those two through a real Chrome over
CDP. That browser is **not** part of the API stack: it is its own compose unit
on a second machine, spawned on the first connection and reaped when idle. The
API needs it only to discover new chapters and to fetch a kagane cover once —
covers are stored, so the library renders in full with the browser switched off.
---
## 1. Backend
@@ -24,23 +38,51 @@ Bromite userscript (isolated world, Shadow DOM UI, localStorage cache)
| Var | Default | Notes |
|-----|---------|-------|
| `API_TOKEN` | *(required)* | Bearer token shared with the userscript. |
| `ALLOWED_ORIGINS` | Asura + Demonic origins | Comma-separated CORS allowlist. |
| `DB_PATH` | `/data/bookmarks.db` | SQLite file location. |
| `TOKEN_KEY` | *(required)* | Secret every Reader's userscript credential is derived from (issue #24); only SHA-256 hashes of credentials are stored. |
| `OWNER_DISCORD_ID` | *(required)* | Discord user ID of the owner: seeded as the first Reader, owns every pre-registration bookmark, and is the only Reader who can revoke another's sessions. |
| `ALLOWED_ORIGINS` | Asura + Demonic + Comix + Kagane origins | Comma-separated CORS allowlist. |
| `DATABASE_URL` | *(required)* | Postgres connection URL, e.g. `postgres://bookmarks:…@postgres:5432/bookmarks?sslmode=disable`. Compose builds it from `POSTGRES_PASSWORD`. |
| `COVER_DIR` | *(required)* | Filesystem volume for immutable, content-addressed Cover bytes. Compose builds the image and mounts `cover-data` at this path; standalone runs may choose another writable durable path. |
| `PORT` | `8080` | Plain HTTP; TLS terminated by the proxy. |
| `BROWSER_WS_URL` | empty | CDP endpoint of the remote browser (`ws://<tailnet IP>:9222`), used to poll Kagane/Novelfull past their JS challenge and to fetch uncached Kagane covers. Must be an IP or `localhost` — Chrome's DevTools handler 500s any other Host header, MagicDNS names included. Unset disables both; stored covers still serve. |
| `DISCORD_CLIENT_ID` | *(required)* | Discord application credentials for the browser sign-in (ADR-0002). |
| `DISCORD_CLIENT_SECRET` | *(required)* | As above. Never logged, never echoed in an error. |
| `DISCORD_GUILD_ID` | *(required)* | The one guild whose membership gates sign-in, checked at login only. Membership *is* registration: any member becomes a Reader on first login. |
| `DISCORD_REDIRECT_URI` | *(required)* | Exact callback URL; Discord matches it verbatim against the registered redirect. |
| `DISCORD_REQUIRED_ROLE` | empty | Role snowflake a member must additionally hold. Empty means guild membership alone suffices. |
| `DISCORD_API_BASE` | `https://discord.com/api/v10` | Test seam — tests point it at a local stub so the real token exchange runs. |
| `USERSCRIPT_PATH` | `/userscript/manga-bookmark.user.js` | Bindmounted file served at `/u/{token}/manga-bookmark.user.js`. |
| `NOVEL_USERSCRIPT_PATH` | `/userscript/novel-bookmark.user.js` | Same, for the novel library. |
| `LATEST_CHAPTER_POLL_ENABLED` | `1` | `0` turns the poller off entirely. |
| `LATEST_CHAPTER_POLL_COOLDOWN` | `1h` | Rest between checks of one plain-TLS series; floor `15m`. |
| `LATEST_CHAPTER_POLL_BROWSER_COOLDOWN` | `6h` | Rest between checks of one browser-backed series; floor `15m`. |
| `LATEST_CHAPTER_POLL_INTERVAL` | `10m` | How often the poller wakes. Cannot shorten either cooldown. |
| `LATEST_CHAPTER_POLL_BATCH` | `14` | Series per wake. Keep `BATCH × STAGGER` under `INTERVAL`. |
| `LATEST_CHAPTER_POLL_STAGGER` | `20s` | Delay between fetches in a batch — this is the outbound request rate. |
Compose reads a few more from the same `.env` that the backend never sees:
`POSTGRES_PASSWORD` (required — `DATABASE_URL` is built from it, and Postgres
only applies it while `postgres-data` is empty), `BOOKMARK_API_HOST` and
`BOOKMARK_WEB_HOST` (required by the prod override), and the optional
`PROXY_NETWORK` / `TRAEFIK_ENTRYPOINT` / `TRAEFIK_CERTRESOLVER`. The browser
unit has its own `chrome/.env` on its own machine — `BROWSER_BIND_ADDR`
(required, the tailnet IP the CDP port is published on) and the optional
`BROWSER_TZ`. Full commentary is in `.env.example` and `chrome/.env.example`;
deployment order is `DEPLOY.md`.
### Endpoints
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/bookmarks` | Bearer | All bookmarks (single-user). |
| `GET` | `/bookmarks` | Bearer | All bookmarks of the acting Reader. |
| `PUT` | `/bookmarks/{key}` | Bearer | Upsert one series; returns the row as stored. |
| `DELETE` | `/bookmarks/{key}` | Bearer | Remove one. |
| `GET` | `/healthz` | none | `200 ok`. |
| `GET` | `/u/{token}/manga-bookmark.user.js` | token in path | Serves the userscript with an mtime-derived `@version`. |
| `GET` | `/u/{token}/manga-bookmark.user.js` | credential in path | Serves the userscript with the requesting Reader's credential substituted in and an mtime-derived `@version`. |
`key` is `<site>:<series_id>` — e.g. `asura:trash-of-the-counts-family-f886a8af`
or `demonic:Infinite-Level-Up-in-Murim`. Sync is last-write-wins.
`key` is `<site>:<series_id>` — e.g. `asura:trash-of-the-counts-family-f886a8af`,
`demonic:Infinite-Level-Up-in-Murim`, `comix:12345`, or
`kagane:3fa85f64-5717-4562-b3fc-2c963f66afa6`. Sync is last-write-wins.
`updated_at` orders the bookmark list, so it moves only on real reading
progress: the server applies its timestamp when the row is new or
@@ -57,19 +99,45 @@ go test ./... # unit + handler tests
CGO_ENABLED=0 go build # static binary
```
**`go test ./...` requires Docker.** The store talks to a real Postgres, so
each test package starts a throwaway `postgres:17-alpine` container and gives
every test its own database inside it (`internal/pgtest`). Nothing is stubbed
and nothing reaches the network beyond the local Docker daemon.
### Run the stack
```bash
cp .env.example .env
# edit .env: set API_TOKEN (openssl rand -hex 32)
# edit .env: set TOKEN_KEY (openssl rand -hex 32) and
# POSTGRES_PASSWORD (openssl rand -hex 24)
docker compose up -d --build # binds 127.0.0.1:8080
```
That brings up two services — the API and Postgres. The browser is deliberately
not one of them; without `BROWSER_WS_URL` the poller logs and skips kagane and
novelfull, and everything else works. To run one locally, publish it on the
Docker bridge gateway so the API container can name it by IP:
```bash
cd chrome
echo 'BROWSER_BIND_ADDR=172.17.0.1' > .env
docker compose up -d --build
# then in the repo's own .env: BROWSER_WS_URL=ws://172.17.0.1:9222
```
Bind it to `127.0.0.1` instead if you only want to drive it from the host, e.g.
`SMOKE_BROWSER_WS_URL=ws://127.0.0.1:9222 go test -run TestSmokeKagane ./internal/latest`.
In production that address is the home machine's tailnet IP and nothing else —
see `DEPLOY.md` §7 and ADR-0006.
Smoke test:
```bash
TOKEN=$(grep '^API_TOKEN=' .env | cut -d= -f2)
# The credential is per Reader and derived, so there is no token in .env to
# grep. Take yours from the Userscripts panel's install link after signing in,
# or read it out of an installed script's API_TOKEN constant.
TOKEN=<your Reader credential>
curl -s localhost:8080/healthz # ok
curl -s localhost:8080/bookmarks # 401
curl -s -H "Authorization: Bearer $TOKEN" localhost:8080/bookmarks # []
@@ -83,7 +151,7 @@ curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \
### Deploy behind your reverse proxy
Route `https://manga-api.<domain>` → the service on `:8080` (TLS at the proxy).
Route `https://bookmark-api.<domain>` → the service on `:8080` (TLS at the proxy).
- **Host proxy** (nginx/Caddy on the host): the base compose already binds
`127.0.0.1:8080`; point the proxy `proxy_pass http://127.0.0.1:8080;`.
@@ -96,7 +164,7 @@ Route `https://manga-api.<domain>` → the service on `:8080` (TLS at the proxy)
```
Set `PROXY_NETWORK` in `.env` if your network isn't named `proxy`.
Verify: `https://manga-api.<domain>/healthz` returns `ok` over valid TLS (no
Verify: `https://bookmark-api.<domain>/healthz` returns `ok` over valid TLS (no
mixed-content), and an `OPTIONS` preflight from a real site origin returns the
CORS headers.
@@ -104,17 +172,18 @@ CORS headers.
## 2. Userscript
### Configure
### Install
Edit the config block at the top of `userscript/manga-bookmark.user.js`:
Sign in to the web UI and open the **Userscripts** panel: it offers one
install link per library. Each link serves a script rendered with your own
credential already inside it — you never see, type or copy a credential. The
served script carries `@downloadURL`/`@updateURL` pointing at its
credential-bearing path, so Violentmonkey keeps auto-updating it.
```js
const API_BASE = "https://manga-api.<domain>"; // no trailing slash
const API_TOKEN = "<same token as backend>";
```
The token lives in the userscript's **isolated world** — the manga sites' own
JS cannot read it.
The bindmounted files carry `__API_TOKEN__` placeholders; the backend
substitutes the requesting Reader's credential at serve time, so no real
credential is ever committed. Rotating the credential (same panel) invalidates
every installed copy immediately — reinstall on all devices.
### Install on Bromite (mobile)
@@ -122,8 +191,8 @@ Bromite runs Chromium's native userscript engine (no Tampermonkey needed):
1. Bromite → **Settings → User scripts** → enable user scripts (allow the
permission prompt).
2. Save the configured `manga-bookmark.user.js` to the device (or open its raw
URL). Bromite detects the `.user.js` and offers to install it.
2. Open the install link from the web UI — Bromite detects the `.user.js` and
offers to install it.
3. Confirm the install; the `@match` list covers both sites.
4. Open a series on either site — a 📑 button appears bottom-right.
@@ -181,7 +250,7 @@ userscript does the looking, from your own browser session:
(`LATEST_CHECK_BATCH` / `LATEST_CHECK_THROTTLE_MS`). Failures are silent and
simply retried after the window.
Freshness is tracked per device in `localStorage` under `mangabm:lastchecked`
Freshness is tracked per device in `localStorage` under `bmgr:manga:lastchecked`
and is deliberately not synced, since each device checks on its own.
This means a bookmark is as current as its last check — not the moment a
@@ -192,20 +261,25 @@ an API.
## Adapter reference (verified live 2026-07-24)
The site adapters key everything off URL regex, with `title`/`cover` from
`og:title` / `og:image`. Confirmed against live pages via Playwright:
The site adapters key everything off URL regex, with `title` from `og:title`
(or the page's own heading where a site ships none). No adapter reads a cover:
the backend acquires, stores and serves every Cover from its own origin
(ADR-0007). Confirmed against live pages via Playwright:
| Site | Series URL | Chapter URL | `series_id` |
|------|-----------|-------------|-------------|
| **Asura** (`asurascans.com`) | `/comics/<slug-hash>` | `/comics/<slug-hash>/chapter/<n>` | `<slug-hash>` |
| **Demonic** (`demonicscans.org`) | `/manga/<slug>` | `/title/<slug>/chapter/<n>/<page>` (`chaptered.php?manga=<id>&chapter=<n>` 301s here) | `<slug>` |
| **Comix** (`comix.to`) | `/title/<id>-<slug>` | `/title/<id>-<slug>/<uploadId>-chapter-<n>` | `<id>` |
| **Kagane** (`kagane.to`) | `/series/<uuid>` | `/series/<uuid>/reader/<bookUuid>` | `<uuid>` |
Notes:
- **`asuracomic.net` deep links are dead (re-checked 2026-07-25).** They 301 to
the `asurascans.com` **root**, discarding the path, at the edge — before the
userscript gets a document — so nothing client-side can rescue them. Reach
series through `asurascans.com`. The host stays matched in case the redirect
starts preserving paths again.
- **`asuracomic.net` is no longer matched (deep links dead, re-checked
2026-07-25).** They 301 to the `asurascans.com` **root**, discarding the path,
at the edge — before the userscript gets a document — so nothing client-side
can rescue them. The backend rejects stored addresses on that host too, since
the poller pins each Site to one hostname. Reach series through
`asurascans.com`.
- Asura `og:title` carries a `Chapter N - Read Online \| Asura Scans` suffix that
the adapter strips; Demonic chapter `og:title` is `<Title> Chapter N`.
- Demonic's `<slug>` is identical on `/manga/…` and the canonical `/title/…`
+228 -102
View File
@@ -9,16 +9,16 @@ Whole thing is ~5 minutes, most of it waiting on `docker build`. Order matters:
**back up before you pull.** A backup taken after a bad migration is a backup of
the damage.
Paths below assume the checkout is at `/opt/mangabm`; substitute your own. The
one absolute rule about paths: **backups live in `../mangabm-backups/`**, a
sibling of the project directory (`/opt/mangabm-backups`), never inside it. It
Paths below assume the checkout is at `~/mangaBookmark`, which is where it lives
on this deployment; substitute your own. The one absolute rule about paths:
**backups live in a `-backups` sibling of the checkout**, never inside it. It
sits outside the repo so `git pull`, `git clean -fd` and a bad `rm -rf` inside
the checkout cannot take the backups with them.
```
/opt/
├── mangabm/ <- the checkout (this repo)
└── mangabm-backups/ <- bookmarks-YYYYmmdd-HHMMSS.db
~/
├── mangaBookmark/ <- the checkout (this repo)
└── mangaBookmark-backups/ <- bookmarks-YYYYmmdd-HHMMSS.dump
```
---
@@ -26,12 +26,12 @@ the checkout cannot take the backups with them.
## 0. Preflight
```bash
cd /opt/mangabm
cd ~/mangaBookmark
# Both -f flags, every time. The prod override is not standalone.
COMPOSE="docker compose -f docker-compose.yml -f docker-compose.prod.yml"
$COMPOSE ps # manga-api should be Up
$COMPOSE ps # bookmark-api should be Up
git status --short # expect empty
git log --oneline -1 # note this hash — it is your rollback target
df -h /var/lib/docker | tail -1 # a build needs room
@@ -44,87 +44,117 @@ dirty tree fails halfway and leaves you in a worse spot than either.
Create the backup directory once, and make sure it is a sibling, not a child:
```bash
mkdir -p ../mangabm-backups
BACKUP_DIR="$(cd .. && pwd)/mangabm-backups" # absolute — Docker needs it
echo "$BACKUP_DIR" # -> /opt/mangabm-backups
BACKUP_DIR="$(cd .. && pwd)/$(basename "$PWD")-backups" # absolute — Docker needs it
mkdir -p "$BACKUP_DIR"
echo "$BACKUP_DIR" # -> /home/sulthan/mangaBookmark-backups
```
---
## 1. Back up the database
The database is a single SQLite file in the named Docker volume, at
`/data/bookmarks.db` inside the container. Find the volume's real name — Compose
prefixes it with the project directory:
The database is Postgres, running as the `postgres` service on the named volume
`postgres-data`. It has **no published port** — nothing outside the internal `db`
network can reach it — so every command below goes in through the container:
```bash
docker volume ls --filter name=bookmarks-data
# -> local mangabm_bookmarks-data
VOL=$(docker volume ls --filter name=bookmarks-data -q | head -1)
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -c '\dt'
# -> bookmarks, covers, readers, schema_migrations, series, sessions
```
### Preferred: hot backup, no downtime
The `covers` table is metadata only after the filesystem cutover: bytes live in
the separate `cover-data` volume. Back that volume up with the database dump;
restoring only Postgres leaves stored Cover addresses without files.
The store runs in **WAL mode**, so recent writes may still be sitting in
`bookmarks.db-wal`. Copying `bookmarks.db` alone while the container runs can
therefore silently drop the newest bookmarks. `VACUUM INTO` folds the WAL in and
writes one consistent file, safe to run against a live database:
Inside the container that connects over the local socket as the `bookmarks`
superuser, so no password is needed anywhere in this section. `-T` is not
optional: without it Compose allocates a TTY, which rewrites `\n` to `\r\n` and
silently corrupts any binary stream flowing back out — see the dump below.
### Preferred: hot dump, no downtime
`pg_dump` runs in a single repeatable-read transaction, so it writes one
point-in-time-consistent snapshot while the API keeps serving. No stopping, no
WAL to worry about — that is the server's problem, not yours.
```bash
STAMP=$(date -u +%Y%m%d-%H%M%S) # UTC, sorts chronologically as text
docker run --rm \
-v "$VOL":/data \
-v "$BACKUP_DIR":/backup \
alpine sh -c "apk add -q sqlite &&
sqlite3 /data/bookmarks.db \"VACUUM INTO '/backup/bookmarks-$STAMP.db'\""
$COMPOSE exec -T postgres pg_dump -U bookmarks -d bookmarks -Fc \
> "$BACKUP_DIR/bookmarks-$STAMP.dump"
ls -lh "$BACKUP_DIR"/bookmarks-$STAMP.db
ls -lh "$BACKUP_DIR"/bookmarks-$STAMP.dump
```
`$STAMP` is the "time in the name" — `bookmarks-20260730-014233.db`. UTC, so the
files sort in real order and never collide across a DST shift.
`-Fc` is the custom archive format rather than plain SQL: it is compressed, and
`pg_restore` can inspect and replay it selectively — list its table of contents,
restore one table, restore schema without data, reorder. A plain `.sql` dump can
only be piped into `psql` whole, and gives you no way to check what is in it
short of reading it.
Note the source volume is mounted **read-write**, which looks wrong for a backup
and is not. Opening a WAL database requires creating the `-shm` shared-memory
file; with `:ro` the command fails with `unable to open database file` and no
backup is produced. `VACUUM INTO` never writes to the source itself.
`$STAMP` is the "time in the name" — `bookmarks-20260730-014233.dump`. UTC, so
the files sort in real order and never collide across a DST shift.
Verify it before you trust it. An unreadable backup is worse than none, because
you will act as though you have one:
```bash
docker run --rm -v "$BACKUP_DIR":/backup alpine sh -c "apk add -q sqlite &&
sqlite3 /backup/bookmarks-$STAMP.db 'PRAGMA integrity_check;' &&
sqlite3 /backup/bookmarks-$STAMP.db 'SELECT count(*) FROM bookmarks;'"
# -> ok
# 1. The dump parses and contains the tables. Uses the same image compose
# already pulls, so nothing new to install.
docker run --rm -v "$BACKUP_DIR":/backup postgres:17-alpine \
pg_restore --list "/backup/bookmarks-$STAMP.dump" | grep 'TABLE DATA'
# -> 1234; 0 0 TABLE DATA public bookmarks bookmarks
# -> 1235; 0 0 TABLE DATA public covers bookmarks
# -> 1236; 0 0 TABLE DATA public readers bookmarks
# -> 1237; 0 0 TABLE DATA public schema_migrations bookmarks
# -> 1238; 0 0 TABLE DATA public series bookmarks
# -> 1239; 0 0 TABLE DATA public sessions bookmarks
# 2. Sanity-check the live row count you just captured.
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks \
-c 'select count(*) from bookmarks'
# -> 37
```
The count should match what the web UI shows. Zero rows on a server you know has
bookmarks means you backed up the wrong volume.
A custom-format archive stores row counts nowhere, so step 1 proves the file is
a readable archive with the right tables in it, not that the rows are there;
step 2 is the number those rows should be. It should match what the web UI
shows. Zero on a server you know has bookmarks means the API and your `psql`
are looking at different databases — check `DATABASE_URL`.
### Fallback: cold copy (no network for `apk add sqlite`)
### Fallback: cold volume archive
Stop the service first, then copy the database **and its sidecars** — the `-wal`
is not optional, it is where the newest writes are:
Use this when you want the whole data directory rather than a logical dump — a
like-for-like restore of the same Postgres major version onto the same host.
**The stack must be stopped first.** A running Postgres has dirty pages in
shared buffers and WAL that has not been replayed into the data files, and `tar`
walks the directory over several seconds while the server keeps writing to it.
The archive you get is torn: files from different instants, possibly a
half-written page. It may restore, start, and be quietly wrong. Online
filesystem-level backup is `pg_basebackup`'s job, not `tar`'s; with the
container stopped the shutdown checkpoint has already flushed everything and a
plain archive of the volume is consistent.
```bash
# Derived exactly, not with a `--filter name=` substring match plus `head -1`:
# that quietly picks the first of however many volumes happen to contain the
# string, and archiving the wrong data directory is not a visible failure.
VOL="$(basename "$PWD" | tr '[:upper:]' '[:lower:]')_postgres-data"
docker volume inspect "$VOL" >/dev/null && echo "$VOL" # -> mangabookmark_postgres-data
$COMPOSE stop
docker run --rm -v "$VOL":/data:ro -v "$BACKUP_DIR":/backup alpine sh -c "
cp /data/bookmarks.db /backup/bookmarks-$STAMP.db
[ -f /data/bookmarks.db-wal ] && cp /data/bookmarks.db-wal /backup/bookmarks-$STAMP.db-wal
[ -f /data/bookmarks.db-shm ] && cp /data/bookmarks.db-shm /backup/bookmarks-$STAMP.db-shm
ls -1 /backup"
docker run --rm -v "$VOL":/from:ro -v "$BACKUP_DIR":/to alpine \
tar czf "/to/postgres-data-$STAMP.tgz" -C /from .
$COMPOSE start
ls -lh "$BACKUP_DIR"/postgres-data-$STAMP.tgz
```
Costs ~10 seconds of downtime. A clean shutdown usually checkpoints the WAL away,
so seeing only the `.db` file is normal and fine — the `[ -f ]` guards exist for
the case where it did not. Restoring this variant means putting whichever files
you got back together, under their original names.
Read-only is safe here precisely because nothing opens the database: it is a file
copy, not a SQLite connection.
Costs ~15 seconds of downtime. Read-only on the source is safe here precisely
because nothing is running against it. Restoring this variant means untarring it
back into an *empty* `postgres-data` volume with the stack down — it is a whole
data directory, not a file you can drop next to the live one, and it will only
start under `postgres:17`.
### Retention
@@ -132,7 +162,21 @@ Keep a month, drop the rest — a bookmark database this small compresses the
decision to "disk is free, but not infinite":
```bash
ls -1t "$BACKUP_DIR"/bookmarks-*.db | tail -n +31 | xargs -r rm -v
ls -1t "$BACKUP_DIR"/bookmarks-*.dump | tail -n +31 | xargs -r rm -v
```
### A note on the old `bookmarks-data` volume
`bookmarks-data` is the **pre-migration SQLite volume**. It is deliberately not
declared in `docker-compose.yml` any more, which is what keeps `docker compose
down -v` from taking it with the rest of the stack. It is not the live database
and nothing reads it — the one-way move out of it is `CUTOVER.md`. Once the
Postgres data has been trusted for a while, remove it by hand — nothing else will.
Its full name is `<compose project>_bookmarks-data`, and the project name is the
lowercased directory name of the checkout:
```bash
docker volume rm "$(basename "$PWD" | tr '[:upper:]' '[:lower:]')_bookmarks-data"
```
---
@@ -171,12 +215,16 @@ rebuilt. The one exception is `userscript/manga-bookmark.user.js`, which is
bindmounted read-only and read fresh per request.
```bash
$COMPOSE ps # Up, and recently (re)created
docker logs manga-api --tail 20 # -> "listening on :8080 ..."
$COMPOSE ps # bookmark-api Up; postgres Up (healthy)
docker logs bookmark-api --tail 20 # -> "listening on :8080 ..."
```
Nothing in the log about the database or the poller failing. The image is tagged
`mangabm-backend:latest`, so the previous image is still on disk untagged —
Nothing in the log about the database, the migrations or the poller failing.
`bookmark-api` waits on `postgres` reporting healthy before it starts and the
binary applies any pending migration before it listens, so an API that never
says "listening" is usually the database, not the code — `$COMPOSE logs
postgres` first. The image is tagged
`bookmarkmanager-backend:latest`, so the previous image is still on disk untagged —
that is what makes the rollback in §6 quick.
---
@@ -186,9 +234,12 @@ that is what makes the rollback in §6 quick.
Same four API checks as `DEPLOY.md` §3, plus the web UI. Set the host names once:
```bash
API=https://manga-api.violetcrown.my.id
WEB=https://manga.violetcrown.my.id
TOKEN=$(grep -E '^API_TOKEN=' .env | cut -d= -f2)
API=https://bookmark-api.violetcrown.my.id
WEB=https://bookmark.violetcrown.my.id
# Your own Reader credential - derived, never stored in .env. Take it from the
# Userscripts panel's install link after signing in, or from an installed
# script's API_TOKEN constant.
TOKEN=<your Reader credential>
curl -s $API/healthz # -> ok
curl -s -o /dev/null -w '%{http_code}\n' $API/bookmarks # -> 401
@@ -199,9 +250,16 @@ curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \
$API/bookmarks/x | grep -i access-control # -> allow-origin echoed
```
`[]` from the third call is the alarm that matters: the volume is not attached
and you are looking at an empty database. Stop and check `$COMPOSE config
--volumes` before touching anything else.
`[]` from the third call is the alarm that matters: you are talking to an empty
database, which means the API found a *different* Postgres than the one holding
your data — a renamed project directory, a fresh `postgres-data`, or a
`DATABASE_URL` override in `.env` pointing elsewhere. Stop and check, before
touching anything else:
```bash
$COMPOSE config --volumes # -> postgres-data
$COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -c 'select count(*) from bookmarks'
```
Web UI and its assets:
@@ -267,33 +325,41 @@ git checkout <previous-hash>
$COMPOSE up -d --build
```
**Database damaged** — restore the backup from §1. Stop first: the running
process holds the WAL, and dropping a file under a live SQLite connection
corrupts what you were trying to save.
**Database damaged** — restore the dump from §1. Stop **only the API**, not the
whole stack: `pg_restore` needs the server up to restore into, and it needs
`bookmark-api`'s connection pool gone, because `--clean` cannot drop a table
other sessions are holding open.
```bash
$COMPOSE stop
$COMPOSE stop bookmark-api
docker run --rm -v "$VOL":/data -v "$BACKUP_DIR":/backup alpine sh -c '
rm -f /data/bookmarks.db /data/bookmarks.db-wal /data/bookmarks.db-shm &&
cp /backup/bookmarks-<STAMP>.db /data/bookmarks.db &&
chown 65532:65532 /data/bookmarks.db &&
ls -l /data'
$COMPOSE exec -T postgres pg_restore -U bookmarks -d bookmarks --clean --if-exists \
< "$BACKUP_DIR/bookmarks-<STAMP>.dump"
$COMPOSE start
docker logs manga-api --tail 20
$COMPOSE start bookmark-api
docker logs bookmark-api --tail 20
curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks | head -c 200
```
Two steps here are easy to skip and both bite:
Three things here are easy to skip and all three bite:
- **Delete the stale `-wal` and `-shm`.** Leaving them beside a restored database
mixes two different histories; SQLite will either refuse to open it or quietly
reapply writes you meant to discard.
- **`chown 65532:65532`.** The image is `distroless/static:nonroot` and runs as
that uid, while the helper container above writes as root. A root-owned
database opens read-only-ish: reads work, so `/bookmarks` looks fine, and then
every write fails. That is the worst possible failure mode — it looks restored.
- **`--clean --if-exists`.** Without `--clean` the dump's rows land *on top of*
what is already there and you get primary-key collisions half way through, a
partially restored database, and a non-zero exit you may not notice.
`--if-exists` only suppresses the "does not exist" noise when the target is
already empty; it is not the part doing the work.
- **`-T` again.** Feeding a custom-format archive into a TTY-allocated `exec`
corrupts it in flight and `pg_restore` fails with a garbled-header error on a
file that is perfectly fine on disk.
- **Stop the API, not Postgres.** `$COMPOSE stop` (everything) leaves you with
nothing to restore into; leaving `bookmark-api` running leaves connections
that block the drops *and* lets the poller write into a half-restored table.
No ownership fixing is needed any more — the Postgres image owns `postgres-data`
itself and `pg_restore` writes through the server, not the filesystem.
`schema_migrations` is inside the dump, so the database comes back at whatever
schema version the backup was taken at; the migration runner applies anything
newer the next time `bookmark-api` starts.
---
@@ -302,24 +368,76 @@ Two steps here are easy to skip and both bite:
For a routine redeploy where nothing needs deciding:
```bash
cd /opt/mangabm
cd /opt/bookmarkmanager
COMPOSE="docker compose -f docker-compose.yml -f docker-compose.prod.yml"
BACKUP_DIR="$(cd .. && pwd)/mangabm-backups"; mkdir -p "$BACKUP_DIR"
VOL=$(docker volume ls --filter name=bookmarks-data -q | head -1)
BACKUP_DIR="$(cd .. && pwd)/bookmarkmanager-backups"; mkdir -p "$BACKUP_DIR"
STAMP=$(date -u +%Y%m%d-%H%M%S)
docker run --rm -v "$VOL":/data -v "$BACKUP_DIR":/backup alpine sh -c \
"apk add -q sqlite && sqlite3 /data/bookmarks.db \"VACUUM INTO '/backup/bookmarks-$STAMP.db'\" &&
sqlite3 /backup/bookmarks-$STAMP.db 'PRAGMA integrity_check;'" &&
$COMPOSE exec -T postgres pg_dump -U bookmarks -d bookmarks -Fc \
> "$BACKUP_DIR/bookmarks-$STAMP.dump" &&
docker run --rm -v "$BACKUP_DIR":/backup postgres:17-alpine \
pg_restore --list "/backup/bookmarks-$STAMP.dump" > /dev/null &&
git pull --ff-only &&
$COMPOSE up -d --build &&
sleep 5 &&
curl -sf https://manga-api.violetcrown.my.id/healthz && echo " deploy ok"
curl -sf https://bookmark-api.violetcrown.my.id/healthz && echo " deploy ok"
```
The `&&` chain is deliberate: if the backup or its integrity check fails,
nothing is pulled and nothing is rebuilt. Then still do §5 by hand — no shell
command can tell you the panel works on the phone.
The `&&` chain is deliberate: if the dump or its `pg_restore --list` check
fails, nothing is pulled and nothing is rebuilt. A failed dump still leaves a
short or empty `.dump` behind — the shell creates the file before `pg_dump`
runs — so delete it rather than letting it sit in the backup directory looking
like a backup. Then still do §5 by hand — no shell command can tell you the
panel works on the phone.
---
## 8. The browser unit (separate machine, separate cadence)
Everything above is the API stack on the VPS. The headless browser is its own
compose unit on the home machine (ADR-0006, `DEPLOY.md` §7) and is redeployed
on its own schedule — it holds no data you can lose, so there is nothing to
back up and no ordering constraint against the API.
```bash
cd ~/mangaBookmark/chrome
git pull --ff-only
docker compose up -d --build
```
Then confirm it answers, and that a stopped-and-restarted Chrome is invisible
to the API:
```bash
curl -s -m 15 http://$(tailscale ip -4):9222/json/version | head -c 120
# -> {"Browser":"Chrome/1xx...","webSocketDebuggerUrl":"ws://...<new uuid>"}
```
The first call takes a few seconds: Chrome is not running until something
connects, and it is reaped again after five idle minutes. The debugger UUID
changes on every start and the API does not care — chromedp re-runs
`/json/version` discovery per fetch, which is exactly why `chromedp.NoModifyURL`
must never be added to `browser.go`.
**Rebuild is the Chrome upgrade path.** The image installs
`google-chrome-stable` unpinned on purpose: a stale browser is what Cloudflare
turns away, and the pinned Chrome 124 in `zenika/alpine-chrome` is the worked
example. The `chrome-profile` volume survives `--build`, so clearance cookies
are reused rather than re-solved.
Two things worth a glance after several days, both from the acceptance criteria
of the move:
```bash
docker inspect bookmark-browser --format '{{.RestartCount}} {{.State.OOMKilled}}'
# -> 0 false
free -m # the Gitea runner should still have its headroom
```
Nothing here needs doing during an API redeploy. The API stack does not
`depends_on` the browser, and an unreachable one degrades exactly as an unset
`BROWSER_WS_URL`: plain-TLS libraries unaffected, kagane and novelfull logged
and skipped, stored covers still served.
---
@@ -327,16 +445,24 @@ command can tell you the panel works on the phone.
| Symptom | Cause / fix |
|---|---|
| `/bookmarks` returns `[]` after redeploy | Volume not attached — check `$COMPOSE config --volumes` and that you passed both `-f` files. Do **not** re-bookmark; the data is still in the volume. |
| `/bookmarks` returns `[]` after redeploy | You are on an empty Postgres. Check `$COMPOSE config --volumes` lists `postgres-data`, that you passed both `-f` files, and that `.env` has no stray `DATABASE_URL` override. Do **not** re-bookmark; the data is still in the volume. |
| UI looks like plain Georgia / system sans | `static/fonts/` missing from the image, or the browser cached an old `style.css`. `/static/*` is served `max-age=3600`, so hard-reload or wait an hour. |
| CSS or template change did not appear | You restarted without `--build`. Assets are `//go:embed`ed. |
| Font answers `application/octet-stream` | Old binary — the `.woff2` MIME registration is in `web.go`. Rebuild. |
| Everyone logged out of the web UI | `API_TOKEN` or `WEB_PASSWORD` changed; sessions are derived from both. Expected, just log in again. |
| `compose` errors about `MANGA_WEB_HOST` | Run from the directory holding `.env`. Both host vars are required even when the web UI is unused. |
| Everyone logged out of the web UI | The `sessions` table was wiped; sessions are database rows, not signed cookies. Expected after a deliberate revoke. |
| `compose` errors about `BOOKMARK_WEB_HOST` | Run from the directory holding `.env`. Both host vars are required even when the web UI is unused. |
| Userscript did not update on the phone | Violentmonkey polls on its own schedule; force a check. `@version` comes from the file's mtime, so confirm the pull actually touched it. |
| `apk add sqlite` fails (no network) | Use the cold-copy fallback in §1 — and copy `bookmarks.db-wal` too. |
| Reads work but every write fails after a restore | Restored file is root-owned; the container is uid 65532. `chown 65532:65532` it (§6). |
| Backup command: `unable to open database file` | Source volume mounted `:ro`. WAL needs to create `-shm`; mount it read-write (§1). |
| `bookmark-api` crash-loops, log says `password authentication failed for user "bookmarks"` | `POSTGRES_PASSWORD` in `.env` no longer matches the one burned into `postgres-data` at first init — Postgres reads that variable only when initialising an empty volume. Put the old value back, or reset the role: `$COMPOSE exec postgres psql -U bookmarks -d bookmarks -c '\password bookmarks'` (prompts, so nothing lands in shell history) and then match `.env` to it. |
| `compose` errors `set POSTGRES_PASSWORD in .env` | Unset. Compose builds the backend's `DATABASE_URL` out of it, so it is required even though you never write that URL yourself. Run from the directory holding `.env`. |
| `postgres` never leaves `starting`; `bookmark-api` never starts either | The healthcheck (`pg_isready`) is failing and `bookmark-api` waits on it. `$COMPOSE logs postgres` — usually `postgres-data` was initialised by a different major version ("database files are incompatible with server"), or the disk is full. |
| `pg_restore`: `cannot drop … other objects depend on it` / `being accessed by other users` | Live connections block `--clean`. `$COMPOSE stop bookmark-api` first (§6). If they persist: `$COMPOSE exec -T postgres psql -U bookmarks -d postgres -c "select pg_terminate_backend(pid) from pg_stat_activity where datname='bookmarks' and pid <> pg_backend_pid()"`. |
| Dump is 0 bytes, or `pg_restore`: `did not find magic string in file header` | You ran `exec` without `-T`. The allocated TTY rewrites newlines in the binary stream and corrupts the archive in flight (§1). |
| `git pull`: `could not read Username for 'https://…'` | The checkout's remote is the HTTPS clone URL and the server has no credential helper, so the pull prompts into a closed stdin. Switch it to SSH once — `git remote set-url origin ssh://git@gitea.violetcrown.my.id:2222/sulthan/mangaBookmark.git`. Gitea's SSH listens on **2222**, not 22; port 22 is the host's own sshd and answers `Permission denied (publickey)` no matter which key is registered. |
| kagane rows stopped updating after a redeploy | Check `BROWSER_WS_URL` survived the `.env` edit and still names the home machine's tailnet **IP**. A hostname 500s at `/json/version`; an empty value disables the browser silently. Plain-TLS sites keep working either way, which is why this is easy to miss. |
| kagane covers went blank in the web UI | Covers use the `cover-data` volume now. Restore/check that volume alongside Postgres; rows in `covers` are metadata only. If the database has rows but files are missing, the next browser-backed request refetches them; without a browser it remains a 404. |
| Browser unit will not start: `set BROWSER_BIND_ADDR to this machine's tailnet IP` | `chrome/.env` is missing or the variable is empty. It has no default on purpose — an unset value must fail the deploy rather than publish an unauthenticated CDP port to the LAN. |
| `bookmark-browser` shows `OOMKilled true` | The cap did its job. Read `docker logs bookmark-browser` before raising it — the sizing and what the cap protects are in ADR-0006. |
Full first-time setup: `DEPLOY.md`. Config reference and endpoints: `README.md`.
Full first-time setup: `DEPLOY.md`. The one-off SQLite→Postgres move:
`CUTOVER.md`. Config reference and endpoints: `README.md`.
UI conventions: `docs/design-system.md`.
+4 -6
View File
@@ -1,10 +1,8 @@
# Only go source + module files, plus the go:embed'd templates/static
# directories, are needed in the build context.
# Only go source + module files, plus internal/ (which carries the
# go:embed'd templates/static directories), are needed in the build context.
*
!go.mod
!go.sum
!*.go
!templates/
!templates/**
!static/
!static/**
!internal/
!internal/**
+186
View File
@@ -0,0 +1,186 @@
Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGENTS.md` for the project-wide architecture diagram, hard constraints, and design system.
- **Backend** (`backend/`): stdlib `net/http` (handful routes, no framework) + Postgres over `jackc/pgx/v5` (pure Go, `CGO_ENABLED=0` -> static binary -> distroless/scratch image). Reverse proxy terminates TLS; Go service listens plain `:8080`.
Single binary, split into packages under `backend/internal/`: `store`
(Bookmark type, Postgres persistence, migration runner), `latest` (background
poller, site parsers, TLS fetcher), `session` (cookie signing, login
rate limiter), `httpmw` (Auth/Gzip/CORS middleware), `api` (JSON
bookmark handlers), `userscript` (userscript-serving handler), `web`
(browser UI handler + `templates/` + `static/`, `go:embed`-ed).
`backend/main.go` is the composition root — the only place that wires
packages together into `newRouter`. Root-level `*_test.go` hold
integration tests that exercise the full router; unit tests for a
package live beside it under `internal/`.
- **Schema is migration-owned.** `internal/store/migrations/*.sql` is
`go:embed`-ed and applied on every start by `store.migrate`: one numbered
file per change, one transaction each, versions recorded in
`schema_migrations`. Files are **append-only** — editing an applied one
changes nothing on a database that already ran it. No column probing, no
data-fixup migrations: both were SQLite-era machinery and are gone.
- **Tests need Docker.** `internal/pgtest` starts one `postgres:17-alpine`
container per test binary (`TestMain` -> `pgtest.Main`) and hands each test
its own database (`pgtest.URL(t)`). A package whose tests touch the store
must have that `TestMain`.
- **Reader-owned store, four tables.** `readers` is keyed by Discord user ID
and carries the SHA-256 of the Reader's userscript credential plus a
`token_epoch` (issue #24). Credentials are derived, never stored: `token.Token(TOKEN_KEY, discord_id, epoch)` (HMAC, `internal/token`), and only its SHA-256 sits in `readers.token_sha256`, so install URLs can be rebuilt after any restart while a database leak yields nothing but hashes. The seed creates the **owner** row at startup; its epoch-0 hash is refreshed on every start **only while the row has never been rotated**, so a restart can never resurrect a rotated-away credential. Every other row is created by that Reader's own first login (`Store.EnsureReader`, idempotent on `discord_id`, and it never rewrites an existing row's hash). Rotation is `Store.RotateToken` (epoch bump + hash rewrite in one transaction), driven by the web UI.
`series` keyed `(site, series_id)`
(`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`) owns the
shared facts — title, cover, canonical URL, `kind` (`manga`|`novel`),
Latest Chapter, `latest_checked_at` — and `bookmarks` holds only what
differs between readers: progress, favourite, lifecycle bucket,
`updated_at`. A bookmark is keyed `(reader_id, site, series_id)` — no
surrogate id; the wire `key` is derived as `site:series_id` on read — and
every store read/write is scoped to the reader it names. Auth resolves the
acting Reader from the presented credential (`httpmw.Auth`) and nothing
else — there is no unauthenticated-by-Reader route and no global token; the
reader id travels in the request context. Sync **last-write-wins**; the wire format
stays flat (ADR-0004). `Store.Upsert` decomposes one flat body across two
tables and enforces the ownership rule: client `title`/`series_url`/`cover`
are written only when the series row is new (ADR-0003).
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- **Web UI:** same binary serve the browser UI on a second
hostname — `GET /` (list, or login page when no session),
`GET /auth/discord` + `GET /auth/discord/callback` (Discord OAuth,
ADR-0002), `POST /logout`, `GET /static/*`, htmx fragment endpoints
under `/ui/*`. Templates + assets `go:embed`-ed under
`backend/internal/web/`, so `backend/Dockerfile` must copy the whole
`internal/` tree, not just `*.go`. Sessions are rows in the `sessions`
table: the cookie carries only an opaque id, looked up (and expiry-
checked) on every request, and deleting the row revokes the session.
Guild membership *is* registration (issue #27): `discordCallback` gates on
membership (and `DISCORD_REQUIRED_ROLE` when set) and then calls
`Store.EnsureReader`, so a refusal creates nothing and a returning Reader
reuses their row. The owner is the only Reader with administrative reach:
`POST /readers/{id}/revoke` (404 for anyone else) drops that Reader's
sessions, and the `readers` panel renders only on the owner's page.
A Reader with no bookmarks at all sees `listView.Fresh`, whose empty state
offers both install links instead of describing a filter.
UI mutations read-modify-write
through `Store.Get` + `Store.Upsert` so `updated_at` rule stays one
place. See `docs/superpowers/specs/2026-07-25-web-ui-design.md`.
**Design-tool caveat:** templates link `/static/style.css` root-absolutely
(correct — served from `/`), but impeccable detector resolves
stylesheet href with `path.resolve(fileDir, href)`, drops directory
on leading `/` and silently skip file. Relative href don't help
either: template's directory isn't its served path. So
`detect.mjs backend/internal/web/templates` reports **false clean** —
always pass `backend/internal/web/static` too. One finding there,
`overused-font` on "Instrument Serif", deliberate identity choice, not debt.
- **Every action that moves series out of list is confirm-gated.**
Archive, finish, remove each open own `.confirm-row` disclosure
(`toggleConfirmRow(key, kind)` in `filter.js`, `kind` ∈
`archive|finish|remove`); restore fire instantly since it's the reversal.
Remove's row wear ember wash, two reversible ones wear `.calm` grey.
`--ember` stay reserved for new-chapter signal: busy bar and inline
error use `--mute`.
- **Latest-chapter poller:** ticker goroutine in same binary re-check
each bookmarked series' newest published chapter from backend's own
network access, so `latest_chapter` stay fresh when user not
browsing. Second, parallel signal — userscript keep own
`maybeCaptureLatestOnSeriesPage`/`backgroundRefreshLatest` logic unchanged.
Two independent clocks: per-series cooldown (`series.latest_checked_at`,
enforced by `Store.DueForLatestCheck`'s WHERE clause) and wake interval.
The poller walks **Series, not Bookmarks** — a series referenced by several
bookmarks is fetched once per cycle, and the due queue orders
`reader_count DESC, latest_checked_at ASC` (ADR-0003). Series row stamped
*before* fetch so broken series wait out full cooldown instead of retrying
every tick; found chapter written straight to the series row via
`Store.SetLatestChapter`, so a bookmark's `updated_at` — and the list
order — is never touched.
Fetches use `bogdanfinn/tls-client` with Chrome profile as defence in depth
against fingerprint-based blocking; any failure log and skip. kagane and
novelfull sit behind Cloudflare JavaScript challenges the TLS client can't
clear, so they are fetched over CDP via `BROWSER_WS_URL`; kagane is simply
not polled when that's unset, while novelfull falls back to a plain-TLS
attempt — its challenge is a live time-varying fact, and its cover bytes
never need the browser. See
`docs/superpowers/specs/2026-07-26-server-latest-chapter-polling-design.md`.
The poller's series write is a single-column UPDATE
(`Store.SetLatestChapter`), not a read-modify-write of the whole bookmark:
it cannot revert read progress or move `updated_at`, so the old
stale-re-read race is gone with the Get+Upsert flow.
- **Covers are acquired at creation, then served from our own origin
(ADR-0007):** the first Bookmark of a Series fires `Store.OnSeriesCreated`,
which `latest.Acquirer` turns into one series-page fetch yielding both the
Latest Chapter and the cover URL; the bytes then go through
`latest.CoverBytesFetcher` into `Store.SetSeriesCover`. It runs in a
goroutine — the Reader's PUT must neither block on a Site nor fail with one
— and every failure is logged and dropped, leaving the Bookmark intact. The
wire's `cover` is the absolute `PUBLIC_BASE_URL + /covers/{sha256}` once
bytes exist and `""` before, never an address that 404s. `GET /covers/{addr}`
is public and uncredentialed: the userscript renders it on a Site's origin,
where no cookie or token of ours travels. A client-sent `cover` is decoded
and discarded, permanently (ADR-0004 compatibility).
Browser-backed Sites join the same pipeline (issue #62): kagane pages *and*
cover bytes go through the browser sidecar (nothing falls back to a plain
fetch, which would only retrieve a challenge page), while novelfull needs
the browser only for its HTML — the cover URL comes out of the
browser-fetched page and the bytes go over plain TLS. With no browser
configured, kagane Covers are simply absent; novelfull still gets one — at
creation and on the poll — when its page body happens to answer a plain
request (the challenge is a live time-varying fact). The old kagane-only
serving path (`/img/kagane/{id}`, template rewrite, `CoverFetcher`) is gone
(issue #63): the one public route serves every Site.
- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
`finished`, orthogonal to `favorite`. Archived and finished appear only in
own tab — not in All, Updated, Favourites, or recent strip. Poller keeps
checking archived series and skip finished ones. `finished` settable
only from web UI; `PUT /bookmarks/{key}` reject it with 400.
**Empty incoming status means "keep stored one"** — resolved on the
`VALUES` side of `Store.Upsert`, not conflict clause, since
`excluded.*` is post-evaluation row and default applied there would
wipe bucket on every PUT from client that predates column. See
`docs/superpowers/specs/2026-07-27-status-buckets-design.md`.
- **Config via env:** `TOKEN_KEY` (derives every Reader's userscript credential;
required), `OWNER_DISCORD_ID` (seeds the owner Reader — the administrator and
the owner of every pre-registration bookmark; required),
`ALLOWED_ORIGINS` (comma list),
`DATABASE_URL` (Postgres connection URL, required — no default),
`COVER_DIR` (required filesystem volume for content-addressed Cover bytes),
`PUBLIC_BASE_URL` (required origin this deployment answers on, trailing
slash trimmed; every Cover URL on the wire is built from it, absolute
because the userscript renders on a Site's origin — ADR-0007),
`PORT` (default `8080`), `DISCORD_CLIENT_ID`/`_CLIENT_SECRET`/`_GUILD_ID`/
`_REDIRECT_URI` (required; Discord OAuth for the browser UI),
`DISCORD_REQUIRED_ROLE` (optional role gate, empty by default),
`DISCORD_API_BASE` (default `https://discord.com/api/v10`),
`LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_BROWSER_COOLDOWN`/`_INTERVAL`/
`_BATCH`/`_STAGGER` (background latest-chapter poller; defaults on,
`1h` plain-TLS cooldown, `6h` browser cooldown, `10m`/`14`/`20s`; both
cooldowns have a `15m` floor).
`USERSCRIPT_PATH` and `NOVEL_USERSCRIPT_PATH` (files served at
`/u/{token}/manga-bookmark.user.js` and `/u/{token}/novel-bookmark.user.js`,
defaults `/userscript/manga-bookmark.user.js` and
`/userscript/novel-bookmark.user.js`, both supplied by bindmount; the
`__API_TOKEN__` placeholder inside them is substituted with the requesting
Reader's credential at serve time).
`BROWSER_WS_URL` (CDP endpoint of the browser, which runs on a **separate
machine** and is reached over the tailnet — ADR-0006, `chrome/docker-compose.yml`.
Used by the poller for kagane and novelfull page fetches and by the cover
pipeline for kagane's image bytes (the browser is the only route that clears
the challenge kagane serves its covers behind); unset — the default —
disables browser polling and leaves kagane Covers blank until stored bytes
exist. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s
`/json/version` for any Host that isn't an IP or `localhost`).
- **No per-Site cover path (issue #63):** every Cover — all six Sites — is
served by the one public `GET /covers/{addr}` route from content-addressed
bytes. There is no proxy, no per-Site rewrite, no second place that decides
a Cover's renderable address: the wire `cover` is it. The only place a Site
name still appears in cover code is the extraction module (`latest`), where
kagane's image URLs are claimed by `browserOnlyCoverURL` — they answer a
plain fetch with a challenge and `cross-origin-resource-policy: same-origin`;
every other Site's CDN answers plain TLS. Templates render `.Cover` — the
wire value — never anything else.
- **Web UI also owns:** session-gated `GET /install/{manga,novel}-bookmark.user.js`
(renders the bindmounted script with the acting Reader's derived credential
substituted in — the credential never appears in page markup, the address
bar, or a redirect; `?download=1` adds `Content-Disposition: attachment` for
mobile Violentmonkey, which ignores a `.user.js` navigation) and
`POST /rotate-token` (atomic epoch bump + hash
rewrite; invalidates every installed copy, so the panel warns to reinstall
on all devices).
Owner-only `POST /readers/{id}/revoke` (drops one Reader's session rows and
re-renders the `readers` panel; 404 for any non-owner) is the only route that
reaches across Readers.
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+13 -13
View File
@@ -1,35 +1,35 @@
# syntax=docker/dockerfile:1
# --- build stage: compile a static, CGO-free binary ---
FROM golang:1.24-alpine AS build
FROM golang:1.26-alpine AS build
ARG COVER_DIR=/covers
WORKDIR /src
# Dependencies first for layer caching (changes rarely).
COPY go.mod go.sum ./
RUN go mod download
# Then source (changes often).
# Source plus the go:embed'd assets. Missing either directory turns the embed
# directive into a build error, so both must be copied before `go build`.
# Then source (changes often). internal/web carries the go:embed'd
# templates/static assets — missing them turns the embed directive into a
# build error, so the whole tree must land before `go build`.
COPY *.go ./
COPY templates/ ./templates/
COPY static/ ./static/
COPY internal/ ./internal/
# Static binary: pure-Go sqlite means CGO_ENABLED=0 -> no libc dependency.
# Static binary: the Postgres driver (jackc/pgx) is pure Go, so CGO_ENABLED=0
# leaves no libc dependency.
# -trimpath + -ldflags strip paths and debug info for a smaller image.
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/server .
# Data dir with the runtime user's ownership so the mounted volume inherits it.
RUN mkdir -p /out/data
# Create the source directory; runtime COPY sets ownership for the named volume.
RUN mkdir -p "$COVER_DIR"
# --- runtime stage: distroless static, non-root ---
FROM gcr.io/distroless/static:nonroot
ARG COVER_DIR=/covers
WORKDIR /
COPY --from=build --chown=65532:65532 ${COVER_DIR} ${COVER_DIR}
COPY --from=build /out/server /server
COPY --from=build --chown=65532:65532 /out/data /data
VOLUME ["/data"]
EXPOSE 8080
USER nonroot:nonroot
ENV DB_PATH=/data/bookmarks.db PORT=8080
ENV PORT=8080
ENTRYPOINT ["/server"]
+680
View File
@@ -0,0 +1,680 @@
package main
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
"bookmarkmanager/backend/internal/pgtest"
"bookmarkmanager/backend/internal/store"
"bookmarkmanager/backend/internal/token"
)
// testTokenKey derives every test Reader's credential; it must match the key
// newTestStoreURL seeds the owner with, or derived credentials authenticate
// nothing.
const testTokenKey = "test-token-key"
// testDiscordID is the owner row's discord_id (newTestStoreURL); the derived
// credential is a function of it.
const testDiscordID = "test-owner"
// testCoverBaseURL is the public origin cover URLs are built from, standing in
// for PUBLIC_BASE_URL.
const testCoverBaseURL = "https://bookmarks.test"
func testConfig() Config {
return Config{
TokenKey: testTokenKey,
AllowedOrigins: []string{"https://asurascans.com", "https://demonicscans.org"},
Port: "8080",
}
}
// ownerCredential is the owner's epoch-0 derived credential: the string the
// install links carry and the userscript routes authenticate.
func ownerCredential() string {
return token.Token([]byte(testTokenKey), testDiscordID, 0)
}
func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
func newTestServer(t *testing.T) http.Handler {
t.Helper()
return newRouter(newTestStore(t), testConfig())
}
func newTestStore(t *testing.T) *store.Store {
t.Helper()
s, _ := newTestStoreURL(t)
return s
}
// newTestStoreURL is newTestStore plus the database URL, for tests that need
// to reach the same database directly.
func newTestStoreURL(t *testing.T) (*store.Store, string) {
t.Helper()
url := pgtest.URL(t)
s, err := store.Open(url, store.Owner{
DiscordID: testDiscordID, TokenHash: token.Hash(ownerCredential()),
}, t.TempDir(), testCoverBaseURL)
if err != nil {
t.Fatalf("store.Open: %v", err)
}
t.Cleanup(func() { s.Close() })
return s, url
}
// auth authenticates a request as the owner Reader, whose derived credential
// is the only thing the API accepts.
func auth(req *http.Request) *http.Request {
req.Header.Set("Authorization", "Bearer "+ownerCredential())
return req
}
func floatPtr(f float64) *float64 { return &f }
// seedForCheck inserts a bookmark (and with it its series) and forces the
// series' latest_checked_at.
func seedForCheck(t *testing.T, s *store.Store, key, seriesURL string, checkedAt int64) {
t.Helper()
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: key,
Site: site,
SeriesID: seriesID,
SeriesURL: seriesURL,
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed %q: %v", key, err)
}
if err := s.MarkLatestChecked(site, seriesID, checkedAt); err != nil {
t.Fatalf("seed mark %q: %v", key, err)
}
}
func readLatestCheckedAt(t *testing.T, s *store.Store, key string) int64 {
t.Helper()
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
t.Fatalf("key %q: no ':' separator", key)
}
ts, err := s.LatestCheckedAt(site, seriesID)
if err != nil {
t.Fatalf("LatestCheckedAt %q: %v", key, err)
}
return ts
}
func TestHealthzNoAuth(t *testing.T) {
srv := newTestServer(t)
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/healthz", nil))
if rr.Code != http.StatusOK {
t.Fatalf("healthz status = %d, want 200", rr.Code)
}
if rr.Body.String() != "ok" {
t.Fatalf("healthz body = %q, want ok", rr.Body.String())
}
}
func TestAuthRequired(t *testing.T) {
srv := newTestServer(t)
cases := []struct {
name string
header string
}{
{"no header", ""},
{"bad token", "Bearer wrong"},
{"not bearer", "Basic " + ownerCredential()},
{"empty bearer", "Bearer "},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/bookmarks", nil)
if tc.header != "" {
req.Header.Set("Authorization", tc.header)
}
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if rr.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401", rr.Code)
}
})
}
}
func TestAuthAccepted(t *testing.T) {
srv := newTestServer(t)
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil)))
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rr.Code)
}
if got := rr.Body.String(); got != "[]\n" {
t.Fatalf("empty list body = %q, want []", got)
}
}
func TestCORSPreflight(t *testing.T) {
srv := newTestServer(t)
req := httptest.NewRequest(http.MethodOptions, "/bookmarks/asura:foo-1", nil)
req.Header.Set("Origin", "https://asurascans.com")
req.Header.Set("Access-Control-Request-Method", "PUT")
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if rr.Code != http.StatusNoContent {
t.Fatalf("preflight status = %d, want 204", rr.Code)
}
if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "https://asurascans.com" {
t.Fatalf("Allow-Origin = %q, want reflected origin", got)
}
if got := rr.Header().Get("Access-Control-Allow-Methods"); got == "" {
t.Fatal("Allow-Methods missing")
}
if got := rr.Header().Get("Access-Control-Allow-Headers"); got == "" {
t.Fatal("Allow-Headers missing")
}
}
func TestCORSDisallowedOrigin(t *testing.T) {
srv := newTestServer(t)
req := httptest.NewRequest(http.MethodOptions, "/bookmarks", nil)
req.Header.Set("Origin", "https://evil.example")
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if got := rr.Header().Get("Access-Control-Allow-Origin"); got != "" {
t.Fatalf("Allow-Origin = %q, want empty for disallowed origin", got)
}
}
func TestBookmarkRoundTrip(t *testing.T) {
srv := newTestServer(t)
key := "asura:solo-leveling-123"
in := store.Bookmark{
Title: "Solo Leveling",
SeriesURL: "https://asurascans.com/series/solo-leveling-123",
Cover: "https://asurascans.com/cover.jpg",
LastChapter: "Chapter 10",
LastChapterNum: 10,
LastChapterURL: "https://asurascans.com/series/solo-leveling-123/chapter/10",
}
body, _ := json.Marshal(in)
// PUT
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body))))
if rr.Code != http.StatusOK {
t.Fatalf("PUT status = %d, want 200", rr.Code)
}
var stored store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &stored); err != nil {
t.Fatalf("decode PUT response: %v", err)
}
if stored.Key != key || stored.Site != "asura" || stored.SeriesID != "solo-leveling-123" {
t.Fatalf("derived fields wrong: %+v", stored)
}
if stored.UpdatedAt == 0 {
t.Fatal("server did not set updated_at")
}
// GET
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil)))
var list []store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil {
t.Fatalf("decode list: %v", err)
}
if len(list) != 1 || list[0].Key != key || list[0].LastChapterNum != 10 {
t.Fatalf("GET list wrong: %+v", list)
}
// PUT again (upsert, progress advance)
in.LastChapter, in.LastChapterNum = "Chapter 11", 11
body, _ = json.Marshal(in)
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body))))
if rr.Code != http.StatusOK {
t.Fatalf("second PUT status = %d", rr.Code)
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil)))
json.Unmarshal(rr.Body.Bytes(), &list)
if len(list) != 1 || list[0].LastChapterNum != 11 {
t.Fatalf("upsert did not update in place: %+v", list)
}
// DELETE
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodDelete, "/bookmarks/"+key, nil)))
if rr.Code != http.StatusNoContent {
t.Fatalf("DELETE status = %d, want 204", rr.Code)
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil)))
json.Unmarshal(rr.Body.Bytes(), &list)
if len(list) != 0 {
t.Fatalf("after delete list = %+v, want empty", list)
}
}
// The wire contract (ADR-0004): GET and PUT speak exactly the flat field set
// they always did, with the series-owned fields as siblings of the bookmark
// fields, not nested. Asserted as a key set, not by inspection.
func TestFlatWireFieldSet(t *testing.T) {
srv := newTestServer(t)
key := "comix:some-title"
in := store.Bookmark{
Key: key,
Site: "comix",
SeriesID: "some-title",
Title: "Some Title",
SeriesURL: "https://comix.to/title/some-title",
Cover: "https://comix.to/covers/some-title.jpg",
LastChapter: "Chapter 7",
LastChapterNum: 7,
LastChapterURL: "https://comix.to/title/some-title/ch/7",
Favorite: true,
LatestChapter: "Chapter 8",
LatestChapterNum: floatPtr(8),
Status: store.StatusArchived,
Kind: store.KindManga,
}
body, _ := json.Marshal(in)
wantKeys := map[string]bool{
"key": true, "site": true, "series_id": true, "title": true,
"series_url": true, "cover": true, "last_chapter": true,
"last_chapter_num": true, "last_chapter_url": true, "favorite": true,
"latest_chapter": true, "latest_chapter_num": true, "updated_at": true,
"status": true, "kind": true,
}
checkFlat := func(t *testing.T, payload []byte) map[string]json.RawMessage {
t.Helper()
var obj map[string]json.RawMessage
if err := json.Unmarshal(payload, &obj); err != nil {
t.Fatalf("decode: %v", err)
}
if len(obj) != len(wantKeys) {
t.Fatalf("field count = %d, want %d (%s)", len(obj), len(wantKeys), payload)
}
for k := range obj {
if !wantKeys[k] {
t.Fatalf("unexpected field %q", k)
}
}
return obj
}
// PUT
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body))))
if rr.Code != http.StatusOK {
t.Fatalf("PUT status = %d, want 200", rr.Code)
}
checkFlat(t, rr.Body.Bytes())
// Every field round-trips with its value, and updated_at is server-stamped.
var stored store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &stored); err != nil {
t.Fatalf("decode PUT response: %v", err)
}
latestNum := floatPtr(8)
want := store.Bookmark{
Key: key, Site: "comix", SeriesID: "some-title",
Title: in.Title, SeriesURL: in.SeriesURL,
LastChapter: in.LastChapter, LastChapterNum: in.LastChapterNum,
LastChapterURL: in.LastChapterURL, Favorite: true,
LatestChapter: in.LatestChapter, LatestChapterNum: latestNum,
Status: store.StatusArchived, Kind: store.KindManga,
}
// Cover is deliberately absent above: the client's cover is discarded, and
// this wiring acquires none, so the field is present and empty (ADR-0007).
if stored.Title != want.Title || stored.SeriesURL != want.SeriesURL || stored.Cover != want.Cover ||
stored.LastChapter != want.LastChapter || stored.LastChapterNum != want.LastChapterNum ||
stored.LastChapterURL != want.LastChapterURL || stored.Favorite != want.Favorite ||
stored.LatestChapter != want.LatestChapter ||
stored.LatestChapterNum == nil || *stored.LatestChapterNum != *want.LatestChapterNum ||
stored.Status != want.Status || stored.Kind != want.Kind {
t.Fatalf("PUT response = %+v, want %+v", stored, want)
}
if stored.UpdatedAt == 0 {
t.Fatal("updated_at not server-stamped")
}
// GET reports the same flat shape.
list := getBookmarks(t, srv)
if len(list) != 1 {
t.Fatalf("list = %d items, want 1", len(list))
}
body2, _ := json.Marshal(list[0])
checkFlat(t, body2)
}
// A PUT naming an existing series must ignore client-supplied title, cover and
// URL — the security boundary from ADR-0003, where a hostile site's scraped
// values could otherwise land on a shared row — while progress still lands.
func TestPutExistingSeriesIgnoresClientTitleCoverURL(t *testing.T) {
srv := newTestServer(t)
key := "asura:solo"
first := putBookmark(t, srv, key, store.Bookmark{
Title: "Solo Leveling",
SeriesURL: "https://asurascans.com/comics/solo",
Cover: "https://asurascans.com/covers/solo.jpg",
LastChapterNum: 10,
})
second := putBookmark(t, srv, key, store.Bookmark{
Title: "Scraped Rename",
SeriesURL: "https://evil.example/solo",
Cover: "https://evil.example/solo.jpg",
LastChapterNum: 11,
})
if second.Title != first.Title || second.SeriesURL != first.SeriesURL || second.Cover != first.Cover {
t.Fatalf("stored = %+v, want original title/url/cover kept", second)
}
if second.LastChapterNum != 11 {
t.Fatalf("LastChapterNum = %v, want 11 — progress must still land", second.LastChapterNum)
}
}
// putBookmark PUTs b at key and returns the bookmark the server echoes back,
// which is the row as actually stored (not the request payload).
func putBookmark(t *testing.T, srv http.Handler, key string, b store.Bookmark) store.Bookmark {
t.Helper()
body, _ := json.Marshal(b)
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body))))
if rr.Code != http.StatusOK {
t.Fatalf("PUT %s status = %d, body = %s", key, rr.Code, rr.Body.String())
}
var out store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &out); err != nil {
t.Fatalf("decode PUT response: %v", err)
}
return out
}
func getBookmarks(t *testing.T, srv http.Handler) []store.Bookmark {
t.Helper()
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodGet, "/bookmarks", nil)))
if rr.Code != http.StatusOK {
t.Fatalf("GET status = %d", rr.Code)
}
var list []store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &list); err != nil {
t.Fatalf("decode list: %v", err)
}
return list
}
// updated_at drives list ordering, so it must move only on a real progress
// advance — never on a favorite toggle or a latest-chapter capture.
func TestUpsertConditionalUpdatedAt(t *testing.T) {
cases := []struct {
name string
mutate func(store.Bookmark) store.Bookmark
wantBumped bool
}{
{
name: "unchanged progress",
mutate: func(b store.Bookmark) store.Bookmark { return b },
wantBumped: false,
},
{
name: "changed progress",
mutate: func(b store.Bookmark) store.Bookmark {
b.LastChapter, b.LastChapterNum = "Chapter 11", 11
return b
},
wantBumped: true,
},
{
name: "favorite only",
mutate: func(b store.Bookmark) store.Bookmark {
b.Favorite = true
return b
},
wantBumped: false,
},
{
name: "latest chapter only",
mutate: func(b store.Bookmark) store.Bookmark {
b.LatestChapter, b.LatestChapterNum = "Chapter 15", floatPtr(15)
return b
},
wantBumped: false,
},
{
name: "unrelated metadata only",
mutate: func(b store.Bookmark) store.Bookmark {
b.Title, b.Cover = "Renamed", "https://example.test/new.jpg"
return b
},
wantBumped: false,
},
}
for i, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
srv := newTestServer(t)
key := fmt.Sprintf("asura:cond-%d", i)
first := putBookmark(t, srv, key, store.Bookmark{
Title: "Test",
LastChapter: "Chapter 10",
LastChapterNum: 10,
})
if first.UpdatedAt == 0 {
t.Fatal("new bookmark did not get updated_at set")
}
// Guarantee a later wall-clock ms so a real bump is observable.
time.Sleep(2 * time.Millisecond)
second := putBookmark(t, srv, key, tc.mutate(first))
if tc.wantBumped && second.UpdatedAt <= first.UpdatedAt {
t.Fatalf("updated_at = %d, want > %d", second.UpdatedAt, first.UpdatedAt)
}
if !tc.wantBumped && second.UpdatedAt != first.UpdatedAt {
t.Fatalf("updated_at = %d, want preserved %d", second.UpdatedAt, first.UpdatedAt)
}
// The PUT response must match what a subsequent GET reports.
list := getBookmarks(t, srv)
if len(list) != 1 {
t.Fatalf("list = %+v, want 1 item", list)
}
if list[0].UpdatedAt != second.UpdatedAt {
t.Fatalf("GET updated_at = %d, PUT echoed %d", list[0].UpdatedAt, second.UpdatedAt)
}
})
}
}
func TestFavoriteRoundTrip(t *testing.T) {
srv := newTestServer(t)
key := "demonic:some-series"
stored := putBookmark(t, srv, key, store.Bookmark{Title: "Fav", Favorite: true})
if !stored.Favorite {
t.Fatalf("PUT response favorite = false, want true")
}
list := getBookmarks(t, srv)
if len(list) != 1 || !list[0].Favorite {
t.Fatalf("favorite did not round-trip: %+v", list)
}
// Unfavoriting must persist too (guards against a write that only ever ORs in true).
stored = putBookmark(t, srv, key, store.Bookmark{Title: "Fav", Favorite: false})
if stored.Favorite {
t.Fatal("PUT response favorite = true after unfavorite")
}
list = getBookmarks(t, srv)
if len(list) != 1 || list[0].Favorite {
t.Fatalf("unfavorite did not round-trip: %+v", list)
}
}
func TestLatestChapterNullable(t *testing.T) {
srv := newTestServer(t)
key := "asura:latest-test"
// Never captured: latest_chapter_num must serialize as JSON null.
body, _ := json.Marshal(store.Bookmark{Title: "No latest yet"})
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, "/bookmarks/"+key, bytes.NewReader(body))))
if rr.Code != http.StatusOK {
t.Fatalf("PUT status = %d", rr.Code)
}
if !strings.Contains(rr.Body.String(), `"latest_chapter_num":null`) {
t.Fatalf("want latest_chapter_num null in response, got %s", rr.Body.String())
}
list := getBookmarks(t, srv)
if len(list) != 1 || list[0].LatestChapterNum != nil {
t.Fatalf("latest_chapter_num = %v, want nil", list[0].LatestChapterNum)
}
// Once captured it round-trips as a value.
stored := putBookmark(t, srv, key, store.Bookmark{
Title: "No latest yet",
LatestChapter: "Chapter 162",
LatestChapterNum: floatPtr(162),
})
if stored.LatestChapterNum == nil || *stored.LatestChapterNum != 162 {
t.Fatalf("PUT response latest_chapter_num = %v, want 162", stored.LatestChapterNum)
}
list = getBookmarks(t, srv)
if len(list) != 1 || list[0].LatestChapterNum == nil || *list[0].LatestChapterNum != 162 {
t.Fatalf("latest chapter did not round-trip: %+v", list)
}
if list[0].LatestChapter != "Chapter 162" {
t.Fatalf("latest_chapter = %q, want %q", list[0].LatestChapter, "Chapter 162")
}
}
func TestLoadConfigDiscord(t *testing.T) {
t.Setenv("DISCORD_CLIENT_ID", "client-1")
t.Setenv("DISCORD_CLIENT_SECRET", "client-secret-1")
t.Setenv("DISCORD_GUILD_ID", "guild-1")
t.Setenv("DISCORD_REQUIRED_ROLE", "role-9")
t.Setenv("DISCORD_REDIRECT_URI", "https://bm.example.com/auth/discord/callback")
t.Setenv("DISCORD_API_BASE", "https://stub.example/api")
if got := loadConfig().Discord; got.ClientID != "client-1" || got.ClientSecret != "client-secret-1" ||
got.GuildID != "guild-1" || got.RequiredRole != "role-9" ||
got.RedirectURI != "https://bm.example.com/auth/discord/callback" ||
got.APIBase != "https://stub.example/api" {
t.Fatalf("Discord config = %+v, want every field set", got)
}
// API base falls back to the Discord default; the role is optional.
t.Setenv("DISCORD_REQUIRED_ROLE", "")
t.Setenv("DISCORD_API_BASE", "")
got := loadConfig().Discord
if got.RequiredRole != "" {
t.Fatalf("RequiredRole = %q, want empty by default", got.RequiredRole)
}
if got.APIBase != "https://discord.com/api/v10" {
t.Fatalf("APIBase = %q, want the Discord default", got.APIBase)
}
}
// A userscript PUT body has no latest_checked_at field. If the column is ever
// moved into bookmarkColumns, this test catches it: the PUT would reset the
// cooldown and the poller would re-fetch that series on every single tick.
func TestPutDoesNotClobberLatestCheckedAt(t *testing.T) {
s := newTestStore(t)
srv := newRouter(s, testConfig())
seedForCheck(t, s, "asura:x", "https://asurascans.com/comics/x", 777)
// Exactly what the userscript sends: no latest_checked_at key at all.
body := `{"key":"asura:x","site":"asura","series_id":"x",
"series_url":"https://asurascans.com/comics/x",
"last_chapter":"Chapter 5","last_chapter_num":5}`
req := httptest.NewRequest(http.MethodPut, "/bookmarks/asura:x", strings.NewReader(body))
req.Header.Set("Authorization", "Bearer "+ownerCredential())
req.Header.Set("Content-Type", "application/json")
rec := httptest.NewRecorder()
srv.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("PUT status = %d, want 200 (body %s)", rec.Code, rec.Body.String())
}
if got := readLatestCheckedAt(t, s, "asura:x"); got != 777 {
t.Fatalf("latest_checked_at = %d after client PUT, want 777 preserved", got)
}
}
// The userscript route is registered outside the web UI's Discord auth, so it
// must keep working whatever the web config — see internal/userscript for the
// handler's own behaviour. The credential in the path is the owner's derived
// one, and the served script carries it substituted in.
func TestUserscriptServedWithWebUIDisabled(t *testing.T) {
path := filepath.Join(t.TempDir(), "manga-bookmark.user.js")
if err := os.WriteFile(path, []byte("const API_TOKEN = \"__API_TOKEN__\";\n"), 0o644); err != nil {
t.Fatalf("write script: %v", err)
}
s := newTestStore(t)
cfg := testConfig() // no Discord config needed for the userscript route
cfg.UserscriptPath = path
rr := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/u/"+ownerCredential()+"/manga-bookmark.user.js", nil)
newRouter(s, cfg).ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rr.Code)
}
if got := rr.Body.String(); !strings.Contains(got, `API_TOKEN = "`+ownerCredential()+`"`) {
t.Fatalf("served script does not carry the requesting Reader's credential:\n%s", got)
}
}
// Both scripts are served from the same handler, outside the web UI's auth —
// a wrong credential is a 404, never a 401.
func TestNovelUserscriptServed(t *testing.T) {
dir := t.TempDir()
novelPath := filepath.Join(dir, "novel-bookmark.user.js")
if err := os.WriteFile(novelPath, []byte("// novel\n"), 0o644); err != nil {
t.Fatalf("write script: %v", err)
}
s := newTestStore(t)
cfg := testConfig()
cfg.NovelUserscriptPath = novelPath
srv := newRouter(s, cfg)
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet,
"/u/"+ownerCredential()+"/novel-bookmark.user.js", nil))
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rr.Code)
}
if ct := rr.Header().Get("Content-Type"); !strings.HasPrefix(ct, "text/javascript") {
t.Fatalf("Content-Type = %q, want text/javascript", ct)
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet,
"/u/wrong-token/novel-bookmark.user.js", nil))
if rr.Code != http.StatusNotFound {
t.Fatalf("wrong token status = %d, want 404", rr.Code)
}
}
+134
View File
@@ -0,0 +1,134 @@
package main
import (
"database/sql"
"net/http"
"net/http/httptest"
"strings"
"testing"
"bookmarkmanager/backend/internal/store"
)
func getCover(t *testing.T, srv http.Handler, path string, cookie *http.Cookie) *httptest.ResponseRecorder {
t.Helper()
req := httptest.NewRequest(http.MethodGet, path, nil)
if cookie != nil {
req.AddCookie(cookie)
}
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
return rr
}
// The acquired Cover is served from this deployment's own origin, to any
// browser rendering a third-party page — no session, no credential (ADR-0007).
func TestPublicCoverServesStoredBytesUnauthenticated(t *testing.T) {
const sourceURL = "https://cdn.asurascans.com/covers/solo.webp"
srv, st := newWebTestServer(t, testConfig())
if err := st.PutCover(sourceURL, []byte("\x00webp-bytes"), "image/webp"); err != nil {
t.Fatalf("PutCover: %v", err)
}
// The wire URL is what a client actually requests, so the path under test
// is taken from it rather than rebuilt by hand.
wire := st.CoverWireURL(store.CoverAddress(sourceURL))
path, ok := strings.CutPrefix(wire, testCoverBaseURL)
if !ok {
t.Fatalf("wire URL %q is not on the public origin %q", wire, testCoverBaseURL)
}
rr := getCover(t, srv, path, nil)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 without any credential", rr.Code)
}
if got := rr.Body.String(); got != "\x00webp-bytes" {
t.Fatalf("body = %q, want the stored bytes", got)
}
if got := rr.Header().Get("Content-Type"); got != "image/webp" {
t.Fatalf("Content-Type = %q, want the stored one", got)
}
// Content-addressed bytes never change, so a client that has them must
// never need to ask again.
if got := rr.Header().Get("Cache-Control"); !strings.Contains(got, "immutable") {
t.Fatalf("Cache-Control = %q, want an immutable cache directive", got)
}
}
func TestPublicCoverRejectsUnknownAddress(t *testing.T) {
srv, _ := newWebTestServer(t, testConfig())
cases := map[string]string{
"unknown": "/covers/" + store.CoverAddress("https://cdn.example/never-stored.jpg"),
"malformed": "/covers/not-an-address",
"traversal": "/covers/../../etc/passwd",
"empty": "/covers/",
}
for name, path := range cases {
t.Run(name, func(t *testing.T) {
if rr := getCover(t, srv, path, nil); rr.Code == http.StatusOK {
t.Fatalf("%s: status = 200, want anything but a served body", path)
}
})
}
}
// A content type outside the image set is never echoed back. The old kagane
// proxy could fetch text/html from a challenged fetch and had to refuse it;
// the general route's only input is the store, and the store refuses to
// record anything that is not an image — but the guarantee is pinned at the
// serving boundary, not the write gate, so a poisoned row (migrated data, a
// writer that skips the gate) is also never served.
func TestPublicCoverNeverEchoesNonImage(t *testing.T) {
const sourceURL = "https://cdn.example/cover"
st, dsn := newTestStoreURL(t)
// The write gate refuses non-image content types outright.
if err := st.PutCover(sourceURL, []byte("<script>"), "text/html"); err == nil {
t.Fatal("PutCover accepted a non-image content type")
}
// A legitimate row, then the content type flipped behind the store's back:
// the bytes exist at the address, so only the type is hostile.
address := store.CoverAddress(sourceURL)
if err := st.SetSeriesCover("asura", "solo", sourceURL, []byte("<script>"), "image/png"); err != nil {
t.Fatalf("seed row: %v", err)
}
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("open %s: %v", dsn, err)
}
defer db.Close()
if _, err := db.Exec(`UPDATE covers SET content_type = 'text/html' WHERE address = $1`, address); err != nil {
t.Fatalf("poison row: %v", err)
}
rr := getCover(t, newRouter(st, testConfig()), "/covers/"+address, nil)
if rr.Code == http.StatusOK {
t.Fatalf("status = 200, want a refusal for a non-image row (body %q)", rr.Body.String())
}
}
// The whole point of acquiring bytes is that the UI shows them: the card's
// <img> must carry the public address, not a third-party URL and not a
// placeholder.
func TestListRendersAcquiredCover(t *testing.T) {
const sourceURL = "https://static.comix.to/039d/i/1/34/6a6742bf15736@280.jpg"
srv, st := newWebTestServer(t, testConfig())
if _, err := st.Upsert(st.OwnerID(), store.Bookmark{
Key: "comix:n8we", Site: "comix", SeriesID: "n8we", Title: "Dungeons and Crayons",
SeriesURL: "https://comix.to/title/n8we", UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
if err := st.SetSeriesCover("comix", "n8we", sourceURL, []byte("\xff\xd8jpeg"), "image/jpeg"); err != nil {
t.Fatalf("SetSeriesCover: %v", err)
}
req := httptest.NewRequest(http.MethodGet, "/ui/list", nil)
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rr.Code)
}
want := `src="` + testCoverBaseURL + "/covers/" + store.CoverAddress(sourceURL) + `"`
if !strings.Contains(rr.Body.String(), want) {
t.Fatalf("rendered list does not contain %s", want)
}
}
+15 -16
View File
@@ -1,11 +1,13 @@
module mangabm/backend
module bookmarkmanager/backend
go 1.24.1
go 1.26
require (
github.com/bogdanfinn/fhttp v0.6.8
github.com/bogdanfinn/tls-client v1.15.1
modernc.org/sqlite v1.34.4
github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f
github.com/chromedp/chromedp v0.16.0
github.com/jackc/pgx/v5 v5.10.0
)
require (
@@ -15,23 +17,20 @@ require (
github.com/bogdanfinn/quic-go-utls v1.0.9-utls // indirect
github.com/bogdanfinn/utls v1.7.7-barnius // indirect
github.com/bogdanfinn/websocket v1.5.5-barnius // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/chromedp/sysutil v1.1.0 // indirect
github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68 // indirect
github.com/gobwas/httphead v0.1.0 // indirect
github.com/gobwas/pool v0.2.1 // indirect
github.com/gobwas/ws v1.4.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/klauspost/compress v1.18.2 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/tam7t/hpkp v0.0.0-20160821193359-2b70b4024ed5 // indirect
golang.org/x/crypto v0.46.0 // indirect
golang.org/x/net v0.48.0 // indirect
golang.org/x/sys v0.39.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.32.0 // indirect
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
modernc.org/libc v1.55.3 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/strutil v1.2.0 // indirect
modernc.org/token v1.1.0 // indirect
)
+34 -46
View File
@@ -14,28 +14,44 @@ github.com/bogdanfinn/utls v1.7.7-barnius h1:OuJ497cc7F3yKNVHRsYPQdGggmk5x6+V5Zl
github.com/bogdanfinn/utls v1.7.7-barnius/go.mod h1:aAK1VZQlpKZClF1WEQeq6kyclbkPq4hz6xTbB5xSlmg=
github.com/bogdanfinn/websocket v1.5.5-barnius h1:bY+qnxpai1qe7Jmjx+Sds/cmOSpuuLoR8x61rWltjOI=
github.com/bogdanfinn/websocket v1.5.5-barnius/go.mod h1:gvvEw6pTKHb7yOiFvIfAFTStQWyrm25BMVCTj5wRSsI=
github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f h1:0Z1zcSLEmnj2c2CmJYBqewtS6pxhB39bNWUSEUAWjgk=
github.com/chromedp/cdproto v0.0.0-20260714215040-dc233986426f/go.mod h1:RwFsSODCtFExll+GhHM6R92SARHR3Z3oipaxLHj46C0=
github.com/chromedp/chromedp v0.16.0 h1:rOO4deOm4CbZgBCa8mD9g2rDyIoNs0BkgvNrlbp5ouk=
github.com/chromedp/chromedp v0.16.0/go.mod h1:rbuGKFT1vMcFcFqKfPIO1GpX/N+2s8onm2qMxZLbU5U=
github.com/chromedp/sysutil v1.1.0 h1:PUFNv5EcprjqXZD9nJb9b/c9ibAbxiYo4exNWZyipwM=
github.com/chromedp/sysutil v1.1.0/go.mod h1:WiThHUdltqCNKGc4gaU50XgYjwjYIhKWoHGPTUfWTJ8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68 h1:KZaTBSyshWX3MP5jukJcNSuXDQTO+rNpt0J564dX/eg=
github.com/go-json-experiment/json v0.0.0-20260623181947-01eb4420fa68/go.mod h1:tphK2c80bpPhMOI4v6bIc2xWywPfbqi1Z06+RcrMkDg=
github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU=
github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og=
github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw=
github.com/gobwas/ws v1.4.0 h1:CTaoG1tojrh4ucGPcoJFiAQUAsEWekEWvLy7GsVNqGs=
github.com/gobwas/ws v1.4.0/go.mod h1:G3gNqMNtPppf5XUz7O4shetPpcZ1VJ7zt18dlUeakrc=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80 h1:6Yzfa6GP0rIo/kULo2bwGEkFvCePZ3qHDDTC3/J9Swo=
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/tam7t/hpkp v0.0.0-20160821193359-2b70b4024ed5 h1:YqAladjX7xpA6BM04leXMWAEjS0mTZ5kUU9KRBriQJc=
@@ -46,8 +62,6 @@ go.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=
go.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=
golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU=
golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0=
golang.org/x/mod v0.30.0 h1:fDEXFVZ/fmCKProc/yAXXUijritrDzahmwwefnjoPFk=
golang.org/x/mod v0.30.0/go.mod h1:lAsf5O2EvJeSFMiBxXDki7sCgAxEUcZHXoXMKT4GJKc=
golang.org/x/net v0.0.0-20211104170005-ce137452f963/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU=
golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY=
@@ -56,40 +70,14 @@ golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk=
golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU=
golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.39.0 h1:ik4ho21kwuQln40uelmciQPp9SipgNDdrafrYA4TmQQ=
golang.org/x/tools v0.39.0/go.mod h1:JnefbkDPyD8UU2kI5fuf8ZX4/yUeh9W877ZeBONxUqQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ=
modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ=
modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y=
modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s=
modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE=
modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ=
modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw=
modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4=
modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U=
modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w=
modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E=
modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc=
modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss=
modernc.org/sqlite v1.34.4 h1:sjdARozcL5KJBvYQvLlZEmctRgW9xqIZc2ncN7PU0P8=
modernc.org/sqlite v1.34.4/go.mod h1:3QQFCG2SEMtc2nv+Wq4cQCH7Hjcg+p/RMlS1XK+zwbk=
modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA=
modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
-112
View File
@@ -1,112 +0,0 @@
package main
import (
"encoding/json"
"log"
"net/http"
"strings"
"time"
)
type bookmarkHandler struct {
store *Store
}
func writeJSON(w http.ResponseWriter, status int, v any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
if v != nil {
if err := json.NewEncoder(w).Encode(v); err != nil {
log.Printf("encode response: %v", err)
}
}
}
// list returns all bookmarks. GET /bookmarks
func (h *bookmarkHandler) list(w http.ResponseWriter, r *http.Request) {
items, err := h.store.List()
if err != nil {
log.Printf("list: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
writeJSON(w, http.StatusOK, items)
}
// put upserts one bookmark. PUT /bookmarks/{key}
func (h *bookmarkHandler) put(w http.ResponseWriter, r *http.Request) {
key := r.PathValue("key")
if key == "" {
http.Error(w, "missing key", http.StatusBadRequest)
return
}
var b Bookmark
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<16)).Decode(&b); err != nil {
http.Error(w, "invalid JSON body", http.StatusBadRequest)
return
}
// Path key is authoritative; derive site/series_id from it when the body
// omits them so the stored row is always self-consistent.
b.Key = key
if b.Site == "" || b.SeriesID == "" {
if site, series, ok := strings.Cut(key, ":"); ok {
if b.Site == "" {
b.Site = site
}
if b.SeriesID == "" {
b.SeriesID = series
}
}
}
// An empty status is "no opinion" and Upsert keeps the stored bucket.
// Finishing a series is a web-UI decision, so the JSON API refuses it
// rather than trusting every client to leave it alone.
switch b.Status {
case "", statusReading, statusArchived:
case statusFinished:
http.Error(w, "status "+statusFinished+" can only be set from the web UI",
http.StatusBadRequest)
return
default:
http.Error(w, "invalid status", http.StatusBadRequest)
return
}
// Candidate timestamp, not a decision: Upsert keeps the stored one unless
// reading progress actually moved. Any client value is ignored.
b.UpdatedAt = time.Now().UnixMilli()
stored, err := h.store.Upsert(b)
if err != nil {
log.Printf("upsert: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
// Echo the stored row: clients adopt this as their cached copy, so it must
// carry the authoritative updated_at rather than the candidate above.
writeJSON(w, http.StatusOK, stored)
}
// delete removes one bookmark. DELETE /bookmarks/{key}
func (h *bookmarkHandler) delete(w http.ResponseWriter, r *http.Request) {
key := r.PathValue("key")
if key == "" {
http.Error(w, "missing key", http.StatusBadRequest)
return
}
if err := h.store.Delete(key); err != nil {
log.Printf("delete: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusNoContent)
}
func healthz(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("ok"))
}
+166
View File
@@ -0,0 +1,166 @@
package api
import (
"encoding/json"
"log"
"net/http"
"strings"
"time"
"bookmarkmanager/backend/internal/httpmw"
"bookmarkmanager/backend/internal/store"
)
// Handler serves the userscript-facing JSON bookmark API.
type Handler struct {
Store *store.Store
}
func writeJSON(w http.ResponseWriter, status int, v any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
if v != nil {
if err := json.NewEncoder(w).Encode(v); err != nil {
log.Printf("encode response: %v", err)
}
}
}
// List returns all bookmarks of the acting Reader. GET /bookmarks
func (h *Handler) List(w http.ResponseWriter, r *http.Request) {
items, err := h.Store.List(httpmw.ReaderID(r))
if err != nil {
log.Printf("list: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
writeJSON(w, http.StatusOK, items)
}
// Put upserts one bookmark. PUT /bookmarks/{key}
func (h *Handler) Put(w http.ResponseWriter, r *http.Request) {
key := r.PathValue("key")
if key == "" {
http.Error(w, "missing key", http.StatusBadRequest)
return
}
var b store.Bookmark
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<16)).Decode(&b); err != nil {
http.Error(w, "invalid JSON body", http.StatusBadRequest)
return
}
// A body may carry a cover, and it is discarded here rather than
// rejected: an older installed userscript may still send one, and
// ADR-0004's compatibility argument depends on those scripts continuing
// to work. The Cover is acquired server-side (ADR-0007), so the field is
// permanently inert - not pending removal, and not a value any later code
// should start reading.
b.Cover = ""
// Path key is authoritative; derive site/series_id from it when the body
// omits them so the stored row is always self-consistent.
b.Key = key
if b.Site == "" || b.SeriesID == "" {
if site, series, ok := strings.Cut(key, ":"); ok {
if b.Site == "" {
b.Site = site
}
if b.SeriesID == "" {
b.SeriesID = series
}
}
}
// An empty status is "no opinion" and Upsert keeps the stored bucket.
// Finishing a series is a web-UI decision, so the JSON API refuses it
// rather than trusting every client to leave it alone.
switch b.Status {
case "", store.StatusReading, store.StatusArchived:
case store.StatusFinished:
http.Error(w, "status "+store.StatusFinished+" can only be set from the web UI",
http.StatusBadRequest)
return
default:
http.Error(w, "invalid status", http.StatusBadRequest)
return
}
// Same rule as status: empty means "keep the stored value". An unknown
// value is a client bug, not something to silently coerce to manga.
switch b.Kind {
case "", store.KindManga, store.KindNovel:
default:
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid kind"})
return
}
// Candidate timestamp, not a decision: Upsert keeps the stored one unless
// reading progress actually moved. Any client value is ignored.
b.UpdatedAt = time.Now().UnixMilli()
stored, err := h.Store.Upsert(httpmw.ReaderID(r), b)
if err != nil {
log.Printf("upsert: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
// Echo the stored row: clients adopt this as their cached copy, so it must
// carry the authoritative updated_at rather than the candidate above.
writeJSON(w, http.StatusOK, stored)
}
// Delete removes one bookmark. DELETE /bookmarks/{key}
func (h *Handler) Delete(w http.ResponseWriter, r *http.Request) {
key := r.PathValue("key")
if key == "" {
http.Error(w, "missing key", http.StatusBadRequest)
return
}
if err := h.Store.Delete(httpmw.ReaderID(r), key); err != nil {
log.Printf("delete: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusNoContent)
}
// Healthz answers the unauthenticated liveness check. GET /healthz
func Healthz(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("ok"))
}
// Cover serves stored cover bytes. GET /covers/{address}
//
// Public on purpose: the userscript renders these on Sites the deployment
// does not control, where no credential of ours may be sent, and the address
// is the SHA-256 of a URL the Site already publishes (ADR-0007). An unknown
// address is a 404 rather than an error - "no Cover yet" is a normal state,
// and the clients fall back to their placeholder.
func (h *Handler) Cover(w http.ResponseWriter, r *http.Request) {
body, contentType, ok, err := h.Store.CoverByAddress(r.PathValue("address"))
if err != nil {
log.Printf("cover: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if !ok {
http.NotFound(w, r)
return
}
// Refuse anything the write gate would not have recorded: a poisoned row
// (migrated data, a writer that skips the gate) must never be echoed back
// as bytes of a type no Cover may have.
if _, ok := store.CoverContentType(contentType); !ok {
log.Printf("cover %s: refusing non-image content type %q", r.PathValue("address"), contentType)
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", contentType)
// Content-addressed, so the bytes at this URL can never change. Public
// rather than private: no credential gates the route.
w.Header().Set("Cache-Control", "public, max-age=604800, immutable")
_, _ = w.Write(body)
}
+147
View File
@@ -0,0 +1,147 @@
package httpmw
import (
"compress/gzip"
"context"
"log"
"net/http"
"strings"
"bookmarkmanager/backend/internal/store"
"bookmarkmanager/backend/internal/token"
)
const bearerPrefix = "Bearer "
type ctxKey int
// readerCtxKey is where Auth stashes the authenticated Reader id.
const readerCtxKey ctxKey = iota
// ReaderID returns the Reader id Auth authenticated, for handlers that take
// the acting Reader from the request rather than from a fixed field.
func ReaderID(r *http.Request) int64 { return r.Context().Value(readerCtxKey).(int64) }
// ResolveReader maps a presented credential to a Reader. The credential is
// hashed and matched against readers.token_sha256 — an equality on 32-byte
// values, never a comparison of the credential itself. The same resolution
// backs the API bearer header and the userscript download path, so a Reader
// has exactly one credential with one blast radius.
func ResolveReader(s *store.Store, cred string) (int64, bool) {
readerID, ok, err := s.ReaderIDForTokenHash(token.Hash(cred))
if err != nil {
log.Printf("auth: reader lookup: %v", err)
return 0, false
}
return readerID, ok
}
// Auth guards a handler with a per-Reader bearer credential. The acting
// Reader travels in the request context, so a handler scopes every store call
// to exactly the Reader that authenticated.
func Auth(s *store.Store, next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
h := r.Header.Get("Authorization")
if !strings.HasPrefix(h, bearerPrefix) {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
readerID, ok := ResolveReader(s, strings.TrimPrefix(h, bearerPrefix))
if !ok {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), readerCtxKey, readerID)))
})
}
// gzipWriter compresses the body and drops Content-Length, which no longer
// describes what goes on the wire. WriteHeader is where the decision is made:
// only then is Content-Type known.
type gzipWriter struct {
http.ResponseWriter
gz *gzip.Writer
decided bool
}
// compressible covers what this server actually serves in bulk: HTML, CSS, JS
// and JSON. Fonts are woff2, which is already compressed — gzipping them costs
// CPU to add bytes.
func compressible(contentType string) bool {
ct, _, _ := strings.Cut(contentType, ";")
switch strings.TrimSpace(ct) {
case "text/html", "text/css", "text/javascript", "application/javascript",
"application/json", "text/plain":
return true
}
return false
}
func (w *gzipWriter) WriteHeader(status int) {
if !w.decided {
w.decided = true
if compressible(w.Header().Get("Content-Type")) {
w.Header().Set("Content-Encoding", "gzip")
w.Header().Del("Content-Length")
w.gz = gzip.NewWriter(w.ResponseWriter)
}
}
w.ResponseWriter.WriteHeader(status)
}
func (w *gzipWriter) Write(b []byte) (int, error) {
if !w.decided {
w.WriteHeader(http.StatusOK)
}
if w.gz != nil {
return w.gz.Write(b)
}
return w.ResponseWriter.Write(b)
}
// Gzip compresses text responses for clients that ask. The templates,
// stylesheet and htmx together are ~120 KB uncompressed and roughly a quarter
// of that gzipped, which is the difference between a fast and a slow first load
// on mobile data.
func Gzip(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
next.ServeHTTP(w, r)
return
}
w.Header().Add("Vary", "Accept-Encoding")
gw := &gzipWriter{ResponseWriter: w}
defer func() {
if gw.gz != nil {
gw.gz.Close()
}
}()
next.ServeHTTP(gw, r)
})
}
// CORS reflects the request Origin only when it is in allowed, answers
// preflight OPTIONS with 204, and passes everything else through. It wraps the
// auth middleware so preflight (which carries no Authorization header) is never
// rejected by auth.
func CORS(allowed []string, next http.Handler) http.Handler {
set := make(map[string]struct{}, len(allowed))
for _, o := range allowed {
set[o] = struct{}{}
}
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
origin := r.Header.Get("Origin")
if _, ok := set[origin]; ok && origin != "" {
w.Header().Set("Access-Control-Allow-Origin", origin)
w.Header().Add("Vary", "Origin")
w.Header().Set("Access-Control-Allow-Methods", "GET,PUT,DELETE,OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Authorization,Content-Type")
w.Header().Set("Access-Control-Max-Age", "86400")
}
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
next.ServeHTTP(w, r)
})
}
+146
View File
@@ -0,0 +1,146 @@
package latest
import (
"context"
"errors"
"log"
"sync"
"time"
"bookmarkmanager/backend/internal/store"
)
// acquireTimeout bounds one creation-time acquisition end to end: the series
// page plus the cover bytes. Nothing is waiting on it — the Reader's write has
// already returned — so this only stops a stalled Site from holding a
// goroutine and a connection open forever.
const acquireTimeout = 45 * time.Second
// Acquirer gives a Series its Latest Chapter and its Cover the moment the
// first Bookmark creates it, instead of leaving the Reader to wait out the
// poll queue — which is ordered by Reader count, so a Series with one Reader
// sits behind every popular one (ADR-0007).
//
// Both facts come from a single series-page fetch, which is also why no
// client-supplied cover hint is worth accepting: the page has to be fetched
// for the chapter signal regardless, so a hint would save no request while
// adding a client-controlled input to a server-side fetch.
//
// Every failure path is "log and move on". The Bookmark, its progress and its
// Latest Chapter are already committed; a Site that is down or a Cover that
// cannot be produced must not disturb any of them, and the Series is simply
// left blank until the poll's own cover pass (#61) fills it.
type Acquirer struct {
Store *store.Store
// Fetch retrieves the series page over plain TLS. Nil with a nil
// BrowserFetch disables acquisition entirely.
Fetch Fetcher
// BrowserFetch retrieves kagane and novelfull pages through the browser
// sidecar, the only thing that clears their Cloudflare challenge. The
// per-site fallback policy lives in fetcherFor. Nil leaves those Sites
// unacquired when no fallback applies.
BrowserFetch Fetcher
// Covers retrieves the cover bytes. Nil leaves the Cover blank and the
// chapter half working.
Covers CoverBytesFetcher
// BrowserCoverFetch retrieves browser-claimed cover bytes through the
// sidecar. Nil leaves those Covers blank; nothing falls back to a plain
// fetch, which would only ever retrieve a challenge page.
BrowserCoverFetch BrowserCoverFetcher
// Ctx cancels in-flight acquisitions at shutdown. A hook signature has
// nowhere to pass one, so it lives here; nil means context.Background.
Ctx context.Context
inflight sync.WaitGroup
}
// acquireSlots caps how many creation-time fetches run at once. A Reader whose
// userscript bulk-syncs creates many Series at once, and a burst of
// simultaneous requests from one server IP is the traffic shape most likely to
// move that IP's bot score — the same reason the poller staggers its batch.
var acquireSlots = make(chan struct{}, 2)
// Acquire starts one acquisition and returns immediately: a Reader's bookmark
// action may not block on a third-party Site's latency, nor fail with it. It
// is the store's OnSeriesCreated hook, so it only ever runs for a Series no
// Reader had bookmarked before.
func (a *Acquirer) Acquire(sr store.Series) {
a.inflight.Add(1)
go func() {
defer a.inflight.Done()
defer func() {
if r := recover(); r != nil {
log.Printf("acquire %q: recovered from panic: %v", sr.Key(), r)
}
}()
parent := a.Ctx
if parent == nil {
parent = context.Background()
}
select {
case acquireSlots <- struct{}{}:
defer func() { <-acquireSlots }()
case <-parent.Done():
return
}
ctx, cancel := context.WithTimeout(parent, acquireTimeout)
defer cancel()
a.acquire(ctx, sr)
}()
}
// Wait blocks until every started acquisition has finished. It exists for
// tests: an asynchronous side effect is otherwise unobservable without
// polling for it.
func (a *Acquirer) Wait() { a.inflight.Wait() }
func (a *Acquirer) acquire(ctx context.Context, sr store.Series) {
if a.Fetch == nil && a.BrowserFetch == nil {
return
}
facts, err := readSeriesPage(ctx, sr.Site, sr.SeriesURL, a.BrowserFetch, a.Fetch)
if err != nil {
switch {
case errors.Is(err, errNotFetchable):
// series_url arrives in a client-supplied PUT body, so without the
// gate a token-holder chooses what the server fetches from its own
// network position.
log.Printf("acquire %q: not fetchable: site=%q url=%q", sr.Key(), sr.Site, sr.SeriesURL)
case errors.Is(err, errNoFetcher):
log.Printf("acquire %q: no fetcher for site %q", sr.Key(), sr.Site)
default:
log.Printf("acquire %q: %v", sr.Key(), err)
}
return
}
// This page just served the same purpose a poll tick would have; without
// the stamp the row stays due and the poller refetches it immediately.
//
// Stamped after success — the reverse of the poller, which stamps before
// the fetch: the Reader is here, watching the Series they just created, so
// a failed acquisition must leave the row due for a fast retry rather than
// consuming the cooldown. The stamp happens even when the page read
// succeeded but produced no facts to persist.
if err := a.Store.MarkLatestChecked(sr.Site, sr.SeriesID, time.Now().UnixMilli()); err != nil {
log.Printf("acquire %q: mark checked: %v", sr.Key(), err)
}
if facts.HasLatest {
if err := a.Store.SetLatestChapter(sr.Site, sr.SeriesID, facts.Latest.Label, facts.Latest.Num); err != nil {
log.Printf("acquire %q: set latest chapter: %v", sr.Key(), err)
}
}
if !facts.HasCover {
return
}
bytes, contentType, err := fetchCoverBytes(ctx, facts.Cover, a.BrowserCoverFetch, a.Covers)
if err != nil {
log.Printf("acquire %q: fetch cover %s: %v", sr.Key(), facts.Cover, err)
return
}
if err := a.Store.SetSeriesCover(sr.Site, sr.SeriesID, facts.Cover, bytes, contentType); err != nil {
log.Printf("acquire %q: persist cover: %v", sr.Key(), err)
}
}
+430
View File
@@ -0,0 +1,430 @@
package latest
import (
"context"
"errors"
"testing"
"time"
"bookmarkmanager/backend/internal/store"
)
// The series page carries both facts, which is the whole argument for taking
// them from one fetch.
const asuraSeriesAndCoverFixture = asuraSeriesFixture + asuraCoverFixture
const (
acquireKey = "asura:chronicles-of-the-demon-faction-f886a8af"
acquireSeriesID = "chronicles-of-the-demon-faction-f886a8af"
acquireSeriesURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
acquireCoverURL = "https://cdn.asurascans.com/asura-images/covers/chronicles-of-the-demon-faction.d4dcb8.webp"
)
const (
kaganeKey = "kagane:019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"
kaganeSeriesID = "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"
kaganeSeriesURL = "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"
kaganeImageID = "019fe11a-84c3-7fc3-a84b-88787374b617"
kaganeCoverSrc = "https://kagane.to/api/v2/image/" + kaganeImageID + "/compressed"
)
// kagane's browser-fetched body is one JSON object carrying both the chapter
// list (series_books) and the cover image ids (series_covers), so the single
// acquisition fetch yields both facts.
const kaganeSeriesAndCoverFixture = `{"series_id":"019f84bc-9ba0-7ed9-86f5-8b905ec7c28b",` +
`"series_books":[{"book_id":"b","title":"Episode 41","chapter_no":"41","sort_no":41}],` +
`"series_covers":[{"cover_id":"019fe11a-84d1-714b-9cf4-2827f277f3c0","language":"en",` +
`"image_id":"019fe11a-84c3-7fc3-a84b-88787374b617"}]}`
const (
novelfullKey = "novelfull:reverend-insanity"
novelfullSeriesID = "reverend-insanity"
novelfullSeriesURI = "https://novelfull.com/reverend-insanity.html"
novelfullCoverURL = "https://novelfull.com/uploads/webp/novel/reverend-insanity-82661d911a.webp"
)
// newAcquirer wires an acquirer onto the store's creation hook, which is how
// main wires it: the write path is what starts an acquisition.
func newAcquirer(s *store.Store, page *fakeFetcher, covers *fakeBytesCoverFetcher) *Acquirer {
a := &Acquirer{Store: s, Fetch: page, Covers: covers}
s.OnSeriesCreated = a.Acquire
return a
}
func bookmarkNewSeries(t *testing.T, s *store.Store, seriesURL string) store.Bookmark {
t.Helper()
stored, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: acquireKey, Site: "asura", SeriesID: acquireSeriesID,
Title: "Chronicles of the Demon Faction", SeriesURL: seriesURL,
Cover: "https://evil.example/client-supplied.jpg", UpdatedAt: 1000,
})
if err != nil {
t.Fatalf("Upsert: %v", err)
}
return stored
}
func readBookmark(t *testing.T, s *store.Store, key string) store.Bookmark {
t.Helper()
b, ok, err := s.Get(s.OwnerID(), key)
if err != nil || !ok {
t.Fatalf("Get %q = %v, %v", key, ok, err)
}
return b
}
func bookmarkNewKaganeSeries(t *testing.T, s *store.Store) store.Bookmark {
t.Helper()
stored, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: kaganeKey, Site: "kagane", SeriesID: kaganeSeriesID,
Title: "Infinite Decryption", SeriesURL: kaganeSeriesURL, UpdatedAt: 1000,
})
if err != nil {
t.Fatalf("Upsert: %v", err)
}
return stored
}
func bookmarkNewNovelfullSeries(t *testing.T, s *store.Store) store.Bookmark {
t.Helper()
stored, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: novelfullKey, Site: "novelfull", SeriesID: novelfullSeriesID,
Title: "Reverend Insanity", SeriesURL: novelfullSeriesURI, UpdatedAt: 1000,
})
if err != nil {
t.Fatalf("Upsert: %v", err)
}
return stored
}
// The reported bug: a Reader bookmarks a Series nobody holds and expects the
// Cover, not a broken image. Both facts come from the one series-page fetch.
func TestAcquireFillsChapterAndCoverFromOneFetch(t *testing.T) {
s, _ := newTestStore(t)
page := &fakeFetcher{body: asuraSeriesAndCoverFixture, status: 200}
covers := &fakeBytesCoverFetcher{body: []byte("cover-bytes"), contentType: "image/jpeg"}
acq := newAcquirer(s, page, covers)
// The write itself must not carry the acquisition: it returns before the
// Cover exists, and the field is empty until the bytes land.
stored := bookmarkNewSeries(t, s, acquireSeriesURL)
if stored.Cover != "" {
t.Fatalf("Cover on the creating write = %q, want empty", stored.Cover)
}
acq.Wait()
if got := page.callCount(); got != 1 {
t.Fatalf("series page fetches = %d, want exactly 1", got)
}
if got := covers.callCount(); got != 1 {
t.Fatalf("cover fetches = %d, want 1", got)
}
got := readBookmark(t, s, acquireKey)
if got.LatestChapterNum == nil || *got.LatestChapterNum != 181 {
t.Fatalf("LatestChapterNum = %v, want 181", got.LatestChapterNum)
}
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(acquireCoverURL); got.Cover != want {
t.Fatalf("Cover = %q, want the absolute address %q", got.Cover, want)
}
body, contentType, ok, err := s.CoverByAddress(store.CoverAddress(acquireCoverURL))
if err != nil || !ok {
t.Fatalf("CoverByAddress = %v, %v", ok, err)
}
if string(body) != "cover-bytes" || contentType != "image/jpeg" {
t.Fatalf("stored cover = (%q, %q), want the fetched bytes", body, contentType)
}
}
// A Series that already exists is not re-acquired: no fetch, and the Cover it
// already has is left alone.
func TestAcquireSkipsAnExistingSeries(t *testing.T) {
s, _ := newTestStore(t)
page := &fakeFetcher{body: asuraSeriesAndCoverFixture, status: 200}
covers := &fakeBytesCoverFetcher{body: []byte("cover-bytes"), contentType: "image/jpeg"}
acq := newAcquirer(s, page, covers)
bookmarkNewSeries(t, s, acquireSeriesURL)
acq.Wait()
bookmarkNewSeries(t, s, acquireSeriesURL)
acq.Wait()
if got := page.callCount(); got != 1 {
t.Fatalf("series page fetches = %d, want 1 — an existing series is not re-acquired", got)
}
if got := covers.callCount(); got != 1 {
t.Fatalf("cover fetches = %d, want 1", got)
}
got := readBookmark(t, s, acquireKey)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(acquireCoverURL); got.Cover != want {
t.Fatalf("Cover = %q, want the acquired one %q", got.Cover, want)
}
}
// A Site that is down costs the Cover and nothing else.
func TestAcquireFailureLeavesTheBookmarkIntact(t *testing.T) {
cases := []struct {
name string
page *fakeFetcher
covers *fakeBytesCoverFetcher
// wantLatest is the chapter that still lands; 0 means none did.
wantLatest float64
}{
{
"the series page is unreachable",
&fakeFetcher{err: errors.New("connection reset")},
&fakeBytesCoverFetcher{body: []byte("cover-bytes"), contentType: "image/jpeg"},
0,
},
{
"the series page answers with a challenge",
&fakeFetcher{body: challengeFixture, status: 200},
&fakeBytesCoverFetcher{body: []byte("cover-bytes"), contentType: "image/jpeg"},
0,
},
{
"only the cover bytes fail",
&fakeFetcher{body: asuraSeriesAndCoverFixture, status: 200},
&fakeBytesCoverFetcher{err: errors.New("403")},
181,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
s, _ := newTestStore(t)
acq := newAcquirer(s, tc.page, tc.covers)
stored := bookmarkNewSeries(t, s, acquireSeriesURL)
acq.Wait()
got := readBookmark(t, s, acquireKey)
if got.Cover != "" {
t.Fatalf("Cover = %q, want empty rather than an address that 404s", got.Cover)
}
if got.Title != stored.Title || got.UpdatedAt != stored.UpdatedAt {
t.Fatalf("bookmark = %+v, want it untouched by the failed acquisition", got)
}
if tc.wantLatest == 0 {
if got.LatestChapterNum != nil {
t.Fatalf("LatestChapterNum = %v, want none captured", *got.LatestChapterNum)
}
return
}
if got.LatestChapterNum == nil || *got.LatestChapterNum != tc.wantLatest {
t.Fatalf("LatestChapterNum = %v, want %v", got.LatestChapterNum, tc.wantLatest)
}
})
}
}
// series_url arrives in a client-supplied body, so the acquisition reuses the
// poller's gate rather than deriving a second one: a non-https scheme, a
// site the parsers do not know, or a host pinned to another site is refused
// before the server spends a request from its own network position.
func TestAcquireRefusesAnUnfetchableSeriesURL(t *testing.T) {
for _, seriesURL := range []string{
"http://asurascans.com/comics/x",
"file:///etc/passwd",
"",
} {
t.Run(seriesURL, func(t *testing.T) {
s, _ := newTestStore(t)
page := &fakeFetcher{body: asuraSeriesAndCoverFixture, status: 200}
acq := newAcquirer(s, page, &fakeBytesCoverFetcher{})
bookmarkNewSeries(t, s, seriesURL)
acq.Wait()
if got := page.callCount(); got != 0 {
t.Fatalf("fetches for %q = %d, want 0", seriesURL, got)
}
})
}
}
// blockingFetcher stands in for a Site that never answers, so a synchronous
// acquisition would be visible as a stalled write rather than a slow one.
type blockingFetcher struct {
release <-chan struct{}
body string
}
func (f *blockingFetcher) Get(ctx context.Context, _ string) (string, int, error) {
select {
case <-f.release:
return f.body, 200, nil
case <-ctx.Done():
return "", 0, ctx.Err()
}
}
// The Reader's write may not wait on a third-party Site: with the acquisition
// wedged on an unanswering page, the PUT still returns.
func TestAcquireDoesNotBlockTheWrite(t *testing.T) {
s, _ := newTestStore(t)
release := make(chan struct{})
acq := &Acquirer{Store: s, Fetch: &blockingFetcher{release: release, body: asuraSeriesAndCoverFixture}}
s.OnSeriesCreated = acq.Acquire
upserted := make(chan error, 1)
go func() {
_, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: acquireKey, Site: "asura", SeriesID: acquireSeriesID,
Title: "Chronicles of the Demon Faction", SeriesURL: acquireSeriesURL, UpdatedAt: 1000,
})
upserted <- err
}()
select {
case err := <-upserted:
if err != nil {
t.Fatalf("Upsert: %v", err)
}
case <-time.After(10 * time.Second):
t.Fatal("the creating write blocked on the acquisition")
}
close(release)
acq.Wait()
}
// The second symptom of #47: a kagane Series bookmarked from a chapter page
// gets its Cover at creation, with the bytes fetched through the browser
// sidecar — the only path that clears the challenge — into the
// content-addressed store.
func TestAcquireKaganeCoverThroughBrowser(t *testing.T) {
s, _ := newTestStore(t)
tlsPage := &fakeFetcher{body: "", status: 403}
browserPage := &fakeFetcher{body: kaganeSeriesAndCoverFixture, status: 200}
covers := &fakeCoverFetcher{body: []byte("cover-bytes"), contentType: "image/webp"}
acq := &Acquirer{
Store: s, Fetch: tlsPage, BrowserFetch: browserPage,
BrowserCoverFetch: covers, Covers: &fakeBytesCoverFetcher{},
}
s.OnSeriesCreated = acq.Acquire
bookmarkNewKaganeSeries(t, s)
acq.Wait()
if got := tlsPage.callCount(); got != 0 {
t.Fatalf("plain-TLS page fetches = %d, want 0 — kagane pages are browser-only", got)
}
if got := browserPage.callCount(); got != 1 {
t.Fatalf("browser page fetches = %d, want 1", got)
}
if got := covers.callCount(); got != 1 {
t.Fatalf("browser cover fetches = %d, want 1", got)
}
if got := covers.calls[0]; got != kaganeCoverSrc {
t.Fatalf("browser cover fetched URL %q, want %q", got, kaganeCoverSrc)
}
got := readBookmark(t, s, kaganeKey)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(kaganeCoverSrc); got.Cover != want {
t.Fatalf("Cover = %q, want the content-addressed URL %q", got.Cover, want)
}
body, contentType, ok, err := s.CoverByAddress(store.CoverAddress(kaganeCoverSrc))
if err != nil || !ok {
t.Fatalf("CoverByAddress = %v, %v", ok, err)
}
if string(body) != "cover-bytes" || contentType != "image/webp" {
t.Fatalf("stored cover = (%q, %q), want the browser-fetched bytes", body, contentType)
}
}
// novelfull needs the browser only for its HTML: the cover URL comes out of
// the browser-fetched page, but the bytes go over plain TLS through the
// ordinary gated fetcher, never through the browser (issue #62).
func TestAcquireNovelfullCoverOverPlainTLS(t *testing.T) {
s, _ := newTestStore(t)
browserPage := &fakeFetcher{body: novelfullSeriesFixture + novelfullCoverFixture, status: 200}
covers := &fakeBytesCoverFetcher{body: []byte("cover-bytes"), contentType: "image/webp"}
acq := &Acquirer{
Store: s, Fetch: &fakeFetcher{body: "", status: 403},
BrowserFetch: browserPage, Covers: covers,
}
s.OnSeriesCreated = acq.Acquire
bookmarkNewNovelfullSeries(t, s)
acq.Wait()
if got := browserPage.callCount(); got != 1 {
t.Fatalf("browser page fetches = %d, want 1", got)
}
if got := covers.callCount(); got != 1 {
t.Fatalf("cover fetches = %d, want 1 — novelfull bytes never touch the browser", got)
}
if got := covers.calls[0]; got != novelfullCoverURL {
t.Fatalf("cover fetched from %q, want %q", got, novelfullCoverURL)
}
got := readBookmark(t, s, novelfullKey)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(novelfullCoverURL); got.Cover != want {
t.Fatalf("Cover = %q, want %q", got.Cover, want)
}
}
// With no browser sidecar configured, kagane is simply not acquired: no
// request is spent on a page that could only ever answer with a challenge,
// and nothing falls back to a plain fetch.
func TestAcquireKaganeSkippedWithoutBrowser(t *testing.T) {
s, _ := newTestStore(t)
tlsPage := &fakeFetcher{body: kaganeSeriesAndCoverFixture, status: 200}
acq := &Acquirer{
Store: s, Fetch: tlsPage,
Covers: &fakeBytesCoverFetcher{body: []byte("x"), contentType: "image/webp"},
}
s.OnSeriesCreated = acq.Acquire
bookmarkNewKaganeSeries(t, s)
acq.Wait()
if got := tlsPage.callCount(); got != 0 {
t.Fatalf("plain-TLS fetches for kagane = %d, want 0", got)
}
if got := readBookmark(t, s, kaganeKey); got.Cover != "" {
t.Fatalf("Cover = %q, want empty without a browser", got.Cover)
}
}
// The byte half of "nothing falls back to a plain fetch": with a browser for
// the page but none for the bytes, a kagane Cover stays absent and the TLS
// cover fetcher is never consulted.
func TestAcquireKaganeBytesNeverFallBackToPlainTLS(t *testing.T) {
s, _ := newTestStore(t)
browserPage := &fakeFetcher{body: kaganeSeriesAndCoverFixture, status: 200}
tlsCovers := &fakeBytesCoverFetcher{body: []byte("cover-bytes"), contentType: "image/webp"}
acq := &Acquirer{
Store: s, Fetch: &fakeFetcher{body: "", status: 403},
BrowserFetch: browserPage, Covers: tlsCovers,
}
s.OnSeriesCreated = acq.Acquire
bookmarkNewKaganeSeries(t, s)
acq.Wait()
if got := tlsCovers.callCount(); got != 0 {
t.Fatalf("plain-TLS cover fetches = %d, want 0 — kagane bytes are browser-only", got)
}
if got := readBookmark(t, s, kaganeKey); got.Cover != "" {
t.Fatalf("Cover = %q, want empty without a browser cover fetcher", got.Cover)
}
}
// novelfull's no-browser degradation differs from kagane's: only its HTML
// needs the sidecar, so when the page body is available — the challenge is a
// live time-varying fact that sometimes answers a plain request — the Cover
// still lands, bytes over plain TLS.
func TestAcquireNovelfullCoverWithoutBrowser(t *testing.T) {
s, _ := newTestStore(t)
tlsPage := &fakeFetcher{body: novelfullSeriesFixture + novelfullCoverFixture, status: 200}
covers := &fakeBytesCoverFetcher{body: []byte("cover-bytes"), contentType: "image/webp"}
acq := &Acquirer{Store: s, Fetch: tlsPage, Covers: covers}
s.OnSeriesCreated = acq.Acquire
bookmarkNewNovelfullSeries(t, s)
acq.Wait()
if got := covers.callCount(); got != 1 {
t.Fatalf("cover fetches = %d, want 1", got)
}
got := readBookmark(t, s, novelfullKey)
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(novelfullCoverURL); got.Cover != want {
t.Fatalf("Cover = %q, want %q", got.Cover, want)
}
}
+338
View File
@@ -0,0 +1,338 @@
package latest
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/url"
"regexp"
"strings"
"sync"
"time"
"github.com/chromedp/cdproto/runtime"
"github.com/chromedp/chromedp"
)
// challengeTimeout bounds one navigate-and-solve. A Cloudflare managed
// challenge clears in a few seconds when it clears at all; anything longer is a
// challenge that is not going to pass, and the caller's cooldown was already
// stamped before this ran.
const challengeTimeout = 45 * time.Second
var kaganeSeriesRe = regexp.MustCompile(`^/series/([0-9a-f-]{36})/?$`)
// BrowserFetcher retrieves pages through a remote headless Chrome over the
// DevTools Protocol.
//
// It exists for one reason: kagane.to and novelfull.com sit behind a
// Cloudflare JavaScript challenge. Verified 2026-08-03 (kagane) and 2026-08-05
// (novelfull) from the deployment host, plain HTTP and bogdanfinn/tls-client
// with a Chrome_133 profile both get 403 with cf-mitigated: challenge on every
// path, including the API, robots.txt and images. Clearing it requires
// executing the challenge script, which only a real browser does.
//
// The request is made *inside* the page rather than by extracting cf_clearance
// and replaying it through TLSFetcher. That cookie is bound to IP, User-Agent
// and often the TLS fingerprint, so replaying it means keeping three things in
// sync that break silently and separately. The browser's own cookie jar
// persists across polls, so the challenge is solved once every few hours.
//
// The two sites differ in how the chapter list is read: kagane serves it from
// a JSON API that must be called from inside the page (so the request carries
// the clearance cookie), while novelfull renders it into the HTML so the
// cleared DOM is the payload.
type BrowserFetcher struct {
allocCtx context.Context
cancel context.CancelFunc
// One page at a time: caps the browser's memory — it runs under a hard
// cgroup cap on a shared machine — and keeps series from sharing page state.
mu sync.Mutex
}
var _ Fetcher = (*BrowserFetcher)(nil)
// NewBrowserFetcher connects to a Chrome over CDP. The browser is not a
// sidecar: it runs on a separate machine and is reached over the tailnet
// (ADR-0006), so wsURL is that machine's tailnet address, e.g.
// ws://100.64.0.5:9222.
//
// It must be an IP, never a hostname — not MagicDNS, not a Docker service
// name. Chrome's DevTools HTTP handler 500s any /json/version request whose
// Host header isn't an IP or "localhost" (confirmed 2026-08-03), so a name
// fails at discovery and surfaces as a dead site rather than a bad URL.
//
// Do not add chromedp.NoModifyURL here: that option skips the /json/version
// discovery request entirely and dials wsURL as if it were already the full
// debugger endpoint, but Chrome only accepts connections at
// /devtools/browser/<uuid>, a path chosen fresh at every Chrome start — dialing
// the bare host:port 404s. The default (discovery) path works precisely
// because Chrome's /json/version response echoes back the Host header of the
// discovery request in webSocketDebuggerUrl, so as long as wsURL is an IP this
// process can reach, the URL chromedp gets back already points at it. That is
// also why a Chrome restarted behind a stable endpoint needs no reconnect
// here: the fresh UUID arrives with the next discovery.
func NewBrowserFetcher(wsURL string) (*BrowserFetcher, error) {
if wsURL == "" {
return nil, fmt.Errorf("empty browser websocket url")
}
ctx, cancel := chromedp.NewRemoteAllocator(context.Background(), wsURL)
return &BrowserFetcher{allocCtx: ctx, cancel: cancel}, nil
}
func (f *BrowserFetcher) Close() {
f.cancel()
}
// Get navigates to seriesURL, lets any challenge resolve, then reads the
// payload the Site's registry entry describes — kagane's chapter-list API from
// inside the page so the request carries the clearance cookie, novelfull's
// served HTML. The returned body is whatever the Site's chapter list lives in,
// which is what the entry's LatestChapter parse expects.
func (f *BrowserFetcher) Get(ctx context.Context, seriesURL string) (string, int, error) {
var body string
// Sorted order (browserBackedSites sorts) makes dispatch deterministic:
// entries' Read funcs are expected to refuse any address owned by another
// Site, and the loop must not depend on that staying true.
for _, name := range browserBackedSites() {
s := sites[name]
read, ok := s.Browser.Read(seriesURL, &body)
if !ok {
continue
}
if err := f.run(ctx, seriesURL, read,
func() bool { return s.Browser.Done(body) }); err != nil {
// Challenge never cleared, or the payload was refused.
// Indistinguishable from here and handled identically by the caller.
if errors.Is(err, errChallengeHeld) {
return "", 403, nil
}
return "", 0, fmt.Errorf("browser fetch %q: %w", seriesURL, err)
}
return body, 200, nil
}
return "", 0, fmt.Errorf("not a fetchable browser series url: %q", seriesURL)
}
// kaganeRead builds the in-tab fetch of kagane's chapter-list API: the
// request must be made from inside the page so it carries the clearance
// cookie, and the API is the only place the list exists. Refusing any other
// address is the per-Site half of the SSRF gate, kept deliberately behind
// fetchableSeriesURL (see browserRead.Read).
func kaganeRead(seriesURL string, out *string) (chromedp.Action, bool) {
apiURL, ok := kaganeAPIURL(seriesURL)
if !ok {
return nil, false
}
return chromedp.Evaluate(
`fetch(`+jsString(apiURL)+`).then(r => r.ok ? r.text() : "")`,
out, awaitPromise), true
}
// novelfullRead reads the cleared DOM. novelfull renders its chapter list
// into the served HTML, so there is no API to call from inside the page — the
// challenge-cleared DOM is the payload.
func novelfullRead(seriesURL string, out *string) (chromedp.Action, bool) {
if !novelfullSeriesURL(seriesURL) {
return nil, false
}
return chromedp.OuterHTML("html", out, chromedp.ByQuery), true
}
// Image retrieves one cover's bytes through the browser sidecar, and its
// content type.
//
// It exists because kagane serves covers behind the same challenge as its
// pages *and* with `cross-origin-resource-policy: same-origin`, so the bytes
// are only reachable from inside a browser that already holds the clearance
// cookie (verified 2026-08-08). Acquisition through the sidecar is the only
// route.
//
// The image URL is navigated to rather than fetched from some other kagane
// page: the challenge only runs on a top-level navigation, and once it clears
// the document *is* the image, so a same-origin fetch of location.href reads
// it straight back out of the cache.
//
// The challenge is not solved by the first read: WaitReady("body") is satisfied
// by the interstitial too. run holds the tab open until the in-page fetch
// succeeds, which is what gives the challenge script the seconds it needs.
func (f *BrowserFetcher) Image(ctx context.Context, imageURL string) ([]byte, string, error) {
m := kaganeImageURLRe.FindStringSubmatch(imageURL)
if m == nil {
return nil, "", fmt.Errorf("not a browser-fetchable cover url: %q", imageURL)
}
imageID := m[1]
var dataURL string
err := f.run(ctx, imageURL,
chromedp.Evaluate(`fetch(location.href).then(r => r.ok
? r.blob().then(b => new Promise(res => {
const fr = new FileReader();
fr.onload = () => res(fr.result);
fr.readAsDataURL(b);
}))
: "")`, &dataURL, awaitPromise),
func() bool { return dataURL != "" })
if err != nil {
return nil, "", fmt.Errorf("browser image %s: %w", imageID, err)
}
// "data:image/webp;base64,<payload>".
head, payload, ok := strings.Cut(dataURL, ";base64,")
if !ok {
return nil, "", fmt.Errorf("browser image %s: not a data url", imageID)
}
raw, err := base64.StdEncoding.DecodeString(payload)
if err != nil {
return nil, "", fmt.Errorf("browser image %s: %w", imageID, err)
}
return raw, strings.TrimPrefix(head, "data:"), nil
}
// errChallengeHeld reports that the budget ran out with the interstitial still
// up. Distinct from a transport failure: it means "this site said no", which
// the poller answers with a 403 and its ordinary cooldown.
var errChallengeHeld = errors.New("challenge held")
// errBrowserInterrupted distinguishes a remote Chrome restart from the
// caller's own deadline. chromedp reports both as context.Canceled.
var errBrowserInterrupted = errors.New("browser interrupted")
func classifyBrowserError(ctx context.Context, browserLost bool, err error) error {
if err == nil || ctx.Err() != nil {
return err
}
if !browserLost {
return err
}
if !errors.Is(err, context.Canceled) {
return err
}
return fmt.Errorf("%w: %w", errBrowserInterrupted, err)
}
func browserConnectionLost(ctx context.Context) bool {
c := chromedp.FromContext(ctx)
if c == nil || c.Browser == nil {
return true
}
select {
case <-c.Browser.LostConnection:
return true
default:
return false
}
}
// challengePollInterval paces re-reads while a challenge solves itself.
const challengePollInterval = 2 * time.Second
// isInterstitial reports whether html is Cloudflare's challenge page rather
// than the site's own. Matched on the challenge runtime's script path, which is
// stable across the interstitial's wording and locale — the visible "Just a
// moment..." title is neither.
func isInterstitial(html string) bool {
return strings.Contains(html, "/cdn-cgi/challenge-platform/")
}
// run navigates to target and re-reads until done reports an answer, bounded by
// challengeTimeout and by the caller's own deadline, in a tab that is closed on
// return so one wedged page cannot poison later calls.
//
// Holding the tab open across re-reads is the whole point. A Cloudflare
// interstitial needs several seconds of a live page to solve itself and write
// clearance into the browser's shared cookie jar; reading once and closing the
// tab — which is what this did before 2026-08-08 — never gives it that window,
// so every fetch lands on the interstitial and the clearance that would have
// unblocked all the later ones is never obtained.
func (f *BrowserFetcher) run(ctx context.Context, target string, read chromedp.Action, done func() bool) error {
f.mu.Lock()
defer f.mu.Unlock()
callerCtx := ctx
ctx, cancel := context.WithTimeout(ctx, challengeTimeout)
defer cancel()
tabCtx, cancelTab := chromedp.NewContext(f.allocCtx)
defer cancelTab()
// Bind the caller's deadline to the tab.
tabCtx, cancelDeadline := context.WithCancel(tabCtx)
defer cancelDeadline()
go func() {
<-ctx.Done()
cancelDeadline()
}()
if err := chromedp.Run(tabCtx,
chromedp.Navigate(target),
chromedp.WaitReady("body", chromedp.ByQuery),
); err != nil {
return classifyBrowserError(callerCtx, browserConnectionLost(tabCtx), err)
}
var lastErr error
for {
// The challenge reloads the page when it passes, which tears down the
// execution context mid-read. That is a retry, not a failure.
if err := chromedp.Run(tabCtx, read); err != nil {
err = classifyBrowserError(callerCtx, browserConnectionLost(tabCtx), err)
if errors.Is(err, errBrowserInterrupted) {
return err
}
lastErr = err
} else if done() {
return nil
}
select {
case <-ctx.Done():
if err := callerCtx.Err(); err != nil {
return err
}
if lastErr != nil {
return fmt.Errorf("%w (last read: %v)", errChallengeHeld, lastErr)
}
return errChallengeHeld
case <-time.After(challengePollInterval):
}
}
}
// kaganeAPIURL maps a stored series_url to the JSON endpoint carrying its
// chapter list. Returning false for anything else is a second line of defence
// behind fetchableSeriesURL: a headless browser is a strong SSRF primitive and
// series_url is client-supplied, so the host is pinned here too.
func kaganeAPIURL(seriesURL string) (string, bool) {
u, err := url.Parse(seriesURL)
if err != nil || u.Scheme != "https" || u.Hostname() != "kagane.to" {
return "", false
}
m := kaganeSeriesRe.FindStringSubmatch(u.Path)
if m == nil {
return "", false
}
return "https://kagane.to/api/v2/series/" + m[1], true
}
// novelfullSeriesURL reports whether seriesURL is a novelfull series page this
// fetcher will open. novelfull's chapter list is in the served HTML, so unlike
// kagane there is no API to call from inside the page — the challenge-cleared
// DOM is the payload. The host is pinned here for the same reason kagane's is:
// series_url is client-supplied and a headless browser is a strong SSRF
// primitive.
func novelfullSeriesURL(seriesURL string) bool {
u, err := url.Parse(seriesURL)
return err == nil && u.Scheme == "https" && u.Hostname() == "novelfull.com" &&
strings.HasSuffix(u.Path, ".html")
}
// awaitPromise makes Evaluate resolve the promise rather than returning a
// serialised Promise object.
func awaitPromise(p *runtime.EvaluateParams) *runtime.EvaluateParams {
return p.WithAwaitPromise(true)
}
// jsString renders s as a JavaScript string literal for embedding in an
// Evaluate expression. The URL is host-pinned by kaganeAPIURL before it gets
// here, but quoting it properly is what keeps that guarantee intact.
func jsString(s string) string {
b, _ := json.Marshal(s)
return string(b)
}
+77
View File
@@ -0,0 +1,77 @@
package latest
import (
"context"
"errors"
"testing"
)
func TestKaganeAPIURL(t *testing.T) {
const uuid = "019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"
tests := []struct {
name string
seriesURL string
want string
wantOK bool
}{
{
name: "series page maps to its API endpoint",
seriesURL: "https://kagane.to/series/" + uuid,
want: "https://kagane.to/api/v2/series/" + uuid,
wantOK: true,
},
{
name: "trailing slash is tolerated",
seriesURL: "https://kagane.to/series/" + uuid + "/",
want: "https://kagane.to/api/v2/series/" + uuid,
wantOK: true,
},
{"not a series path", "https://kagane.to/search", "", false},
{"foreign host", "https://evil.example/series/" + uuid, "", false},
{"garbage", "://", "", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, ok := kaganeAPIURL(tt.seriesURL)
if ok != tt.wantOK || got != tt.want {
t.Errorf("kaganeAPIURL(%q) = %q, %v; want %q, %v",
tt.seriesURL, got, ok, tt.want, tt.wantOK)
}
})
}
}
func TestNovelfullSeriesURL(t *testing.T) {
cases := []struct {
name string
url string
want bool
}{
{"series page", "https://novelfull.com/reverend-insanity.html", true},
{"foreign host", "https://evil.example/reverend-insanity.html", false},
{"not https", "http://novelfull.com/reverend-insanity.html", false},
{"not a series page", "https://novelfull.com/genre/Fantasy", false},
{"garbage", "://nope", false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := novelfullSeriesURL(tc.url); got != tc.want {
t.Fatalf("novelfullSeriesURL(%q) = %v, want %v", tc.url, got, tc.want)
}
})
}
}
func TestClassifyBrowserInterruption(t *testing.T) {
if err := classifyBrowserError(context.Background(), true, context.Canceled); !errors.Is(err, errBrowserInterrupted) {
t.Fatalf("classifyBrowserError(context.Canceled) = %v, want browser interruption", err)
}
if err := classifyBrowserError(context.Background(), false, context.Canceled); errors.Is(err, errBrowserInterrupted) {
t.Fatalf("ordinary cancellation misclassified as browser interruption: %v", err)
}
caller, cancel := context.WithCancel(context.Background())
cancel()
if err := classifyBrowserError(caller, true, context.Canceled); errors.Is(err, errBrowserInterrupted) {
t.Fatalf("caller cancellation misclassified as browser interruption: %v", err)
}
}
+204
View File
@@ -0,0 +1,204 @@
package latest
import (
"context"
"errors"
"fmt"
"io"
"mime"
"net"
"net/http"
"net/netip"
"net/url"
"strings"
"time"
"bookmarkmanager/backend/internal/store"
)
// CoverBytesFetcher retrieves one cover from its source URL. The caller owns
// persistence; this seam keeps network policy independent from the store.
type CoverBytesFetcher interface {
Fetch(ctx context.Context, sourceURL string) (body []byte, contentType string, err error)
}
// fetchCoverBytes routes a cover's byte retrieval by URL shape, not by Site
// name: the browser fetcher's module claims the addresses only it can fetch
// (kagane's image route answers a plain fetch with a challenge and
// `cross-origin-resource-policy: same-origin`), and everything else goes over
// plain TLS. Missing fetchers degrade to an error the caller logs, never a
// fallback onto a path that cannot succeed. One routing rule for the poll and
// the acquirer, so the two cannot drift apart.
func fetchCoverBytes(ctx context.Context, cover string, browser BrowserCoverFetcher, tls CoverBytesFetcher) ([]byte, string, error) {
if browserOnlyCoverURL(cover) {
if browser == nil {
return nil, "", errors.New("no cover fetcher")
}
return browser.Image(ctx, cover)
}
if tls == nil {
return nil, "", errors.New("no cover fetcher")
}
return tls.Fetch(ctx, cover)
}
// CoverResolver resolves a host before any connection is attempted. Tests
// inject it to exercise hostile DNS results without touching the live network.
type CoverResolver func(context.Context, string) ([]netip.Addr, error)
// TLSCoverFetcher retrieves image bytes with the standard HTTPS client. Unlike
// TLSFetcher, it does not need a browser fingerprint: cover hosts are public
// CDNs and the response is accepted only after the destination gate passes.
type TLSCoverFetcher struct {
client *http.Client
resolve CoverResolver
}
var _ CoverBytesFetcher = (*TLSCoverFetcher)(nil)
const coverRequestTimeout = 30 * time.Second
var carrierGradeNAT = netip.MustParsePrefix("100.64.0.0/10")
// NewCoverFetcher builds the production cover client with the real resolver.
func NewCoverFetcher() *TLSCoverFetcher {
return NewCoverFetcherWithResolver(nil)
}
// NewCoverFetcherWithResolver builds a cover client using resolve, or the real
// system resolver when resolve is nil.
func NewCoverFetcherWithResolver(resolve CoverResolver) *TLSCoverFetcher {
if resolve == nil {
resolve = defaultCoverResolver
}
return newCoverFetcher(newCoverHTTPClient(resolve), resolve)
}
func newCoverFetcher(client *http.Client, resolve CoverResolver) *TLSCoverFetcher {
f := &TLSCoverFetcher{client: client, resolve: resolve}
client.CheckRedirect = func(req *http.Request, _ []*http.Request) error {
if err := f.validateURL(req.Context(), req.URL); err != nil {
return fmt.Errorf("redirect destination: %w", err)
}
return nil
}
return f
}
func defaultCoverResolver(ctx context.Context, host string) ([]netip.Addr, error) {
return net.DefaultResolver.LookupNetIP(ctx, "ip", host)
}
func newCoverHTTPClient(resolve CoverResolver) *http.Client {
base, ok := http.DefaultTransport.(*http.Transport)
if !ok {
base = &http.Transport{}
}
transport := base.Clone()
// A proxy would make the dial target the proxy rather than the cover host,
// defeating destination classification. Cover fetching is direct by design.
transport.Proxy = nil
dialer := &net.Dialer{}
transport.DialContext = func(ctx context.Context, network, address string) (net.Conn, error) {
host, port, err := net.SplitHostPort(address)
if err != nil {
return nil, fmt.Errorf("split cover address %q: %w", address, err)
}
addrs, err := resolveCoverHost(ctx, host, resolve)
if err != nil {
return nil, err
}
for _, addr := range addrs {
if !publicCoverAddress(addr) {
return nil, fmt.Errorf("cover host resolves to refused address %s", addr)
}
conn, err := dialer.DialContext(ctx, network, net.JoinHostPort(addr.String(), port))
if err == nil {
return conn, nil
}
}
return nil, fmt.Errorf("cover host %q has no reachable address", host)
}
return &http.Client{Transport: transport, Timeout: coverRequestTimeout}
}
func (f *TLSCoverFetcher) Fetch(ctx context.Context, sourceURL string) ([]byte, string, error) {
u, err := url.Parse(sourceURL)
if err != nil {
return nil, "", fmt.Errorf("parse cover URL: %w", err)
}
if err := f.validateURL(ctx, u); err != nil {
return nil, "", err
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
return nil, "", fmt.Errorf("build cover request: %w", err)
}
resp, err := f.client.Do(req)
if err != nil {
return nil, "", fmt.Errorf("fetch cover: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, "", fmt.Errorf("fetch cover: status %d", resp.StatusCode)
}
raw, _, err := mime.ParseMediaType(resp.Header.Get("Content-Type"))
if err != nil {
return nil, "", fmt.Errorf("fetch cover: unsupported content type %q", resp.Header.Get("Content-Type"))
}
contentType, ok := store.CoverContentType(raw)
if !ok {
return nil, "", fmt.Errorf("fetch cover: unsupported content type %q", raw)
}
if resp.ContentLength > maxBodyBytes {
return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxBodyBytes)
}
body, err := io.ReadAll(io.LimitReader(resp.Body, maxBodyBytes+1))
if err != nil {
return nil, "", fmt.Errorf("read cover: %w", err)
}
if len(body) > maxBodyBytes {
return nil, "", fmt.Errorf("fetch cover: response exceeds %d bytes", maxBodyBytes)
}
return body, contentType, nil
}
// This gate deliberately differs from fetchableSeriesURL: cover hosts are
// site-independent CDNs, so a Site host allowlist would reject valid covers.
func (f *TLSCoverFetcher) validateURL(ctx context.Context, u *url.URL) error {
if u == nil || u.Scheme != "https" || u.Host == "" || u.User != nil {
return errors.New("cover URL must use HTTPS without credentials")
}
host := u.Hostname()
if host == "" {
return errors.New("cover URL has no host")
}
addrs, err := resolveCoverHost(ctx, host, f.resolve)
if err != nil {
return fmt.Errorf("resolve cover host %q: %w", host, err)
}
if len(addrs) == 0 {
return fmt.Errorf("resolve cover host %q: no addresses", host)
}
for _, addr := range addrs {
if !publicCoverAddress(addr) {
return fmt.Errorf("cover host %q resolves to refused address %s", host, addr)
}
}
return nil
}
func resolveCoverHost(ctx context.Context, host string, resolve CoverResolver) ([]netip.Addr, error) {
if literal, err := netip.ParseAddr(host); err == nil {
return []netip.Addr{literal.Unmap()}, nil
}
return resolve(ctx, strings.TrimSuffix(host, "."))
}
func publicCoverAddress(addr netip.Addr) bool {
addr = addr.Unmap()
return addr.IsValid() && addr.IsGlobalUnicast() &&
!addr.IsLoopback() && !addr.IsPrivate() && !addr.IsLinkLocalUnicast() &&
!carrierGradeNAT.Contains(addr)
}
+226
View File
@@ -0,0 +1,226 @@
package latest
import (
"bytes"
"context"
"crypto/tls"
"io"
"net"
"net/http"
"net/http/httptest"
"net/netip"
"testing"
)
func TestCoverFetcherFetchesPublicHTTPSImage(t *testing.T) {
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.TLS == nil {
t.Fatal("cover request was not made over TLS")
}
w.Header().Set("Content-Type", "image/jpeg")
io.WriteString(w, "cover-bytes")
}))
defer server.Close()
transport := server.Client().Transport.(*http.Transport).Clone()
transport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true} // test server certificate
transport.DialContext = func(ctx context.Context, network, _ string) (net.Conn, error) {
return (&net.Dialer{}).DialContext(ctx, network, server.Listener.Addr().String())
}
client := &http.Client{Transport: transport}
fetcher := newCoverFetcher(client, func(context.Context, string) ([]netip.Addr, error) {
return []netip.Addr{netip.MustParseAddr("198.51.100.10")}, nil
})
body, contentType, err := fetcher.Fetch(context.Background(), "https://cdn.example/cover.jpg")
if err != nil {
t.Fatalf("Fetch: %v", err)
}
if string(body) != "cover-bytes" || contentType != "image/jpeg" {
t.Fatalf("Fetch = (%q, %q), want (cover-bytes, image/jpeg)", body, contentType)
}
}
func TestNewCoverFetcherRechecksResolverBeforeConnection(t *testing.T) {
var requests int
server := httptest.NewTLSServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
requests++
}))
defer server.Close()
_, port, err := net.SplitHostPort(server.Listener.Addr().String())
if err != nil {
t.Fatalf("server address: %v", err)
}
resolves := 0
fetcher := NewCoverFetcherWithResolver(func(context.Context, string) ([]netip.Addr, error) {
resolves++
if resolves == 1 {
return []netip.Addr{netip.MustParseAddr("198.51.100.10")}, nil
}
return []netip.Addr{netip.MustParseAddr("127.0.0.1")}, nil
})
_, _, err = fetcher.Fetch(context.Background(), "https://cdn.example:"+port+"/cover.jpg")
if err == nil {
t.Fatal("Fetch accepted a destination that became private")
}
if resolves != 2 {
t.Fatalf("resolver calls = %d, want preflight and dial checks", resolves)
}
if requests != 0 {
t.Fatalf("requests = %d, want 0", requests)
}
}
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) }
func coverResponse(status int, contentType, location string, body []byte) *http.Response {
header := make(http.Header)
if contentType != "" {
header.Set("Content-Type", contentType)
}
if location != "" {
header.Set("Location", location)
}
return &http.Response{
StatusCode: status,
Status: http.StatusText(status),
Header: header,
Body: io.NopCloser(bytes.NewReader(body)),
ContentLength: int64(len(body)),
}
}
func TestCoverFetcherRefusesUnsafeDestinationsBeforeRequest(t *testing.T) {
var calls int
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
calls++
return coverResponse(http.StatusOK, "image/jpeg", "", []byte("must not reach network")), nil
})}
resolve := func(_ context.Context, host string) ([]netip.Addr, error) {
switch host {
case "loopback.example":
return []netip.Addr{netip.MustParseAddr("127.0.0.1")}, nil
case "private.example":
return []netip.Addr{netip.MustParseAddr("10.0.0.1")}, nil
case "linklocal.example":
return []netip.Addr{netip.MustParseAddr("169.254.1.1")}, nil
case "unique-local.example":
return []netip.Addr{netip.MustParseAddr("fc00::1")}, nil
case "cgnat.example":
return []netip.Addr{netip.MustParseAddr("100.64.0.1")}, nil
default:
return []netip.Addr{netip.MustParseAddr("198.51.100.10")}, nil
}
}
fetcher := newCoverFetcher(client, resolve)
tests := []string{
"http://public.example/cover.jpg",
"https://127.0.0.1/cover.jpg",
"https://10.0.0.1/cover.jpg",
"https://169.254.1.1/cover.jpg",
"https://[fc00::1]/cover.jpg",
"https://100.64.0.1/cover.jpg",
"https://loopback.example/cover.jpg",
"https://private.example/cover.jpg",
"https://linklocal.example/cover.jpg",
"https://unique-local.example/cover.jpg",
"https://cgnat.example/cover.jpg",
}
for _, sourceURL := range tests {
t.Run(sourceURL, func(t *testing.T) {
calls = 0
if _, _, err := fetcher.Fetch(context.Background(), sourceURL); err == nil {
t.Fatal("Fetch accepted refused destination")
}
if calls != 0 {
t.Fatalf("network calls = %d, want 0", calls)
}
})
}
}
func TestCoverFetcherStopsRedirectIntoPrivateAddress(t *testing.T) {
var calls int
client := &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
calls++
if req.URL.Hostname() != "cdn.example" {
t.Fatalf("redirect reached %s", req.URL)
}
return coverResponse(http.StatusFound, "", "https://internal.example/cover.jpg", nil), nil
})}
fetcher := newCoverFetcher(client, func(_ context.Context, host string) ([]netip.Addr, error) {
if host == "internal.example" {
return []netip.Addr{netip.MustParseAddr("192.168.1.1")}, nil
}
return []netip.Addr{netip.MustParseAddr("198.51.100.10")}, nil
})
if _, _, err := fetcher.Fetch(context.Background(), "https://cdn.example/cover.jpg"); err == nil {
t.Fatal("Fetch followed redirect into private address")
}
if calls != 1 {
t.Fatalf("network calls = %d, want only public first hop", calls)
}
}
func TestCoverFetcherRejectsOversizedBody(t *testing.T) {
var calls int
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
calls++
response := coverResponse(http.StatusOK, "image/webp", "", bytes.Repeat([]byte("x"), maxBodyBytes+1))
response.ContentLength = -1
return response, nil
})}
fetcher := newCoverFetcher(client, func(context.Context, string) ([]netip.Addr, error) {
return []netip.Addr{netip.MustParseAddr("198.51.100.10")}, nil
})
if _, _, err := fetcher.Fetch(context.Background(), "https://cdn.example/large.webp"); err == nil {
t.Fatal("Fetch accepted oversized body")
}
if calls != 1 {
t.Fatalf("network calls = %d, want 1", calls)
}
}
func TestCoverFetcherRejectsNonImage(t *testing.T) {
var calls int
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
calls++
return coverResponse(http.StatusOK, "text/html", "", []byte("challenge")), nil
})}
fetcher := newCoverFetcher(client, func(context.Context, string) ([]netip.Addr, error) {
return []netip.Addr{netip.MustParseAddr("198.51.100.10")}, nil
})
if _, _, err := fetcher.Fetch(context.Background(), "https://cdn.example/challenge"); err == nil {
t.Fatal("Fetch accepted non-image response")
}
if calls != 1 {
t.Fatalf("network calls = %d, want 1", calls)
}
}
// comix labels its covers "image/jpg", which is not a registered type but is
// what the Site actually answers with; the bytes are stored under the real
// name so one image cannot land under two spellings.
func TestCoverFetcherCanonicalisesJpgAlias(t *testing.T) {
client := &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
return coverResponse(http.StatusOK, "image/jpg", "", []byte("cover-bytes")), nil
})}
fetcher := newCoverFetcher(client, func(context.Context, string) ([]netip.Addr, error) {
return []netip.Addr{netip.MustParseAddr("198.51.100.10")}, nil
})
body, contentType, err := fetcher.Fetch(context.Background(), "https://static.comix.to/cover.jpg")
if err != nil {
t.Fatalf("Fetch: %v", err)
}
if string(body) != "cover-bytes" || contentType != "image/jpeg" {
t.Fatalf("Fetch = (%q, %q), want (cover-bytes, image/jpeg)", body, contentType)
}
}
@@ -1,4 +1,4 @@
package main
package latest
import (
"context"
@@ -11,8 +11,9 @@ import (
)
// maxBodyBytes caps what a single series page can cost in memory. Real pages
// measured 100-400 KB on 2026-07-26, so this is roughly 10x headroom and mostly
// guards against a proxy handing back something enormous.
// measured 100-400 KB on 2026-07-26; lightnovelworld runs larger — 685 KB and
// 1.18 MB measured 2026-08-11 — so the headroom there is roughly 3.5x, and the
// cap mostly guards against a proxy handing back something enormous.
const maxBodyBytes = 4 << 20
// chromeUA matches the client profile below. A Chrome fingerprint paired with a
@@ -20,20 +21,20 @@ const maxBodyBytes = 4 << 20
const chromeUA = "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 " +
"(KHTML, like Gecko) Chrome/133.0.0.0 Mobile Safari/537.36"
// tlsFetcher fetches series pages with a Chrome TLS fingerprint.
// TLSFetcher fetches series pages with a Chrome TLS fingerprint.
//
// Plain net/http was verified working against both sites on 2026-07-26, so this
// is not fixing an observed block — it is deliberate defence-in-depth against a
// future fingerprint-based one, chosen up front rather than reacted to later.
// The library is pure Go, so CGO_ENABLED=0, the static binary, and the
// distroless image are all unaffected.
type tlsFetcher struct {
type TLSFetcher struct {
client tls_client.HttpClient
}
var _ fetcher = (*tlsFetcher)(nil)
var _ Fetcher = (*TLSFetcher)(nil)
func newTLSFetcher() (*tlsFetcher, error) {
func NewTLSFetcher() (*TLSFetcher, error) {
c, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(),
tls_client.WithTimeoutSeconds(30),
tls_client.WithClientProfile(profiles.Chrome_133),
@@ -41,13 +42,13 @@ func newTLSFetcher() (*tlsFetcher, error) {
if err != nil {
return nil, fmt.Errorf("new tls client: %w", err)
}
return &tlsFetcher{client: c}, nil
return &TLSFetcher{client: c}, nil
}
// Get fetches url and returns the body and status. Redirects are followed: the
// demonic chapter anchors are a redirect form, and asura has moved domains
// before.
func (f *tlsFetcher) Get(ctx context.Context, url string) (string, int, error) {
func (f *TLSFetcher) Get(ctx context.Context, url string) (string, int, error) {
req, err := fhttp.NewRequest(fhttp.MethodGet, url, nil)
if err != nil {
return "", 0, fmt.Errorf("build request %q: %w", url, err)
+297
View File
@@ -0,0 +1,297 @@
package latest
import (
"context"
"errors"
"log"
"net/url"
"time"
"bookmarkmanager/backend/internal/store"
)
// Fetcher retrieves a series page. It exists as an interface so tests can inject
// a fake: nothing in the test suite may touch the network or the TLS client.
type Fetcher interface {
Get(ctx context.Context, url string) (body string, status int, err error)
}
// BrowserCoverFetcher retrieves one cover's bytes through the browser-backed
// path — the only route that clears the challenge kagane's image URLs answer
// a plain fetch with. Satisfied by BrowserFetcher.
type BrowserCoverFetcher interface {
Image(ctx context.Context, imageURL string) (body []byte, contentType string, err error)
}
// Poller re-checks each bookmarked series' newest published chapter on a
// schedule, independent of the userscript's own in-browser checks. The two run
// in parallel and report the same observable fact, so whichever writes last wins
// and neither needs to know about the other.
//
// Two clocks, deliberately independent:
//
// - Interval is how often this goroutine wakes up and looks.
// - Cooldowns are how long a series rests since its own last check. Browser-
// backed sites use the longer BrowserCooldown.
//
// Cooldowns are enforced by the WHERE clause in DueForLatestCheck rather than
// by any timer. Shortening Interval therefore cannot shorten anyone's cooldown;
// it only makes the poller wake up and find nothing due more often.
type Poller struct {
Store *store.Store
Fetch Fetcher
// BrowserFetch handles sites behind a JavaScript challenge that Fetch
// cannot clear. Nil disables those sites entirely rather than falling back
// to Fetch, which would only ever retrieve a challenge page.
BrowserFetch Fetcher
// CoverFetch is optional; failures are logged and never affect the chapter poll.
CoverFetch BrowserCoverFetcher
// CoverBytesFetch is optional; it handles plain-TLS sources through the
// same failure-isolated prefetch path.
CoverBytesFetch CoverBytesFetcher
Now func() time.Time // injected so tests can freeze it
Cooldown time.Duration
BrowserCooldown time.Duration
Interval time.Duration
Stagger time.Duration
Batch int
}
// fillBlankCover gives a Series its Cover when it has none. The blank state is
// what "no Cover yet" means on the wire (ADR-0007): permanently-blank rows
// created before acquisition existed, and rows whose creation-time fetch
// failed, both heal here. A non-blank CoverAddress is left alone — refetching
// would add a request per Series per cycle and change artwork under the Reader
// for no visible reason. A row that already carries a source URL is owned by
// prefetchCover instead; this path only records a Cover address already
// extracted from the series page.
//
// Failures are logged against the Series and never returned: the chapter poll
// must not notice. A failed fill is retried the next time this Series is due;
// there is no separate retry queue.
func (p *Poller) fillBlankCover(ctx context.Context, sr store.Series, cover string) {
if sr.CoverAddress != "" || sr.Cover != "" {
return
}
if cover == "" {
return
}
p.storeCover(ctx, sr, cover)
}
// prefetchCover heals Series that already carry a third-party source URL but
// no stored address — the state left by client-supplied covers before
// acquisition moved server-side. Every Site takes the same path; fetchCoverBytes
// routes by URL shape, so browser-claimed URLs still need the sidecar. New
// blanks have no source URL and go through fillBlankCover from the series page
// instead.
func (p *Poller) prefetchCover(ctx context.Context, sr store.Series) {
if sr.Cover == "" || sr.CoverAddress != "" {
return
}
body, contentType, found, err := p.Store.GetCover(sr.Cover)
if err != nil {
log.Printf("latest poll %q: read cover: %v", sr.Key(), err)
return
}
if found {
if err := p.Store.SetSeriesCover(sr.Site, sr.SeriesID, sr.Cover, body, contentType); err != nil {
log.Printf("latest poll %q: persist cover: %v", sr.Key(), err)
}
return
}
p.storeCover(ctx, sr, sr.Cover)
}
// storeCover fetches bytes for sourceURL and points the Series at them. Every
// failure is logged against the Series and swallowed so the chapter poll
// cannot see it.
func (p *Poller) storeCover(ctx context.Context, sr store.Series, sourceURL string) {
bytes, contentType, err := fetchCoverBytes(ctx, sourceURL, p.CoverFetch, p.CoverBytesFetch)
if err != nil {
log.Printf("latest poll %q: fetch cover %s: %v", sr.Key(), sourceURL, err)
return
}
if err := p.Store.SetSeriesCover(sr.Site, sr.SeriesID, sourceURL, bytes, contentType); err != nil {
log.Printf("latest poll %q: persist cover: %v", sr.Key(), err)
}
}
// fetcherFor returns the fetcher a site's page needs, or nil when the site
// cannot be fetched at all right now. A Site whose registry entry carries a
// Browser read — kagane and novelfull, both behind a Cloudflare JavaScript
// challenge no TLS fingerprint clears — prefers the browser; when it is
// absent, the entry's Fallback decides whether plain TLS may take over. One
// routing rule for the poll and the acquirer, so the two cannot drift apart.
func fetcherFor(site string, browser, tls Fetcher) Fetcher {
s, known := sites[site]
if !known {
// No registry entry means nothing to fetch or parse; fail closed even
// though the only caller gates first, so a future caller that skips
// the gate cannot hand an arbitrary https URL to the TLS fetcher.
return nil
}
if s.Browser == nil {
return tls
}
if browser != nil {
return browser
}
if s.Browser.Fallback {
return tls
}
return nil
}
// Run polls until ctx is cancelled.
//
// runOnce is called synchronously, so a batch that overruns the tick delays the
// next one instead of stacking a second batch on top of it. That is the intended
// failure mode for a misconfigured batch x stagger: a slower cadence, never
// concurrent fetch storms.
func (p *Poller) Run(ctx context.Context) {
log.Printf("latest-chapter poller: interval=%s cooldown=%s browser-cooldown=%s batch=%d stagger=%s",
p.Interval, p.Cooldown, p.BrowserCooldown, p.Batch, p.Stagger)
t := time.NewTicker(p.Interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
log.Println("latest-chapter poller: stopped")
return
case <-t.C:
p.runOnce(ctx)
}
}
}
// runOnce processes one batch of due series.
func (p *Poller) runOnce(ctx context.Context) {
now := p.Now()
cutoff := now.Add(-p.Cooldown).UnixMilli()
browserCutoff := now.Add(-p.BrowserCooldown).UnixMilli()
due, err := p.Store.DueForLatestCheck(cutoff, browserCutoff, browserBackedSites(), p.Batch)
if err != nil {
log.Printf("latest poll: due query: %v", err)
return
}
checked := 0
for i, sr := range due {
if ctx.Err() != nil {
break
}
// Staggered rather than fired together: a burst of simultaneous requests
// from one server IP is the traffic shape most likely to move that IP's
// bot score. This is the server-side analogue of the userscript's "one
// series per navigation ... indistinguishable from browsing" (L455-456).
stopped := false
if i > 0 && p.Stagger > 0 {
select {
case <-ctx.Done():
stopped = true
case <-time.After(p.Stagger):
}
}
if stopped {
break
}
p.checkOne(ctx, sr)
checked++
}
// due vs checked is how you tell which constraint is binding: ticks that
// report due=0 mean the cooldown is the limit, ticks that report due==batch
// every time mean throughput is.
log.Printf("latest poll: due=%d checked=%d", len(due), checked)
}
// checkOne re-checks one series. Every failure path here is "log and move on":
// the poller is a best-effort enhancement, and no single bad series may stall a
// batch or take down the process.
func (p *Poller) checkOne(ctx context.Context, sr store.Series) {
defer func() {
if r := recover(); r != nil {
log.Printf("latest poll %q: recovered from panic: %v", sr.Key(), r)
}
}()
// Stamped before the fetch, not after, so an error, a timeout, or a shutdown
// mid-request still consumes the cooldown. Otherwise a renamed or deleted
// series would be retried on every single tick forever. The userscript
// stamps in the same order and for the same reason (L471-473).
if err := p.Store.MarkLatestChecked(sr.Site, sr.SeriesID, p.Now().UnixMilli()); err != nil {
log.Printf("latest poll %q: mark checked: %v", sr.Key(), err)
return
}
facts, err := readSeriesPage(ctx, sr.Site, sr.SeriesURL, p.BrowserFetch, p.Fetch)
if err != nil {
switch {
case errors.Is(err, errNotFetchable):
// The cooldown above is already consumed, so a row that never
// passes the gate is retried at cooldown pace rather than
// hot-looping.
log.Printf("latest poll %q: not fetchable: site=%q url=%q", sr.Key(), sr.Site, sr.SeriesURL)
return
case errors.Is(err, errNoFetcher):
log.Printf("latest poll %q: no fetcher for site %q", sr.Key(), sr.Site)
return
}
// A legacy cover heals independently of the page read: its source may
// answer — a CDN — while the origin does not, so a fetch failure does
// not skip the heal, matching the order the shared read replaced.
p.prefetchCover(ctx, sr)
log.Printf("latest poll %q: %v", sr.Key(), err)
return
}
// A legacy cover source is healed independently of the page read.
p.prefetchCover(ctx, sr)
// Cover fill is independent of the chapter signal: a page that lost its
// chapter list may keep its og:image, and a blank Series heals either way.
p.fillBlankCover(ctx, sr, facts.Cover)
if !facts.HasLatest {
// Most likely a challenge page or a layout change. Either way the row is
// already stamped, so this waits out a cooldown instead of hot-looping.
log.Printf("latest poll %q: no chapter links in %d bytes", sr.Key(), facts.BodyLen)
return
}
// Equality, not >, mirroring the userscript (L427): a site that retracts a
// chapter should correct the stored number downward. The comparison is
// against the due-query snapshot; a concurrent write in between only costs
// one redundant UPDATE of the same absolute value, never a wrong one.
if sr.LatestChapterNum != nil && *sr.LatestChapterNum == facts.Latest.Num {
return
}
// Series-level write: the row is shared, so one update refreshes every
// bookmark joining to it, and the bookmark's updated_at is never touched —
// a newly published chapter is not reading progress and must not reorder
// the list.
if err := p.Store.SetLatestChapter(sr.Site, sr.SeriesID, facts.Latest.Label, facts.Latest.Num); err != nil {
log.Printf("latest poll %q: set latest chapter: %v", sr.Key(), err)
return
}
log.Printf("latest poll %q: latest is now %s", sr.Key(), facts.Latest.Label)
}
// fetchableSeriesURL reports whether site is a Site the registry knows and
// seriesURL is safe to hand to a fetcher: an https URL whose host matches the
// Site's pinned hostname exactly. series_url comes from client-supplied PUT
// bodies, so this is a defence against the poller being used to probe
// arbitrary hosts from the server's own network position, not just a check
// against wasted requests. The pin guards different things per Site — a
// browser Site guards a control that executes JavaScript and carries cookies,
// a parser Site guards a wasted request — but the rule is one rule, from the
// registry.
func fetchableSeriesURL(site, seriesURL string) bool {
s, known := sites[site]
if !known {
return false
}
u, err := url.Parse(seriesURL)
if err != nil {
return false
}
return u.Scheme == "https" && u.Hostname() == s.Host
}
File diff suppressed because it is too large Load Diff
+58
View File
@@ -0,0 +1,58 @@
package latest
import (
"context"
"errors"
"fmt"
)
// seriesRead carries the two facts the poll and the acquirer both extract
// from a series page. Persistence, stamps and scheduling stay with the
// callers, so the policies that keep the two flows distinct (stamp order,
// cooldowns) are not swallowed by the module.
type seriesRead struct {
Latest latestChapter
HasLatest bool
Cover string
HasCover bool
// BodyLen is the fetched body's length, surfaced because the no-chapter
// log uses it to tell a markup change from a body the size cap cut short.
BodyLen int
}
// errNotFetchable and errNoFetcher separate the gate and the route from fetch
// failures so each caller keeps its own distinct log line for all three.
var (
errNotFetchable = errors.New("series url not fetchable")
errNoFetcher = errors.New("no fetcher for site")
)
// readSeriesPage performs the series-page read the poll and the acquirer have
// in common: gate the address, choose the route, fetch the page, extract the
// Latest Chapter and the Cover address. It persists nothing and stamps
// nothing.
//
// series_url arrives in a client-supplied PUT body (PUT /bookmarks/{key}
// accepts any string), so the gate is not an optimisation against burning a
// request on an unknown site: without it, the server would issue a GET from
// its own network position to whatever URL a token-holder writes, including
// link-local/internal addresses or non-https schemes.
func readSeriesPage(ctx context.Context, site, seriesURL string, browser, tls Fetcher) (seriesRead, error) {
if !fetchableSeriesURL(site, seriesURL) {
return seriesRead{}, fmt.Errorf("%w: site=%q url=%q", errNotFetchable, site, seriesURL)
}
f := fetcherFor(site, browser, tls)
if f == nil {
return seriesRead{}, fmt.Errorf("%w: site %q", errNoFetcher, site)
}
body, status, err := f.Get(ctx, seriesURL)
if err != nil {
return seriesRead{}, fmt.Errorf("fetch %s: %w", seriesURL, err)
}
if status != 200 {
return seriesRead{}, fmt.Errorf("fetch %s: status %d", seriesURL, status)
}
latest, hasLatest := latestChapterFrom(site, seriesURL, body)
cover, hasCover := coverFrom(site, seriesURL, body)
return seriesRead{Latest: latest, HasLatest: hasLatest, Cover: cover, HasCover: hasCover, BodyLen: len(body)}, nil
}
+437
View File
@@ -0,0 +1,437 @@
package latest
import (
"encoding/json"
"html"
"log"
"net/url"
"regexp"
"sort"
"strconv"
"strings"
"github.com/chromedp/chromedp"
)
// latestChapter is the newest chapter a series page advertises.
type latestChapter struct {
Num float64
Label string
}
// site answers the fixed questions every series-page read asks of its Site
// (ADR-0009): the host its addresses must carry, how to find the Latest
// Chapter and the Cover address in a body, and — for a Site behind a
// JavaScript challenge — how to read its payload from a cleared tab. One
// entry describes everything about one Site, and nowhere else gets to compare
// the site string.
type site struct {
// Host is the exact hostname a series_url for this Site must carry.
Host string
// LatestChapter finds the newest chapter in a fetched body.
LatestChapter func(seriesURL, body string) (latestChapter, bool)
// Cover finds the Cover address in a fetched body.
Cover func(seriesURL, body string) (string, bool)
// Browser reads this Site's payload from a cleared browser tab; nil
// means the page is fetched over plain TLS.
Browser *browserRead
}
type browserRead struct {
// Read builds the tab read for seriesURL, refusing (false) an address
// this Site will not open in a browser — the per-Site half of the SSRF
// gate, kept deliberately behind fetchableSeriesURL: a headless browser
// executes JavaScript and carries cookies, and series_url is
// client-supplied.
Read func(seriesURL string, out *string) (chromedp.Action, bool)
// Done reports whether the payload arrived.
Done func(body string) bool
// Fallback allows the plain-TLS fetcher when no browser is configured.
// False skips the Site instead. kagane is false — a plain fetch would
// only ever retrieve a challenge page — and novelfull is true, because
// its challenge is a live time-varying fact (AGENTS.md).
Fallback bool
}
// asuraSlugRe pulls the series slug out of a stored series_url.
// Shape verified live 2026-07-26: https://asurascans.com/comics/<slug>, where
// the slug carries a trailing build-hash suffix (e.g. "-f886a8af") that
// rotates on every site redeploy — callers must strip it (asuraBuildHash)
// before using the slug to scope anything.
var asuraSlugRe = regexp.MustCompile(`/comics/([^/?#]+)`)
// asuraBuildHash matches the trailing "-xxxxxxxx" site-wide build ID Asura
// appends to every series slug. It rotates on each site redeploy, so it is
// never part of a stable series_id. Must stay in sync with stripBuildHash in
// userscript/manga-bookmark.user.js.
var asuraBuildHash = regexp.MustCompile(`-[0-9a-f]{8}$`)
// demonicChapterRe matches the pre-redirect anchors demonic series pages link
// through. Both the raw "&" and the HTML-escaped "&amp;" forms occur.
var demonicChapterRe = regexp.MustCompile(`chaptered\.php\?manga=\d+&(?:amp;)?chapter=([0-9.]+)`)
// comixSlugRe pulls the "<id>-<slug>" segment out of a stored series_url.
// Only the id prefix is stable; the slug tail follows the title.
var comixSlugRe = regexp.MustCompile(`/title/([^/?#]+)`)
func comixSeriesID(seriesURL string) (string, bool) {
m := comixSlugRe.FindStringSubmatch(seriesURL)
if m == nil {
return "", false
}
id := m[1]
if i := strings.Index(id, "-"); i != -1 {
id = id[:i]
}
return id, true
}
// kaganeChapterRe matches the chapter numbers in a kagane API response. This
// branch is fed by the browser fetcher, so the body is JSON rather than HTML —
// there are no anchors to scan.
var kaganeChapterRe = regexp.MustCompile(`"chapter_no":"([0-9.]+)"`)
// novelfullSlugRe pulls the series slug out of a stored series_url. novelfull
// series pages are "/<slug>.html"; their chapter anchors are
// "/<slug>/chapter-<n>[-<title-slug>].html". Verified live 2026-08-05.
var novelfullSlugRe = regexp.MustCompile(`^/([^/?#]+)\.html$`)
// lnwChapterRe matches any chapter-shaped address on lightnovelworld. Unlike
// asura, novelfull and comix — which scope to their stored series slug so a
// foreign chapter link cannot contribute — this Site's chapter addresses carry
// the Chapter Slug, which is not the Series identity: one Series may publish
// under several Chapter Slugs (measured 2026-08-11: a sampled novel serves
// 1-99 under one slug and 100-423 under another), so no stored-slug pattern can
// cover a Series' whole list. An unscoped match is safe because
// lnwLatestChapter truncates the body at the comment thread before scanning
// (lnwCommentMarker); without that, a visitor's comment could set the Latest
// Chapter on the shared Series row.
var lnwChapterRe = regexp.MustCompile(`lightnovelworld\.net/[a-z0-9-]+-chapter-([0-9.]+)/`)
// lnwCommentMarker is the boundary of lightnovelworld's server-rendered
// wpdiscuz comment thread. It occurs exactly once per page and follows every
// chapter anchor (measured 2026-08-11,
// docs/research/lightnovelworld-chapter-vs-series-slug.md §6), so cutting the
// body at its first occurrence keeps the whole chapter list while excluding a
// region any visitor can write to. Absent means the page shape changed: the
// body is skipped, never scanned whole.
const lnwCommentMarker = "wpd-threads"
// maxChapter returns the highest chapter number the regex finds in body. A
// maximum rather than a first or last, ported from the userscript's
// latestChapterFromAnchors (asura L123-133, demonic L183-193): neither site
// lists chapters in a dependable order.
//
// The userscript's asura rule additionally requires the anchor text to match
// /Chapter\s+[\d.]+/i. That check exists only to skip the "First Chapter"
// shortcut, which points at chapter/1 and therefore can never win a maximum,
// so it is redundant once a maximum is taken.
func maxChapter(re *regexp.Regexp, body string) (latestChapter, bool) {
var best latestChapter
found := false
for _, m := range re.FindAllStringSubmatch(body, -1) {
// [0-9.]+ can swallow a trailing separator, e.g. "chapter/12." in a
// sentence; ParseFloat would reject the whole match.
raw := strings.Trim(m[1], ".")
num, err := strconv.ParseFloat(raw, 64)
if err != nil {
continue
}
if !found || num > best.Num {
best = latestChapter{Num: num, Label: "Chapter " + raw}
found = true
}
}
return best, found
}
// asuraLatestChapter scopes chapter links to this series' own slug, which
// replaces the userscript's anchor-text check with a stronger guarantee: a
// chapter link belonging to some other series cannot contribute even if the
// page starts carrying them.
func asuraLatestChapter(seriesURL, body string) (latestChapter, bool) {
m := asuraSlugRe.FindStringSubmatch(seriesURL)
if m == nil {
return latestChapter{}, false
}
// Stored URLs predating a redeploy may carry a stale build hash; chapter
// hrefs in the fetched body carry the current one. Strip to the stable ID
// and make the hash optional in the pattern, so scoping survives
// rotations.
slug := asuraBuildHash.ReplaceAllString(m[1], "")
// Compiled per call rather than cached: this runs once per fetch, which is
// at most a few times a minute, and the slug varies per series.
re := regexp.MustCompile(`/comics/` + regexp.QuoteMeta(slug) + `(?:-[0-9a-f]{8})?/chapter/([0-9.]+)`)
return maxChapter(re, body)
}
// demonicLatestChapter is not scoped: demonicChapterRe matches any
// chaptered.php?manga=<id> anchor, because the stored series_id is a slug,
// not the numeric id the URL carries, so it cannot be scoped.
func demonicLatestChapter(_, body string) (latestChapter, bool) {
return maxChapter(demonicChapterRe, body)
}
// comixLatestChapter reads comix's SPA: the served HTML carries a JSON state
// blob instead of chapter anchors, and latestChapterUrl is the only place the
// newest chapter appears. Scoping to this series' id prefix keeps a
// "recommended" strip's entries from winning the maximum.
func comixLatestChapter(seriesURL, body string) (latestChapter, bool) {
id, ok := comixSeriesID(seriesURL)
if !ok {
return latestChapter{}, false
}
re := regexp.MustCompile(`"latestChapterUrl":"/title/` + regexp.QuoteMeta(id) + `-[^"]*-chapter-([0-9.]+)"`)
return maxChapter(re, body)
}
// kaganeLatestChapter scans the kagane series API JSON that the browser read
// fetched from inside the page; the match rides on the property name,
// regardless of the surrounding JSON shape.
func kaganeLatestChapter(_, body string) (latestChapter, bool) {
return maxChapter(kaganeChapterRe, body)
}
// novelfullLatestChapter is scoped to this series' slug for the same reason
// asura is: page 1 carries a "latest chapters" widget and a "you may also
// like" strip, and neither may contribute to the maximum.
func novelfullLatestChapter(seriesURL, body string) (latestChapter, bool) {
u, err := url.Parse(seriesURL)
if err != nil {
return latestChapter{}, false
}
m := novelfullSlugRe.FindStringSubmatch(u.Path)
if m == nil {
return latestChapter{}, false
}
re := regexp.MustCompile(`/` + regexp.QuoteMeta(m[1]) + `/chapter-([0-9.]+)`)
return maxChapter(re, body)
}
// lnwLatestChapter truncates the body at the comment thread before scanning:
// it is the one region of the page any visitor can write to (see lnwChapterRe).
// A body without the marker is skipped, never scanned whole — a redesign must
// degrade into staleness, not into a wrong shared value; the logged body length
// tells a markup change from a body the size cap cut short.
func lnwLatestChapter(seriesURL, body string) (latestChapter, bool) {
i := strings.Index(body, lnwCommentMarker)
if i < 0 {
log.Printf("latest poll %q: no %s marker in %d bytes", seriesURL, lnwCommentMarker, len(body))
return latestChapter{}, false
}
return maxChapter(lnwChapterRe, body[:i])
}
// latestChapterFrom returns the highest chapter number body advertises for this
// series, via the Site's registry entry. ok is false when the body yields
// nothing usable — an unknown site, an empty body, a Cloudflare challenge page,
// and a site redesign all land here, and the caller treats all four identically.
func latestChapterFrom(site, seriesURL, body string) (latestChapter, bool) {
if fn := sites[site].LatestChapter; fn != nil {
return fn(seriesURL, body)
}
return latestChapter{}, false
}
var metaTagRe = regexp.MustCompile(`(?is)<meta\b[^>]*>`)
var doubleQuotedMetaAttrRe = regexp.MustCompile(`(?is)([a-z][a-z0-9:_-]*)\s*=\s*"([^"]*)"`)
var singleQuotedMetaAttrRe = regexp.MustCompile(`(?is)([a-z][a-z0-9:_-]*)\s*=\s*'([^']*)'`)
// comix's server-rendered page embeds query data in this JSON script; parsing
// the target detail entry avoids matching posters from recommended results.
var comixInitialDataRe = regexp.MustCompile(`(?is)<script\b[^>]*\bid\s*=\s*["']initial-data["'][^>]*>(.*?)</script>`)
// kaganeImageURLRe matches the canonical compressed image route kagane's API
// publishes — the only cover URL form the extractor emits and the browser
// fetcher accepts. The URL is matched in full (scheme, host, id shape) rather
// than trusted: the value a fetcher is pointed at may have been client-
// supplied, and a headless browser is a strong SSRF primitive.
var kaganeImageURLRe = regexp.MustCompile(`^https://kagane\.to/api/v2/image/([0-9a-f-]{36})/compressed$`)
// browserOnlyCoverURL reports whether the browser sidecar is the only fetcher
// for cover bytes at imageURL. kagane's image route answers a plain fetch with
// a challenge and `cross-origin-resource-policy: same-origin`, so a TLS fetch
// would only ever retrieve a challenge page and must not be attempted
// (ADR-0007). This is the byte-fetch router's per-Site knowledge; it lives in
// the extraction module, which owns kagane's URL shapes.
func browserOnlyCoverURL(imageURL string) bool {
return kaganeImageURLRe.MatchString(imageURL)
}
// kagane's browser-fetched series response publishes cover image IDs under
// series_covers. The API's canonical compressed image route is the only URL
// form accepted by the store and browser fetcher; no rendition is guessed.
func kaganeCoverURL(body string) string {
var response struct {
SeriesCovers []struct {
ImageID string `json:"image_id"`
} `json:"series_covers"`
}
if err := json.Unmarshal([]byte(body), &response); err != nil {
return ""
}
for _, cover := range response.SeriesCovers {
// Validate the assembled URL against the same regex the browser
// fetcher enforces, so the extractor can never emit an address the
// fetch would refuse.
imageURL := "https://kagane.to/api/v2/image/" + cover.ImageID + "/compressed"
if kaganeImageURLRe.MatchString(imageURL) {
return imageURL
}
}
return ""
}
func comixCoverURL(seriesURL, body string) string {
id, ok := comixSeriesID(seriesURL)
if !ok {
return ""
}
data := comixInitialDataRe.FindStringSubmatch(body)
if data == nil {
return ""
}
var state struct {
Queries map[string]json.RawMessage `json:"queries"`
}
if err := json.Unmarshal([]byte(data[1]), &state); err != nil {
return ""
}
raw := state.Queries[`["manga","detail","`+id+`"]`]
if len(raw) == 0 {
return ""
}
var detail struct {
Poster struct {
Medium string `json:"medium"`
} `json:"poster"`
}
if err := json.Unmarshal(raw, &detail); err != nil {
return ""
}
return publishedCoverURL(detail.Poster.Medium)
}
// ogImageCover reads the og:image metadata shared by asura, demonic and
// lightnovelworld.
func ogImageCover(_, body string) (string, bool) {
cover := metaContent(body, "property", "og:image")
return cover, cover != ""
}
func novelfullCoverEntry(_, body string) (string, bool) {
cover := metaContent(body, "name", "image")
return cover, cover != ""
}
func comixCoverEntry(seriesURL, body string) (string, bool) {
cover := comixCoverURL(seriesURL, body)
return cover, cover != ""
}
func kaganeCoverEntry(_, body string) (string, bool) {
cover := kaganeCoverURL(body)
return cover, cover != ""
}
// coverFrom reports false for unknown sites, challenge bodies, and pages with
// no usable cover, via the Site's registry entry.
func coverFrom(site, seriesURL, body string) (string, bool) {
if fn := sites[site].Cover; fn != nil {
return fn(seriesURL, body)
}
return "", false
}
// metaContent returns the content of the first <meta> whose attrName is
// attrValue. It keeps scanning after an empty match so a later published cover
// is not hidden by an empty tag.
func metaContent(body, attrName, attrValue string) string {
for _, tag := range metaTagRe.FindAllString(body, -1) {
attrs := make(map[string]string)
for _, m := range doubleQuotedMetaAttrRe.FindAllStringSubmatch(tag, -1) {
attrs[strings.ToLower(m[1])] = m[2]
}
for _, m := range singleQuotedMetaAttrRe.FindAllStringSubmatch(tag, -1) {
attrs[strings.ToLower(m[1])] = m[2]
}
if strings.EqualFold(attrs[strings.ToLower(attrName)], attrValue) {
if cover := publishedCoverURL(attrs["content"]); cover != "" {
return cover
}
}
}
return ""
}
func publishedCoverURL(value string) string {
value = strings.TrimSpace(html.UnescapeString(value))
return strings.ReplaceAll(value, " ", "%20")
}
// sites is the registry: one entry per Site, keyed by the stored site string.
// Adding a Site means adding an entry here and nowhere else — the dispatch
// functions above and the poller's route list are lookups into this map. An
// unknown site string resolves to the zero entry, which fails the existing
// not-fetchable and no-fetcher paths unchanged.
var sites = map[string]site{
"asura": {
Host: "asurascans.com",
LatestChapter: asuraLatestChapter,
Cover: ogImageCover,
},
"demonic": {
Host: "demonicscans.org",
LatestChapter: demonicLatestChapter,
Cover: ogImageCover,
},
"comix": {
Host: "comix.to",
LatestChapter: comixLatestChapter,
Cover: comixCoverEntry,
},
"kagane": {
Host: "kagane.to",
LatestChapter: kaganeLatestChapter,
Cover: kaganeCoverEntry,
Browser: &browserRead{
Read: kaganeRead,
Done: func(body string) bool { return body != "" },
// Never falls back: a plain fetch of a kagane page or cover would
// only ever retrieve a challenge page (verified 2026-08-03).
Fallback: false,
},
},
"novelfull": {
Host: "novelfull.com",
LatestChapter: novelfullLatestChapter,
Cover: novelfullCoverEntry,
Browser: &browserRead{
Read: novelfullRead,
// The interstitial has a DOM too, so "the payload arrived" has to
// exclude it explicitly.
Done: func(body string) bool { return body != "" && !isInterstitial(body) },
Fallback: true,
},
},
"lightnovelworld": {
Host: "lightnovelworld.net",
LatestChapter: lnwLatestChapter,
Cover: ogImageCover,
},
}
// browserBackedSites is derived from the registry: the Sites whose pages are
// read through the browser sidecar, which are also the ones granted the longer
// cooldown. Sorted so callers that range it (the due query, the browser
// fetcher's dispatch) see a stable order instead of map-iteration noise.
func browserBackedSites() []string {
out := make([]string, 0, len(sites))
for name, s := range sites {
if s.Browser != nil {
out = append(out, name)
}
}
sort.Strings(out)
return out
}
+468
View File
@@ -0,0 +1,468 @@
package latest
import (
"strings"
"testing"
)
// Trimmed from https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af
// fetched 2026-07-26. The first anchor is the "First Chapter" shortcut: it is a
// real chapter link with no "Chapter N" text, and it must not be mistaken for
// the latest just because it parses.
const asuraSeriesFixture = `
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/1" class="py-3 rounded-md bg-[#E8E8E8]"><svg class="w-4 h-4"></svg>First Chapter</a>
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/179" data-astro-prefetch="hover" class="group flex"><span class="font-medium">Chapter 179</span></a>
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/181" data-astro-prefetch="hover" class="group flex"><span class="font-medium">Chapter 181</span></a>
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/180" data-astro-prefetch="hover" class="group flex"><span class="font-medium">Chapter 180</span></a>
`
// A chapter link belonging to a different series, of the kind a "you might also
// like" strip would introduce. Slug scoping must exclude it.
const asuraCrossSeriesFixture = asuraSeriesFixture + `
<a href="/comics/some-other-series-aabbccdd/chapter/999" class="group flex"><span>Chapter 999</span></a>
`
// Trimmed from https://demonicscans.org/manga/Catastrophic-Necromancer fetched
// 2026-07-26. Note the raw "&", the doubled space after <a, and the decimal
// chapters, all as they appear live.
const demonicSeriesFixture = `
<a href="/chaptered.php?manga=11799&chapter=0.5" class="chplinks" title="Catastrophic Necromancer 0.5">Chapter 0.5</a>
<a href="/chaptered.php?manga=11799&chapter=294" class="chplinks" title="Catastrophic Necromancer 294">Chapter 294</a>
<a href="/chaptered.php?manga=11799&amp;chapter=296" class="chplinks" title="Catastrophic Necromancer 296">Chapter 296</a>
<a href="/chaptered.php?manga=11799&chapter=295" class="chplinks" title="Catastrophic Necromancer 295">Chapter 295</a>
`
// What Cloudflare serves instead of the page when an IP's bot score flips.
const challengeFixture = `<!DOCTYPE html><html><head><title>Just a moment...</title>
<script src="/cdn-cgi/challenge-platform/h/b/orchestrate/chl_page/v1"></script></head>
<body><div id="challenge-running">Checking your browser</div></body></html>`
// Trimmed from the server-rendered HTML of
// https://comix.to/title/n8we-dungeons-and-crayons fetched 2026-08-03. comix is
// an SPA: the page ships a JSON state blob rather than a list of chapter
// anchors, and latestChapterUrl is where the newest chapter actually lives.
const comixSeriesFixture = `
{"firstChapterUrl":"/title/n8we-dungeons-and-crayons/5038739-chapter-1","latestChapterUrl":"/title/n8we-dungeons-and-crayons/11139891-chapter-80"},
{""manga","recommended","n8we",1]":{"items":[{"latestChapterUrl":"/title/qqwrm-full-time-awakening/99999999-chapter-999"}]}
`
// The kagane branch is fed by the browser fetcher, so the body is API JSON, not
// HTML. Trimmed from GET /api/v2/series/<uuid> on 2026-08-03.
const kaganeAPIFixture = `
{"series_id":"019f84bc-9ba0-7ed9-86f5-8b905ec7c28b","title":"Infinite Decryption",
"series_books":[{"book_id":"a","title":"Episode 1","chapter_no":"1","sort_no":1},
{"book_id":"b","title":"Episode 41","chapter_no":"41","sort_no":41},
{"book_id":"c","title":"Episode 40.5","chapter_no":"40.5","sort_no":40}]}
`
// Trimmed from https://novelfull.com/reverend-insanity.html fetched 2026-08-05.
// The page carries a newest-first "latest chapters" widget above an
// oldest-first paginated list, so the newest anchor is deliberately NOT last —
// only a maximum finds it. The final anchor belongs to another series and must
// be excluded by slug scoping.
const novelfullSeriesFixture = `
<div class="l-chapters">
<a href="/reverend-insanity/chapter-2334-fang-yuan-and-giant-sun.html">Chapter 2334</a>
<a href="/reverend-insanity/chapter-2333-three-venerables.html">Chapter 2333</a>
</div>
<ul class="list-chapter">
<li><a href="/reverend-insanity/chapter-1.html">Chapter 1</a></li>
<li><a href="/reverend-insanity/chapter-2.html">Chapter 2</a></li>
</ul>
<a href="/release-that-witch/chapter-9999.html">Chapter 9999</a>
`
// Trimmed from
// https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/
// fetched 2026-08-11, whole document (~307 KB decoded; the wire body is ~32 KB
// zstd-compressed). This
// novel publishes its chapters under two Chapter Slugs: 1–99 at
// …-not-chapter-<n>/ and 100–423 at …-not-them-all-chapter-<n>/, and the page
// lists them newest-first, so the …-not-them-all anchors precede the …-not
// anchors. Every anchor through the comment-thread marker is verbatim page
// text (the site renders this novel's chapter titles as "[ ... words ]"). The
// comment block after the marker is the real wpdiscuz comment #wpd-comm-358_0
// from https://lightnovelworld.net/novel/the-sword-illuminates-the-great-wilderness/
// — the pinned page serves zero comments — with its share/link/vote/reply
// boilerplate trimmed. The comment's body carried no link, so the bare <a
// href> to https://lightnovelworld.net/overgeared-chapter-2059/ inside
// wpd-comment-text is the one composed element; that URL is a real chapter of
// a real different novel (overgeared; fetched, HTTP 200).
const lnwSeriesFixture = `
<li data-ID="102741">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-404/">
<div class="epl-num">Vol. 1 Ch. 404</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">April 12, 2026</div>
</a>
</li>
<li data-ID="102780">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-423/">
<div class="epl-num">Vol. 1 Ch. 423</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">April 7, 2026</div>
</a>
</li>
<li data-ID="102527">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-300/">
<div class="epl-num">Vol. 1 Ch. 300</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">April 4, 2026</div>
</a>
</li>
<li data-ID="102325">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-200/">
<div class="epl-num">Vol. 1 Ch. 200</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">March 29, 2026</div>
</a>
</li>
<li data-ID="102121">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all-chapter-100/">
<div class="epl-num">Vol. 1 Ch. 100</div>
<div class="epl-title">[ ... words ]</div>
<div class="epl-date">March 22, 2026</div>
</a>
</li>
<li data-ID="26014">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-chapter-99/">
<div class="epl-num">Vol. 1 Ch. 99</div>
<div class="epl-title">Chapter 99</div>
<div class="epl-date">November 5, 2025</div>
</a>
</li>
<li data-ID="25916">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-chapter-50/">
<div class="epl-num">Vol. 1 Ch. 50</div>
<div class="epl-title">Chapter 50</div>
<div class="epl-date">October 29, 2025</div>
</a>
</li>
<li class='tseplsfrst' data-ID="25818">
<a href="https://lightnovelworld.net/all-jobs-and-classes-i-just-wanted-one-skill-not-chapter-1/">
<div class="epl-num">Vol. 1 Ch. 1</div>
<div class="epl-title">Chapter 01</div>
<div class="epl-date">October 11, 2025</div>
</a>
</li>
<div id="wpd-threads" class="wpd-thread-wrapper">
<div class="wpd-thread-list">
<div id='wpd-comm-358_0' class='comment byuser comment-author-jimbear even thread-even depth-1 wpd-comment wpd_comment_level-1'><div class="wpd-comment-wrap wpd-blog-user wpd-blog-subscriber">
<div class="wpd-comment-left ">
<div class="wpd-avatar ">
<img alt='hasbi asy' src='https://secure.gravatar.com/avatar/3c1792cb31cab842f90e7c463f0948e98536cf938aebbd2f678f937ca60fb799?s=64&#038;d=mm&#038;r=g' srcset='https://secure.gravatar.com/avatar/3c1792cb31cab842f90e7c463f0948e98536cf938aebbd2f678f937ca60fb799?s=128&#038;d=mm&#038;r=g 2x' class='avatar avatar-64 photo' height='64' width='64' decoding='async'/>
</div>
<div class="wpd-comment-label" wpd-tooltip="Member" wpd-tooltip-position="right">
<span>Member</span>
</div>
</div>
<div id="comment-358" class="wpd-comment-right">
<div class="wpd-comment-header">
<div class="wpd-comment-author ">
hasbi asy
</div>
<div class="wpd-comment-date" title="July 9, 2026 1:44 am">
<i class='far fa-clock' aria-hidden='true'></i>
1 month ago
</div>
</div>
<div class="wpd-comment-text">
<p>where&#8217;s everyone</p>
<a href="https://lightnovelworld.net/overgeared-chapter-2059/">https://lightnovelworld.net/overgeared-chapter-2059/</a>
</div>
</div>
</div>
<div id='wpdiscuz_form_anchor-358_0'></div>
</div>
</div>
`
// Trimmed from https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af
// (redirected to ...-00dcbf97) on 2026-08-10.
const asuraCoverFixture = `<meta property="og:image" content="https://cdn.asurascans.com/asura-images/covers/chronicles-of-the-demon-faction.d4dcb8.webp">`
// Trimmed from https://demonicscans.org/manga/Catastrophic-Necromancer on 2026-08-10.
// The source publishes the raw space in this URL.
const demonicCoverFixture = `<meta property="og:image" content="https://readermc.org/images/thumbnails/Catastrophic Necromancer.webp">`
// Trimmed from https://comix.to/title/n8we-dungeons-and-crayons on 2026-08-10.
// The state includes a recommended poster before the target detail object and
// nested IDs inside that object; no og:image is present.
const comixCoverFixture = `<script type="application/json" id="initial-data">{"queries":{"[\"manga\",\"recommended\",\"n8we\",1]":{"poster":{"medium":"https://static.comix.to/recommended@280.jpg","large":"https://static.comix.to/recommended.jpg"}},"[\"manga\",\"detail\",\"n8we\"]":{"chapters":[{"hid":"nested"}],"poster":{"medium":"https://static.comix.to/039d/i/1/34/6a6742bf15736@280.jpg","large":"https://static.comix.to/039d/i/1/34/6a6742bf15736.jpg"}}}}</script>`
// Trimmed from GET https://kagane.to/api/v2/series/019fe11a-8670-7cf3-8343-0b02057d3787 on 2026-08-10.
const kaganeCoverFixture = `{"series_covers":[{"cover_id":"019fe11a-84d1-714b-9cf4-2827f277f3c0","language":"en","volume_number":"1","chapter_number":null,"note":null,"image_id":"019fe11a-84c3-7fc3-a84b-88787374b617"}]}`
// Trimmed from https://novelfull.com/reverend-insanity.html on 2026-08-10.
const novelfullCoverFixture = `<meta name="image" content="https://novelfull.com/uploads/webp/novel/reverend-insanity-82661d911a.webp">`
// Trimmed from
// https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/
// on 2026-08-11.
const lnwCoverFixture = `<meta property="og:image" content="https://i1.wp.com/lightnovelworld.net/wp-content/uploads/2025/10/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all.jpg" />`
func TestCoverFrom(t *testing.T) {
const comixURL = "https://comix.to/title/n8we-dungeons-and-crayons"
tests := []struct {
name string
site string
seriesURL string
body string
wantOK bool
wantCover string
}{
{
name: "asura uses published metadata URL",
site: "asura", body: asuraCoverFixture, wantOK: true,
wantCover: "https://cdn.asurascans.com/asura-images/covers/chronicles-of-the-demon-faction.d4dcb8.webp",
},
{
name: "demonic escapes raw spaces",
site: "demonic", body: demonicCoverFixture, wantOK: true,
wantCover: "https://readermc.org/images/thumbnails/Catastrophic%20Necromancer.webp",
},
{
name: "comix takes target medium poster",
site: "comix", seriesURL: comixURL, body: comixCoverFixture, wantOK: true,
wantCover: "https://static.comix.to/039d/i/1/34/6a6742bf15736@280.jpg",
},
{
name: "kagane reads API cover image ID",
site: "kagane", body: kaganeCoverFixture, wantOK: true,
wantCover: "https://kagane.to/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed",
},
{
name: "novelfull reads image metadata",
site: "novelfull", body: novelfullCoverFixture, wantOK: true,
wantCover: "https://novelfull.com/uploads/webp/novel/reverend-insanity-82661d911a.webp",
},
{
name: "lightnovelworld reads og image",
site: "lightnovelworld", body: lnwCoverFixture, wantOK: true,
wantCover: "https://i1.wp.com/lightnovelworld.net/wp-content/uploads/2025/10/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all.jpg",
},
{
name: "later metadata cover survives empty match",
site: "asura",
body: `<meta property="og:image" content="">` + asuraCoverFixture,
wantOK: true,
wantCover: "https://cdn.asurascans.com/asura-images/covers/chronicles-of-the-demon-faction.d4dcb8.webp",
},
{
name: "page without cover is empty",
site: "asura", body: `<meta property="og:title" content="No Cover">`,
},
{
name: "unknown site is empty",
site: "unknown", body: asuraCoverFixture,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, ok := coverFrom(tt.site, tt.seriesURL, tt.body)
if ok != tt.wantOK {
t.Fatalf("ok = %v, want %v (got %q)", ok, tt.wantOK, got)
}
if got != tt.wantCover {
t.Errorf("cover = %q, want %q", got, tt.wantCover)
}
})
}
}
func TestCoverFromChallenge(t *testing.T) {
tests := []struct {
site string
seriesURL string
}{
{"asura", "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"},
{"demonic", "https://demonicscans.org/manga/Catastrophic-Necromancer"},
{"comix", "https://comix.to/title/n8we-dungeons-and-crayons"},
{"kagane", "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"},
{"novelfull", "https://novelfull.com/reverend-insanity.html"},
{"lightnovelworld", "https://lightnovelworld.net/novel/a-will-eternal/"},
}
for _, tt := range tests {
t.Run(tt.site, func(t *testing.T) {
if got, ok := coverFrom(tt.site, tt.seriesURL, challengeFixture); ok || got != "" {
t.Fatalf("cover = %q, ok = %v, want empty", got, ok)
}
})
}
}
func TestLatestChapterFrom(t *testing.T) {
const asuraURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
const demonicURL = "https://demonicscans.org/manga/Catastrophic-Necromancer"
tests := []struct {
name string
site string
seriesURL string
body string
wantOK bool
wantNum float64
wantLabel string
}{
{
name: "asura takes the max, not the last listed",
site: "asura", seriesURL: asuraURL, body: asuraSeriesFixture,
wantOK: true, wantNum: 181, wantLabel: "Chapter 181",
},
{
name: "asura ignores another series' chapter links",
site: "asura", seriesURL: asuraURL, body: asuraCrossSeriesFixture,
wantOK: true, wantNum: 181, wantLabel: "Chapter 181",
},
{
name: "asura scoping survives a build-hash rotation",
site: "asura",
seriesURL: "https://asurascans.com/comics/chronicles-of-the-demon-faction-059befe1",
body: asuraCrossSeriesFixture,
wantOK: true, wantNum: 181, wantLabel: "Chapter 181",
},
{
name: "asura with an unparseable series url",
site: "asura", seriesURL: "https://asurascans.com/", body: asuraSeriesFixture,
wantOK: false,
},
{
name: "demonic takes the max across raw and escaped ampersands",
site: "demonic", seriesURL: demonicURL, body: demonicSeriesFixture,
wantOK: true, wantNum: 296, wantLabel: "Chapter 296",
},
{
name: "demonic keeps decimal chapters parseable",
site: "demonic", seriesURL: demonicURL,
body: `<a href="/chaptered.php?manga=11799&chapter=0.5">Chapter 0.5</a>`,
wantOK: true, wantNum: 0.5, wantLabel: "Chapter 0.5",
},
{
name: "empty body",
site: "asura", seriesURL: asuraURL, body: "",
wantOK: false,
},
{
name: "cloudflare challenge page",
site: "asura", seriesURL: asuraURL, body: challengeFixture,
wantOK: false,
},
{
name: "demonic markup handed to the asura rule",
site: "asura", seriesURL: asuraURL, body: demonicSeriesFixture,
wantOK: false,
},
{
name: "unknown site",
site: "mangadex", seriesURL: "https://example.com/x", body: asuraSeriesFixture,
wantOK: false,
},
{
name: "comix reads latestChapterUrl, scoped to this series",
site: "comix",
seriesURL: "https://comix.to/title/n8we-dungeons-and-crayons",
body: comixSeriesFixture,
wantOK: true, wantNum: 80, wantLabel: "Chapter 80",
},
{
name: "comix yields nothing on a challenge page",
site: "comix",
seriesURL: "https://comix.to/title/n8we-dungeons-and-crayons",
body: challengeFixture,
wantOK: false,
},
{
name: "kagane takes the max chapter_no from API json",
site: "kagane",
seriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b",
body: kaganeAPIFixture,
wantOK: true, wantNum: 41, wantLabel: "Chapter 41",
},
{
name: "kagane yields nothing on a challenge page",
site: "kagane",
seriesURL: "https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b",
body: challengeFixture,
wantOK: false,
},
{
name: "novelfull takes the max and ignores another series",
site: "novelfull",
seriesURL: "https://novelfull.com/reverend-insanity.html",
body: novelfullSeriesFixture,
wantOK: true, wantNum: 2334, wantLabel: "Chapter 2334",
},
{
name: "novelfull yields nothing on a challenge page",
site: "novelfull",
seriesURL: "https://novelfull.com/reverend-insanity.html",
body: challengeFixture,
wantOK: false,
},
{
name: "novelfull with an unparseable series url",
site: "novelfull",
seriesURL: "https://novelfull.com/genre/Fantasy",
body: novelfullSeriesFixture,
wantOK: false,
},
// Stored before the slug split, so the address carries the ...-not
// Chapter Slug; the 100-423 block under the other slug must still win.
// The body is the chapter-list portion of lnwSeriesFixture with the
// comment block omitted; the marker is kept, because a body without it
// is skipped, not scanned.
{
name: "lightnovelworld max spans both chapter slugs",
site: "lightnovelworld",
seriesURL: "https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not/",
body: strings.SplitN(lnwSeriesFixture, lnwCommentMarker, 2)[0] + lnwCommentMarker,
wantOK: true, wantNum: 423, wantLabel: "Chapter 423",
},
{
name: "lightnovelworld comment anchor cannot set the latest chapter",
site: "lightnovelworld",
seriesURL: "https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/",
body: lnwSeriesFixture,
wantOK: true, wantNum: 423, wantLabel: "Chapter 423",
},
// Marker removed from the fixture, comment block still present: a
// redesign must degrade into a skip, never into the comment's number.
{
name: "lightnovelworld body without the comment marker is skipped",
site: "lightnovelworld",
seriesURL: "https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/",
body: strings.ReplaceAll(lnwSeriesFixture, lnwCommentMarker, ""),
wantOK: false,
},
{
name: "lightnovelworld yields nothing on a challenge page",
site: "lightnovelworld",
seriesURL: "https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/",
body: challengeFixture,
wantOK: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, ok := latestChapterFrom(tt.site, tt.seriesURL, tt.body)
if ok != tt.wantOK {
t.Fatalf("ok = %v, want %v (got %+v)", ok, tt.wantOK, got)
}
if !tt.wantOK {
return
}
if got.Num != tt.wantNum {
t.Errorf("Num = %v, want %v", got.Num, tt.wantNum)
}
if got.Label != tt.wantLabel {
t.Errorf("Label = %q, want %q", got.Label, tt.wantLabel)
}
})
}
}
+157
View File
@@ -0,0 +1,157 @@
package latest
import (
"context"
"net/http"
"os"
"testing"
"time"
"bookmarkmanager/backend/internal/store"
)
// TestSmokeKaganeImage is the live proof that the acquisition path's browser
// fetch actually clears Cloudflare and returns image bytes. It needs the real
// browser unit with outbound network, so it runs only when SMOKE_BROWSER_WS_URL
// is set:
//
// cd chrome && BROWSER_BIND_ADDR=127.0.0.1 docker compose up -d --build
// SMOKE_BROWSER_WS_URL=ws://127.0.0.1:9222 go test -run TestSmokeKaganeImage ./internal/latest
//
// Not chromedp/headless-shell: its challenge never clears (see chrome/Dockerfile),
// so a red run there proves nothing about kagane.
func TestSmokeKaganeImage(t *testing.T) {
ws := os.Getenv("SMOKE_BROWSER_WS_URL")
if ws == "" {
t.Skip("SMOKE_BROWSER_WS_URL unset")
}
const imageURL = "https://kagane.to/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed" // SP Baby's cover
// The same URL through a plain client is what any other fetcher would get.
// Asserting on it keeps the test honest about why the browser is needed.
req, err := http.NewRequest(http.MethodGet, imageURL, nil)
if err != nil {
t.Fatal(err)
}
if res, err := (&http.Client{Timeout: 15 * time.Second}).Do(req); err == nil {
res.Body.Close()
if res.StatusCode == http.StatusOK {
t.Log("note: kagane answered a plain request 200 — the challenge is not up right now")
}
}
f, err := NewBrowserFetcher(ws)
if err != nil {
t.Fatalf("NewBrowserFetcher: %v", err)
}
defer f.Close()
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
body, contentType, err := f.Image(ctx, imageURL)
if err != nil {
t.Fatalf("Image: %v", err)
}
if len(body) < 1000 {
t.Fatalf("body is %d bytes, want a real image", len(body))
}
if contentType != "image/webp" {
t.Fatalf("content type = %q, want image/webp", contentType)
}
// WebP files start with "RIFF....WEBP".
if string(body[:4]) != "RIFF" || string(body[8:12]) != "WEBP" {
t.Fatalf("body is not a WebP: % x", body[:12])
}
t.Logf("fetched %d bytes of %s", len(body), contentType)
// The browser module claims only the cover URL shape it can clear a
// challenge for; anything else must be refused before any navigation.
if _, _, err := f.Image(ctx, "https://cdn.example/cover.jpg"); err == nil {
t.Fatal("Image accepted a cover URL the browser module does not claim")
}
}
// Control for the test above: the poller's own kagane path, same sidecar. If
// this fails too, the sidecar is not clearing the challenge at all and the
// image result says nothing about Image itself.
func TestSmokeKaganeGet(t *testing.T) {
ws := os.Getenv("SMOKE_BROWSER_WS_URL")
if ws == "" {
t.Skip("SMOKE_BROWSER_WS_URL unset")
}
f, err := NewBrowserFetcher(ws)
if err != nil {
t.Fatalf("NewBrowserFetcher: %v", err)
}
defer f.Close()
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
body, status, err := f.Get(ctx, "https://kagane.to/series/019fe11a-8670-7cf3-8343-0b02057d3787")
if err != nil {
t.Fatalf("Get: %v", err)
}
t.Logf("status=%d bytes=%d head=%.80q", status, len(body), body)
if status != 200 {
t.Fatalf("status = %d, want 200 — the sidecar is not clearing the challenge", status)
}
}
// TestSmokeAcquireKaganeCover proves the #62 acquisition path end to end
// against the real browser: a kagane Series bookmarked at creation gets its
// Cover, bytes fetched through the sidecar into the content-addressed store.
// Same SMOKE_BROWSER_WS_URL gate as the tests above; a red run means the
// challenge is not clearing from this IP (a live fact to re-check), not
// necessarily a defect in the pipeline.
func TestSmokeAcquireKaganeCover(t *testing.T) {
ws := os.Getenv("SMOKE_BROWSER_WS_URL")
if ws == "" {
t.Skip("SMOKE_BROWSER_WS_URL unset")
}
const (
seriesID = "019fe11a-8670-7cf3-8343-0b02057d3787"
coverURL = "https://kagane.to/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed"
)
s, _ := newTestStore(t)
bf, err := NewBrowserFetcher(ws)
if err != nil {
t.Fatalf("NewBrowserFetcher: %v", err)
}
defer bf.Close()
tlsF, err := NewTLSFetcher()
if err != nil {
t.Fatalf("NewTLSFetcher: %v", err)
}
acq := &Acquirer{
Store: s, Fetch: tlsF, BrowserFetch: bf,
BrowserCoverFetch: bf, Covers: NewCoverFetcher(),
}
s.OnSeriesCreated = acq.Acquire
if _, err := s.Upsert(s.OwnerID(), store.Bookmark{
Key: "kagane:" + seriesID, Site: "kagane", SeriesID: seriesID,
Title: "smoke", SeriesURL: "https://kagane.to/series/" + seriesID, UpdatedAt: 1000,
}); err != nil {
t.Fatalf("Upsert: %v", err)
}
acq.Wait()
got, found, err := s.Get(s.OwnerID(), "kagane:"+seriesID)
if err != nil || !found {
t.Fatalf("Get: %v found=%v", err, found)
}
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(coverURL); got.Cover != want {
t.Fatalf("Cover = %q, want %q — the acquire path did not store the browser-fetched bytes", got.Cover, want)
}
body, contentType, ok, err := s.CoverByAddress(store.CoverAddress(coverURL))
if err != nil || !ok {
t.Fatalf("CoverByAddress: %v found=%v", err, ok)
}
if len(body) < 1000 {
t.Fatalf("stored cover is %d bytes, want a real image", len(body))
}
if contentType != "image/webp" {
t.Fatalf("content type = %q, want image/webp", contentType)
}
t.Logf("stored %d bytes of %s", len(body), contentType)
}
+101
View File
@@ -0,0 +1,101 @@
package latest
import (
"context"
"fmt"
"net/http"
"os"
"strings"
"testing"
"time"
)
// lnwSeriesPageFloor is the smallest body that can still be a whole
// lightnovelworld Series page. Whole pages measured 685 KB..1.18 MB on
// 2026-08-11 and carry the marker at ~94% of the document, so a body under
// 100 KB is a challenge, a notice, or a truncated read — asserting on it
// would report the marker missing when it was never fetched.
const lnwSeriesPageFloor = 100 << 10
// TestSmokeLnwCommentBoundary is the live proof that the comment-thread marker
// the lightnovelworld chapter scan truncates at (lnwCommentMarker,
// "wpd-threads") still holds on the Site. The scan depends on it: when the
// marker vanishes every Series is skipped and logged — correct, but silent
// until a Reader notices their Latest Chapter has stopped moving. It runs only
// when SMOKE_LNW_SERIES_URL is set — the URL of the live Series page to check.
// The immortality-simulator page measured 2026-08-11
// (docs/research/lightnovelworld-chapter-vs-series-slug.md) is the default to
// point it at:
//
// SMOKE_LNW_SERIES_URL=https://lightnovelworld.net/novel/immortality-simulator/ go test -v -run TestSmokeLnwCommentBoundary ./internal/latest
//
// A red run means the Site's markup has moved — the marker is gone, occurs
// more than once, or no longer follows the last chapter anchor — and the scan
// in sites.go is now skipping this Site. Revisit sites.go before anything
// else; the test is not flaky. A Cloudflare challenge or a non-200 is
// distinguished from a marker failure by the "not a marker failure" messages
// below, which carry the observed status and body length.
func TestSmokeLnwCommentBoundary(t *testing.T) {
seriesURL := os.Getenv("SMOKE_LNW_SERIES_URL")
if seriesURL == "" {
t.Skip("SMOKE_LNW_SERIES_URL unset")
}
if !fetchableSeriesURL("lightnovelworld", seriesURL) {
t.Fatalf("%q is not a fetchable lightnovelworld series URL", seriesURL)
}
f, err := NewTLSFetcher()
if err != nil {
t.Fatalf("NewTLSFetcher: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
body, status, err := f.Get(ctx, seriesURL)
if err != nil {
t.Fatalf("Get: %v", err)
}
if status != http.StatusOK {
t.Fatalf("status = %d, body %d bytes — not a marker failure; the Site did not answer this IP with a Series page", status, len(body))
}
if len(body) < lnwSeriesPageFloor {
t.Fatalf("body %d bytes — not a whole Series page (measured 685 KB..1.18 MB); not a marker failure, likely a Cloudflare challenge or a non-Series response", len(body))
}
if !lnwChapterRe.MatchString(body) {
t.Fatalf("no chapter anchor in %d bytes — not a lightnovelworld Series page; not a marker failure, likely a Cloudflare challenge or a non-Series response", len(body))
}
markerIdx := strings.Index(body, lnwCommentMarker)
if failures := checkLnwCommentBoundary(body); len(failures) > 0 {
t.Fatalf("%s (body %d bytes)", strings.Join(failures, "; "), len(body))
}
t.Logf("ok: %q once at byte %d, body %d bytes", lnwCommentMarker, markerIdx, len(body))
}
// checkLnwCommentBoundary verifies the three marker assertions against a
// fetched Series body: the marker occurs exactly once, every chapter anchor
// precedes it, and at least one anchor precedes it at all. It returns one
// human-readable failure per broken assertion — with observed offsets and body
// length — and empty when the page is healthy.
func checkLnwCommentBoundary(body string) []string {
markerIdx := strings.Index(body, lnwCommentMarker)
switch n := strings.Count(body, lnwCommentMarker); {
case n == 0:
return []string{fmt.Sprintf("%q occurs 0 times in %d bytes, want exactly 1", lnwCommentMarker, len(body))}
case n != 1:
return []string{fmt.Sprintf("%q occurs %d times in %d bytes (first at byte %d), want exactly 1", lnwCommentMarker, n, len(body), markerIdx)}
}
lastAnchor, anchorsBefore := -1, 0
for _, m := range lnwChapterRe.FindAllStringIndex(body, -1) {
if m[0] < markerIdx {
anchorsBefore++
}
lastAnchor = m[0]
}
var failures []string
if lastAnchor >= markerIdx {
failures = append(failures, fmt.Sprintf("last chapter anchor at byte %d does not precede the marker at byte %d", lastAnchor, markerIdx))
}
if anchorsBefore == 0 {
failures = append(failures, fmt.Sprintf("no chapter anchor before the marker at byte %d — the truncated prefix the scan sees yields nothing", markerIdx))
}
return failures
}
+120
View File
@@ -0,0 +1,120 @@
// Package pgtest runs the Postgres the test suite needs: one throwaway
// container per test binary, one fresh database per test. Docker is therefore
// a hard prerequisite for `go test ./...`.
//
// Rolled by hand rather than pulled in as a dependency — it is one `docker
// run`, one `docker port` and a ping loop, against a module list that is
// otherwise stdlib plus what the poller genuinely needs.
package pgtest
import (
"database/sql"
"fmt"
"os/exec"
"strconv"
"strings"
"sync/atomic"
"testing"
"time"
_ "github.com/jackc/pgx/v5/stdlib"
)
const (
image = "postgres:17-alpine"
readyLimit = 60 * time.Second
)
var (
adminURL string
dbSeq atomic.Int64
)
// Main starts the container, runs the package's tests and tears the container
// down. Every test package that touches the store calls it from TestMain:
//
// func TestMain(m *testing.M) { os.Exit(pgtest.Main(m)) }
func Main(m *testing.M) int {
id, url, err := start()
if err != nil {
fmt.Println("pgtest:", err)
return 1
}
defer exec.Command("docker", "rm", "-f", id).Run()
adminURL = url
return m.Run()
}
// URL creates a database of its own for t and returns a connection URL for it.
// Nothing drops it again: the container goes away wholesale when Main returns.
func URL(t testing.TB) string {
t.Helper()
if adminURL == "" {
t.Fatal("pgtest: no container; this package needs TestMain to call pgtest.Main")
}
// Generated, never derived from the test name, so it needs no quoting and
// cannot collide when tests run in parallel.
name := "test_" + strconv.FormatInt(dbSeq.Add(1), 10)
admin, err := sql.Open("pgx", adminURL)
if err != nil {
t.Fatalf("pgtest: open admin connection: %v", err)
}
defer admin.Close()
if _, err := admin.Exec(`CREATE DATABASE ` + name); err != nil {
t.Fatalf("pgtest: create database %s: %v", name, err)
}
return strings.Replace(adminURL, "/postgres?", "/"+name+"?", 1)
}
// start launches the container and waits for it to accept queries, returning
// its id and a connection URL for the default database.
func start() (id, url string, err error) {
out, err := exec.Command("docker", "run", "-d", "--rm",
"-e", "POSTGRES_PASSWORD=pgtest",
"-P", image,
// Durability buys nothing for a database that dies with the test
// binary, and turning it off is most of the container's start-up cost.
"-c", "fsync=off", "-c", "full_page_writes=off",
).Output()
if err != nil {
return "", "", fmt.Errorf("docker run %s: %w", image, err)
}
id = strings.TrimSpace(string(out))
port, err := exec.Command("docker", "port", id, "5432/tcp").Output()
if err != nil {
exec.Command("docker", "rm", "-f", id).Run()
return "", "", fmt.Errorf("docker port: %w", err)
}
// "0.0.0.0:32768" (and possibly a second, IPv6 line); the port is all we want.
first, _, _ := strings.Cut(strings.TrimSpace(string(port)), "\n")
url = fmt.Sprintf("postgres://postgres:pgtest@127.0.0.1:%s/postgres?sslmode=disable",
first[strings.LastIndex(first, ":")+1:])
if err := waitReady(url); err != nil {
exec.Command("docker", "rm", "-f", id).Run()
return "", "", err
}
return id, url, nil
}
func waitReady(url string) error {
db, err := sql.Open("pgx", url)
if err != nil {
return err
}
defer db.Close()
deadline := time.Now().Add(readyLimit)
for {
if err = db.Ping(); err == nil {
return nil
}
if time.Now().After(deadline) {
return fmt.Errorf("postgres not ready after %s: %w", readyLimit, err)
}
time.Sleep(200 * time.Millisecond)
}
}
@@ -1,64 +1,30 @@
package main
package session
import (
"crypto/hmac"
"crypto/sha256"
"crypto/subtle"
"encoding/base64"
"crypto/rand"
"encoding/hex"
"net"
"net/http"
"strconv"
"strings"
"sync"
"time"
)
const (
sessionCookieName = "mangabm_session"
CookieName = "bmgr_session"
// 60 days: long enough that a phone stays logged in between reading spells.
sessionTTL = 60 * 24 * time.Hour
// Domain separation, so the session key can never collide with any other
// use of the secrets it is derived from. Changing this string logs
// everyone out.
sessionKeyPurpose = "mangabm-web-session-v1"
SessionTTL = 60 * 24 * time.Hour
)
// sessionKey derives the cookie-signing key from both secrets. Sessions are
// stateless — there is no session table — so rotating either API_TOKEN or
// WEB_PASSWORD invalidates every outstanding cookie at once. The \x00
// separator prevents the concatenation ambiguity a bare apiToken+webPassword
// would have (e.g. "ab"+"c" colliding with "a"+"bc").
func sessionKey(apiToken, webPassword string) []byte {
sum := sha256.Sum256([]byte(apiToken + "\x00" + webPassword + sessionKeyPurpose))
return sum[:]
// NewID returns an opaque session id: 32 random bytes, hex-encoded. The id is
// all the cookie carries and all the sessions table keys on, so its entropy is
// what stops a guessed id from being someone else's session.
func NewID() string {
var b [32]byte
if _, err := rand.Read(b[:]); err != nil {
panic("session id: " + err.Error())
}
// signSession encodes "<expiryMs>.<base64url HMAC(expiryMs)>".
func signSession(key []byte, expiryMs int64) string {
payload := strconv.FormatInt(expiryMs, 10)
return payload + "." + sessionMAC(key, payload)
}
func sessionMAC(key []byte, payload string) string {
mac := hmac.New(sha256.New, key)
mac.Write([]byte(payload))
return base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
}
// verifySession checks shape, then expiry, then the signature — in that order.
// The signature comparison is constant-time; the checks before it only look at
// data the holder already supplied, so their timing leaks nothing.
func verifySession(key []byte, value string, nowMs int64) bool {
payload, sig, ok := strings.Cut(value, ".")
if !ok {
return false
}
expiry, err := strconv.ParseInt(payload, 10, 64)
if err != nil || expiry <= nowMs {
return false
}
want := sessionMAC(key, payload)
return subtle.ConstantTimeCompare([]byte(sig), []byte(want)) == 1
return hex.EncodeToString(b[:])
}
// isHTTPS reports whether the browser's connection is encrypted. Behind Traefik
@@ -69,21 +35,23 @@ func isHTTPS(r *http.Request) bool {
return r.TLS != nil || r.Header.Get("X-Forwarded-Proto") == "https"
}
func setSessionCookie(w http.ResponseWriter, r *http.Request, key []byte) {
// SetCookie writes the session cookie. The value is the session id and nothing
// else; the row behind it is looked up on every request.
func SetCookie(w http.ResponseWriter, r *http.Request, id string) {
http.SetCookie(w, &http.Cookie{
Name: sessionCookieName,
Value: signSession(key, time.Now().Add(sessionTTL).UnixMilli()),
Name: CookieName,
Value: id,
Path: "/",
MaxAge: int(sessionTTL / time.Second),
MaxAge: int(SessionTTL / time.Second),
HttpOnly: true,
Secure: isHTTPS(r),
SameSite: http.SameSiteLaxMode,
})
}
func clearSessionCookie(w http.ResponseWriter, r *http.Request) {
func ClearCookie(w http.ResponseWriter, r *http.Request) {
http.SetCookie(w, &http.Cookie{
Name: sessionCookieName,
Name: CookieName,
Value: "",
Path: "/",
MaxAge: -1,
@@ -94,11 +62,11 @@ func clearSessionCookie(w http.ResponseWriter, r *http.Request) {
}
const (
loginMaxFailures = 10
loginWindow = 20 * time.Minute
MaxFailures = 10
Window = 20 * time.Minute
)
// clientIP returns the address the reverse proxy actually observed.
// ClientIP returns the address the reverse proxy actually observed.
//
// Traefik appends the peer address to whatever X-Forwarded-For the client sent,
// so the leftmost entry is attacker-controlled and the rightmost is not. Go's
@@ -106,7 +74,7 @@ const (
// by sending its own; Values covers every line so the true last hop is found.
// RemoteAddr is useless behind the proxy — it is always the Traefik container —
// so it serves only as the direct-connection fallback for local development.
func clientIP(r *http.Request) string {
func ClientIP(r *http.Request) string {
if vals := r.Header.Values("X-Forwarded-For"); len(vals) > 0 {
hops := strings.Split(vals[len(vals)-1], ",")
if ip := strings.TrimSpace(hops[len(hops)-1]); ip != "" {
@@ -120,46 +88,46 @@ func clientIP(r *http.Request) string {
return host
}
// loginLimiter throttles password guessing: loginMaxFailures failures inside a
// rolling loginWindow blocks further attempts from that IP until the oldest one
// LoginLimiter throttles failed sign-in attempts: MaxFailures failures inside
// a rolling Window blocks further attempts from that IP until the oldest one
// ages out. There is no permanent ban and no unlock step.
//
// Behind carrier-grade NAT this budget is shared with every other subscriber on
// the same public address, so a stranger can lock the owner out for up to one
// window. That is accepted: the block self-heals, and ten attempts is generous
// for a mistyped password.
// for the occasional fumbled sign-in.
//
// State is in memory and per-process, so a restart clears it. Entries are
// pruned lazily on access; for a single-user deployment the map cannot grow
// past the handful of addresses that ever attempt a login.
type loginLimiter struct {
type LoginLimiter struct {
mu sync.Mutex
failures map[string][]time.Time
}
func newLoginLimiter() *loginLimiter {
return &loginLimiter{failures: make(map[string][]time.Time)}
func NewLoginLimiter() *LoginLimiter {
return &LoginLimiter{failures: make(map[string][]time.Time)}
}
// retryAfter returns how long ip must wait, or zero when it may try now.
func (l *loginLimiter) retryAfter(ip string, now time.Time) time.Duration {
func (l *LoginLimiter) RetryAfter(ip string, now time.Time) time.Duration {
l.mu.Lock()
defer l.mu.Unlock()
recent := l.pruneLocked(ip, now)
if len(recent) < loginMaxFailures {
if len(recent) < MaxFailures {
return 0
}
return recent[0].Add(loginWindow).Sub(now)
return recent[0].Add(Window).Sub(now)
}
func (l *loginLimiter) fail(ip string, now time.Time) {
func (l *LoginLimiter) Fail(ip string, now time.Time) {
l.mu.Lock()
defer l.mu.Unlock()
l.failures[ip] = append(l.pruneLocked(ip, now), now)
}
func (l *loginLimiter) reset(ip string) {
func (l *LoginLimiter) Reset(ip string) {
l.mu.Lock()
defer l.mu.Unlock()
delete(l.failures, ip)
@@ -167,8 +135,8 @@ func (l *loginLimiter) reset(ip string) {
// pruneLocked drops attempts older than the window and returns what is left.
// The caller must hold l.mu.
func (l *loginLimiter) pruneLocked(ip string, now time.Time) []time.Time {
cutoff := now.Add(-loginWindow)
func (l *LoginLimiter) pruneLocked(ip string, now time.Time) []time.Time {
cutoff := now.Add(-Window)
// In-place filter: kept reuses the backing array of the slice being
// ranged over. Safe to alias because append writes at index len(kept),
// which is always <= the range index i, and element i is read before
@@ -1,4 +1,4 @@
package main
package session
import (
"crypto/tls"
@@ -9,66 +9,19 @@ import (
"time"
)
func TestSessionRoundTrip(t *testing.T) {
key := sessionKey("token-abc", "pw-abc")
now := time.Now().UnixMilli()
value := signSession(key, now+60_000)
if !verifySession(key, value, now) {
t.Fatal("verifySession = false for a freshly signed cookie, want true")
func TestNewID(t *testing.T) {
a := NewID()
b := NewID()
if a == b {
t.Fatal("NewID returned the same value twice")
}
if len(a) != 64 { // 32 random bytes, hex
t.Fatalf("NewID() length = %d, want 64", len(a))
}
func TestSessionRejects(t *testing.T) {
key := sessionKey("token-abc", "pw-abc")
now := time.Now().UnixMilli()
valid := signSession(key, now+60_000)
payload, sig, _ := strings.Cut(valid, ".")
cases := []struct {
name string
value string
}{
{"empty", ""},
{"no separator", payload + sig},
{"unparseable expiry", "notanumber." + sig},
{"expired", signSession(key, now-1)},
{"tampered signature", payload + "." + flipLastChar(sig)},
{"tampered expiry", "99999999999999." + sig},
{"signed with another key", signSession(sessionKey("other-token", "pw-abc"), now+60_000)},
for _, r := range a {
if !strings.ContainsRune("0123456789abcdef", r) {
t.Fatalf("NewID() = %q, want hex", a)
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if verifySession(key, tc.value, now) {
t.Fatalf("verifySession(%q) = true, want false", tc.value)
}
})
}
}
func flipLastChar(s string) string {
if s == "" {
return "x"
}
last := s[len(s)-1]
if last == 'A' {
return s[:len(s)-1] + "B"
}
return s[:len(s)-1] + "A"
}
func TestSessionKeyDependsOnToken(t *testing.T) {
a := sessionKey("token-a", "pw-abc")
b := sessionKey("token-b", "pw-abc")
if string(a) == string(b) {
t.Fatal("sessionKey collided for different API tokens")
}
}
func TestSessionKeyDependsOnWebPassword(t *testing.T) {
a := sessionKey("token-abc", "pw-a")
b := sessionKey("token-abc", "pw-b")
if string(a) == string(b) {
t.Fatal("sessionKey collided for different web passwords with the same API token")
}
}
@@ -86,7 +39,7 @@ func TestSetSessionCookieAttributes(t *testing.T) {
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
r := httptest.NewRequest(http.MethodPost, "/login", nil)
r := httptest.NewRequest(http.MethodPost, "/", nil)
if tc.tls {
r.TLS = &tls.ConnectionState{}
}
@@ -94,15 +47,18 @@ func TestSetSessionCookieAttributes(t *testing.T) {
r.Header.Set("X-Forwarded-Proto", tc.forwarded)
}
rr := httptest.NewRecorder()
setSessionCookie(rr, r, sessionKey("token-abc", "pw-abc"))
SetCookie(rr, r, "abc123")
cookies := rr.Result().Cookies()
if len(cookies) != 1 {
t.Fatalf("got %d cookies, want 1", len(cookies))
}
c := cookies[0]
if c.Name != sessionCookieName {
t.Fatalf("cookie name = %q, want %q", c.Name, sessionCookieName)
if c.Name != CookieName {
t.Fatalf("cookie name = %q, want %q", c.Name, CookieName)
}
if c.Value != "abc123" {
t.Fatalf("cookie value = %q, want the session id verbatim", c.Value)
}
if !c.HttpOnly {
t.Fatal("cookie HttpOnly = false, want true")
@@ -116,8 +72,8 @@ func TestSetSessionCookieAttributes(t *testing.T) {
if c.Secure != tc.wantSecure {
t.Fatalf("cookie Secure = %v, want %v", c.Secure, tc.wantSecure)
}
if c.MaxAge != int(sessionTTL/time.Second) {
t.Fatalf("cookie MaxAge = %d, want %d", c.MaxAge, int(sessionTTL/time.Second))
if c.MaxAge != int(SessionTTL/time.Second) {
t.Fatalf("cookie MaxAge = %d, want %d", c.MaxAge, int(SessionTTL/time.Second))
}
})
}
@@ -126,7 +82,7 @@ func TestSetSessionCookieAttributes(t *testing.T) {
func TestClearSessionCookie(t *testing.T) {
r := httptest.NewRequest(http.MethodPost, "/logout", nil)
rr := httptest.NewRecorder()
clearSessionCookie(rr, r)
ClearCookie(rr, r)
cookies := rr.Result().Cookies()
if len(cookies) != 1 {
@@ -163,70 +119,70 @@ func TestClientIP(t *testing.T) {
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
r := httptest.NewRequest(http.MethodPost, "/login", nil)
r := httptest.NewRequest(http.MethodPost, "/", nil)
r.RemoteAddr = tc.remoteAddr
for _, v := range tc.xff {
r.Header.Add("X-Forwarded-For", v)
}
if got := clientIP(r); got != tc.want {
t.Fatalf("clientIP() = %q, want %q", got, tc.want)
if got := ClientIP(r); got != tc.want {
t.Fatalf("ClientIP() = %q, want %q", got, tc.want)
}
})
}
}
func TestLoginLimiterBlocksAfterMaxFailures(t *testing.T) {
l := newLoginLimiter()
l := NewLoginLimiter()
now := time.Now()
for i := 0; i < loginMaxFailures; i++ {
if wait := l.retryAfter("1.2.3.4", now); wait != 0 {
t.Fatalf("blocked after %d failures, want block only after %d", i, loginMaxFailures)
for i := 0; i < MaxFailures; i++ {
if wait := l.RetryAfter("1.2.3.4", now); wait != 0 {
t.Fatalf("blocked after %d failures, want block only after %d", i, MaxFailures)
}
l.fail("1.2.3.4", now)
l.Fail("1.2.3.4", now)
}
wait := l.retryAfter("1.2.3.4", now)
wait := l.RetryAfter("1.2.3.4", now)
if wait <= 0 {
t.Fatalf("retryAfter = %v after %d failures, want > 0", wait, loginMaxFailures)
t.Fatalf("retryAfter = %v after %d failures, want > 0", wait, MaxFailures)
}
if wait > loginWindow {
t.Fatalf("retryAfter = %v, want <= %v", wait, loginWindow)
if wait > Window {
t.Fatalf("retryAfter = %v, want <= %v", wait, Window)
}
}
func TestLoginLimiterWindowExpires(t *testing.T) {
l := newLoginLimiter()
l := NewLoginLimiter()
start := time.Now()
for i := 0; i < loginMaxFailures; i++ {
l.fail("1.2.3.4", start)
for i := 0; i < MaxFailures; i++ {
l.Fail("1.2.3.4", start)
}
if l.retryAfter("1.2.3.4", start) == 0 {
if l.RetryAfter("1.2.3.4", start) == 0 {
t.Fatal("expected block immediately after the failures")
}
later := start.Add(loginWindow + time.Second)
if wait := l.retryAfter("1.2.3.4", later); wait != 0 {
later := start.Add(Window + time.Second)
if wait := l.RetryAfter("1.2.3.4", later); wait != 0 {
t.Fatalf("retryAfter = %v once the window passed, want 0", wait)
}
}
func TestLoginLimiterResetClearsCounter(t *testing.T) {
l := newLoginLimiter()
l := NewLoginLimiter()
now := time.Now()
for i := 0; i < loginMaxFailures; i++ {
l.fail("1.2.3.4", now)
for i := 0; i < MaxFailures; i++ {
l.Fail("1.2.3.4", now)
}
l.reset("1.2.3.4")
if wait := l.retryAfter("1.2.3.4", now); wait != 0 {
l.Reset("1.2.3.4")
if wait := l.RetryAfter("1.2.3.4", now); wait != 0 {
t.Fatalf("retryAfter = %v after reset, want 0", wait)
}
}
func TestLoginLimiterIsPerIP(t *testing.T) {
l := newLoginLimiter()
l := NewLoginLimiter()
now := time.Now()
for i := 0; i < loginMaxFailures; i++ {
l.fail("1.2.3.4", now)
for i := 0; i < MaxFailures; i++ {
l.Fail("1.2.3.4", now)
}
if wait := l.retryAfter("5.6.7.8", now); wait != 0 {
if wait := l.RetryAfter("5.6.7.8", now); wait != 0 {
t.Fatalf("retryAfter for a different IP = %v, want 0", wait)
}
}
@@ -0,0 +1,28 @@
-- One row per tracked series, keyed "<site>:<series_id>".
--
-- Everything is NOT NULL with a default except latest_chapter_num, where NULL
-- is a distinct state: nothing has been captured yet, which is not the same as
-- chapter zero.
--
-- Timestamps are unix milliseconds as bigint, not timestamptz: the userscripts
-- send Date.now() over the wire and the ordering rule compares them directly.
CREATE TABLE bookmarks (
key text PRIMARY KEY,
site text NOT NULL,
series_id text NOT NULL,
title text NOT NULL DEFAULT '',
series_url text NOT NULL DEFAULT '',
cover text NOT NULL DEFAULT '',
last_chapter text NOT NULL DEFAULT '',
last_chapter_num double precision NOT NULL DEFAULT 0,
last_chapter_url text NOT NULL DEFAULT '',
favorite boolean NOT NULL DEFAULT false,
latest_chapter text NOT NULL DEFAULT '',
latest_chapter_num double precision,
-- When the server last polled this series, unix ms; 0 means never, and sorts
-- first so a new bookmark is picked up on the next tick with no special case.
latest_checked_at bigint NOT NULL DEFAULT 0,
status text NOT NULL DEFAULT 'reading',
kind text NOT NULL DEFAULT 'manga',
updated_at bigint NOT NULL
);
@@ -0,0 +1,45 @@
-- One row per distinct work, shared by every bookmark that tracks it
-- (ADR-0003). Keyed (site, series_id), the pair a bookmark key decomposes
-- into. title/series_url/cover are written once, at creation, and never
-- again: client-supplied values are ignored once the row exists and the
-- poller is the only party that may change them. kind and the latest-chapter
-- fields are last-write-wins like the bookmark's own fields.
CREATE TABLE series (
site text NOT NULL,
series_id text NOT NULL,
title text NOT NULL DEFAULT '',
series_url text NOT NULL DEFAULT '',
cover text NOT NULL DEFAULT '',
kind text NOT NULL DEFAULT 'manga',
latest_chapter text NOT NULL DEFAULT '',
latest_chapter_num double precision,
-- When the server last polled this series, unix ms; 0 means never, and sorts
-- first so a new bookmark is picked up on the next tick with no special case.
latest_checked_at bigint NOT NULL DEFAULT 0,
PRIMARY KEY (site, series_id)
);
-- Backfill from today's rows. The bookmark key's uniqueness makes
-- (site, series_id) unique in practice; DISTINCT is belt and braces.
INSERT INTO series (site, series_id, title, series_url, cover, kind,
latest_chapter, latest_chapter_num, latest_checked_at)
SELECT DISTINCT site, series_id, title, series_url, cover, kind,
latest_chapter, latest_chapter_num, latest_checked_at
FROM bookmarks;
-- The bookmark keeps only what differs between readers (ADR-0003): progress,
-- favourite, lifecycle bucket. The dropped columns now live on series.
ALTER TABLE bookmarks
DROP COLUMN title,
DROP COLUMN series_url,
DROP COLUMN cover,
DROP COLUMN kind,
DROP COLUMN latest_chapter,
DROP COLUMN latest_chapter_num,
DROP COLUMN latest_checked_at;
-- A bookmark may not point at a series that does not exist. No cascade: a
-- series outlives its last bookmark, and deleting one is not a store operation.
ALTER TABLE bookmarks
ADD CONSTRAINT bookmarks_series_fk
FOREIGN KEY (site, series_id) REFERENCES series (site, series_id);
@@ -0,0 +1,17 @@
-- One row per person. Keyed by their Discord user ID; carries the SHA-256 of
-- their userscript token and when they were created. Hashed because a token
-- in the database is a token anyone with the database can replay; SHA-256 is
-- enough because the tokens are high-entropy random values with nothing to
-- brute-force. No one can register yet, so this table holds exactly the one
-- owner row the seed creates at startup (see Store.Open).
CREATE TABLE readers (
id bigserial PRIMARY KEY,
discord_id text NOT NULL UNIQUE,
token_sha256 bytea NOT NULL UNIQUE,
created_at timestamptz NOT NULL DEFAULT now()
);
-- Every bookmark now belongs to a reader. Added nullable: rows created before
-- this migration have no owner yet — 0004 attaches them to the seeded owner
-- before NOT NULL and the composite key land.
ALTER TABLE bookmarks ADD COLUMN reader_id bigint;
@@ -0,0 +1,19 @@
-- Attach every pre-existing bookmark to the owner reader, seeded between the
-- two migrate passes (Store.Open). The oldest reader is the owner by
-- construction: only the seed creates readers, and it runs once per database.
-- Run-once via the version table, like every migration.
UPDATE bookmarks SET reader_id = (SELECT id FROM readers ORDER BY id LIMIT 1);
-- Ownership lands structurally: reader_id becomes part of the key, so a
-- bookmark is one Reader's progress on one Series and a duplicate for the
-- same pair is impossible at the database level. Deleting a Reader takes
-- their bookmarks with them. The old text key is gone — the wire "key" is
-- derived as site:series_id on read, and nothing references the column.
-- Dropping it drops the primary key it carried; the composite key replaces
-- it, and the FK index the series constraint needs is created automatically.
ALTER TABLE bookmarks
ALTER COLUMN reader_id SET NOT NULL,
DROP COLUMN key,
ADD PRIMARY KEY (reader_id, site, series_id),
ADD CONSTRAINT bookmarks_reader_fk
FOREIGN KEY (reader_id) REFERENCES readers (id) ON DELETE CASCADE;
@@ -0,0 +1,11 @@
-- One row per browser session. The id is an opaque random value the cookie
-- carries verbatim; a request is authenticated by looking the row up, and
-- deleting the row is how a session is revoked. Expired rows are removed
-- lazily on lookup and swept by the next login, so nothing runs a background
-- cleanup.
CREATE TABLE sessions (
id text PRIMARY KEY,
reader_id bigint NOT NULL REFERENCES readers (id) ON DELETE CASCADE,
created_at timestamptz NOT NULL DEFAULT now(),
expires_at timestamptz NOT NULL
);
@@ -0,0 +1,7 @@
-- Rotation is an epoch bump: a Reader's credential is derived from the
-- deployment secret, their Discord id and this epoch, so bumping it issues a
-- new credential and the rewritten token_sha256 invalidates the old one the
-- moment the transaction commits. The seed's ON CONFLICT refresh (Store.Open)
-- is gated on this being 0, so a restart can never undo a rotation by
-- restoring the epoch-0 hash.
ALTER TABLE readers ADD COLUMN token_epoch bigint NOT NULL DEFAULT 0;
@@ -0,0 +1,8 @@
-- Kagane cover bytes belong in their own table so image blobs never enter the
-- series queries that drive the latest-chapter poller.
CREATE TABLE covers (
image_id text PRIMARY KEY,
body bytea NOT NULL,
content_type text NOT NULL,
fetched_at timestamptz NOT NULL DEFAULT now()
);
@@ -0,0 +1,9 @@
-- Cover bytes move out of Postgres. Existing rows are intentionally dropped:
-- the old kagane path already refetches missing Covers on demand.
DROP TABLE covers;
CREATE TABLE covers (
address text PRIMARY KEY,
path text NOT NULL,
content_type text NOT NULL
);
@@ -0,0 +1,8 @@
-- The Cover splits into two facts. `cover` keeps the third-party address the
-- bytes come from, which is what the acquisition path refetches and dedupes
-- on; `cover_address` is the content address of the bytes once they are
-- actually stored, and is what the wire's absolute URL is built from.
--
-- Empty `cover_address` therefore means "no Cover yet" rather than "a Cover
-- that 404s", which is the distinction the API and the UI both depend on.
ALTER TABLE series ADD COLUMN cover_address text NOT NULL DEFAULT '';
+80
View File
@@ -0,0 +1,80 @@
package store
import (
"database/sql"
"fmt"
"time"
)
// Session is one browser login: an opaque id the cookie carries verbatim,
// the Reader it belongs to, and when it stops being valid.
type Session struct {
ID string
ReaderID int64
ExpiresAt time.Time
}
// CreateSession stores a new session row for reader. The id is generated by
// the caller (session.NewID) — the store only persists it. Expired rows that
// were never looked up are swept in the same transaction: this is the one
// write every login makes, so the table stays bounded without a background
// job.
func (s *Store) CreateSession(id string, readerID int64, ttl time.Duration) (Session, error) {
tx, err := s.db.Begin()
if err != nil {
return Session{}, err
}
defer tx.Rollback()
expires := time.Now().Add(ttl)
if _, err := tx.Exec(`INSERT INTO sessions (id, reader_id, expires_at) VALUES ($1, $2, $3)`,
id, readerID, expires); err != nil {
return Session{}, err
}
if _, err := tx.Exec(`DELETE FROM sessions WHERE expires_at < now()`); err != nil {
return Session{}, err
}
if err := tx.Commit(); err != nil {
return Session{}, err
}
return Session{ID: id, ReaderID: readerID, ExpiresAt: expires}, nil
}
// GetSession returns the live session row for id, or ok=false when the id is
// unknown or expired. An expired row is deleted on the way out, so the table
// never grows past sessions that are still valid.
func (s *Store) GetSession(id string, now time.Time) (Session, bool, error) {
var sess Session
err := s.db.QueryRow(
`SELECT id, reader_id, expires_at FROM sessions WHERE id = $1`, id,
).Scan(&sess.ID, &sess.ReaderID, &sess.ExpiresAt)
if err == sql.ErrNoRows {
return Session{}, false, nil
}
if err != nil {
return Session{}, false, err
}
if !sess.ExpiresAt.After(now) {
// Best-effort: the row is dead either way; failing the request over a
// cleanup delete would only hide the real error. CreateSession's
// sweep catches anything this misses.
_, _ = s.db.Exec(`DELETE FROM sessions WHERE id = $1`, id)
return Session{}, false, nil
}
return sess, true, nil
}
// DeleteSession revokes one session. Deleting an unknown id is not an error.
func (s *Store) DeleteSession(id string) error {
_, err := s.db.Exec(`DELETE FROM sessions WHERE id = $1`, id)
return err
}
// DeleteReaderSessions revokes every session one Reader holds — the owner's
// remedy when a Reader's browser must be logged out everywhere at once. The
// next request carrying any of those cookies finds no row and is rejected.
func (s *Store) DeleteReaderSessions(readerID int64) error {
if _, err := s.db.Exec(`DELETE FROM sessions WHERE reader_id = $1`, readerID); err != nil {
return fmt.Errorf("delete sessions for reader %d: %w", readerID, err)
}
return nil
}
+84
View File
@@ -0,0 +1,84 @@
package store
import (
"testing"
"time"
)
func TestCreateAndGetSession(t *testing.T) {
s := newTestStore(t)
owner := s.OwnerID()
sess, err := s.CreateSession("sess-1", owner, time.Hour)
if err != nil {
t.Fatalf("CreateSession: %v", err)
}
if sess.ID != "sess-1" || sess.ReaderID != owner {
t.Fatalf("CreateSession returned %+v, want id sess-1 reader %d", sess, owner)
}
got, ok, err := s.GetSession("sess-1", time.Now())
if err != nil || !ok {
t.Fatalf("GetSession: ok=%v err=%v, want ok", ok, err)
}
if got.ReaderID != owner {
t.Fatalf("session reader = %d, want %d", got.ReaderID, owner)
}
}
func TestGetSessionUnknownID(t *testing.T) {
s := newTestStore(t)
if _, ok, err := s.GetSession("nope", time.Now()); err != nil || ok {
t.Fatalf("GetSession(unknown) = ok=%v err=%v, want ok=false", ok, err)
}
}
func TestExpiredSessionIsGone(t *testing.T) {
s := newTestStore(t)
owner := s.OwnerID()
if _, err := s.CreateSession("sess-exp", owner, -time.Minute); err != nil {
t.Fatalf("CreateSession: %v", err)
}
now := time.Now()
if _, ok, err := s.GetSession("sess-exp", now); err != nil || ok {
t.Fatalf("GetSession(expired) = ok=%v err=%v, want ok=false", ok, err)
}
// The expired row is deleted on lookup, so the next call cannot revive it.
if _, ok, err := s.GetSession("sess-exp", now.Add(-time.Hour)); err != nil || ok {
t.Fatalf("GetSession(expired again) = ok=%v err=%v, want ok=false", ok, err)
}
}
func TestDeleteSessionRevokes(t *testing.T) {
s := newTestStore(t)
owner := s.OwnerID()
if _, err := s.CreateSession("sess-del", owner, time.Hour); err != nil {
t.Fatalf("CreateSession: %v", err)
}
if err := s.DeleteSession("sess-del"); err != nil {
t.Fatalf("DeleteSession: %v", err)
}
if _, ok, err := s.GetSession("sess-del", time.Now()); err != nil || ok {
t.Fatalf("GetSession after delete = ok=%v err=%v, want ok=false", ok, err)
}
// Deleting twice is not an error.
if err := s.DeleteSession("sess-del"); err != nil {
t.Fatalf("DeleteSession twice: %v", err)
}
}
func TestDeleteSessionIsPerReader(t *testing.T) {
s := newTestStore(t)
other := secondReader(t, s)
if _, err := s.CreateSession("sess-other", other, time.Hour); err != nil {
t.Fatalf("CreateSession: %v", err)
}
got, ok, err := s.GetSession("sess-other", time.Now())
if err != nil || !ok {
t.Fatalf("GetSession: ok=%v err=%v, want ok", ok, err)
}
if got.ReaderID != other {
t.Fatalf("session reader = %d, want %d", got.ReaderID, other)
}
}
+994
View File
@@ -0,0 +1,994 @@
package store
import (
"crypto/sha256"
"database/sql"
"embed"
"encoding/hex"
"errors"
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
"regexp"
"slices"
"strconv"
"strings"
"github.com/jackc/pgx/v5/pgtype"
_ "github.com/jackc/pgx/v5/stdlib"
)
// Bookmark is one tracked series, keyed "<site>:<series_id>" across both sites.
//
// LastChapter* is the user's read progress; LatestChapter* is the newest
// chapter the site has published, captured opportunistically by the userscript.
//
// Title, SeriesURL, Cover, Kind and LatestChapter* live on the shared Series
// row (ADR-0003) and are joined in on read; Bookmark carries only what differs
// between readers: progress, favourite, lifecycle bucket, updated_at. The wire
// format stays flat regardless — see ADR-0004.
type Bookmark struct {
Key string `json:"key"`
Site string `json:"site"`
SeriesID string `json:"series_id"`
Title string `json:"title"`
SeriesURL string `json:"series_url"`
// Cover is the wire value: an absolute URL on this deployment's own
// origin once the bytes exist, and "" until they do — never a third-party
// address and never an address that 404s (ADR-0007). A client may still
// send this field and it is discarded on the way in; see Upsert.
Cover string `json:"cover"`
LastChapter string `json:"last_chapter"`
LastChapterNum float64 `json:"last_chapter_num"`
LastChapterURL string `json:"last_chapter_url"`
Favorite bool `json:"favorite"`
LatestChapter string `json:"latest_chapter"`
LatestChapterNum *float64 `json:"latest_chapter_num"` // nil until first captured
UpdatedAt int64 `json:"updated_at"` // unix ms; see Upsert
// Status is the lifecycle bucket: reading, archived, or finished.
// Archived series stay polled for new chapters; finished ones do not.
// Empty on the way in means "no opinion" — see Upsert.
Status string `json:"status"`
// Kind is the library bucket: manga or novel. Empty on the way in means
// "no opinion" — see Upsert.
Kind string `json:"kind"`
}
// Series is one distinct work, shared by every bookmark that tracks it. It is
// keyed (site, series_id) — the pair a bookmark key decomposes into — and
// exists once no matter how many bookmarks point at it (ADR-0003).
//
// Title, SeriesURL and Cover are written once, at creation: a PUT naming an
// existing Series has them ignored, and only the backend's own Poll may change
// them. Kind and the latest-chapter fields are last-write-wins like the
// bookmark's own fields. Never serialized: the wire format is the flat
// Bookmark (ADR-0004).
type Series struct {
Site string
SeriesID string
Title string
SeriesURL string
// Cover is the third-party source address the bytes come from, and
// CoverAddress the content address they are stored under. A blank
// CoverAddress is what "no Cover yet" means: the poll fills it and never
// replaces a filled one (ADR-0007).
Cover string
CoverAddress string
Kind string
LatestChapter string
LatestChapterNum *float64 // nil until first captured
LatestCheckedAt int64 // unix ms; see MarkLatestChecked
// readerCount is the number of bookmarks referencing this series, filled
// only by the due-queue query that orders on it.
readerCount int
}
// Key returns the canonical identity in bookmark-key form ("<site>:<series_id>"),
// used by the poller's logs and by tests asserting on the due queue.
func (s Series) Key() string { return s.Site + ":" + s.SeriesID }
// HasNewChapter reports whether the site has published past the read point.
// A nil LatestChapterNum means nothing has been captured yet, which is not the
// same as "nothing new".
func (b Bookmark) HasNewChapter() bool {
return b.LatestChapterNum != nil && *b.LatestChapterNum > b.LastChapterNum
}
// chapterLeadIn matches the prefix the userscript and the poller both write
// ("Chapter 250"), so the UI can add exactly one "Ch " of its own instead of
// doubling it. A manual edit through the web UI stores a bare "250", which is
// the same string minus the lead-in.
var chapterLeadIn = regexp.MustCompile(`(?i)^\s*(?:chapter|ch\.?)\s*`)
func displayChapter(raw string, num float64) string {
rest := strings.TrimSpace(chapterLeadIn.ReplaceAllString(raw, ""))
if rest == "" {
rest = strconv.FormatFloat(num, 'f', -1, 64)
}
// "Ch " only makes sense in front of a number; anything else is a label the
// site gave us, so pass it through as written.
if rest[0] < '0' || rest[0] > '9' {
return rest
}
return "Ch " + rest
}
// DisplayChapter is the read-progress line: one canonical "Ch N" whatever
// format the write came in as.
func (b Bookmark) DisplayChapter() string {
return displayChapter(b.LastChapter, b.LastChapterNum)
}
// DisplayLatest is the same for the newest published chapter, which arrives
// with the same "Chapter N" lead-in from both the userscript and the poller.
func (b Bookmark) DisplayLatest() string {
var num float64
if b.LatestChapterNum != nil {
num = *b.LatestChapterNum
}
return displayChapter(b.LatestChapter, num)
}
// ContinueURL is where the Continue button points: the chapter last read, or
// the series page when no chapter URL was ever captured.
func (b Bookmark) ContinueURL() string {
if b.LastChapterURL != "" {
return b.LastChapterURL
}
return b.SeriesURL
}
// Initial is the monogram the web UI shows in place of a cover when the
// source site never gave us an og:image. First rune, uppercased; "?" when even
// the title is missing, so the slot is never empty.
func (b Bookmark) Initial() string {
for _, r := range b.Title {
return strings.ToUpper(string(r))
}
return "?"
}
// CoverContentType canonicalises a fetched response's media type and reports
// whether the bytes are safe to store and serve. comix answers "image/jpg",
// which no standard lists but browsers accept; it is stored as the real name
// rather than passed through, so one image never lands under two spellings.
func CoverContentType(contentType string) (string, bool) {
switch contentType {
case "image/jpg":
return "image/jpeg", true
case "image/webp", "image/jpeg", "image/png", "image/avif", "image/gif":
return contentType, true
default:
return "", false
}
}
// Library buckets. A bookmark is in exactly one. This cannot be derived from
// Site: asurascans serves manga and novels from the same /comics/ path, so the
// userscript that recorded the page is the only party that knows which.
const (
KindManga = "manga"
KindNovel = "novel"
)
// Lifecycle buckets. A bookmark is in exactly one; favorite is orthogonal.
const (
StatusReading = "reading"
StatusArchived = "archived"
StatusFinished = "finished"
)
//go:embed migrations/*.sql
var migrations embed.FS
// bookmarkColumns is the only value ever concatenated into query text. It is a
// compile-time constant; every request value is bound as a parameter. The
// series-owned fields are joined in from the series table, in scanBookmark
// order, so the flat Bookmark reads back whole despite the split (ADR-0004).
const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover_address,
b.last_chapter, b.last_chapter_num, b.last_chapter_url,
b.favorite, s.latest_chapter, s.latest_chapter_num, b.updated_at, b.status, s.kind`
// seriesColumns is the series row in scanSeries order, used by the poller's
// due query. latest_checked_at lives only on series — see MarkLatestChecked
// for why it stays off every client-visible write.
const seriesColumns = `s.site, s.series_id, s.title, s.series_url, s.cover, s.cover_address,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at`
// Owner is the person running the service: the first Reader, seeded at startup
// so a fresh deployment has a library before anyone logs in. The seed makes
// sure exactly one readers row matches their Discord ID, carrying the SHA-256
// of their epoch-0 userscript credential (derived by internal/token). Every
// other Reader is created by their own first login (EnsureReader).
type Owner struct {
DiscordID string
// TokenHash is the SHA-256 of the epoch-0 credential; the array shape
// makes it a compile error to store anything that is not a hash.
TokenHash [32]byte
}
// Store is the Postgres-backed bookmark store.
type Store struct {
db *sql.DB
// ownerID is the seeded owner Reader (issue #22) — the only Reader with
// administrative reach (revoking another Reader's sessions). Every store
// method takes a reader id explicitly, so ownership is never implicit.
ownerID int64
coverDir string
// coverBaseURL is this deployment's public origin. Cover addresses are
// absolute because the userscript renders them on third-party origins,
// where a relative path would resolve against the Site (ADR-0007).
coverBaseURL string
// OnSeriesCreated fires once, after commit, for a Series no Reader had
// bookmarked before. It is how creation-time Cover and Latest Chapter
// acquisition is triggered without the write waiting on a third-party
// Site; nil disables it, which is what every test that does not care
// about acquisition leaves it as.
OnSeriesCreated func(Series)
}
// OwnerID returns the seeded owner Reader's id: the administrator, and the
// Reader every pre-registration bookmark belongs to.
func (s *Store) OwnerID() int64 { return s.ownerID }
// ReaderIDForTokenHash resolves the Reader whose stored credential hash
// matches, reporting absence with ok=false. The comparison is an equality on
// the 32-byte SHA-256 of the presented credential — never on the credential
// itself — and the indexed lookup reveals only whether some Reader matches,
// which the 401/200 split has to reveal anyway. An attacker's probe is the
// hash of their guess, so even the index's prefix comparisons leak nothing
// about the real credential.
func (s *Store) ReaderIDForTokenHash(hash [32]byte) (int64, bool, error) {
var id int64
err := s.db.QueryRow(
`SELECT id FROM readers WHERE token_sha256 = $1`, hash[:]).Scan(&id)
if errors.Is(err, sql.ErrNoRows) {
return 0, false, nil
}
if err != nil {
return 0, false, fmt.Errorf("reader by token hash: %w", err)
}
return id, true, nil
}
// ReaderTokenInfo returns the identity halves a Reader's credential is
// derived from (internal/token.Token): their Discord id and token epoch. The
// web UI needs these to rebuild the install URL — the only place a credential
// is ever produced in plaintext.
func (s *Store) ReaderTokenInfo(readerID int64) (string, int64, error) {
var (
discordID string
epoch int64
)
err := s.db.QueryRow(
`SELECT discord_id, token_epoch FROM readers WHERE id = $1`, readerID).
Scan(&discordID, &epoch)
if err != nil {
return "", 0, fmt.Errorf("reader %d token info: %w", readerID, err)
}
return discordID, epoch, nil
}
// RotateToken bumps a Reader's token epoch and rewrites the stored hash in
// one statement, so the new hash always matches the new epoch. expectedEpoch
// is the epoch the caller derived newHash for (ReaderTokenInfo + 1); a
// concurrent rotation — or an unknown reader — leaves the row untouched and
// is reported as an error rather than silently succeeding.
func (s *Store) RotateToken(readerID, expectedEpoch int64, newHash [32]byte) error {
var epoch int64
err := s.db.QueryRow(`
UPDATE readers SET token_epoch = token_epoch + 1, token_sha256 = $3
WHERE id = $1 AND token_epoch = $2
RETURNING token_epoch`, readerID, expectedEpoch, newHash[:]).Scan(&epoch)
if errors.Is(err, sql.ErrNoRows) {
return fmt.Errorf("rotate token for reader %d: concurrent rotation or unknown reader", readerID)
}
if err != nil {
return fmt.Errorf("rotate token for reader %d: %w", readerID, err)
}
return nil
}
// EnsureReader returns the Reader registered to discordID, creating the row on
// first sight. Registration is open to every guild member (issue #27), and the
// Discord identity is the only thing that decides which Reader a login is: one
// code path serves the first login and every later one, so a returning Reader
// can never end up with a second library.
//
// epochZeroHash is only used for a brand-new row. An existing row keeps its
// stored hash untouched, or a login would silently undo a rotation and revive
// the credential the Reader rotated away from.
func (s *Store) EnsureReader(discordID string, epochZeroHash [32]byte) (int64, error) {
var id int64
// DO UPDATE rather than DO NOTHING because only an updated row is
// returned by RETURNING; assigning the column to itself is the no-op that
// makes the existing id come back.
err := s.db.QueryRow(`
INSERT INTO readers (discord_id, token_sha256) VALUES ($1, $2)
ON CONFLICT (discord_id) DO UPDATE SET discord_id = readers.discord_id
RETURNING id`, discordID, epochZeroHash[:]).Scan(&id)
if err != nil {
return 0, fmt.Errorf("ensure reader: %w", err)
}
return id, nil
}
// ReaderSummary is one Reader as the owner's administration panel sees them:
// who they are and how many live sessions they hold. No credential material,
// hashed or otherwise, is exposed.
type ReaderSummary struct {
ID int64
DiscordID string
// Sessions counts unexpired session rows — what the owner revokes.
Sessions int
}
// Readers lists every Reader with their live session count, oldest first, so
// the owner row (always the oldest) heads the list.
func (s *Store) Readers() ([]ReaderSummary, error) {
rows, err := s.db.Query(`
SELECT r.id, r.discord_id,
count(sess.id) FILTER (WHERE sess.expires_at > now()) AS sessions
FROM readers r
LEFT JOIN sessions sess ON sess.reader_id = r.id
GROUP BY r.id, r.discord_id
ORDER BY r.id`)
if err != nil {
return nil, fmt.Errorf("query readers: %w", err)
}
defer rows.Close()
out := []ReaderSummary{}
for rows.Next() {
var r ReaderSummary
if err := rows.Scan(&r.ID, &r.DiscordID, &r.Sessions); err != nil {
return nil, fmt.Errorf("scan reader: %w", err)
}
out = append(out, r)
}
return out, rows.Err()
}
// readersMigration is the version that creates the readers table. The owner
// seed runs between two migrate passes, so that the run-once migration which
// attaches existing bookmarks (0004) finds the owner row.
const readersMigration = 3
// allMigrations is the migrate() cap that applies every pending version.
const allMigrations = 0
// Open connects to Postgres at url — a libpq connection URL such as
// "postgres://user:pass@host:5432/bookmarks?sslmode=disable" — brings its
// schema up to date, seeds the owner Reader, and prepares cover storage.
func Open(url string, owner Owner, coverDir, coverBaseURL string) (*Store, error) {
if strings.TrimSpace(coverDir) == "" {
return nil, errors.New("cover directory is required")
}
// Every wire Cover is this string with a path glued on, rendered by a
// userscript on a Site's own origin: anything but an absolute origin
// produces addresses no client can load, silently (ADR-0007).
base := strings.TrimRight(coverBaseURL, "/")
if host, ok := strings.CutPrefix(base, "https://"); !ok || host == "" {
if host, ok := strings.CutPrefix(base, "http://"); !ok || host == "" {
return nil, fmt.Errorf("cover base URL %q is not an absolute http(s) origin", coverBaseURL)
}
}
if err := os.MkdirAll(coverDir, 0o755); err != nil {
return nil, fmt.Errorf("create cover directory: %w", err)
}
info, err := os.Stat(coverDir)
if err != nil {
return nil, fmt.Errorf("stat cover directory: %w", err)
}
if !info.IsDir() {
return nil, fmt.Errorf("cover directory %q is not a directory", coverDir)
}
db, err := sql.Open("pgx", url)
if err != nil {
return nil, fmt.Errorf("open postgres: %w", err)
}
// Schema runs in two passes with the seed between: 0003 creates the
// readers table, the owner row must exist before 0004 attaches the
// existing bookmarks to it. Anything past 0004 is applied by the second
// pass.
if err := migrate(db, readersMigration); err != nil {
db.Close()
return nil, fmt.Errorf("migrate schema: %w", err)
}
// The owner row must exist before 0004 attaches the existing bookmarks to
// it. The hash refresh is a separate statement after all migrations: the
// token_epoch column 0006 adds does not exist yet at this point, and the
// refresh only ever concerns rows that have never been rotated.
if err := seedOwner(db, owner); err != nil {
db.Close()
return nil, fmt.Errorf("seed owner: %w", err)
}
if err := migrate(db, allMigrations); err != nil {
db.Close()
return nil, fmt.Errorf("migrate: %w", err)
}
if err := refreshOwnerToken(db, owner); err != nil {
db.Close()
return nil, fmt.Errorf("refresh owner token: %w", err)
}
var ownerID int64
if err := db.QueryRow(
`SELECT id FROM readers WHERE discord_id = $1`, owner.DiscordID).Scan(&ownerID); err != nil {
db.Close()
return nil, fmt.Errorf("resolve owner: %w", err)
}
return &Store{
db: db, ownerID: ownerID, coverDir: coverDir, coverBaseURL: base,
}, nil
}
// seedOwner makes sure the configured owner exists as exactly one readers row.
// The hash is only ever written here for a brand-new row; existing rows keep
// what they have until refreshOwnerToken decides otherwise, so the seed can
// never clobber a rotation.
func seedOwner(db *sql.DB, o Owner) error {
if _, err := db.Exec(`
INSERT INTO readers (discord_id, token_sha256) VALUES ($1, $2)
ON CONFLICT (discord_id) DO NOTHING`,
o.DiscordID, o.TokenHash[:]); err != nil {
return fmt.Errorf("seed owner: %w", err)
}
return nil
}
// refreshOwnerToken brings a never-rotated owner row's hash current with the
// configured credential. That is the cutover path: a database seeded under
// the retired global token still carries its hash at epoch 0, and the
// epoch-0 derivation is the caller's TokenHash. A rotated row (epoch > 0) is
// left alone — a restart must not resurrect the old credential by
// overwriting the hash a rotation wrote.
func refreshOwnerToken(db *sql.DB, o Owner) error {
if _, err := db.Exec(`
UPDATE readers SET token_sha256 = $2
WHERE discord_id = $1 AND token_epoch = 0`,
o.DiscordID, o.TokenHash[:]); err != nil {
return fmt.Errorf("refresh owner token: %w", err)
}
return nil
}
// migrate applies every embedded migration this database has not recorded, in
// filename order, each in its own transaction. upto caps the highest version
// applied; 0 means all. Files are named "<version>_<name>.sql" and are
// append-only: editing an applied file changes nothing, because
// schema_migrations is how a database remembers what it ran. Runs on every
// start and is a no-op once current.
func migrate(db *sql.DB, upto int64) error {
if _, err := db.Exec(`CREATE TABLE IF NOT EXISTS schema_migrations (
version bigint PRIMARY KEY,
applied_at timestamptz NOT NULL DEFAULT now())`); err != nil {
return fmt.Errorf("create version table: %w", err)
}
names, err := fs.Glob(migrations, "migrations/*.sql")
if err != nil {
return err
}
slices.Sort(names)
for _, name := range names {
version, err := strconv.ParseInt(strings.SplitN(path.Base(name), "_", 2)[0], 10, 64)
if err != nil {
return fmt.Errorf("migration %q: filename must start with a version number", name)
}
if upto > 0 && version > upto {
continue
}
body, err := migrations.ReadFile(name)
if err != nil {
return err
}
if err := applyMigration(db, version, string(body)); err != nil {
return fmt.Errorf("migration %q: %w", name, err)
}
}
return nil
}
// applyMigration runs one migration and records its version in the same
// transaction, so an interrupted start leaves neither half behind.
func applyMigration(db *sql.DB, version int64, body string) error {
tx, err := db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
var applied bool
if err := tx.QueryRow(
`SELECT EXISTS (SELECT 1 FROM schema_migrations WHERE version = $1)`,
version).Scan(&applied); err != nil {
return err
}
if applied {
return nil
}
// No parameters, so this goes over the simple protocol and a migration may
// hold more than one statement.
if _, err := tx.Exec(body); err != nil {
return err
}
if _, err := tx.Exec(`INSERT INTO schema_migrations (version) VALUES ($1)`, version); err != nil {
return err
}
return tx.Commit()
}
// scanBookmark reads one row in bookmarkColumns order. Every column is NOT
// NULL except latest_chapter_num, where NULL means "never captured" — a
// distinct state from chapter zero, and the reason for the pointer.
func (s *Store) scanBookmark(scan func(...any) error) (Bookmark, error) {
var (
b Bookmark
coverAddress string
latestChapterNum sql.NullFloat64
)
if err := scan(
&b.Site, &b.SeriesID, &b.Title, &b.SeriesURL, &coverAddress,
&b.LastChapter, &b.LastChapterNum, &b.LastChapterURL,
&b.Favorite, &b.LatestChapter, &latestChapterNum, &b.UpdatedAt, &b.Status, &b.Kind,
); err != nil {
return Bookmark{}, err
}
b.Cover = s.CoverWireURL(coverAddress)
if latestChapterNum.Valid {
b.LatestChapterNum = &latestChapterNum.Float64
}
// The wire identity is derived: there is no stored key column, the
// bookmark is keyed (reader_id, site, series_id) (issue #22).
b.Key = b.Site + ":" + b.SeriesID
// An unrecognised bucket (a hand-edited row) would leave the row in no list
// at all, so anything outside the three known buckets reads as the default
// rather than being passed through.
if b.Status != StatusReading && b.Status != StatusArchived && b.Status != StatusFinished {
b.Status = StatusReading
}
return b, nil
}
// scanSeries reads one row in seriesColumns order, plus the due query's
// reader_count column. latest_chapter_num is NULL until the first capture,
// same as on the bookmark read path.
func scanSeries(scan func(...any) error) (Series, error) {
var (
sr Series
latestChapterNum sql.NullFloat64
)
if err := scan(
&sr.Site, &sr.SeriesID, &sr.Title, &sr.SeriesURL, &sr.Cover, &sr.CoverAddress,
&sr.Kind, &sr.LatestChapter, &latestChapterNum, &sr.LatestCheckedAt,
&sr.readerCount,
); err != nil {
return Series{}, err
}
if latestChapterNum.Valid {
sr.LatestChapterNum = &latestChapterNum.Float64
}
return sr, nil
}
// Close releases the underlying database handle.
func (s *Store) Close() error { return s.db.Close() }
func coverSourceAddress(sourceURL string) string {
sum := sha256.Sum256([]byte(sourceURL))
return hex.EncodeToString(sum[:])
}
func coverRelativePath(address string) string {
return address[:2] + "/" + address[2:4] + "/" + address
}
func (s *Store) getCover(sourceURL string) ([]byte, string, bool, error) {
return s.getCoverByAddress(coverSourceAddress(sourceURL))
}
func (s *Store) getCoverByAddress(address string) ([]byte, string, bool, error) {
var relativePath, contentType string
err := s.db.QueryRow(
`SELECT path, content_type FROM covers WHERE address = $1`, address,
).Scan(&relativePath, &contentType)
if errors.Is(err, sql.ErrNoRows) {
return nil, "", false, nil
}
if err != nil {
return nil, "", false, fmt.Errorf("get cover %q: %w", address, err)
}
expectedPath := coverRelativePath(address)
if relativePath != expectedPath {
return nil, "", false, fmt.Errorf("cover %q has unexpected path %q", address, relativePath)
}
body, err := os.ReadFile(filepath.Join(s.coverDir, filepath.FromSlash(relativePath)))
if errors.Is(err, fs.ErrNotExist) {
return nil, "", false, nil
}
if err != nil {
return nil, "", false, fmt.Errorf("read cover %q: %w", address, err)
}
return body, contentType, true, nil
}
func (s *Store) putCover(sourceURL string, body []byte, contentType string) error {
stored, ok := CoverContentType(contentType)
if !ok {
return fmt.Errorf("put cover %q: unsupported content type %q", sourceURL, contentType)
}
contentType = stored
address := coverSourceAddress(sourceURL)
relativePath := coverRelativePath(address)
coverPath := filepath.Join(s.coverDir, filepath.FromSlash(relativePath))
if err := os.MkdirAll(filepath.Dir(coverPath), 0o755); err != nil {
return fmt.Errorf("create cover shard: %w", err)
}
tmp, err := os.CreateTemp(filepath.Dir(coverPath), ".cover-*")
if err != nil {
return fmt.Errorf("create cover temp file: %w", err)
}
tmpName := tmp.Name()
defer os.Remove(tmpName)
if _, err := tmp.Write(body); err != nil {
tmp.Close()
return fmt.Errorf("write cover temp file: %w", err)
}
if err := tmp.Sync(); err != nil {
tmp.Close()
return fmt.Errorf("sync cover temp file: %w", err)
}
if err := tmp.Close(); err != nil {
return fmt.Errorf("close cover temp file: %w", err)
}
if err := os.Link(tmpName, coverPath); err != nil && !errors.Is(err, fs.ErrExist) {
return fmt.Errorf("install cover file: %w", err)
}
if _, err := s.db.Exec(`
INSERT INTO covers (address, path, content_type)
VALUES ($1, $2, $3)
ON CONFLICT (address) DO NOTHING`, address, relativePath, contentType); err != nil {
return fmt.Errorf("record cover %q: %w", address, err)
}
return nil
}
// GetCover returns the immutable object addressed by its source URL. Missing
// files are reported with ok=false so callers can retry acquisition later.
func (s *Store) GetCover(sourceURL string) ([]byte, string, bool, error) {
return s.getCover(sourceURL)
}
// PutCover persists bytes under the source URL's content address. A later
// write for the same URL cannot replace the immutable object.
func (s *Store) PutCover(sourceURL string, body []byte, contentType string) error {
return s.putCover(sourceURL, body, contentType)
}
// CoverAddress is the content address bytes fetched from sourceURL are stored
// under. It is a pure function of the URL, so the acquisition path can name a
// Cover before it has the bytes.
func CoverAddress(sourceURL string) string { return coverSourceAddress(sourceURL) }
// coverAddressRe is the shape of a stored address: the hex SHA-256 of a source
// URL. Request paths reach CoverByAddress, so the shape is checked before the
// value is ever turned into a filesystem path.
var coverAddressRe = regexp.MustCompile(`^[0-9a-f]{64}$`)
// CoverByAddress returns the immutable object at one content address. An
// address that is not a stored one - malformed, unknown, or recorded but with
// its file gone - is reported with ok=false rather than as an error.
func (s *Store) CoverByAddress(address string) ([]byte, string, bool, error) {
if !coverAddressRe.MatchString(address) {
return nil, "", false, nil
}
return s.getCoverByAddress(address)
}
// CoverWireURL is the absolute URL a client renders for a stored Cover, and ""
// for a Series that has none yet. A blank is a real state, not a placeholder
// address: it is what tells both clients to draw their own fallback instead of
// requesting bytes that do not exist (ADR-0007).
func (s *Store) CoverWireURL(address string) string {
if address == "" {
return ""
}
return s.coverBaseURL + "/covers/" + address
}
// SetSeriesCover stores the bytes and points the Series at them, but only
// while the Series has no Cover: acquisition at creation and the poll both
// call this, and whichever arrives second must not overwrite the first. The
// bytes themselves are content-addressed and immutable, so storing them twice
// is free.
func (s *Store) SetSeriesCover(site, seriesID, sourceURL string, body []byte, contentType string) error {
if err := s.putCover(sourceURL, body, contentType); err != nil {
return err
}
if _, err := s.db.Exec(`
UPDATE series SET cover = $3, cover_address = $4
WHERE site = $1 AND series_id = $2 AND cover_address = ''`,
site, seriesID, sourceURL, coverSourceAddress(sourceURL)); err != nil {
return fmt.Errorf("set cover for %q: %w", site+":"+seriesID, err)
}
return nil
}
// List returns every bookmark of one reader, newest activity first.
// Series-owned fields are joined in, so each Bookmark reads back whole and
// flat (ADR-0004).
func (s *Store) List(readerID int64) ([]Bookmark, error) {
rows, err := s.db.Query(`SELECT `+bookmarkColumns+`
FROM bookmarks b
JOIN series s ON s.site = b.site AND s.series_id = b.series_id
WHERE b.reader_id = $1
ORDER BY b.updated_at DESC`, readerID)
if err != nil {
return nil, fmt.Errorf("query bookmarks: %w", err)
}
defer rows.Close()
out := []Bookmark{}
for rows.Next() {
b, err := s.scanBookmark(rows.Scan)
if err != nil {
return nil, fmt.Errorf("scan bookmark: %w", err)
}
out = append(out, b)
}
return out, rows.Err()
}
// Get returns one bookmark of one reader by key. A missing key is not an
// error: ok is false and err is nil. UI mutations read-modify-write through
// this so they preserve the fields they do not touch.
func (s *Store) Get(readerID int64, key string) (Bookmark, bool, error) {
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
return Bookmark{}, false, nil
}
b, err := s.scanBookmark(s.db.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks b
JOIN series s ON s.site = b.site AND s.series_id = b.series_id
WHERE b.reader_id = $1 AND b.site = $2 AND b.series_id = $3`,
readerID, site, seriesID).Scan)
if errors.Is(err, sql.ErrNoRows) {
return Bookmark{}, false, nil
}
if err != nil {
return Bookmark{}, false, fmt.Errorf("get %q: %w", key, err)
}
return b, true, nil
}
// Upsert inserts or replaces one reader's bookmark by key (last-write-wins)
// and returns the row as actually stored — one flat object with the
// series-owned fields joined in, exactly as GET reports it (ADR-0004). A
// bookmark is keyed (reader_id, site, series_id), so the same key upserts two
// independent rows for two readers.
//
// The flat body is decomposed across two tables in one transaction. The series
// row is written first (the bookmarks FK requires it to exist), then the
// bookmark row. On the series side, title/series_url/cover are applied only
// when the row is brand new: once a series exists, client-supplied values are
// ignored, because the row is shared and the values are scraped page content —
// see ADR-0003. Kind and the latest-chapter fields are last-write-wins.
//
// b.UpdatedAt is only a candidate: it is applied when the row is new or when
// last_chapter_num changes, and otherwise the stored value is kept. Clients
// order their list by updated_at, so favoriting a series or recording a newly
// published chapter must not disturb that order — only real reading progress
// does. Callers must therefore use the returned bookmark, not the argument.
func (s *Store) Upsert(readerID int64, b Bookmark) (Bookmark, error) {
tx, err := s.db.Begin()
if err != nil {
return Bookmark{}, fmt.Errorf("begin %q: %w", b.Key, err)
}
defer tx.Rollback()
var latestNum any
if b.LatestChapterNum != nil {
latestNum = *b.LatestChapterNum
}
// The kind column resolves on the VALUES side, not in the conflict clause:
// excluded.* is the row *after* these expressions are evaluated, so a
// default applied there would look identical to a real 'manga' and would
// overwrite a novel series on every PUT from a client that knows nothing
// about the column. Resolved once here, an empty incoming kind means "keep
// what is stored", and only a brand-new row falls through to the literal
// default. The subquery runs inside this transaction, so it sees the row
// this statement is about to conflict with. Same pattern as the status
// COALESCE on the bookmark insert below.
//
// The ::text casts are load-bearing: inside COALESCE/NULLIF there is no
// target column to infer the parameter type from, and Postgres rejects the
// statement rather than guessing.
//
// The cover columns are absent on purpose: the Cover is acquired
// server-side (ADR-0007), so a client-supplied one is not written even
// when the row is brand new.
//
// xmax is zero only on a row this statement inserted, which is how a
// Series nobody had bookmarked before is told apart from one that already
// existed — DO UPDATE returns a row either way.
var created bool
if err := tx.QueryRow(`
INSERT INTO series (site, series_id, title, series_url, kind,
latest_chapter, latest_chapter_num)
VALUES ($1, $2, $3, $4,
COALESCE(NULLIF($5::text, ''), (SELECT kind FROM series WHERE site = $1 AND series_id = $2), 'manga'),
$6, $7)
ON CONFLICT (site, series_id) DO UPDATE SET
kind=excluded.kind,
latest_chapter=excluded.latest_chapter,
latest_chapter_num=excluded.latest_chapter_num
RETURNING xmax = 0`,
b.Site, b.SeriesID, b.Title, b.SeriesURL, b.Kind,
b.LatestChapter, latestNum).Scan(&created); err != nil {
return Bookmark{}, fmt.Errorf("upsert series for %q: %w", b.Key, err)
}
// IS DISTINCT FROM is Postgres's null-safe comparison, and it is what
// implements the ordering rule. Within DO UPDATE, a bare column is the
// stored row and excluded.* is the incoming one; a brand-new key never
// reaches this clause, so it keeps the fresh timestamp from VALUES.
if _, err := tx.Exec(`
INSERT INTO bookmarks (reader_id, site, series_id, last_chapter, last_chapter_num,
last_chapter_url, favorite, status, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7,
COALESCE(NULLIF($8::text, ''), (SELECT status FROM bookmarks WHERE reader_id = $1 AND site = $2 AND series_id = $3), 'reading'),
$9)
ON CONFLICT (reader_id, site, series_id) DO UPDATE SET
last_chapter=excluded.last_chapter, last_chapter_num=excluded.last_chapter_num,
last_chapter_url=excluded.last_chapter_url,
favorite=excluded.favorite,
status=excluded.status,
updated_at=CASE
WHEN bookmarks.last_chapter_num IS DISTINCT FROM excluded.last_chapter_num
THEN excluded.updated_at
ELSE bookmarks.updated_at
END`,
readerID, b.Site, b.SeriesID,
b.LastChapter, b.LastChapterNum, b.LastChapterURL,
b.Favorite, b.Status, b.UpdatedAt); err != nil {
return Bookmark{}, fmt.Errorf("upsert %q: %w", b.Key, err)
}
stored, err := s.scanBookmark(tx.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks b
JOIN series s ON s.site = b.site AND s.series_id = b.series_id
WHERE b.reader_id = $1 AND b.site = $2 AND b.series_id = $3`,
readerID, b.Site, b.SeriesID).Scan)
if err != nil {
return Bookmark{}, fmt.Errorf("read back %q: %w", b.Key, err)
}
if err := tx.Commit(); err != nil {
return Bookmark{}, fmt.Errorf("commit %q: %w", b.Key, err)
}
// After commit, never inside the transaction: the hook reaches a
// third-party Site, and the Reader's write must not wait on it.
if created && s.OnSeriesCreated != nil {
s.OnSeriesCreated(Series{
Site: b.Site, SeriesID: b.SeriesID, Title: stored.Title,
SeriesURL: stored.SeriesURL, Kind: stored.Kind,
})
}
return stored, nil
}
// Delete removes one reader's bookmark by key. Deleting a missing key is not
// an error.
func (s *Store) Delete(readerID int64, key string) error {
site, seriesID, ok := strings.Cut(key, ":")
if !ok {
return nil
}
if _, err := s.db.Exec(
`DELETE FROM bookmarks WHERE reader_id = $1 AND site = $2 AND series_id = $3`,
readerID, site, seriesID); err != nil {
return fmt.Errorf("delete %q: %w", key, err)
}
return nil
}
// DueForLatestCheck returns series whose server-side latest-chapter check has
// aged past the appropriate cutoff, ordered by how many bookmarks reference
// them (descending) then least-recently-checked first, at most limit of them.
// Browser-backed sites use browserCutoffMs; every other site uses cutoffMs.
//
// The reader_count ordering is the point of the split (ADR-0003): a series
// shared by several readers is fetched once per due cycle, and the popular
// ones stay freshest while the long tail absorbs any shortfall. Within one
// reader count, oldest-first keeps the poll fair when the backlog outgrows
// throughput: the most neglected series is always next, so a large collection
// refreshes uniformly slower rather than leaving a tail that never refreshes
// at all. The userscript sorts its own queue the same way (L453).
//
// Series with no series_url are skipped — there is nothing to fetch, which is
// the same filter the userscript applies at L452. Series whose only bookmarks
// are finished are skipped too: nothing more is coming, so fetching them only
// burns requests. Archived bookmarks still count — knowing what a shelved
// series is up to is the whole reason for archiving instead of deleting.
// A series with no bookmarks at all never appears: the join excludes it.
func (s *Store) DueForLatestCheck(cutoffMs, browserCutoffMs int64, browserSites []string, limit int) ([]Series, error) {
rows, err := s.db.Query(`SELECT `+seriesColumns+`, COUNT(*) AS reader_count
FROM series s
JOIN bookmarks b ON b.site = s.site AND b.series_id = s.series_id
WHERE s.series_url <> ''
AND s.latest_checked_at <= CASE
WHEN s.site = ANY($3::text[]) THEN $2::bigint
ELSE $1::bigint
END
GROUP BY s.site, s.series_id, s.title, s.series_url, s.cover,
s.kind, s.latest_chapter, s.latest_chapter_num, s.latest_checked_at
HAVING COUNT(*) FILTER (WHERE b.status <> 'finished') > 0
ORDER BY reader_count DESC, s.latest_checked_at ASC
LIMIT $4`, cutoffMs, browserCutoffMs, pgtype.FlatArray[string](browserSites), limit)
if err != nil {
return nil, fmt.Errorf("query due series: %w", err)
}
defer rows.Close()
out := []Series{}
for rows.Next() {
sr, err := scanSeries(rows.Scan)
if err != nil {
return nil, fmt.Errorf("scan due series: %w", err)
}
out = append(out, sr)
}
return out, rows.Err()
}
// MarkLatestChecked records that the server looked at a series at ts, whatever
// the look turned up. Marking a missing series is not an error: the row may
// have been orphaned while a fetch was in flight.
//
// This is the one write that does not go through Upsert, and the column is kept
// out of the client-visible read path on purpose. PUT /bookmarks/{key} decodes
// a whole Bookmark from the client and Upsert writes every series column it
// knows about, so a userscript PUT — which has no idea this field exists —
// would write a zero and reset the cooldown, making the poller re-fetch that
// series every tick for as long as the user kept reading it.
func (s *Store) MarkLatestChecked(site, seriesID string, ts int64) error {
if _, err := s.db.Exec(
`UPDATE series SET latest_checked_at = $1 WHERE site = $2 AND series_id = $3`,
ts, site, seriesID); err != nil {
return fmt.Errorf("mark checked %s:%s: %w", site, seriesID, err)
}
return nil
}
// LatestCheckedAt reads the column MarkLatestChecked writes. It exists for
// tests outside this package (the poller's own tests assert on cooldown
// bookkeeping) — see MarkLatestChecked for why the field stays off the
// client-visible row.
func (s *Store) LatestCheckedAt(site, seriesID string) (int64, error) {
var ts int64
if err := s.db.QueryRow(
`SELECT latest_checked_at FROM series WHERE site = $1 AND series_id = $2`,
site, seriesID).Scan(&ts); err != nil {
return 0, fmt.Errorf("latest checked at %s:%s: %w", site, seriesID, err)
}
return ts, nil
}
// SetLatestChapter records the newest chapter the poll found on a series page.
// The poller walks Series rather than Bookmarks, so this is a series-level
// write: the row is shared, and updating it once refreshes every bookmark that
// joins to it. Touching a missing series is not an error.
func (s *Store) SetLatestChapter(site, seriesID, label string, num float64) error {
if _, err := s.db.Exec(
`UPDATE series SET latest_chapter = $3, latest_chapter_num = $4
WHERE site = $1 AND series_id = $2`,
site, seriesID, label, num); err != nil {
return fmt.Errorf("set latest chapter %s:%s: %w", site, seriesID, err)
}
return nil
}
File diff suppressed because it is too large Load Diff
+37
View File
@@ -0,0 +1,37 @@
package token
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"strconv"
)
// Token derives one Reader's userscript credential from the deployment
// secret, the Reader's Discord id and their token epoch.
//
// The credential is deterministic rather than stored random because the
// server must be able to rebuild the install URL after a restart while the
// database holds only hashes: a random token with no plaintext copy anywhere
// would be unreconstructible, and keeping plaintext in memory would break
// every install link on restart. HMAC output is high-entropy, indistinguishable
// from random to anyone without the secret, and changes whenever the epoch
// does — which is what rotation is. The stored form is Hash of this value,
// so a database leak yields nothing but hashes of unguessable strings.
func Token(key []byte, discordID string, epoch int64) string {
mac := hmac.New(sha256.New, key)
// The separator is unambiguous: discord ids are decimal snowflakes and
// epochs are plain integers, so no two (id, epoch) pairs can collide.
mac.Write([]byte(discordID))
mac.Write([]byte{0})
mac.Write([]byte(strconv.FormatInt(epoch, 10)))
return hex.EncodeToString(mac.Sum(nil))
}
// Hash is the SHA-256 of a credential — the only form that ever touches the
// database (readers.token_sha256). SHA-256 rather than a password hash is
// deliberate: these are unguessable values with nothing to brute-force, so a
// slow hash would only add per-request cost.
func Hash(cred string) [32]byte {
return sha256.Sum256([]byte(cred))
}
+53
View File
@@ -0,0 +1,53 @@
package token
import (
"bytes"
"crypto/sha256"
"testing"
)
func TestTokenDeterministicPerReaderAndEpoch(t *testing.T) {
key := []byte("deployment-secret")
a := Token(key, "reader-1", 0)
b := Token(key, "reader-1", 0)
if a != b {
t.Fatal("same (reader, epoch) derived different credentials")
}
if a == Token(key, "reader-2", 0) {
t.Fatal("different readers derived the same credential")
}
if a == Token(key, "reader-1", 1) {
t.Fatal("rotation epoch derived the same credential")
}
}
func TestTokenChangesWithSecret(t *testing.T) {
a := Token([]byte("key-1"), "reader-1", 0)
b := Token([]byte("key-2"), "reader-1", 0)
if a == b {
t.Fatal("different secrets derived the same credential")
}
}
func TestTokenFormat(t *testing.T) {
cred := Token([]byte("key"), "reader-1", 0)
// 32 bytes of HMAC-SHA256, hex-encoded: the length the install URL and
// the committed placeholder both assume.
if len(cred) != 64 {
t.Fatalf("credential length = %d, want 64", len(cred))
}
for _, c := range cred {
if !(c >= '0' && c <= '9' || c >= 'a' && c <= 'f') {
t.Fatalf("credential contains non-hex byte %q", c)
}
}
}
func TestHashIsSha256OfCredential(t *testing.T) {
cred := Token([]byte("key"), "reader-1", 0)
got := Hash(cred)
want := sha256.Sum256([]byte(cred))
if !bytes.Equal(got[:], want[:]) {
t.Fatal("Hash is not the SHA-256 of the credential")
}
}
+100
View File
@@ -0,0 +1,100 @@
package userscript
import (
"bytes"
"log"
"net/http"
"os"
"regexp"
"time"
"bookmarkmanager/backend/internal/httpmw"
"bookmarkmanager/backend/internal/store"
)
// tokenPlaceholder is what the bindmounted userscript carries where the
// Reader's credential goes: in the API_TOKEN constant and in the @downloadURL
// and @updateURL metadata lines. The handler substitutes the requesting
// Reader's credential for it at serve time, so no credential literal is ever
// committed or deployed, and each Reader's copy carries exactly their own.
var tokenPlaceholder = []byte("__API_TOKEN__")
// versionLine matches the userscript metadata block's @version directive.
var versionLine = regexp.MustCompile(`(?m)^// @version[ \t]+.*$`)
// stampVersion replaces the served @version with one derived from the file's
// mtime, discarding whatever the file body says.
//
// Violentmonkey only updates when the served version sorts higher than the
// installed one. Deriving it from the body means one accidental downgrade or
// typo freezes updates forever; an mtime-derived version is monotonic by
// construction, so any later write always outranks any earlier one.
//
// A file with no @version line is returned untouched: such a script never
// auto-updates anyway, and inventing a metadata block is not this handler's job.
func stampVersion(src []byte, mod time.Time) []byte {
return versionLine.ReplaceAll(src, []byte("// @version "+mod.UTC().Format("2006.01.02.1504")))
}
// substituteToken replaces every tokenPlaceholder with the Reader's
// credential. A file without the placeholder is returned unchanged so Render
// can warn about it rather than silently serving a credential-less script.
func substituteToken(src []byte, credential string) []byte {
return bytes.ReplaceAll(src, tokenPlaceholder, []byte(credential))
}
// Render writes one userscript file with the credential substituted and the
// mtime-derived version stamped. Shared by the download path (Handler) and
// the web UI's install endpoints, so both serve byte-identical scripts.
//
// The file is read per request — that is what lets a bindmounted copy be
// edited on the host without a restart. It is ~50 KB and polled about once a
// day.
func Render(w http.ResponseWriter, r *http.Request, path, credential string) {
info, err := os.Stat(path)
if err != nil {
log.Printf("userscript: stat %s: %v", path, err)
http.NotFound(w, r)
return
}
src, err := os.ReadFile(path)
if err != nil {
log.Printf("userscript: read %s: %v", path, err)
http.NotFound(w, r)
return
}
rendered := substituteToken(src, credential)
if bytes.Equal(rendered, src) {
// The bindmounted file was not built for per-Reader rendering. Serving
// it as written is the operator's freedom, but a credential-less copy
// is a deployment bug worth one log line — the symptom (silent 401s on
// every device) is otherwise indistinguishable from a network fault.
log.Printf("userscript: %s has no %s placeholder; serving as written", path, tokenPlaceholder)
}
w.Header().Set("Content-Type", "text/javascript; charset=utf-8")
w.Header().Set("Cache-Control", "no-cache")
w.Write(stampVersion(rendered, info.ModTime()))
}
// Handler serves the userscript to Violentmonkey's updater, rendered for the
// Reader whose credential is in the path.
//
// The credential lives in the path because the update poll sends no
// Authorization header, and the rendered file embeds the credential in
// plaintext, so an open path would hand it to anyone who guessed the URL. A
// mismatch answers 404 rather than 401: a prober learns nothing about whether
// the route exists. The same credential authenticates the API bearer header,
// so the two are one secret with one blast radius.
//
// The path segment is the credential itself, so once it resolves it is also
// exactly what the served copy must carry — no re-derivation needed.
func Handler(s *store.Store, path string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
cred := r.PathValue("token")
if _, ok := httpmw.ResolveReader(s, cred); !ok {
http.NotFound(w, r)
return
}
Render(w, r, path, cred)
}
}
@@ -0,0 +1,67 @@
package userscript
import (
"strings"
"testing"
"time"
)
// sampleScript is a stand-in for the real userscript: a metadata block with a
// @version line, the credential placeholder in its metadata and body, plus
// content that must survive the rewrites untouched.
const sampleScript = `// ==UserScript==
// @name Manga Bookmark Sync
// @version 1.5.0
// @downloadURL https://api.example/u/__API_TOKEN__/manga-bookmark.user.js
// @match https://asurascans.com/*
// ==/UserScript==
(function () { "use strict";
const API_TOKEN = "__API_TOKEN__";
})();
`
func TestStampVersionReplacesVersionLineOnly(t *testing.T) {
mod := time.Date(2026, 7, 28, 16, 42, 0, 0, time.UTC)
got := string(stampVersion([]byte(sampleScript), mod))
if !strings.Contains(got, "// @version "+mod.UTC().Format("2006.01.02.1504")) {
t.Errorf("body has no stamped version:\n%s", got)
}
if strings.Contains(got, "1.5.0") {
t.Errorf("body still carries the file's own version:\n%s", got)
}
// Everything outside the @version line is served verbatim, including the
// placeholder — stamping must not do the substitution's job.
if !strings.Contains(got, `const API_TOKEN = "__API_TOKEN__";`) {
t.Errorf("body was altered beyond the version line:\n%s", got)
}
}
func TestStampVersionWithoutVersionLineServedUnmodified(t *testing.T) {
const noVersion = "// ==UserScript==\n// @name x\n// ==/UserScript==\nconsole.log(1);\n"
if got := string(stampVersion([]byte(noVersion), time.Now())); got != noVersion {
t.Errorf("stampVersion altered a file with no @version line:\n%s", got)
}
}
func TestSubstituteTokenReplacesEveryPlaceholder(t *testing.T) {
got := string(substituteToken([]byte(sampleScript), "abc123"))
if strings.Contains(got, "__API_TOKEN__") {
t.Errorf("placeholder survived substitution:\n%s", got)
}
// The credential lands in the constant and in both metadata lines.
if want := `const API_TOKEN = "abc123";`; !strings.Contains(got, want) {
t.Errorf("no substituted constant %q:\n%s", want, got)
}
if want := "https://api.example/u/abc123/manga-bookmark.user.js"; !strings.Contains(got, want) {
t.Errorf("no substituted download URL %q:\n%s", want, got)
}
}
func TestSubstituteTokenWithoutPlaceholderServedUnmodified(t *testing.T) {
const noPlaceholder = "// ==UserScript==\n// @name x\n// ==/UserScript==\n"
if got := string(substituteToken([]byte(noPlaceholder), "abc123")); got != noPlaceholder {
t.Errorf("substituteToken altered a file without the placeholder:\n%s", got)
}
}
+320
View File
@@ -0,0 +1,320 @@
package web
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"net/http"
"net/url"
"slices"
"strconv"
"strings"
"sync"
"time"
"bookmarkmanager/backend/internal/session"
"bookmarkmanager/backend/internal/token"
)
const (
// oauthStateTTL bounds how long a started sign-in stays valid. Ten
// minutes is generous for Discord's round trip and short enough that a
// captured state is stale before it is worth replaying.
oauthStateTTL = 10 * time.Minute
// maxStates caps the state map so a flood of /auth/discord hits cannot
// grow memory; past the cap the oldest state is evicted, which at worst
// invalidates an in-flight sign-in.
maxStates = 256
// maxResponseBytes caps Discord API bodies; they are small, and an
// unbounded read is an OOM handed to Discord's CDN.
maxResponseBytes = 1 << 20
// discordTimeout keeps a hung Discord request from hanging the login
// callback forever.
discordTimeout = 15 * time.Second
)
// DiscordConfig is the OAuth application this service registers as, plus the
// guild that gates access.
type DiscordConfig struct {
ClientID string
ClientSecret string
GuildID string
// RequiredRole, when non-empty, is a role ID a member must hold on top of
// guild membership. Empty by default: membership alone suffices.
RequiredRole string
// APIBase is the Discord API root; configurable so tests run the whole
// flow against a local stub.
APIBase string
// RedirectURI is the full public URL of the callback — Discord requires
// the exact string, so it is configured, never derived from headers.
RedirectURI string
}
// oauthStates stores one-time sign-in states. A state is generated at
// /auth/discord, echoed back by Discord at the callback, and consumed there.
type oauthStates struct {
mu sync.Mutex
expiry map[string]time.Time
}
func newOAuthStates() *oauthStates {
return &oauthStates{expiry: make(map[string]time.Time)}
}
func (s *oauthStates) put(state string, expires time.Time) {
s.mu.Lock()
defer s.mu.Unlock()
now := time.Now()
for k, at := range s.expiry {
if !at.After(now) {
delete(s.expiry, k)
}
}
// Evict the state closest to expiring when full, so a flood of starts
// cannot grow memory; at worst it invalidates an in-flight sign-in.
if len(s.expiry) >= maxStates {
var oldest string
var oldestAt time.Time
for k, at := range s.expiry {
if oldest == "" || at.Before(oldestAt) {
oldest, oldestAt = k, at
}
}
delete(s.expiry, oldest)
}
s.expiry[state] = expires
}
// take validates and consumes a state in one step: a state works exactly
// once, which is what makes a replayed callback useless.
func (s *oauthStates) take(state string) bool {
s.mu.Lock()
defer s.mu.Unlock()
expires, ok := s.expiry[state]
if !ok || !expires.After(time.Now()) {
return false
}
delete(s.expiry, state)
return true
}
// discordStart begins the authorization code grant: a fresh state, then a
// redirect to Discord's authorize page.
func (h *Handler) discordStart(w http.ResponseWriter, r *http.Request) {
state := session.NewID()
h.states.put(state, time.Now().Add(oauthStateTTL))
u := h.discord.APIBase + "/oauth2/authorize?" + url.Values{
"client_id": {h.discord.ClientID},
"redirect_uri": {h.discord.RedirectURI},
"response_type": {"code"},
"scope": {"identify guilds.members.read"},
"state": {state},
}.Encode()
http.Redirect(w, r, u, http.StatusSeeOther)
}
// discordCallback completes the grant: exchange the code, verify identity,
// membership and role, then mint a session. Every failure path renders the
// login page with an author-written message — nothing Discord supplied is
// ever interpolated into a page, and no secret reaches a log line.
func (h *Handler) discordCallback(w http.ResponseWriter, r *http.Request) {
ip := session.ClientIP(r)
if wait := h.limiter.RetryAfter(ip, time.Now()); wait > 0 {
secs := int(wait.Seconds()) + 1
w.Header().Set("Retry-After", strconv.Itoa(secs))
h.renderLogin(w, http.StatusTooManyRequests,
"Too many attempts. Try again in "+strconv.Itoa((secs+59)/60)+" min.")
return
}
// Discord refuses the grant (the reader hit cancel, or the application
// was misconfigured). The state is consumed so the flow is cleanly over;
// this makes no Discord calls, so it is not a failure the limiter counts.
if oerr := r.URL.Query().Get("error"); oerr != "" {
h.states.take(r.URL.Query().Get("state"))
h.renderLogin(w, http.StatusBadRequest, "Sign-in was cancelled.")
return
}
code := r.URL.Query().Get("code")
if code == "" || !h.states.take(r.URL.Query().Get("state")) {
h.limiter.Fail(ip, time.Now())
h.renderLogin(w, http.StatusBadRequest,
"This sign-in link was invalid or already used. Start again.")
return
}
tok, err := h.exchangeToken(r.Context(), code)
if err != nil {
h.limiter.Fail(ip, time.Now())
log.Printf("discord token exchange: %v", err)
h.renderLogin(w, http.StatusBadGateway,
"Discord sign-in is unavailable right now. Try again in a moment.")
return
}
userID, err := h.discordUserID(r.Context(), tok.AccessToken)
if err != nil {
h.limiter.Fail(ip, time.Now())
log.Printf("discord users/@me: %v", err)
h.renderLogin(w, http.StatusBadGateway,
"Discord sign-in is unavailable right now. Try again in a moment.")
return
}
member, isMember, err := h.discordMember(r.Context(), tok.AccessToken)
if err != nil {
h.limiter.Fail(ip, time.Now())
log.Printf("discord member check: %v", err)
h.renderLogin(w, http.StatusBadGateway,
"Discord sign-in is unavailable right now. Try again in a moment.")
return
}
// The refusal is the same for a non-member and a member without the
// required role, and it names neither the guild nor its id: an outsider
// cannot tell whether the guild exists, let alone which one gates.
//
// It also returns before EnsureReader, so a refused sign-in leaves no
// Reader row behind — the gate is the only thing standing between guild
// membership and a library.
if !isMember || (h.discord.RequiredRole != "" && !slices.Contains(member.Roles, h.discord.RequiredRole)) {
h.limiter.Fail(ip, time.Now())
h.renderLogin(w, http.StatusForbidden,
"This Discord account is not a member of this community.")
return
}
// Registration is the login (issue #27): first sight of a guild member
// creates their Reader, every later sight returns the same one. Their
// userscript credential is derived at epoch 0 the way the owner's is, so
// the install links work before they have read anything.
readerID, err := h.store.EnsureReader(userID, token.Hash(token.Token(h.tokenKey, userID, 0)))
if err != nil {
log.Printf("register reader: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.limiter.Reset(ip)
sess, err := h.store.CreateSession(session.NewID(), readerID, session.SessionTTL)
if err != nil {
log.Printf("create session: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
session.SetCookie(w, r, sess.ID)
http.Redirect(w, r, "/", http.StatusSeeOther)
}
// exchangeToken trades an authorization code for an access token. The body is
// form-encoded because that is what Discord accepts — it rejects a JSON
// payload — so the wire format is fixed here, not in a client library.
func (h *Handler) exchangeToken(ctx context.Context, code string) (discordToken, error) {
form := url.Values{
"client_id": {h.discord.ClientID},
"client_secret": {h.discord.ClientSecret},
"grant_type": {"authorization_code"},
"code": {code},
"redirect_uri": {h.discord.RedirectURI},
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
h.discord.APIBase+"/oauth2/token", strings.NewReader(form.Encode()))
if err != nil {
return discordToken{}, err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.Header.Set("Accept", "application/json")
resp, err := h.httpClient.Do(req)
if err != nil {
return discordToken{}, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return discordToken{}, fmt.Errorf("status %d", resp.StatusCode)
}
var tok discordToken
if err := json.NewDecoder(io.LimitReader(resp.Body, maxResponseBytes)).Decode(&tok); err != nil {
return discordToken{}, err
}
if tok.AccessToken == "" {
return discordToken{}, errors.New("empty access token")
}
return tok, nil
}
// discordUserID fetches the signed-in user's id via the identify scope.
func (h *Handler) discordUserID(ctx context.Context, accessToken string) (string, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet,
h.discord.APIBase+"/users/@me", nil)
if err != nil {
return "", err
}
req.Header.Set("Authorization", "Bearer "+accessToken)
req.Header.Set("Accept", "application/json")
resp, err := h.httpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("status %d", resp.StatusCode)
}
var u struct {
ID string `json:"id"`
}
if err := json.NewDecoder(io.LimitReader(resp.Body, maxResponseBytes)).Decode(&u); err != nil {
return "", err
}
if u.ID == "" {
return "", errors.New("empty user id")
}
return u.ID, nil
}
type discordMember struct {
Roles []string `json:"roles"`
}
// discordMember fetches the current user's membership in the configured guild.
//
// This is the OAuth endpoint (Get Current User Guild Member), the one the
// guilds.members.read scope grants. Its bot-side twin, GET /guilds/{id}/
// members/{user}, reads almost identically and is the wrong one: it wants a
// Bot token and the application present in the guild, and answers a user
// Bearer token with 401 — which fails as an outage rather than a refusal, so
// nobody could sign in at all.
//
// A 404 or 403 (not in the guild, or the token lacks the scope) is a
// non-member, not an error.
func (h *Handler) discordMember(ctx context.Context, accessToken string) (discordMember, bool, error) {
u := h.discord.APIBase + "/users/@me/guilds/" +
url.PathEscape(h.discord.GuildID) + "/member"
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
if err != nil {
return discordMember{}, false, err
}
req.Header.Set("Authorization", "Bearer "+accessToken)
req.Header.Set("Accept", "application/json")
resp, err := h.httpClient.Do(req)
if err != nil {
return discordMember{}, false, err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusForbidden {
return discordMember{}, false, nil
}
if resp.StatusCode != http.StatusOK {
return discordMember{}, false, fmt.Errorf("status %d", resp.StatusCode)
}
var m discordMember
if err := json.NewDecoder(io.LimitReader(resp.Body, maxResponseBytes)).Decode(&m); err != nil {
return discordMember{}, false, err
}
return m, true, nil
}
type discordToken struct {
AccessToken string `json:"access_token"`
}
+55
View File
@@ -0,0 +1,55 @@
package web
import (
"testing"
"time"
)
func TestOAuthStateSingleUse(t *testing.T) {
s := newOAuthStates()
s.put("st", time.Now().Add(time.Minute))
if !s.take("st") {
t.Fatal("take of a fresh state = false, want true")
}
if s.take("st") {
t.Fatal("take of a consumed state = true, want false")
}
}
func TestOAuthStateUnknownOrExpired(t *testing.T) {
s := newOAuthStates()
if s.take("never-seen") {
t.Fatal("take of an unknown state = true, want false")
}
s.put("stale", time.Now().Add(-time.Minute))
if s.take("stale") {
t.Fatal("take of an expired state = true, want false")
}
}
// The map is capped: a flood of starts evicts older states instead of growing,
// and consumed states must not change that.
func TestOAuthStateEviction(t *testing.T) {
s := newOAuthStates()
key := func(i, salt int) string {
return string(rune('a'+i%26)) + string(rune('0'+i/26+salt*16))
}
now := time.Now().Add(time.Hour)
for i := 0; i < maxStates*2; i++ {
s.put(key(i, 0), now)
}
if got := len(s.expiry); got != maxStates {
t.Fatalf("states after a flood = %d, want %d", got, maxStates)
}
// Consume everything, then flood again: the map stays bounded.
for state := range s.expiry {
s.take(state)
}
for i := 0; i < maxStates; i++ {
s.put(key(i, 1), now)
}
if got := len(s.expiry); got != maxStates {
t.Fatalf("states after consume+flood = %d, want %d", got, maxStates)
}
}
+188
View File
@@ -0,0 +1,188 @@
// Title search runs entirely in the browser: the full list is already in the
// DOM, so filtering it needs no request.
(function () {
function applyFilter() {
var box = document.getElementById("search");
if (!box) return;
var query = box.value.trim();
var needle = query.toLowerCase();
var cards = document.querySelectorAll(".card");
var visible = 0;
cards.forEach(function (card) {
var title = (card.dataset.title || "").toLowerCase();
card.hidden = needle !== "" && title.indexOf(needle) === -1;
if (!card.hidden) visible++;
});
// The server decides what the strip holds — it ships empty for every tab
// but All, and out of band after every mutation. All this has to do is keep
// it down while a filter is active, since the strip is never filtered and
// would otherwise put non-matching covers above an empty list.
var recent = document.querySelector(".recent");
if (recent) {
recent.hidden = needle !== "" || !recent.querySelector(".recent-card");
}
// An empty bucket already explains itself server-side; this only speaks
// when the filter is what emptied the screen.
var none = document.getElementById("no-match");
if (none) {
none.hidden = !(needle !== "" && cards.length > 0 && visible === 0);
if (!none.hidden) none.querySelector(".no-match-q").textContent = query;
}
}
document.addEventListener("input", function (e) {
if (e.target && e.target.id === "search") applyFilter();
});
document.addEventListener("click", function (e) {
if (!e.target || !e.target.classList.contains("clear-search")) return;
var box = document.getElementById("search");
box.value = "";
applyFilter();
box.focus();
});
// htmx replaces the list on a tab switch, so re-apply to the new cards.
document.body.addEventListener("htmx:afterSwap", applyFilter);
document.addEventListener("bmgr:refilter", applyFilter);
})();
function setActiveTab(el) {
el.parentElement.querySelectorAll("a").forEach(function (t) {
var on = t === el;
t.classList.toggle("active", on);
if (on) t.setAttribute("aria-current", "page");
else t.removeAttribute("aria-current");
});
// The strip is outside the swapped region, so its visibility is re-decided
// here rather than by the server that just answered.
document.dispatchEvent(new Event("bmgr:refilter"));
}
// The chapter-edit form and the archive/finish/remove confirm rows are the
// per-card disclosure panels; only one makes sense open at a time. The button
// that owns an open panel carries .open, which is how the strip shows which
// cell the panel belongs to.
function closeCardPanels(key) {
var card = document.getElementById("card-" + key);
if (!card) return;
card.querySelectorAll(".chapter-form, .confirm-row").forEach(function (p) {
p.hidden = true;
});
card.querySelectorAll(".actions .open").forEach(function (b) {
b.classList.remove("open");
b.setAttribute("aria-expanded", "false");
});
}
function togglePanel(key, panelId, buttonSelector) {
var panel = document.getElementById(panelId);
if (!panel) return null;
var opening = panel.hidden;
closeCardPanels(key);
panel.hidden = !opening;
var card = document.getElementById("card-" + key);
var button = card && card.querySelector(buttonSelector);
if (button) {
button.classList.toggle("open", opening);
button.setAttribute("aria-expanded", opening ? "true" : "false");
}
return panel;
}
function toggleChapterForm(key) {
var form = togglePanel(key, "chapter-form-" + key, ".actions .pencil");
if (form && !form.hidden) form.querySelector("input").focus();
}
// kind is "archive" | "finish" | "remove" — the panel id and the owning action
// cell share it.
function toggleConfirmRow(key, kind) {
var cls = { archive: ".box", finish: ".finish", remove: ".remove" }[kind];
var row = togglePanel(key, "confirm-" + kind + "-" + key, ".actions " + cls);
// Focus the answer rather than trusting aria-live on a container that merely
// unhides: it makes the announcement deterministic, keeps tab order inside
// the confirm instead of running on into the next card, and means the row
// cannot be opened and scrolled past unnoticed.
// The reversible rows open on their affirmative; remove opens on Cancel.
// Focusing the first button in DOM order would hand the irreversible action
// a pre-armed Enter, which is the opposite of what a confirm gate is for.
if (row && !row.hidden) {
row.querySelector(row.classList.contains("calm") ? "button" : "button + button").focus();
}
}
// Esc closes whichever panel this card has open and hands focus back to the
// cell that owns it — otherwise the only way out is finding that exact cell
// again.
document.addEventListener("keydown", function (e) {
if (e.key !== "Escape" || !e.target.closest) return;
var card = e.target.closest(".card");
var owner = card && card.querySelector(".actions .open");
if (!owner) return;
closeCardPanels(card.id.replace(/^card-/, ""));
owner.focus();
});
// A failed favourite/chapter/delete request leaves the card in place (htmx
// does not swap on a non-2xx response) but otherwise gives no sign anything
// went wrong. Surface it inline instead of leaving the tap looking ignored.
(function () {
function showError(elt, message, linkHref, linkText) {
var card = elt.closest(".card");
var slot = card && card.querySelector(".error-inline");
if (!slot) return;
slot.textContent = message;
if (linkHref) {
var a = document.createElement("a");
a.href = linkHref;
a.textContent = linkText;
a.className = "error-link";
slot.append(" ", a);
}
slot.hidden = false;
// No self-destruct timer: this reader gets interrupted mid-tap, and a
// notice that expires after 5s leaves a failed write with no trace at all
// — the star is back off and nothing says why. The notice stays until the
// next request from this card clears it (or a successful one swaps the
// whole card away).
slot.scrollIntoView({
block: "nearest",
behavior: matchMedia("(prefers-reduced-motion: reduce)").matches ? "auto" : "smooth",
});
}
document.body.addEventListener("htmx:beforeRequest", function (e) {
var card = e.detail.elt.closest(".card");
var slot = card && card.querySelector(".error-inline");
if (slot) slot.hidden = true;
});
// The handlers answer a bad value with http.Error, i.e. a short plain-text
// line — worth showing verbatim. Anything long or HTML-ish is an error page,
// not a reason, so fall back to the generic copy.
function reasonFrom(xhr) {
var body = (xhr.responseText || "").trim();
if (!body || body.length > 120 || body.indexOf("<") === 0) return "";
return body.charAt(0).toUpperCase() + body.slice(1);
}
document.body.addEventListener("htmx:responseError", function (e) {
var xhr = e.detail.xhr;
if (xhr.status === 401) {
showError(e.detail.elt, "Session expired — nothing was saved.", "/login", "Log in again");
return;
}
if (xhr.status === 400) {
var reason = reasonFrom(xhr);
showError(e.detail.elt, reason ? reason + "." : "That value wasn't accepted — check it and try again.");
return;
}
showError(e.detail.elt, "Couldn't save — try again.");
});
document.body.addEventListener("htmx:sendError", function (e) {
showError(e.detail.elt, "No connection — try again.");
});
})();
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.7 MiB

+23
View File
@@ -0,0 +1,23 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 200 172" role="img" aria-label="BookmarkManager">
<title>BookmarkManager</title>
<g fill="#100f0e" stroke="#f2ece5" stroke-width="5" stroke-linejoin="round" stroke-linecap="round">
<path fill="none" d="M28 36H4v114h192V36h-24"></path>
<path fill="none" d="M28 23H17v127h166V23h-11"></path>
<g id="mb-half">
<path d="M28 7 88 55v97L28 138z"></path>
<g fill="#f2ece5" stroke="none">
<path d="M37 25 55 39v41L37 66z"></path>
<path d="M60 42 79 57v42L60 84z"></path>
<path d="M37 75 79 108v13L37 88z"></path>
<path d="M37 98 79 129v11L37 131z"></path>
</g>
</g>
<use href="#mb-half" transform="matrix(-1 0 0 1 200 0)"></use>
<g stroke="#e0452c">
<path d="M100 4l9 5v11l-9 5-9-5V9z"></path>
<path d="M94 24h12v24H94z"></path>
<path d="M70 47h60v14H70z"></path>
<path d="M91 61h18v87l-9 20-9-20z"></path>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 977 B

@@ -64,7 +64,9 @@
--paper-hot: #f0d3cb; /* title of a series with a new chapter */
--paper-dim: #ddd5cb; /* resting title */
--mute: #8d857c; /* secondary text, idle icons */
--mute-2: #5a5450; /* labels, hints */
/* Every mute-2 use is 10px mono, so it has to clear 4.5:1 on --ink even
though it reads as the quietest step. */
--mute-2: #877f76; /* labels, hints */
--faint: #3a3733; /* meta separators */
--faint-2: #57504b; /* cover monogram */
@@ -72,11 +74,32 @@
--ember-wash: #1a1211; /* ember-tinted surface */
--ember-ink: #150907; /* text on solid ember */
--ember-soft: #eda798; /* text on ember wash */
/* Destruction is hot but not ember: a duller oxblood, so a remove confirm is
never mistaken across the room for an unread chapter. */
--danger: #cf5c4d;
--danger-wash: #211311;
--danger-ink: #150808;
--danger-soft: #e2aaa1;
--brass: #b8912f; /* favourite — a second, cooler metal */
--trash: #6b5450; /* remove, resting */
/* One accent per action, so a press says which lane it belongs to. All three
are held at the same weight as --brass: muted, no ember competition. */
--slate: #7fa0c0; /* archive */
--moss: #7fae86; /* finished */
--clay: #b5906f; /* set chapter */
--trash: #977671; /* remove, resting — icons need 3:1, not 4.5:1 */
/* Desktop cell borders for the two coloured action states. */
--play-hot-line: #3a1d18;
--fav-line: #332b14;
--asura: #7d93a5;
--demonic: #a98a78;
--comix: #8a9a7d;
--kagane: #9a8aa5;
/* novel sources: same muted family, two hues the manga sites do not use */
--novelfull: #a59a7d;
--lightnovelworld: #7da59a;
/* Covers are often missing; the hatch keeps the slot honest instead of
faking artwork. */
@@ -84,6 +107,10 @@
--hatch-dim: repeating-linear-gradient(135deg, #1b1918 0 5px, #151313 5px 10px);
--measure: 760px;
/* Cover width + row gap: the disclosure panels indent past the cover on
desktop, so both live here rather than as magic numbers. */
--cover-w: 93px;
--row-gap: 14px;
}
/* Light mode: same rules, cooler paper. Hues are re-tuned, not reused — the
@@ -102,7 +129,7 @@
--paper-hot: #a33018;
--paper-dim: #191715;
--mute: #6b645d;
--mute-2: #857d75;
--mute-2: #6c655e; /* clears 4.5:1 on --ash too, not just --ink */
--faint: #c9c2ba;
--faint-2: #a8a098;
@@ -110,11 +137,25 @@
--ember-wash: #fbeee9;
--ember-ink: #fff;
--ember-soft: #8d2c17;
--danger: #97362a;
--danger-wash: #fbe9e5;
--danger-ink: #fff;
--danger-soft: #7c2c22;
--brass: #8a681c;
--trash: #a98276;
--slate: #3f6689;
--moss: #3d6c46;
--clay: #7c5533;
--trash: #8c6558;
--play-hot-line: #f0cfc6;
--fav-line: #e3d3a4;
--asura: #4f6b80;
--demonic: #8a6a55;
--comix: #5f7250;
--kagane: #6f5f7d;
--novelfull: #7d6f4f;
--lightnovelworld: #4f7d70;
--hatch: repeating-linear-gradient(135deg, #e6e0d8 0 5px, #efeae3 5px 10px);
--hatch-dim: repeating-linear-gradient(135deg, #ebe6de 0 5px, #f2eee8 5px 10px);
@@ -140,9 +181,16 @@ button { cursor: pointer; }
/* Every hideable thing here is a flex container, and display beats hidden. */
[hidden] { display: none !important; }
@keyframes barSlide { from { transform: translateX(-100%) } to { transform: translateX(320%) } }
/* Ends flush with the right edge of the card (30% wide × 233% travel), so the
card itself never needs overflow: hidden to contain it. */
@keyframes barSlide { from { transform: translateX(-100%) } to { transform: translateX(233%) } }
@keyframes sheetIn { from { opacity: 0; transform: translateY(-4px) } to { opacity: 1; transform: none } }
@keyframes emberPulse { 0%, 100% { opacity: .5 } 50% { opacity: 1 } }
/* The law: ember means "new chapter", and nothing else on a list screen —
busy, error and destruction all stay off it, so a tired glance never
misreads a system state as unread heat. The one exception is the brand
itself (the wordmark, and the login screen, which shows no series at all).
Destruction has its own token, --danger. */
@keyframes mutePulse { 0%, 100% { opacity: .35 } 50% { opacity: 1 } }
/* One measured column, edges drawn rather than boxed. */
.sheet {
@@ -159,14 +207,21 @@ button { cursor: pointer; }
/* ---- brand + chrome ---- */
.brand {
margin: 0;
display: flex;
align-items: center;
gap: 10px;
font: 400 26px/1 var(--font-display);
color: var(--paper);
}
.brand em { color: var(--ember); font-style: italic; }
.brand .mark { width: 30px; height: 26px; flex: none; overflow: visible; }
/* The line art is drawn at a 5px stroke on a 200-unit grid; at brand size that
thins out, so it is nudged up rather than scaled down blindly. */
.brand .mark g { stroke-width: 6.5; }
.topbar {
display: flex;
align-items: baseline;
align-items: center;
justify-content: space-between;
gap: 16px;
padding: 22px 20px 14px;
@@ -174,6 +229,7 @@ button { cursor: pointer; }
.topbar form { margin: 0; }
.ghost {
position: relative;
padding: 0;
border: none;
border-bottom: 1px solid var(--field-line);
@@ -184,6 +240,82 @@ button { cursor: pointer; }
text-transform: uppercase;
}
.ghost:hover { color: var(--paper); border-bottom-color: var(--paper); }
/* The label is 15px tall by design; the thumb gets 44 without moving it. */
.ghost::after { content: ""; position: absolute; inset: -15px -12px; }
/* ---- userscript setup: collapsed by default, one hairline, no card ---- */
.setup {
margin: 0 20px;
padding: 12px 0 0;
border-bottom: 1px solid var(--rule);
color: var(--mute);
}
.setup summary {
display: flex;
align-items: center;
min-height: 44px;
padding: 0;
font: 500 10px/1 var(--font-mono);
letter-spacing: .2em;
text-transform: uppercase;
color: var(--mute-2);
cursor: pointer;
list-style: none;
}
.setup summary::-webkit-details-marker { display: none; }
.setup summary:hover { color: var(--paper); }
.setup[open] { padding-bottom: 16px; }
.setup-copy {
margin: 0;
padding: 4px 0 12px;
font: 14px/1.55 var(--font-body);
color: var(--mute);
}
.setup-links {
display: flex;
flex-wrap: wrap;
gap: 8px 20px;
margin: 0 0 14px;
}
.setup-links .ghost { font-size: 11px; }
.setup-rotate { margin: 0; }
/* Rotation confirmation: the one hot state the panel wears, and it is
destruction, not new-chapter signal — danger, never ember. */
.setup-warn {
margin: 0;
padding: 10px 12px;
border: 1px solid var(--danger);
color: var(--danger);
font: 500 12px/1.5 var(--font-mono);
letter-spacing: .04em;
}
/* ---- reader roster (owner only): same hairline panel, one row per Reader ---- */
.readerlist { margin: 0; padding: 0; list-style: none; }
.readerlist li {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 4px 16px;
min-height: 44px;
border-top: 1px solid var(--rule);
}
.readerlist form { margin: 0 0 0 auto; }
.reader-id {
font: 500 13px/1.4 var(--font-mono);
letter-spacing: .04em;
color: var(--paper);
}
.reader-sessions {
font: 500 10px/1 var(--font-mono);
letter-spacing: .14em;
text-transform: uppercase;
color: var(--mute);
}
/* Revocation cuts someone off, so it wears --danger. Ember stays reserved for
the new-chapter signal. */
.ghost.danger { color: var(--danger); }
.ghost.danger:hover { color: var(--danger); border-bottom-color: var(--danger); }
.chrome { display: flex; flex-direction: column; }
@@ -196,6 +328,9 @@ button { cursor: pointer; }
border-bottom: 1px solid var(--rule);
color: var(--mute-2);
}
/* The input drops its own outline, so the bar it sits in carries the focus
ring — same move the two other inputs make with their border. */
.searchbar:focus-within { border-bottom-color: var(--paper); color: var(--paper); }
.searchbar svg { width: 15px; height: 15px; flex: none; }
.search {
flex: 1;
@@ -220,18 +355,21 @@ button { cursor: pointer; }
border-bottom: 1px solid var(--rule);
}
.tabs::-webkit-scrollbar { display: none; }
.tabs [role="tab"] {
.tabs a {
flex: none;
display: flex;
align-items: center;
justify-content: center;
gap: 7px;
/* "All" is two characters in a 17px serif — 15px of target without this. */
min-width: 44px;
padding: 8px 0 12px;
color: var(--mute);
font: 400 17px var(--font-display);
white-space: nowrap;
}
.tabs [role="tab"]:hover { color: var(--paper-dim); }
.tabs [role="tab"].active {
.tabs a:hover { color: var(--paper-dim); }
.tabs a.active {
color: var(--paper);
border-bottom: 2px solid var(--paper);
margin-bottom: -1px;
@@ -246,6 +384,35 @@ button { cursor: pointer; }
border: 1px solid currentColor;
}
/* ---- action key: one permanent line under the tabs, so the icon strip below
never has to be guessed at. Lean on a phone (28px, edge to edge), a step
bigger on desktop where there is room to read it. ---- */
.keyrow {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: 10px;
padding: 9px 18px 10px;
border-bottom: 1px solid var(--rule);
}
/* On a phone each key stacks: icon over word, so the word gets the full cell
width and can stay the long form. */
.keyrow .pair {
display: flex;
flex-direction: column;
align-items: center;
gap: 6px;
color: var(--mute);
}
.keyrow .pair svg { width: 14px; height: 14px; flex: none; }
.keyrow .pair span {
font: 500 10px/1 var(--font-mono);
letter-spacing: .04em;
text-transform: uppercase;
}
.keyrow .pair.brass svg { color: var(--brass); }
.keyrow .pair.trash svg { color: var(--trash); }
/* ---- continue reading ---- */
.recent {
display: flex;
@@ -272,7 +439,7 @@ button { cursor: pointer; }
}
.recent-strip::-webkit-scrollbar { display: none; }
.recent-card {
width: 92px;
width: var(--cover-w);
flex: none;
display: flex;
flex-direction: column;
@@ -280,8 +447,8 @@ button { cursor: pointer; }
}
.recent-cover {
position: relative;
width: 92px;
height: 123px;
width: var(--cover-w);
height: calc(var(--cover-w) * 4 / 3);
background: var(--hatch);
display: grid;
place-items: center;
@@ -289,7 +456,7 @@ button { cursor: pointer; }
}
.recent-cover img { width: 100%; height: 100%; object-fit: cover; }
.recent-title {
font: 400 15px/1.2 var(--font-display);
font: 400 16px/1.25 var(--font-display);
color: var(--paper-dim);
display: -webkit-box;
-webkit-line-clamp: 2;
@@ -297,7 +464,7 @@ button { cursor: pointer; }
overflow: hidden;
}
.recent-chapter {
font: 500 10px/1 var(--font-mono);
font: 500 11px/1 var(--font-mono);
letter-spacing: .1em;
color: var(--mute-2);
}
@@ -335,12 +502,12 @@ button { cursor: pointer; }
}
.card.is-dim { background: var(--dim); }
.row { display: flex; flex-wrap: wrap; gap: 14px; }
.row { display: flex; flex-wrap: wrap; align-items: center; gap: var(--row-gap); }
.cover {
position: relative;
width: 66px;
height: 88px;
width: var(--cover-w);
height: calc(var(--cover-w) * 4 / 3);
flex: none;
background: var(--hatch);
display: grid;
@@ -348,7 +515,7 @@ button { cursor: pointer; }
overflow: hidden;
}
.cover img { width: 100%; height: 100%; object-fit: cover; }
.cover .monogram { font-size: 24px; }
.cover .monogram { font-size: 32px; }
.is-dim .cover { background: var(--hatch-dim); filter: grayscale(1); opacity: .85; }
.body {
@@ -356,6 +523,7 @@ button { cursor: pointer; }
min-width: 0;
display: flex;
flex-direction: column;
justify-content: center;
gap: 8px;
}
@@ -364,7 +532,7 @@ button { cursor: pointer; }
margin: 0;
flex: 1;
min-width: 0;
font: 400 19px/1.2 var(--font-display);
font: 400 21px/1.2 var(--font-display);
color: var(--paper-dim);
}
/* Heat: crimson title over an ember hairline sized to the text, not the row. */
@@ -377,7 +545,9 @@ button { cursor: pointer; }
}
.is-dim .title { font-style: italic; color: var(--mute); }
.is-dim .fav-mark { color: var(--mute-2); }
.fav-mark { flex: none; width: 13px; height: 13px; margin-top: 5px; color: var(--brass); }
/* The mark always sits at the right edge of the measure, not after the last
word — a new-chapter title shrink-wraps, so without this it drifts. */
.fav-mark { flex: none; margin-left: auto; width: 14px; height: 14px; margin-top: 5px; color: var(--brass); }
.meta {
margin: 0;
@@ -385,7 +555,7 @@ button { cursor: pointer; }
align-items: center;
gap: 9px;
flex-wrap: wrap;
font: 500 10px/1 var(--font-mono);
font: 500 11px/1 var(--font-mono);
letter-spacing: .12em;
text-transform: uppercase;
color: var(--mute);
@@ -393,11 +563,53 @@ button { cursor: pointer; }
.meta .sep { color: var(--faint); }
.site-asura { color: var(--asura); }
.site-demonic { color: var(--demonic); }
.site-comix { color: var(--comix); }
.site-kagane { color: var(--kagane); }
.site-novelfull { color: var(--novelfull); }
.site-lightnovelworld { color: var(--lightnovelworld); }
.new-chapter { color: var(--ember); }
.state { display: flex; align-items: center; gap: 4px; color: var(--mute); }
.state svg { width: 9px; height: 9px; }
.state svg { width: 10px; height: 10px; }
.is-dim .meta { color: var(--mute-2); }
.is-dim .site-asura, .is-dim .site-demonic { color: var(--mute); filter: grayscale(.6); }
.is-dim .site-asura, .is-dim .site-demonic,
.is-dim .site-comix, .is-dim .site-kagane,
.is-dim .site-novelfull, .is-dim .site-lightnovelworld { color: var(--mute); filter: grayscale(.6); }
/* ---- library switch: manga and novels are separate libraries, so the pair
sits in the topbar next to the wordmark rather than among the buckets. ---- */
.libswitch {
display: flex;
margin-left: auto;
border: 1px solid var(--field-line);
}
.libswitch a {
padding: 7px 13px;
font: 500 10px/1 var(--font-mono);
letter-spacing: .12em;
text-transform: uppercase;
color: var(--mute);
text-decoration: none;
}
.libswitch a + a { border-left: 1px solid var(--field-line); }
.libswitch a:hover { color: var(--paper-dim); }
/* The library you are in carries the ember, the same heat the wordmark and the
Updated tab use — it is the one piece of chrome that has to be unmistakable. */
.libswitch a.active {
background: var(--ember-wash);
color: var(--ember);
box-shadow: inset 0 -2px 0 var(--ember);
}
.topbar form { margin-left: 18px; }
/* At phone width brand + switch + Log out do not fit on one line, so the
switch takes its own row under the wordmark rather than pushing Log out
off-screen. */
@media (max-width: 719px) {
.topbar { flex-wrap: wrap; row-gap: 12px; }
.brand { flex: 1 1 auto; min-width: 0; }
.libswitch { order: 3; margin-left: 0; }
.libswitch a { flex: 1; text-align: center; padding: 8px 14px; }
.topbar form { margin-left: 12px; }
}
/* ---- action strip: full-width on a phone, hairline-divided cells ---- */
.actions {
@@ -420,17 +632,34 @@ button { cursor: pointer; }
.actions > *:last-child { border-right: none; }
.actions svg { width: 17px; height: 17px; }
.actions > *:hover { color: var(--paper); }
/* Per-action accent on hover and press: gold favourite, slate archive, moss
finished, clay chapter. Remove keeps --danger, play keeps paper/ember. */
.actions .fav:hover, .actions .fav:active, .actions .fav:focus-visible { color: var(--brass); }
.actions .pencil:hover, .actions .pencil:active, .actions .pencil:focus-visible { color: var(--clay); }
.actions .box:hover, .actions .box:active, .actions .box:focus-visible { color: var(--slate); }
.actions .finish:hover, .actions .finish:active, .actions .finish:focus-visible { color: var(--moss); }
.actions .play { color: var(--paper); }
.is-new .actions .play { color: var(--ember); }
.actions .play:hover { background: var(--hover); }
.actions .on { color: var(--brass); }
.actions .restore { color: var(--paper); }
.actions .remove { color: var(--trash); }
.actions .remove:hover { color: var(--ember); }
.actions .remove:hover { color: var(--danger); }
.actions .open { background: var(--hover); color: var(--paper); }
.actions .remove.open { background: var(--ember-wash); color: var(--ember); }
/* Lifecycle cells already sit on --ash, which is what --hover resolves to, so
an open one needs the next step up to stay legible as the panel's owner. */
.actions .lifecycle.open { background: var(--rule); }
.actions .remove.open { background: var(--danger-wash); color: var(--danger); }
.is-dim .actions > * { color: var(--mute-2); }
/* Three clusters by consequence: navigate (play) | organize (favourite,
chapter) | lifecycle (archive/restore, finish, remove). The lifecycle cells
sit on a recessed ground so the thumb reads "this one moves the series"
before it reads which icon it landed on. */
.actions > .lifecycle { background: var(--ash); }
.actions > .play + *,
.actions > *:not(.lifecycle) + .lifecycle { box-shadow: inset 1px 0 0 var(--field-line); }
/* ---- disclosure panels ---- */
.chapter-form, .confirm-row, .error-inline { animation: sheetIn .18s ease-out; }
@@ -461,7 +690,9 @@ button { cursor: pointer; }
font: 500 17px var(--font-mono);
outline: none;
}
.chapter-form input:focus { border-color: var(--ember); }
/* Focus follows the .searchbar idiom — paper, not heat: a red border on a
valid number field reads as "invalid". */
.chapter-form input:focus { border-color: var(--paper); }
.chapter-form input::-webkit-outer-spin-button,
.chapter-form input::-webkit-inner-spin-button { -webkit-appearance: none; margin: 0; }
.chapter-form button {
@@ -477,14 +708,17 @@ button { cursor: pointer; }
display: flex;
align-items: center;
justify-content: space-between;
flex-wrap: wrap;
gap: 12px;
padding: 14px;
background: var(--ember-wash);
background: var(--danger-wash);
}
.confirm-row span { font: 400 17px var(--font-display); color: var(--ember-soft); }
.confirm-row div { display: flex; gap: 8px; }
/* The remove question names the series, so it has to be able to take the row
to itself and push the buttons onto their own line. */
.confirm-row span { flex: 1 1 16ch; font: 400 17px/1.3 var(--font-display); color: var(--danger-soft); }
.confirm-row div { display: flex; flex: none; gap: 12px; margin-left: auto; }
.confirm-row button {
height: 40px;
height: 46px;
padding: 0 14px;
border: 1px solid var(--field-line);
background: transparent;
@@ -494,10 +728,21 @@ button { cursor: pointer; }
.confirm-row button:hover { color: var(--paper); }
.confirm-row .danger-solid {
border: none;
background: var(--ember);
color: var(--ember-ink);
background: var(--danger);
color: var(--danger-ink);
font-weight: 600;
}
/* Archive and finish are reversible, so their confirm asks in grey — only the
irreversible remove gets the danger wash. */
.confirm-row.calm { background: var(--ash); }
.confirm-row.calm span { color: var(--paper-dim); }
.confirm-row .go {
border: none;
background: var(--paper);
color: var(--ink);
font-weight: 600;
}
.confirm-row .go:hover { color: var(--ink); }
.error-inline {
display: flex;
@@ -505,10 +750,10 @@ button { cursor: pointer; }
gap: 9px;
margin: 0;
padding: 11px 13px;
background: var(--ember-wash);
border-left: 2px solid var(--ember);
background: var(--ash);
border-left: 2px solid var(--mute);
font: 400 16px var(--font-display);
color: var(--ember-soft);
color: var(--paper-dim);
}
.error-inline::before {
content: "";
@@ -516,12 +761,16 @@ button { cursor: pointer; }
width: 5px;
height: 5px;
border-radius: 50%;
background: var(--ember);
animation: emberPulse 1.4s ease-in-out infinite;
background: var(--mute);
animation: mutePulse 1.4s ease-in-out infinite;
}
.error-inline .error-link {
border-bottom: 1px solid var(--field-line);
color: var(--paper);
}
/* Busy: the hairline at the top of the sheet burns across it. */
.card.htmx-request { overflow: hidden; }
/* Busy: a grey hairline slides across the top of the sheet — deliberately not
ember, which on a list screen only ever means "new chapter". */
.card.htmx-request .actions { pointer-events: none; opacity: .5; }
.card.htmx-request::before {
content: "";
@@ -530,7 +779,7 @@ button { cursor: pointer; }
left: 0;
width: 30%;
height: 1px;
background: var(--ember);
background: var(--mute);
animation: barSlide 1.15s linear infinite;
}
@@ -542,7 +791,17 @@ button { cursor: pointer; }
padding: 40px 20px 48px;
}
.empty strong { font: 400 20px var(--font-display); color: var(--paper); }
.empty.hot strong { color: var(--ember); }
.empty .clear-search {
align-self: flex-start;
margin-top: 4px;
height: 44px;
padding: 0 16px;
border: 1px solid var(--field-line);
background: transparent;
color: var(--paper-dim);
font: 400 16px var(--font-display);
}
.empty .clear-search:hover { color: var(--paper); border-color: var(--paper); }
.empty p {
margin: 0;
max-width: 44ch;
@@ -574,32 +833,26 @@ button { cursor: pointer; }
color: var(--paper);
}
.login-card h1 em { color: var(--ember); font-style: italic; }
.login-card form { display: flex; flex-direction: column; gap: 18px; }
.login-card label {
font: 500 10px/1 var(--font-mono);
letter-spacing: .16em;
text-transform: uppercase;
color: var(--mute);
.login-art {
margin: 8px auto 0;
width: 240px;
aspect-ratio: 1;
display: grid;
place-items: center;
background: radial-gradient(circle, var(--ember-wash) 0%, transparent 70%);
}
.login-card input {
.login-art img {
width: 100%;
height: 54px;
margin-top: 9px;
padding: 0 2px;
border: none;
border-bottom: 1px solid var(--field-line);
background: transparent;
color: var(--paper);
font: 500 20px var(--font-mono);
letter-spacing: .16em;
outline: none;
height: 100%;
object-fit: contain;
filter: drop-shadow(0 0 34px var(--ember-wash)) drop-shadow(0 18px 24px rgba(0,0,0,.5));
}
.login-card input:focus { border-bottom-color: var(--ember); }
.login-card form { display: flex; flex-direction: column; gap: 18px; }
.login-card .error {
margin: 0;
min-height: 20px;
font: 400 13px/1.4 var(--font-body);
color: var(--ember);
color: var(--danger);
}
.login-card button {
height: 54px;
@@ -611,13 +864,31 @@ button { cursor: pointer; }
.login-card button:hover {
background: var(--ember);
border-color: var(--ember);
color: #fff;
color: var(--ember-ink);
}
.login-card .login-note {
margin: 14px 0 0;
text-align: center;
font: 400 12px/1.4 var(--font-body);
color: var(--mute);
}
/* ---- laptop and up: the whole sheet is drawn 20% larger, which is what
reading it at 120% zoom on a 1920-wide screen was doing by hand. Everything
in this file is sized in px, so scaling the root is the one adjustment that
keeps every proportion — hairlines, cover ratios, hit targets —
intact. ---- */
@media (min-width: 1280px) {
:root { zoom: 1.2; }
}
/* ---- desktop: same measure, actions fold up beside the row ---- */
@media (min-width: 720px) {
:root { --cover-w: 80px; --row-gap: 20px; }
.topbar { padding: 26px 32px 18px; }
.brand { font-size: 30px; }
.brand .mark { width: 35px; height: 30px; }
.libswitch a { padding: 9px 16px; font-size: 11px; }
.chrome {
flex-direction: row;
@@ -627,31 +898,56 @@ button { cursor: pointer; }
border-bottom: 1px solid var(--rule);
}
.tabs { order: 1; flex: none; gap: 20px; padding: 0; border-bottom: none; }
.tabs [role="tab"] { padding: 10px 0 14px; font-size: 18px; }
.tabs a { padding: 10px 0 14px; font-size: 18px; }
.searchbar { order: 2; flex: 1; margin: 0; border-bottom: none; }
.recent h2, .recent-strip { padding-left: 32px; padding-right: 32px; }
.recent-card, .recent-cover { width: 100px; }
.recent-cover { height: 133px; }
.keyrow {
align-items: center;
justify-content: flex-start;
gap: 26px;
height: 36px;
padding: 0 32px;
}
.keyrow .pair { flex-direction: row; gap: 7px; }
.keyrow .pair svg { width: 14px; height: 14px; }
.keyrow .pair span { font-size: 11px; letter-spacing: .1em; }
.keyrow .full { display: inline; }
.recent-card, .recent-cover { width: var(--cover-w); }
.recent-cover { height: calc(var(--cover-w) * 4 / 3); }
.recent-title { font-size: 17px; }
.card { padding: 18px 32px; }
.row { flex-wrap: nowrap; align-items: center; gap: 20px; }
.cover { width: 74px; height: 100px; }
.cover .monogram { font-size: 26px; }
.title { font-size: 21px; }
.row { flex-wrap: nowrap; align-items: center; }
.cover .monogram { font-size: 28px; }
.title { font-size: 22px; }
.actions { flex: none; gap: 4px; border-top: none; }
.actions > * {
flex: none;
width: 40px;
height: 40px;
width: 44px;
height: 44px;
border: 1px solid var(--rule);
}
.is-new .actions .play { border-color: #3a1d18; }
.actions .on { border-color: #332b14; }
/* Cells are already gapped here, so the clusters separate by space rather
than by the hairline the phone layout uses. */
.actions > .play + *,
.actions > *:not(.lifecycle) + .lifecycle { margin-left: 10px; box-shadow: none; }
.is-new .actions .play { border-color: var(--play-hot-line); }
.actions .on { border-color: var(--fav-line); }
/* The cell border follows the icon on hover, so the accent reads as a state
rather than a stray colour. */
.actions .fav:hover, .actions .pencil:hover,
.actions .box:hover, .actions .finish:hover { border-color: currentColor; }
.chapter-form, .confirm-row, .error-inline { margin-left: 94px; }
/* Panels line up with the body text, i.e. past the cover and its gap. */
.chapter-form, .confirm-row, .error-inline {
margin-left: calc(var(--cover-w) + var(--row-gap));
}
}
@media (prefers-reduced-motion: reduce) {
* { animation: none !important; transition: none !important; }
/* Pseudo-elements need naming explicitly — `*` does not match them, and the
busy bar and error dot are both ::before. Their static form still reads:
the bar stays drawn and .actions stays dimmed. */
*, *::before, *::after { animation: none !important; transition: none !important; }
}
+91
View File
@@ -0,0 +1,91 @@
{{define "app"}}
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
<meta name="color-scheme" content="dark light">
<title>BookmarkManager</title>
<link rel="icon" href="/static/logo.svg" type="image/svg+xml">
<link rel="stylesheet" href="/static/style.css">
<link rel="preload" href="/static/fonts/instrument-serif-400-latin.woff2" as="font" type="font/woff2" crossorigin>
{{/* Body text before meta lines: DM Sans is the biggest face and the one
most of the page is set in; the mono is small and arrives from CSS. */}}
<link rel="preload" href="/static/fonts/dm-sans-var-latin.woff2" as="font" type="font/woff2" crossorigin>
<script src="/static/htmx.min.js" defer></script>
<script src="/static/filter.js" defer></script>
</head>
<body>
{{template "icons" .}}
<div class="sheet">
<header class="topbar">
<h1 class="brand">{{template "mark" .}}<span>Bookmark<em>Manager</em></span></h1>
{{/* Plain full-page links, not htmx swaps: switching library replaces the
tab row and the chrome, which is a page, not a fragment. */}}
<nav class="libswitch" aria-label="Library">
<a href="/?tab=all" class="{{if eq .Lib "manga"}}active{{end}}"
{{if eq .Lib "manga"}}aria-current="page"{{end}}>Manga</a>
<a href="/?lib=novel&amp;tab=all" class="{{if eq .Lib "novel"}}active{{end}}"
{{if eq .Lib "novel"}}aria-current="page"{{end}}>Novels</a>
</nav>
<form method="post" action="/logout">
<button type="submit" class="ghost">Log out</button>
</form>
</header>
{{/* Search sits above the tabs on a phone and folds into the tab row on a
wider screen — one flex container, order swapped in CSS. */}}
<div class="chrome">
<div class="searchbar">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-search"/></svg>
<input id="search" class="search" type="search" placeholder="Find a title"
autocomplete="off" aria-label="Search titles">
</div>
{{/* These are real links with real hrefs that change the URL, so they are
navigation, not an ARIA tablist — aria-current carries "which bucket am
I in" without owing a tabpanel contract we do not implement. */}}
<nav class="tabs" aria-label="Bookmark buckets">
<a href="{{.PageURL "all"}}" class="{{if eq .Tab "all"}}active{{end}}"
{{if eq .Tab "all"}}aria-current="page"{{end}}
hx-get="{{.ListURL "all"}}" hx-target="#list" hx-swap="innerHTML"
hx-push-url="{{.PageURL "all"}}" hx-on::after-request="setActiveTab(this)">All</a>
{{/* The one bucket novels do not have: without a poller-fed "what is out
that I have not read", the tab would only ever restate All. */}}
{{if eq .Lib "manga"}}
<a href="{{.PageURL "new"}}" class="tab-new {{if eq .Tab "new"}}active{{end}}"
{{if eq .Tab "new"}}aria-current="page"{{end}}
hx-get="{{.ListURL "new"}}" hx-target="#list" hx-swap="innerHTML"
hx-push-url="{{.PageURL "new"}}" hx-on::after-request="setActiveTab(this)">Updated
{{template "newcount" .}}</a>
{{end}}
<a href="{{.PageURL "fav"}}" class="{{if eq .Tab "fav"}}active{{end}}"
{{if eq .Tab "fav"}}aria-current="page"{{end}}
hx-get="{{.ListURL "fav"}}" hx-target="#list" hx-swap="innerHTML"
hx-push-url="{{.PageURL "fav"}}" hx-on::after-request="setActiveTab(this)">Favourites</a>
<a href="{{.PageURL "archived"}}" class="{{if eq .Tab "archived"}}active{{end}}"
{{if eq .Tab "archived"}}aria-current="page"{{end}}
hx-get="{{.ListURL "archived"}}" hx-target="#list" hx-swap="innerHTML"
hx-push-url="{{.PageURL "archived"}}" hx-on::after-request="setActiveTab(this)">Archived</a>
<a href="{{.PageURL "finished"}}" class="{{if eq .Tab "finished"}}active{{end}}"
{{if eq .Tab "finished"}}aria-current="page"{{end}}
hx-get="{{.ListURL "finished"}}" hx-target="#list" hx-swap="innerHTML"
hx-push-url="{{.PageURL "finished"}}" hx-on::after-request="setActiveTab(this)">Finished</a>
</nav>
</div>
{{template "setup" .}}
{{if .Owner}}{{template "readers" .}}{{end}}
{{template "keyrow" .}}
{{template "recent" .}}
<main id="list" class="list">
{{template "list" .}}
</main>
</div>
</body>
</html>
{{end}}
@@ -7,7 +7,10 @@
<a class="cover" href="{{.ContinueURL}}" target="_blank" rel="noopener noreferrer"
tabindex="-1" aria-hidden="true">
{{if .Cover}}<img src="{{.Cover}}" alt="" loading="lazy">
{{else}}<span class="monogram">{{.Initial}}</span>{{end}}
{{/* aria-hidden on the cover link is not enough — Chromium still exposes
the letter because the link is programmatically focusable — so the
monogram carries its own, same as the recent strip's. */}}
{{else}}<span class="monogram" aria-hidden="true">{{.Initial}}</span>{{end}}
{{if and (eq .Status "reading") .HasNewChapter}}<span class="foot-rule"></span>
{{else if .Favorite}}<span class="foot-rule brass"></span>{{end}}
</a>
@@ -21,10 +24,10 @@
<p class="meta">
<span class="site-{{.Site}}">{{.Site}}</span>
<span class="sep">/</span>
<span class="chapter">Ch {{.LastChapter}}</span>
<span class="chapter">{{.DisplayChapter}}</span>
{{if and (eq .Status "reading") .HasNewChapter}}
<span class="sep">/</span>
<span class="new-chapter">Ch {{.LatestChapter}} out</span>
<span class="new-chapter">{{.DisplayLatest}} out</span>
{{end}}
{{if eq .Status "archived"}}
<span class="sep">/</span>
@@ -40,7 +43,7 @@
title="Continue reading" aria-label="Continue reading">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-play"/></svg>
</a>
<button class="{{if .Favorite}}on{{end}}"
<button class="fav{{if .Favorite}} on{{end}}"
title="{{if .Favorite}}Remove from favourites{{else}}Add to favourites{{end}}"
aria-label="Toggle favourite"
hx-post="/ui/bookmarks/{{.Key}}/favorite"
@@ -52,38 +55,32 @@
onclick="toggleChapterForm('{{.Key}}')">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-pencil"/></svg>
</button>
{{if eq .Status "finished"}}
<button class="restore" title="Restore to reading" aria-label="Restore to reading"
hx-post="/ui/bookmarks/{{.Key}}/status" hx-vals='{"status":"reading"}'
hx-target="[id='card-{{.Key}}']" hx-swap="outerHTML"
hx-indicator="[id='card-{{.Key}}']" hx-disabled-elt="this">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-undo"/></svg>
</button>
{{else}}
{{if eq .Status "archived"}}
<button class="restore" title="Restore to reading" aria-label="Restore to reading"
hx-post="/ui/bookmarks/{{.Key}}/status" hx-vals='{"status":"reading"}'
hx-target="[id='card-{{.Key}}']" hx-swap="outerHTML"
hx-indicator="[id='card-{{.Key}}']" hx-disabled-elt="this">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-undo"/></svg>
</button>
{{else}}
<button title="Archive" aria-label="Archive"
hx-post="/ui/bookmarks/{{.Key}}/status" hx-vals='{"status":"archived"}'
hx-target="[id='card-{{.Key}}']" hx-swap="outerHTML"
hx-indicator="[id='card-{{.Key}}']" hx-disabled-elt="this">
{{/* Restore is a reversal, so it fires straight away; every move *out* of
the list (archive, finish, remove) goes through a confirm row. */}}
{{if eq .Status "reading"}}
<button class="lifecycle box" title="Archive" aria-label="Archive"
aria-expanded="false" aria-controls="confirm-archive-{{.Key}}"
onclick="toggleConfirmRow('{{.Key}}', 'archive')">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-box"/></svg>
</button>
{{end}}
<button title="Mark finished" aria-label="Mark finished"
hx-post="/ui/bookmarks/{{.Key}}/status" hx-vals='{"status":"finished"}'
{{else}}
<button class="lifecycle restore" title="Restore to reading" aria-label="Restore to reading"
hx-post="/ui/bookmarks/{{.Key}}/status" hx-vals='{"status":"reading"}'
hx-target="[id='card-{{.Key}}']" hx-swap="outerHTML"
hx-indicator="[id='card-{{.Key}}']" hx-disabled-elt="this">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-undo"/></svg>
</button>
{{end}}
{{if ne .Status "finished"}}
<button class="lifecycle finish" title="Mark finished" aria-label="Mark finished"
aria-expanded="false" aria-controls="confirm-finish-{{.Key}}"
onclick="toggleConfirmRow('{{.Key}}', 'finish')">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-check"/></svg>
</button>
{{end}}
<button class="remove" title="Remove" aria-label="Remove"
onclick="toggleConfirmRow('{{.Key}}')">
<button class="lifecycle remove" title="Remove" aria-label="Remove"
aria-expanded="false" aria-controls="confirm-remove-{{.Key}}"
onclick="toggleConfirmRow('{{.Key}}', 'remove')">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-trash"/></svg>
</button>
</div>
@@ -92,23 +89,57 @@
hx-post="/ui/bookmarks/{{.Key}}/chapter"
hx-target="[id='card-{{.Key}}']" hx-swap="outerHTML"
hx-indicator="[id='card-{{.Key}}']" hx-disabled-elt="input, button">
{{if .LatestChapter}}<p class="hint">Latest known: Ch {{.LatestChapter}}</p>{{end}}
{{/* The field holds your progress; "Latest known" is the published chapter.
Those are different numbers whenever this form is worth opening, so the
label names the field and the latest sits after it as context. */}}
<label class="hint" for="chapter-{{.Key}}">Chapter you're on</label>
<div class="field">
<input name="chapter" type="number" step="0.1" min="0"
value="{{.LastChapterNum}}" aria-label="Chapter number" required>
{{/* max is a fat-finger guard, not a real ceiling — no series is near it. */}}
<input id="chapter-{{.Key}}" name="chapter" type="number" step="0.1" min="0" max="9999"
value="{{.LastChapterNum}}" required>
<button type="submit">Save</button>
</div>
{{if .LatestChapter}}<p class="hint">Latest known: {{.DisplayLatest}}</p>{{end}}
</form>
<div class="confirm-row" id="confirm-row-{{.Key}}" hidden>
<span>Remove this?</span>
{{/* One confirm row per way a series leaves the list. aria-live announces the
step to a screen reader, which otherwise gets no word that the tap
opened a second question. */}}
{{if eq .Status "reading"}}
<div class="confirm-row calm" id="confirm-archive-{{.Key}}" role="group" aria-live="polite" hidden>
<span>Archive this?</span>
<div>
<button class="go"
hx-post="/ui/bookmarks/{{.Key}}/status" hx-vals='{"status":"archived"}'
hx-target="[id='card-{{.Key}}']" hx-swap="outerHTML"
hx-indicator="[id='card-{{.Key}}']" hx-disabled-elt="this">Archive</button>
<button type="button" onclick="toggleConfirmRow('{{.Key}}', 'archive')">Cancel</button>
</div>
</div>
{{end}}
{{if ne .Status "finished"}}
<div class="confirm-row calm" id="confirm-finish-{{.Key}}" role="group" aria-live="polite" hidden>
<span>Mark finished?</span>
<div>
<button class="go"
hx-post="/ui/bookmarks/{{.Key}}/status" hx-vals='{"status":"finished"}'
hx-target="[id='card-{{.Key}}']" hx-swap="outerHTML"
hx-indicator="[id='card-{{.Key}}']" hx-disabled-elt="this">Finish</button>
<button type="button" onclick="toggleConfirmRow('{{.Key}}', 'finish')">Cancel</button>
</div>
</div>
{{end}}
<div class="confirm-row" id="confirm-remove-{{.Key}}" role="group" aria-live="polite" hidden>
<span>Remove “{{.Title}}”? Chapter progress is lost.</span>
<div>
<button class="danger-solid"
hx-delete="/ui/bookmarks/{{.Key}}"
hx-target="[id='card-{{.Key}}']" hx-swap="outerHTML"
hx-indicator="[id='card-{{.Key}}']" hx-disabled-elt="this">Remove</button>
<button type="button" onclick="toggleConfirmRow('{{.Key}}')">Cancel</button>
<button type="button" onclick="toggleConfirmRow('{{.Key}}', 'remove')">Cancel</button>
</div>
</div>
<p class="error-inline" hidden></p>
{{/* role=status announces a failed write; without it the tap just looks
ignored to a screen reader. */}}
<p class="error-inline" role="status" hidden></p>
</article>
{{end}}
@@ -0,0 +1,78 @@
{{/* The regions that live outside the swapped #list: the "Continue reading"
strip, the Updated badge and the action key. All are rendered inline by
app.html and again, out of band, on every /ui/ response — a mutation must
not leave them describing the library as it was before the tap.
All always render, hidden when they have nothing to say, so an out-of-band
swap always has an element with the right id to replace. */}}
{{define "recent"}}
<section class="recent" id="recent"{{if .OOB}} hx-swap-oob="true"{{end}}{{if not .Recent}} hidden{{end}}>
<h2>Continue reading</h2>
<div class="recent-strip">
{{range .Recent}}
<a class="recent-card {{if .HasNewChapter}}is-new{{end}}" href="{{.ContinueURL}}"
target="_blank" rel="noopener noreferrer">
<span class="recent-cover">
{{if .Cover}}<img src="{{.Cover}}" alt="" loading="lazy">
{{else}}<span class="monogram" aria-hidden="true">{{.Initial}}</span>{{end}}
{{if .HasNewChapter}}<span class="foot-rule"></span>
{{else if .Favorite}}<span class="foot-rule brass"></span>{{end}}
</span>
<span class="recent-title">{{.Title}}</span>
<span class="recent-chapter">{{.DisplayChapter}}{{if .HasNewChapter}} · New{{end}}</span>
</a>
{{end}}
</div>
</section>
{{end}}
{{/* The action key. The icon strip on a card is unlabelled, so one permanent
line under the tabs names every glyph. It follows the tab rather than the
row: the archived and finished buckets swap Archive for Restore, and a
finished series has no Done to offer. */}}
{{define "keyrow"}}
<div class="keyrow" id="keyrow" aria-label="Action key"{{if .OOB}} hx-swap-oob="true"{{end}}>
<span class="pair"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-play"/></svg><span>Read</span></span>
<span class="pair brass"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-star"/></svg><span>Fav</span></span>
<span class="pair"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-pencil"/></svg><span>Chapter</span></span>
{{if or (eq .Tab "archived") (eq .Tab "finished")}}
<span class="pair"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-undo"/></svg><span>Restore</span></span>
{{else}}
<span class="pair"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-box"/></svg><span>Archive</span></span>
{{end}}
{{if ne .Tab "finished"}}
<span class="pair"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-check"/></svg><span>Done</span></span>
{{end}}
<span class="pair trash"><svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-trash"/></svg><span>Delete</span></span>
</div>
{{end}}
{{/* The brand mark, inline so it takes the page's ink and ember rather than the
fixed palette of /static/logo.svg (which the favicon needs). */}}
{{define "mark"}}
<svg class="mark" viewBox="0 0 200 172" aria-hidden="true">
<g fill="var(--ink)" stroke="currentColor" stroke-width="5" stroke-linejoin="round" stroke-linecap="round">
<path fill="none" d="M28 36H4v114h192V36h-24"></path>
<path fill="none" d="M28 23H17v127h166V23h-11"></path>
<g id="mb-half">
<path d="M28 7 88 55v97L28 138z"></path>
<g fill="currentColor" stroke="none">
<path d="M37 25 55 39v41L37 66z"></path>
<path d="M60 42 79 57v42L60 84z"></path>
<path d="M37 75 79 108v13L37 88z"></path>
<path d="M37 98 79 129v11L37 131z"></path>
</g>
</g>
<use href="#mb-half" transform="matrix(-1 0 0 1 200 0)"></use>
<g stroke="var(--ember)">
<path d="M100 4l9 5v11l-9 5-9-5V9z"></path>
<path d="M94 24h12v24H94z"></path>
<path d="M70 47h60v14H70z"></path>
<path d="M91 61h18v87l-9 20-9-20z"></path>
</g>
</g>
</svg>
{{end}}
{{define "newcount"}}<span class="count" id="new-count"{{if .OOB}} hx-swap-oob="true"{{end}}{{if not .NewCount}} hidden{{end}}>{{.NewCount}}</span>{{end}}
+33
View File
@@ -0,0 +1,33 @@
{{define "list"}}
{{if .Items}}
{{range .Items}}{{template "card" .}}{{end}}
{{/* The client filter only hides cards, so without this the list area goes
blank on a query that matches nothing. filter.js fills in the query and
unhides it; it lives inside #list so a tab swap re-creates it. */}}
<div class="empty" id="no-match" hidden>
<strong>No titles match “<span class="no-match-q"></span>”.</strong>
<button type="button" class="clear-search">Clear search</button>
</div>
{{else if eq .Tab "fav"}}
<div class="empty"><strong>No favourites yet.</strong><p>Star a series to pin it here.</p></div>
{{else if eq .Tab "new"}}
<div class="empty"><strong>Nothing new.</strong><p>Every series is caught up to its latest chapter.</p></div>
{{else if eq .Tab "archived"}}
<div class="empty"><strong>Nothing archived.</strong><p>Shelve a series to park it here — it keeps getting checked for new chapters.</p></div>
{{else if eq .Tab "finished"}}
<div class="empty"><strong>Nothing finished yet.</strong><p>Mark a series finished and it moves out of your reading list.</p></div>
{{else if .EmptyLibrary}}
{{/* Nothing in either library, so the links are the only thing this page can
usefully say. Both scripts: the two libraries are separate installs. */}}
<div class="empty">
<strong>Nothing here yet.</strong>
<p>Install the userscripts, then open a series and read a chapter — bookmarks arrive on their own.</p>
<p class="setup-links">
<a class="ghost" href="/install/manga-bookmark.user.js">Install Manga script</a>
<a class="ghost" href="/install/novel-bookmark.user.js">Install Novels script</a>
</p>
</div>
{{else}}
<div class="empty"><strong>Nothing here yet.</strong><p>Bookmarks appear once the userscript records a chapter.</p></div>
{{end}}
{{end}}
+32
View File
@@ -0,0 +1,32 @@
{{define "login"}}
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
<meta name="color-scheme" content="dark light">
<title>BookmarkManager</title>
<link rel="icon" href="/static/logo.svg" type="image/svg+xml">
<link rel="stylesheet" href="/static/style.css">
<link rel="preload" href="/static/fonts/instrument-serif-400-latin.woff2" as="font" type="font/woff2" crossorigin>
</head>
<body>
<main class="login-card">
<div>
<span class="eyebrow">Private library</span>
<h1 class="brand">{{template "mark" .}}<span>Bookmark<em>Manager</em></span></h1>
</div>
<figure class="login-art" aria-hidden="true">
<img src="/static/login-art.png" alt="">
</figure>
<form method="get" action="/auth/discord">
{{/* The page reloads on a failed sign-in, so the message is present from
the start; role=alert is what gets it announced anyway. */}}
<p class="error" role="alert">{{.Error}}</p>
<button type="submit">Continue with Discord</button>
</form>
<p class="login-note">Guild membership is required to sign in.</p>
</main>
</body>
</html>
{{end}}
@@ -0,0 +1,29 @@
{{/* The owner's Reader roster. Rendered only for the owner (listView.Owner),
and re-rendered whole as the response to a revocation so the session
counts cannot describe the state before the tap. Revocation is
confirm-gated: it signs someone out of every device at once. */}}
{{define "readers"}}
<details class="setup" id="readers">
<summary>Readers</summary>
<p class="setup-copy">Everyone who has signed in through Discord. Revoking
signs a Reader out of every device; their library and bookmarks are
untouched, and they can sign in again.</p>
<ul class="readerlist">
{{range .Readers}}
<li>
<span class="reader-id">{{.DiscordID}}</span>
<span class="reader-sessions">{{.Sessions}} session{{if ne .Sessions 1}}s{{end}}</span>
{{/* The owner's own row never offers Revoke: it is the one row where the
button would sign the tapping browser out, and the endpoint refuses
it anyway. Logout is the deliberate way to do that. */}}
{{if and .Sessions (ne .ID $.OwnerID)}}
<form hx-post="/readers/{{.ID}}/revoke" hx-target="#readers" hx-swap="outerHTML"
hx-confirm="Revoking signs this Reader out on every device immediately. Revoke?">
<button type="submit" class="ghost danger">Revoke sessions</button>
</form>
{{end}}
</li>
{{end}}
</ul>
</details>
{{end}}
+34
View File
@@ -0,0 +1,34 @@
{{/* The userscript install panel. Each link serves the script rendered
with the acting Reader's credential inside it, so the credential never
appears in this page's markup, the address bar, or a redirect. Rotation
is confirm-gated because it invalidates every installed copy at once;
the response swaps this same panel open with the reinstall warning. */}}
{{define "setup"}}
<details class="setup" id="setup"{{if .Rotated}} open{{end}}>
<summary>Userscripts</summary>
<p class="setup-copy">Install each script once per device. They keep your
bookmarks in sync across every site and update themselves from here.</p>
<p class="setup-links">
<a class="ghost" href="/install/manga-bookmark.user.js">Install Manga script</a>
<a class="ghost" href="/install/novel-bookmark.user.js">Install Novels script</a>
</p>
<p class="setup-copy">On mobile, Violentmonkey does not pick up the install
links — the script opens as text. Download the file instead, then add it
from Violentmonkey's own menu.</p>
<p class="setup-links">
<a class="ghost" href="/install/manga-bookmark.user.js?download=1">Download Manga script</a>
<a class="ghost" href="/install/novel-bookmark.user.js?download=1">Download Novels script</a>
</p>
{{if .Rotated}}
<p class="setup-warn" role="status">Credential rotated — the old one no
longer works. Reinstall both scripts on every device now, or they will
silently stop syncing.</p>
{{else}}
<form class="setup-rotate" hx-post="/rotate-token" hx-target="#setup"
hx-swap="outerHTML"
hx-confirm="Rotation invalidates the current credential on every device immediately. You will have to reinstall both scripts everywhere. Rotate?">
<button type="submit" class="ghost">Rotate credential</button>
</form>
{{end}}
</details>
{{end}}
+657
View File
@@ -0,0 +1,657 @@
package web
import (
"context"
"embed"
"html/template"
"io/fs"
"log"
"math"
"mime"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"bookmarkmanager/backend/internal/session"
"bookmarkmanager/backend/internal/store"
"bookmarkmanager/backend/internal/token"
"bookmarkmanager/backend/internal/userscript"
)
//go:embed templates
var templateFS embed.FS
//go:embed static
var staticFS embed.FS
// RecentCount is how many series the "Continue reading" strip shows.
const RecentCount = 5
// Handler serves the browser UI: full pages at / and htmx fragments at /ui/.
// It is a separate handler from api.Handler because the two speak different
// representations (HTML versus JSON) to different clients under different auth.
type Handler struct {
store *store.Store
// tokenKey derives Readers' userscript credentials (internal/token): the
// install endpoints render the scripts with the credential inside, which
// is the one place the UI needs the secret.
tokenKey []byte
// mangaUserscriptPath / novelUserscriptPath are the bindmounted script
// files the install endpoints render — the same files the /u/ download
// paths serve.
mangaUserscriptPath string
novelUserscriptPath string
tmpl *template.Template
discord DiscordConfig
states *oauthStates
limiter *session.LoginLimiter
// httpClient is the plain stdlib client that talks to Discord. It is not
// an injected interface: tests point APIBase at a stub server instead.
httpClient *http.Client
}
// listView is what every list-rendering template receives.
type listView struct {
// Lib is the library this view renders: store.KindManga or store.KindNovel.
// Manga is the default and carries no query parameter, so every pre-novel
// URL keeps meaning exactly what it did.
Lib string
Tab string // "all", "fav", or "new"
Recent []store.Bookmark
Items []store.Bookmark
// NewCount is the badge on the Updated tab: how many series being read
// have a chapter out that has not been read. It is counted over the whole
// reading set, not the active tab, so the badge does not change meaning as
// the user moves between tabs.
NewCount int
// OOB marks a render of the chrome partials as an out-of-band swap rather
// than the inline copy app.html lays out.
OOB bool
// Rotated marks the setup panel as having just rotated the credential:
// it swaps the reinstall warning in over the button row.
Rotated bool
// EmptyLibrary means this Reader holds no bookmarks in either library, so
// the empty state can offer the installs instead of reporting on a filter.
// It is not "newly registered": a Reader who deletes their last bookmark is
// in the same position and needs the same links.
EmptyLibrary bool
// Owner marks the acting Reader as the deployment's owner, which unlocks
// the Readers panel. Nothing else in the UI differs.
Owner bool
// Readers is the owner's roster, populated only for the owner's own page
// render and the revocation fragment. OwnerID travels with it so the roster
// can tell the owner's own row apart from the Readers they may revoke.
Readers []store.ReaderSummary
OwnerID int64
}
// PageURL and ListURL are the two link shapes every tab needs. Building them
// here rather than concatenating in the template is what keeps the library
// parameter from being dropped on one link out of ten.
func (v listView) PageURL(tab string) string {
if v.Lib == store.KindNovel {
return "/?lib=novel&tab=" + tab
}
return "/?tab=" + tab
}
func (v listView) ListURL(tab string) string {
if v.Lib == store.KindNovel {
return "/ui/list?lib=novel&tab=" + tab
}
return "/ui/list?tab=" + tab
}
// loginView is what the login template receives.
type loginView struct {
Error string
}
// New parses every template up front so a broken one kills the process at
// startup rather than the first request that touches it.
func New(s *store.Store, discord DiscordConfig, tokenKey []byte, mangaPath, novelPath string) (*Handler, error) {
tmpl, err := template.ParseFS(templateFS, "templates/*.html")
if err != nil {
return nil, err
}
return &Handler{
store: s,
tokenKey: tokenKey,
mangaUserscriptPath: mangaPath,
novelUserscriptPath: novelPath,
tmpl: tmpl,
discord: discord,
states: newOAuthStates(),
limiter: session.NewLoginLimiter(),
httpClient: &http.Client{Timeout: discordTimeout},
}, nil
}
func (h *Handler) Register(mux *http.ServeMux) {
mux.HandleFunc("GET /{$}", h.index)
mux.HandleFunc("GET /auth/discord", h.discordStart)
mux.HandleFunc("GET /auth/discord/callback", h.discordCallback)
mux.HandleFunc("POST /logout", h.logout)
mux.Handle("GET /static/", staticHandler())
mux.HandleFunc("GET /ui/list", h.requireSession(h.uiList))
mux.HandleFunc("POST /ui/bookmarks/{key}/favorite", h.requireSession(h.uiFavorite))
mux.HandleFunc("POST /ui/bookmarks/{key}/status", h.requireSession(h.uiStatus))
mux.HandleFunc("POST /ui/bookmarks/{key}/chapter", h.requireSession(h.uiChapter))
mux.HandleFunc("DELETE /ui/bookmarks/{key}", h.requireSession(h.uiDelete))
// Install endpoints render the script directly under the session: the
// credential travels inside the served bytes, never in the address bar or
// the page markup. Updates after install use the credential-bearing /u/
// path the script embeds, which needs no session.
mux.HandleFunc("GET /install/manga-bookmark.user.js", h.requireSession(h.installUserscript("manga-bookmark.user.js")))
mux.HandleFunc("GET /install/novel-bookmark.user.js", h.requireSession(h.installUserscript("novel-bookmark.user.js")))
mux.HandleFunc("POST /rotate-token", h.requireSession(h.rotateToken))
// Owner-only: the one place the UI crosses the Reader boundary.
mux.HandleFunc("POST /readers/{id}/revoke", h.requireSession(h.revokeReaderSessions))
}
// staticHandler serves the embedded assets. An hour, not longer: assets are
// not fingerprinted, and embed.FS reports a zero ModTime, so http.FileServer
// emits no Last-Modified or ETag and a client has no way to revalidate a
// cached copy after a deploy short of waiting out max-age.
func staticHandler() http.Handler {
sub, err := fs.Sub(staticFS, "static")
if err != nil {
panic("embed static: " + err.Error())
}
// Go's built-in table has no .woff2 and the scratch image has no
// /etc/mime.types, so without this the fonts go out as
// application/octet-stream.
if err := mime.AddExtensionType(".woff2", "font/woff2"); err != nil {
panic("woff2 mime: " + err.Error())
}
files := http.FileServer(http.FS(sub))
return http.StripPrefix("/static/", http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Cache-Control", "public, max-age=3600")
files.ServeHTTP(w, r)
}))
}
type ctxKey int
// readerCtxKey is where requireSession stashes the authenticated Reader id.
const readerCtxKey ctxKey = iota
// sessionReader reports whether the request carries a live session, and for
// whom. The cookie holds only the session id; the row behind it is looked up
// on every request, so deleting a session takes effect immediately. Expiry is
// enforced here, in the store, which also removes rows that have lapsed.
func (h *Handler) sessionReader(r *http.Request) (int64, bool) {
c, err := r.Cookie(session.CookieName)
if err != nil {
return 0, false
}
sess, ok, err := h.store.GetSession(c.Value, time.Now())
if err != nil {
log.Printf("session lookup: %v", err)
return 0, false
}
return sess.ReaderID, ok
}
// requireSession guards the fragment endpoints. It answers 401 rather than
// redirecting, because htmx swaps whatever body it receives into the page and a
// redirected login page would be spliced into the card list.
func (h *Handler) requireSession(next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
readerID, ok := h.sessionReader(r)
if !ok {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
next(w, r.WithContext(context.WithValue(r.Context(), readerCtxKey, readerID)))
}
}
// readerOf returns the authenticated Reader id requireSession stashed.
func readerOf(r *http.Request) int64 { return r.Context().Value(readerCtxKey).(int64) }
func (h *Handler) render(w http.ResponseWriter, status int, name string, data any) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(status)
if err := h.tmpl.ExecuteTemplate(w, name, data); err != nil {
// The status line is already sent, so this can only be logged.
log.Printf("render %s: %v", name, err)
}
}
// index renders the list, or the login page when there is no session. The login
// page is served at / with status 200 rather than as a redirect to a separate
// URL: one page, no redirect loop to reason about.
func (h *Handler) index(w http.ResponseWriter, r *http.Request) {
readerID, ok := h.sessionReader(r)
if !ok {
h.render(w, http.StatusOK, "login", loginView{})
return
}
view, err := h.buildListView(readerID, libOf(r.URL.Query().Get("lib")), r.URL.Query().Get("tab"))
if err != nil {
log.Printf("index: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if readerID == h.store.OwnerID() {
view.Owner, view.OwnerID = true, readerID
if view.Readers, err = h.store.Readers(); err != nil {
log.Printf("index readers: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
}
h.render(w, http.StatusOK, "app", view)
}
// filterBookmarks returns the subset keep reports true for, preserving order.
// It always returns a non-nil slice so an empty tab renders its empty state.
func filterBookmarks(all []store.Bookmark, keep func(store.Bookmark) bool) []store.Bookmark {
out := []store.Bookmark{}
for _, b := range all {
if keep(b) {
out = append(out, b)
}
}
return out
}
// kindOf reads a bookmark's library. A row cached or written before the kind
// column existed has none; every one of those is manga, which is what the
// column default says too.
func kindOf(b store.Bookmark) string {
if b.Kind == "" {
return store.KindManga
}
return b.Kind
}
// libOf normalises the query parameter. Anything that is not the novel library
// is the manga one, so a typo lands on the default page rather than an empty
// list.
func libOf(q string) string {
if q == store.KindNovel {
return store.KindNovel
}
return store.KindManga
}
// buildListView loads one reader's list once and derives both the tab-filtered
// items and the recent strip from it.
//
// Archived and finished series appear in their own tab and nowhere else — not
// in All, not in Updated, not in Favourites, and not in the recent strip. An
// archived favourite therefore shows only under Archived: Favourites means
// "favourites I am currently reading".
func (h *Handler) buildListView(readerID int64, lib, tab string) (listView, error) {
all, err := h.store.List(readerID) // already ordered updated_at DESC
if err != nil {
return listView{}, err
}
// Taken before the filter narrows the slice: a Reader with novels but no
// manga has a working install already, and does not need to be told to go
// and get one.
emptyLibrary := len(all) == 0
// Narrow to one library first: reading, withNew and recent all derive from
// this slice, so doing it later would let the other library's rows into the
// strip and the Updated badge.
all = filterBookmarks(all, func(b store.Bookmark) bool { return kindOf(b) == lib })
// Novels do not offer an Updated tab, so a hand-typed one lands on All.
if lib == store.KindNovel && tab == "new" {
tab = "all"
}
reading := filterBookmarks(all, func(b store.Bookmark) bool { return b.Status == store.StatusReading })
withNew := filterBookmarks(reading, func(b store.Bookmark) bool { return b.HasNewChapter() })
var items []store.Bookmark
switch tab {
case "fav":
items = filterBookmarks(reading, func(b store.Bookmark) bool { return b.Favorite })
case "new":
items = withNew
case "archived":
items = filterBookmarks(all, func(b store.Bookmark) bool { return b.Status == store.StatusArchived })
case "finished":
items = filterBookmarks(all, func(b store.Bookmark) bool { return b.Status == store.StatusFinished })
default:
tab = "all"
items = reading
}
// The strip is scoped to series with a chapter waiting, which is the one
// question the list below it does not already answer: the list is ordered by
// reading recency, so the head of it *is* the strip whenever the strip is
// just "the most recent rows". Only on All — on Updated it would render the
// same set twice, and on the other tabs it would contradict the bucket.
//
// It therefore disappears entirely on a library with nothing new. That is
// the intended reading: an empty strip has nothing to say, and the ~240px it
// costs on a phone belongs to the list.
var recent []store.Bookmark
if tab == "all" {
recent = withNew
if len(recent) > RecentCount {
recent = recent[:RecentCount]
}
}
return listView{Lib: lib, Tab: tab, Recent: recent, Items: items,
NewCount: len(withNew), EmptyLibrary: emptyLibrary}, nil
}
func (h *Handler) uiList(w http.ResponseWriter, r *http.Request) {
view, err := h.buildListView(readerOf(r), libOf(r.URL.Query().Get("lib")), r.URL.Query().Get("tab"))
if err != nil {
log.Printf("ui list: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.render(w, http.StatusOK, "list", view)
// The chrome is outside this response's swap target, so without this a tab
// switch would leave the strip and badge from whichever tab the page was
// loaded on — the same URL would render differently depending on how the
// reader got there.
h.writeChromeOOB(w, view)
}
// currentTab is the tab the reader is looking at, read from htmx's own header,
// so out-of-band chrome is rebuilt for that view rather than for a default.
func currentTab(r *http.Request) string {
u, err := url.Parse(r.Header.Get("HX-Current-URL"))
if err != nil {
return ""
}
return u.Query().Get("tab")
}
// currentLib is the library the reader is looking at, read from htmx's own
// header for the same reason currentTab is: out-of-band chrome must be rebuilt
// for that view rather than for the default one.
func currentLib(r *http.Request) string {
u, err := url.Parse(r.Header.Get("HX-Current-URL"))
if err != nil {
return store.KindManga
}
return libOf(u.Query().Get("lib"))
}
// writeChromeOOB appends the regions that live outside #list — the recent
// strip, the Updated badge and the action key — as out-of-band swaps, so a
// mutation cannot leave them describing the library as it was before the tap.
// The key is in here because it is tab-shaped too: archived and finished swap
// Archive for Restore.
func (h *Handler) writeChromeOOB(w http.ResponseWriter, view listView) {
view.OOB = true
names := []string{"recent", "keyrow"}
if view.Lib == store.KindManga {
names = append(names, "newcount")
}
for _, name := range names {
if err := h.tmpl.ExecuteTemplate(w, name, view); err != nil {
// The card is already written; stale chrome beats a torn response.
log.Printf("render %s oob: %v", name, err)
return
}
}
}
// refreshChrome rebuilds the chrome for the reader's current tab after a
// mutation and appends it to the response.
func (h *Handler) refreshChrome(w http.ResponseWriter, r *http.Request) {
view, err := h.buildListView(readerOf(r), currentLib(r), currentTab(r))
if err != nil {
log.Printf("ui chrome: %v", err)
return
}
h.writeChromeOOB(w, view)
}
// renderLogin renders the login page with an error message, for refused or
// failed sign-ins. Every message is author-written text — nothing Discord
// supplied is ever interpolated into a page.
func (h *Handler) renderLogin(w http.ResponseWriter, status int, msg string) {
h.render(w, status, "login", loginView{Error: msg})
}
// logout revokes the session row and clears the cookie in one step: the next
// request finds no row and is rejected.
func (h *Handler) logout(w http.ResponseWriter, r *http.Request) {
if c, err := r.Cookie(session.CookieName); err == nil {
if err := h.store.DeleteSession(c.Value); err != nil {
log.Printf("delete session: %v", err)
}
}
session.ClearCookie(w, r)
http.Redirect(w, r, "/", http.StatusSeeOther)
}
// loadForMutation fetches the row a mutation targets, writing the error
// response itself when there is nothing to mutate.
func (h *Handler) loadForMutation(w http.ResponseWriter, r *http.Request) (store.Bookmark, bool) {
key := r.PathValue("key")
if key == "" {
http.Error(w, "missing key", http.StatusBadRequest)
return store.Bookmark{}, false
}
b, ok, err := h.store.Get(readerOf(r), key)
if err != nil {
log.Printf("ui get %q: %v", key, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return store.Bookmark{}, false
}
if !ok {
http.Error(w, "not found", http.StatusNotFound)
return store.Bookmark{}, false
}
return b, true
}
// saveAndRenderCard upserts and renders the row as stored, then refreshes the
// chrome. Upsert decides whether updated_at moves, so the argument's timestamp
// is only a candidate and the response must come from the return value.
//
// ponytail: the swapped card stays put even when its new status no longer
// matches the active tab. That much is deliberate — the card showing its new
// state is the feedback for the tap. The strip and the badge are not: they
// describe the whole library, so they are rebuilt out of band on every
// mutation, at the cost of one extra list read per toggle.
func (h *Handler) saveAndRenderCard(w http.ResponseWriter, r *http.Request, b store.Bookmark) {
stored, err := h.store.Upsert(readerOf(r), b)
if err != nil {
log.Printf("ui upsert %q: %v", b.Key, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.render(w, http.StatusOK, "card", stored)
h.refreshChrome(w, r)
}
// uiFavorite flips the favourite flag. last_chapter_num is untouched, so
// Upsert keeps the stored updated_at and the list does not reorder.
func (h *Handler) uiFavorite(w http.ResponseWriter, r *http.Request) {
b, ok := h.loadForMutation(w, r)
if !ok {
return
}
b.Favorite = !b.Favorite
b.UpdatedAt = time.Now().UnixMilli()
h.saveAndRenderCard(w, r, b)
}
// uiStatus moves a bookmark between lifecycle buckets. This is the only place
// a series can be marked finished — the JSON API refuses that value, so the
// userscript cannot set it even by accident.
//
// last_chapter_num is untouched, so Upsert keeps the stored updated_at and the
// list does not reorder.
func (h *Handler) uiStatus(w http.ResponseWriter, r *http.Request) {
b, ok := h.loadForMutation(w, r)
if !ok {
return
}
if err := r.ParseForm(); err != nil {
http.Error(w, "invalid form", http.StatusBadRequest)
return
}
switch s := r.PostFormValue("status"); s {
case store.StatusReading, store.StatusArchived, store.StatusFinished:
b.Status = s
default:
http.Error(w, "invalid status", http.StatusBadRequest)
return
}
b.UpdatedAt = time.Now().UnixMilli()
h.saveAndRenderCard(w, r, b)
}
// uiChapter forces the read chapter to a value the user typed.
//
// Writing the number also clears last_chapter_url: that URL points at the
// chapter actually read, and once the number is forced elsewhere it would send
// the reader backwards. ContinueURL then falls back to the series page, which
// is always right.
//
// A submit that does not change the number touches nothing. The form is
// pre-filled, so a bare tap of Save is an easy accidental submit; it must not
// destroy last_chapter_url, nor rewrite the last_chapter display string ("45.0"
// to "45") behind a frozen updated_at.
func (h *Handler) uiChapter(w http.ResponseWriter, r *http.Request) {
b, ok := h.loadForMutation(w, r)
if !ok {
return
}
if err := r.ParseForm(); err != nil {
http.Error(w, "invalid form", http.StatusBadRequest)
return
}
raw := strings.TrimSpace(r.PostFormValue("chapter"))
num, err := strconv.ParseFloat(raw, 64)
if err != nil || num < 0 || math.IsNaN(num) || math.IsInf(num, 0) {
http.Error(w, "chapter must be a non-negative number", http.StatusBadRequest)
return
}
if num != b.LastChapterNum {
b.LastChapterURL = ""
b.LastChapter = raw
b.LastChapterNum = num
}
b.UpdatedAt = time.Now().UnixMilli()
h.saveAndRenderCard(w, r, b)
}
// uiDelete removes the row and answers with an empty body, which htmx swaps in
// place of the card — removing it from the page.
func (h *Handler) uiDelete(w http.ResponseWriter, r *http.Request) {
key := r.PathValue("key")
if key == "" {
http.Error(w, "missing key", http.StatusBadRequest)
return
}
if err := h.store.Delete(readerOf(r), key); err != nil {
log.Printf("ui delete %q: %v", key, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(http.StatusOK)
// The empty body is what removes the card; the chrome still has to be told
// the library got smaller.
h.refreshChrome(w, r)
}
// installUserscript renders the bindmounted script with the acting Reader's
// derived credential substituted in. The credential is derived, not stored,
// so installs work after any restart; the Reader never types or copies it —
// clicking Install is the whole setup.
//
// ?download=1 forces a save instead. Mobile Violentmonkey (Chromium) does not
// intercept navigation to a .user.js URL, so the Install link only renders the
// source as text there; the Reader needs the file on disk to add it by hand.
func (h *Handler) installUserscript(name string) http.HandlerFunc {
path := h.mangaUserscriptPath
if name == "novel-bookmark.user.js" {
path = h.novelUserscriptPath
}
return func(w http.ResponseWriter, r *http.Request) {
discordID, epoch, err := h.store.ReaderTokenInfo(readerOf(r))
if err != nil {
log.Printf("install %s: %v", name, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if r.URL.Query().Has("download") {
w.Header().Set("Content-Disposition", `attachment; filename="`+name+`"`)
}
userscript.Render(w, r, path, token.Token(h.tokenKey, discordID, epoch))
}
}
// rotateToken issues the acting Reader a new credential: the epoch bumps and
// the stored hash is rewritten, so the old credential stops authenticating
// the moment the statement commits. Every device must reinstall, or its
// script keeps failing silently — the setup panel states that warning next
// to the button, and the response repeats it as confirmation.
func (h *Handler) rotateToken(w http.ResponseWriter, r *http.Request) {
readerID := readerOf(r)
discordID, epoch, err := h.store.ReaderTokenInfo(readerID)
if err != nil {
log.Printf("rotate token: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
// The hash is computed for epoch+1 and guarded by it in the store, so a
// concurrent rotation cannot leave the stored hash describing another
// epoch.
if err := h.store.RotateToken(readerID, epoch, token.Hash(token.Token(h.tokenKey, discordID, epoch+1))); err != nil {
log.Printf("rotate token: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
view := listView{Lib: store.KindManga, Rotated: true}
h.render(w, http.StatusOK, "setup", view)
}
// revokeReaderSessions logs one Reader out of every browser they are signed
// in on. Owner-only: it reaches across the Reader boundary every other handler
// respects, so the guard is a comparison against the seeded owner rather than
// a role a Reader could acquire. A non-owner gets 404 — the panel does not
// exist for them, so neither should the endpoint.
func (h *Handler) revokeReaderSessions(w http.ResponseWriter, r *http.Request) {
if readerOf(r) != h.store.OwnerID() {
http.NotFound(w, r)
return
}
target, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil {
http.Error(w, "bad reader id", http.StatusBadRequest)
return
}
// The owner is not one of the Readers this endpoint reaches: revoking
// themselves would sign out the browser making the request, which is what
// logout is for. The roster hides the button; this refuses the hand-rolled
// POST behind it.
if target == h.store.OwnerID() {
http.NotFound(w, r)
return
}
if err := h.store.DeleteReaderSessions(target); err != nil {
log.Printf("revoke sessions: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
readers, err := h.store.Readers()
if err != nil {
log.Printf("revoke sessions: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.render(w, http.StatusOK, "readers", listView{Owner: true, Readers: readers, OwnerID: h.store.OwnerID()})
}
-201
View File
@@ -1,201 +0,0 @@
package main
import (
"context"
"log"
"net/url"
"time"
)
// fetcher retrieves a series page. It exists as an interface so tests can inject
// a fake: nothing in the test suite may touch the network or the TLS client.
type fetcher interface {
Get(ctx context.Context, url string) (body string, status int, err error)
}
// latestPoller re-checks each bookmarked series' newest published chapter on a
// schedule, independent of the userscript's own in-browser checks. The two run
// in parallel and report the same observable fact, so whichever writes last wins
// and neither needs to know about the other.
//
// Two clocks, deliberately independent:
//
// - interval is how often this goroutine wakes up and looks.
// - cooldown is how long one bookmark rests since its own last check.
//
// Only the cooldown is per bookmark, and it is enforced by the WHERE clause in
// DueForLatestCheck rather than by any timer. Shortening interval therefore
// cannot shorten anyone's cooldown; it only makes the poller wake up and find
// nothing due more often.
type latestPoller struct {
store *Store
fetch fetcher
now func() time.Time // injected so tests can freeze it
cooldown time.Duration
interval time.Duration
stagger time.Duration
batch int
}
// Run polls until ctx is cancelled.
//
// runOnce is called synchronously, so a batch that overruns the tick delays the
// next one instead of stacking a second batch on top of it. That is the intended
// failure mode for a misconfigured batch x stagger: a slower cadence, never
// concurrent fetch storms.
func (p *latestPoller) Run(ctx context.Context) {
log.Printf("latest-chapter poller: interval=%s cooldown=%s batch=%d stagger=%s",
p.interval, p.cooldown, p.batch, p.stagger)
t := time.NewTicker(p.interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
log.Println("latest-chapter poller: stopped")
return
case <-t.C:
p.runOnce(ctx)
}
}
}
// runOnce processes one batch of due bookmarks.
func (p *latestPoller) runOnce(ctx context.Context) {
cutoff := p.now().Add(-p.cooldown).UnixMilli()
due, err := p.store.DueForLatestCheck(cutoff, p.batch)
if err != nil {
log.Printf("latest poll: due query: %v", err)
return
}
checked := 0
for i, b := range due {
if ctx.Err() != nil {
break
}
// Staggered rather than fired together: a burst of simultaneous requests
// from one server IP is the traffic shape most likely to move that IP's
// bot score. This is the server-side analogue of the userscript's "one
// series per navigation ... indistinguishable from browsing" (L455-456).
stopped := false
if i > 0 && p.stagger > 0 {
select {
case <-ctx.Done():
stopped = true
case <-time.After(p.stagger):
}
}
if stopped {
break
}
p.checkOne(ctx, b)
checked++
}
// due vs checked is how you tell which constraint is binding: ticks that
// report due=0 mean the cooldown is the limit, ticks that report due==batch
// every time mean throughput is.
log.Printf("latest poll: due=%d checked=%d", len(due), checked)
}
// checkOne re-checks one series. Every failure path here is "log and move on":
// the poller is a best-effort enhancement, and no single bad series may stall a
// batch or take down the process.
func (p *latestPoller) checkOne(ctx context.Context, b Bookmark) {
defer func() {
if r := recover(); r != nil {
log.Printf("latest poll %q: recovered from panic: %v", b.Key, r)
}
}()
// Stamped before the fetch, not after, so an error, a timeout, or a shutdown
// mid-request still consumes the cooldown. Otherwise a renamed or deleted
// series would be retried on every single tick forever. The userscript
// stamps in the same order and for the same reason (L471-473).
if err := p.store.MarkLatestChecked(b.Key, p.now().UnixMilli()); err != nil {
log.Printf("latest poll %q: mark checked: %v", b.Key, err)
return
}
// series_url is client-supplied (PUT /bookmarks/{key} accepts any string),
// so this is not just an optimisation against burning a request on an
// unknown site: without it, the server would issue a GET from its own
// network position to whatever URL a token-holder writes, including
// link-local/internal addresses or non-https schemes. The cooldown above
// is already consumed, so a row that never passes this check is retried at
// cooldown pace rather than hot-looping.
if !fetchableSeriesURL(b.Site, b.SeriesURL) {
log.Printf("latest poll %q: not fetchable: site=%q url=%q", b.Key, b.Site, b.SeriesURL)
return
}
body, status, err := p.fetch.Get(ctx, b.SeriesURL)
if err != nil {
log.Printf("latest poll %q: fetch %s: %v", b.Key, b.SeriesURL, err)
return
}
if status != 200 {
log.Printf("latest poll %q: fetch %s: status %d", b.Key, b.SeriesURL, status)
return
}
latest, ok := latestChapterFrom(b.Site, b.SeriesURL, body)
if !ok {
// Most likely a challenge page or a layout change. Either way the row is
// already stamped, so this waits out a cooldown instead of hot-looping.
log.Printf("latest poll %q: no chapter links in %d bytes", b.Key, len(body))
return
}
// Re-read: the row may have been updated or deleted while the fetch was in
// flight, and writing b back wholesale would undo that.
//
// ponytail: non-transactional read-modify-write, wrap Get+Upsert in a tx if
// this ever runs for more than one user. A client PUT that commits between
// these two statements is lost to the stale re-read — reverting read
// progress or a status change, and moving updated_at because the stored
// value now differs. Accepted for a single-user deployment: the window is
// milliseconds and the loser is one poll cycle.
cur, found, err := p.store.Get(b.Key)
if err != nil {
log.Printf("latest poll %q: reread: %v", b.Key, err)
return
}
if !found {
return
}
// Equality, not >, mirroring the userscript (L427): a site that retracts a
// chapter should correct the stored number downward.
if cur.LatestChapterNum != nil && *cur.LatestChapterNum == latest.Num {
return
}
num := latest.Num
cur.LatestChapter = latest.Label
cur.LatestChapterNum = &num
// A candidate only. last_chapter_num is untouched, so the CASE in Upsert
// keeps the stored updated_at and the bookmark list does not reorder.
cur.UpdatedAt = p.now().UnixMilli()
if _, err := p.store.Upsert(cur); err != nil {
log.Printf("latest poll %q: upsert: %v", b.Key, err)
return
}
log.Printf("latest poll %q: latest is now %s", b.Key, latest.Label)
}
// fetchableSeriesURL reports whether site is a site latestChapterFrom knows how
// to parse and seriesURL is safe to hand to the fetcher: an https URL with a
// non-empty host. series_url comes from client-supplied PUT bodies, so this is
// a defence against the poller being used to probe arbitrary hosts from the
// server's own network position, not just a check against wasted requests.
func fetchableSeriesURL(site, seriesURL string) bool {
switch site {
case "asura", "demonic":
default:
return false
}
u, err := url.Parse(seriesURL)
if err != nil {
return false
}
return u.Scheme == "https" && u.Host != ""
}
-82
View File
@@ -1,82 +0,0 @@
package main
import (
"regexp"
"strconv"
"strings"
)
// latestChapter is the newest chapter a series page advertises.
type latestChapter struct {
Num float64
Label string
}
// asuraSlugRe pulls the series slug out of a stored series_url.
// Shape verified live 2026-07-26: https://asurascans.com/comics/<slug>, where
// the slug carries a trailing build-hash suffix (e.g. "-f886a8af") that
// rotates on every site redeploy — callers must strip it (asuraBuildHash)
// before using the slug to scope anything.
var asuraSlugRe = regexp.MustCompile(`/comics/([^/?#]+)`)
// demonicChapterRe matches the pre-redirect anchors demonic series pages link
// through. Both the raw "&" and the HTML-escaped "&amp;" forms occur.
var demonicChapterRe = regexp.MustCompile(`chaptered\.php\?manga=\d+&(?:amp;)?chapter=([0-9.]+)`)
// latestChapterFrom returns the highest chapter number body advertises for this
// series. ok is false when the body yields nothing usable — an unknown site, an
// empty body, a Cloudflare challenge page, and a site redesign all land here,
// and the caller treats all four identically.
//
// Ported from the userscript's latestChapterFromAnchors (asura L123-133,
// demonic L183-193), including its reason for taking a maximum rather than a
// first or last: neither site lists chapters in a dependable order.
//
// The userscript's asura rule additionally requires the anchor text to match
// /Chapter\s+[\d.]+/i. That check exists only to skip the "First Chapter"
// shortcut, which points at chapter/1 and therefore can never win a maximum, so
// it is redundant here. For asura, scoping the pattern to this series' own slug
// replaces it with a stronger guarantee: a chapter link belonging to some other
// series cannot contribute even if the page starts carrying them. demonic has no
// such guarantee — demonicChapterRe matches any chaptered.php?manga=<id> anchor
// with no per-series scoping, because the stored series_id for demonic is a
// slug, not the numeric id the URL carries, so it cannot easily be scoped.
func latestChapterFrom(site, seriesURL, body string) (latestChapter, bool) {
var re *regexp.Regexp
switch site {
case "asura":
m := asuraSlugRe.FindStringSubmatch(seriesURL)
if m == nil {
return latestChapter{}, false
}
// Stored URLs predating a redeploy may carry a stale build hash;
// chapter hrefs in the fetched body carry the current one. Strip to
// the stable ID (same rule as migrateAsuraKeys) and make the hash
// optional in the pattern, so scoping survives rotations.
slug := asuraBuildHash.ReplaceAllString(m[1], "")
// Compiled per call rather than cached: this runs once per fetch, which
// is at most a few times a minute, and the slug varies per series.
re = regexp.MustCompile(`/comics/` + regexp.QuoteMeta(slug) + `(?:-[0-9a-f]{8})?/chapter/([0-9.]+)`)
case "demonic":
re = demonicChapterRe
default:
return latestChapter{}, false
}
var best latestChapter
found := false
for _, m := range re.FindAllStringSubmatch(body, -1) {
// [0-9.]+ can swallow a trailing separator, e.g. "chapter/12." in a
// sentence; ParseFloat would reject the whole match.
raw := strings.Trim(m[1], ".")
num, err := strconv.ParseFloat(raw, 64)
if err != nil {
continue
}
if !found || num > best.Num {
best = latestChapter{Num: num, Label: "Chapter " + raw}
found = true
}
}
return best, found
}
-122
View File
@@ -1,122 +0,0 @@
package main
import "testing"
// Trimmed from https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af
// fetched 2026-07-26. The first anchor is the "First Chapter" shortcut: it is a
// real chapter link with no "Chapter N" text, and it must not be mistaken for
// the latest just because it parses.
const asuraSeriesFixture = `
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/1" class="py-3 rounded-md bg-[#E8E8E8]"><svg class="w-4 h-4"></svg>First Chapter</a>
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/179" data-astro-prefetch="hover" class="group flex"><span class="font-medium">Chapter 179</span></a>
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/181" data-astro-prefetch="hover" class="group flex"><span class="font-medium">Chapter 181</span></a>
<a href="/comics/chronicles-of-the-demon-faction-f886a8af/chapter/180" data-astro-prefetch="hover" class="group flex"><span class="font-medium">Chapter 180</span></a>
`
// A chapter link belonging to a different series, of the kind a "you might also
// like" strip would introduce. Slug scoping must exclude it.
const asuraCrossSeriesFixture = asuraSeriesFixture + `
<a href="/comics/some-other-series-aabbccdd/chapter/999" class="group flex"><span>Chapter 999</span></a>
`
// Trimmed from https://demonicscans.org/manga/Catastrophic-Necromancer fetched
// 2026-07-26. Note the raw "&", the doubled space after <a, and the decimal
// chapters, all as they appear live.
const demonicSeriesFixture = `
<a href="/chaptered.php?manga=11799&chapter=0.5" class="chplinks" title="Catastrophic Necromancer 0.5">Chapter 0.5</a>
<a href="/chaptered.php?manga=11799&chapter=294" class="chplinks" title="Catastrophic Necromancer 294">Chapter 294</a>
<a href="/chaptered.php?manga=11799&amp;chapter=296" class="chplinks" title="Catastrophic Necromancer 296">Chapter 296</a>
<a href="/chaptered.php?manga=11799&chapter=295" class="chplinks" title="Catastrophic Necromancer 295">Chapter 295</a>
`
// What Cloudflare serves instead of the page when an IP's bot score flips.
const challengeFixture = `<!DOCTYPE html><html><head><title>Just a moment...</title>
<script src="/cdn-cgi/challenge-platform/h/b/orchestrate/chl_page/v1"></script></head>
<body><div id="challenge-running">Checking your browser</div></body></html>`
func TestLatestChapterFrom(t *testing.T) {
const asuraURL = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
const demonicURL = "https://demonicscans.org/manga/Catastrophic-Necromancer"
tests := []struct {
name string
site string
seriesURL string
body string
wantOK bool
wantNum float64
wantLabel string
}{
{
name: "asura takes the max, not the last listed",
site: "asura", seriesURL: asuraURL, body: asuraSeriesFixture,
wantOK: true, wantNum: 181, wantLabel: "Chapter 181",
},
{
name: "asura ignores another series' chapter links",
site: "asura", seriesURL: asuraURL, body: asuraCrossSeriesFixture,
wantOK: true, wantNum: 181, wantLabel: "Chapter 181",
},
{
name: "asura scoping survives a build-hash rotation",
site: "asura",
seriesURL: "https://asurascans.com/comics/chronicles-of-the-demon-faction-059befe1",
body: asuraCrossSeriesFixture,
wantOK: true, wantNum: 181, wantLabel: "Chapter 181",
},
{
name: "asura with an unparseable series url",
site: "asura", seriesURL: "https://asurascans.com/", body: asuraSeriesFixture,
wantOK: false,
},
{
name: "demonic takes the max across raw and escaped ampersands",
site: "demonic", seriesURL: demonicURL, body: demonicSeriesFixture,
wantOK: true, wantNum: 296, wantLabel: "Chapter 296",
},
{
name: "demonic keeps decimal chapters parseable",
site: "demonic", seriesURL: demonicURL,
body: `<a href="/chaptered.php?manga=11799&chapter=0.5">Chapter 0.5</a>`,
wantOK: true, wantNum: 0.5, wantLabel: "Chapter 0.5",
},
{
name: "empty body",
site: "asura", seriesURL: asuraURL, body: "",
wantOK: false,
},
{
name: "cloudflare challenge page",
site: "asura", seriesURL: asuraURL, body: challengeFixture,
wantOK: false,
},
{
name: "demonic markup handed to the asura rule",
site: "asura", seriesURL: asuraURL, body: demonicSeriesFixture,
wantOK: false,
},
{
name: "unknown site",
site: "mangadex", seriesURL: "https://example.com/x", body: asuraSeriesFixture,
wantOK: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, ok := latestChapterFrom(tt.site, tt.seriesURL, tt.body)
if ok != tt.wantOK {
t.Fatalf("ok = %v, want %v (got %+v)", ok, tt.wantOK, got)
}
if !tt.wantOK {
return
}
if got.Num != tt.wantNum {
t.Errorf("Num = %v, want %v", got.Num, tt.wantNum)
}
if got.Label != tt.wantLabel {
t.Errorf("Label = %q, want %q", got.Label, tt.wantLabel)
}
})
}
}
-320
View File
@@ -1,320 +0,0 @@
package main
import (
"context"
"errors"
"sync"
"testing"
"time"
)
// fakeFetcher stands in for the network. Every poller test uses it, so nothing
// in this file can reach tls-client or a real site.
type fakeFetcher struct {
mu sync.Mutex
calls []string
body string
status int
err error
// perURL overrides body/status/err for specific URLs.
perURL map[string]fakeResponse
}
type fakeResponse struct {
body string
status int
err error
}
func (f *fakeFetcher) Get(ctx context.Context, url string) (string, int, error) {
f.mu.Lock()
f.calls = append(f.calls, url)
f.mu.Unlock()
if r, ok := f.perURL[url]; ok {
return r.body, r.status, r.err
}
return f.body, f.status, f.err
}
func (f *fakeFetcher) callCount() int {
f.mu.Lock()
defer f.mu.Unlock()
return len(f.calls)
}
// newTestPoller wires a poller with a frozen clock and no stagger, so tests run
// instantly and deterministically.
func newTestPoller(t *testing.T, s *Store, f fetcher, at time.Time) *latestPoller {
t.Helper()
return &latestPoller{
store: s,
fetch: f,
now: func() time.Time { return at },
cooldown: time.Hour,
interval: 10 * time.Minute,
stagger: 0,
batch: 14,
}
}
func TestRunOnceRecordsLatestChapter(t *testing.T) {
s := newTestStore(t)
const url = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
seedForCheck(t, s, "asura:chronicles-of-the-demon-faction-f886a8af", url, 0)
now := time.UnixMilli(5_000_000)
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).runOnce(context.Background())
b, ok, err := s.Get("asura:chronicles-of-the-demon-faction-f886a8af")
if err != nil || !ok {
t.Fatalf("Get: %v ok=%v", err, ok)
}
if b.LatestChapterNum == nil || *b.LatestChapterNum != 181 {
t.Fatalf("LatestChapterNum = %v, want 181", b.LatestChapterNum)
}
if b.LatestChapter != "Chapter 181" {
t.Fatalf("LatestChapter = %q, want %q", b.LatestChapter, "Chapter 181")
}
if got := readLatestCheckedAt(t, s, "asura:chronicles-of-the-demon-faction-f886a8af"); got != now.UnixMilli() {
t.Fatalf("latest_checked_at = %d, want %d", got, now.UnixMilli())
}
}
// The whole point of the updated_at CASE in Upsert: a newly published chapter is
// not reading progress and must not move the series up the list.
func TestRunOnceDoesNotReorderList(t *testing.T) {
s := newTestStore(t)
const url = "https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af"
const key = "asura:chronicles-of-the-demon-faction-f886a8af"
// "other" is the most recently read, so it must stay at the top of List().
if _, err := s.Upsert(Bookmark{
Key: "asura:other", Site: "asura", SeriesID: "other",
SeriesURL: "https://asurascans.com/comics/other", UpdatedAt: 9_000_000,
}); err != nil {
t.Fatalf("seed other: %v", err)
}
seedForCheck(t, s, key, url, 0)
before, _, err := s.Get(key)
if err != nil {
t.Fatalf("Get before: %v", err)
}
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, time.UnixMilli(9_999_999)).runOnce(context.Background())
after, _, err := s.Get(key)
if err != nil {
t.Fatalf("Get after: %v", err)
}
if after.UpdatedAt != before.UpdatedAt {
t.Fatalf("updated_at moved from %d to %d on a latest-chapter bump",
before.UpdatedAt, after.UpdatedAt)
}
list, err := s.List()
if err != nil {
t.Fatalf("List: %v", err)
}
if list[0].Key != "asura:other" {
t.Fatalf("list reordered: head is %q, want asura:other", list[0].Key)
}
}
// A failed fetch must still consume the cooldown, or a renamed series gets
// retried on every tick forever.
func TestRunOnceMarksCheckedOnFailure(t *testing.T) {
tests := []struct {
name string
resp fakeResponse
}{
{"network error", fakeResponse{err: errors.New("dial tcp: refused")}},
{"non-200", fakeResponse{body: "nope", status: 503}},
{"challenge page", fakeResponse{body: challengeFixture, status: 200}},
{"empty body", fakeResponse{body: "", status: 200}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := newTestStore(t)
const url = "https://asurascans.com/comics/x"
seedForCheck(t, s, "asura:x", url, 0)
now := time.UnixMilli(7_000_000)
f := &fakeFetcher{perURL: map[string]fakeResponse{url: tt.resp}}
newTestPoller(t, s, f, now).runOnce(context.Background())
if got := readLatestCheckedAt(t, s, "asura:x"); got != now.UnixMilli() {
t.Fatalf("latest_checked_at = %d, want %d", got, now.UnixMilli())
}
b, _, err := s.Get("asura:x")
if err != nil {
t.Fatalf("Get: %v", err)
}
if b.LatestChapterNum != nil {
t.Fatalf("LatestChapterNum = %v, want nil on a failed check", *b.LatestChapterNum)
}
})
}
}
func TestRunOnceRespectsBatchLimit(t *testing.T) {
s := newTestStore(t)
for i := 0; i < 20; i++ {
key := "asura:s" + string(rune('a'+i))
seedForCheck(t, s, key, "https://asurascans.com/comics/"+key, 0)
}
f := &fakeFetcher{body: "", status: 200}
p := newTestPoller(t, s, f, time.UnixMilli(5_000_000))
p.batch = 5
p.runOnce(context.Background())
if got := f.callCount(); got != 5 {
t.Fatalf("fetched %d series, want 5 (batch limit)", got)
}
}
// One unreachable series must not abandon the rest of the batch.
func TestRunOnceOneBadSeriesDoesNotStallBatch(t *testing.T) {
s := newTestStore(t)
keys := []string{"asura:a", "asura:b", "asura:c", "asura:d", "asura:e"}
for _, k := range keys {
seedForCheck(t, s, k, "https://asurascans.com/comics/"+k, 0)
}
now := time.UnixMilli(6_000_000)
f := &fakeFetcher{
body: "", status: 200,
perURL: map[string]fakeResponse{
"https://asurascans.com/comics/asura:b": {err: errors.New("boom")},
},
}
newTestPoller(t, s, f, now).runOnce(context.Background())
if got := f.callCount(); got != 5 {
t.Fatalf("fetched %d series, want all 5 attempted", got)
}
for _, k := range keys {
if got := readLatestCheckedAt(t, s, k); got != now.UnixMilli() {
t.Fatalf("%s latest_checked_at = %d, want %d", k, got, now.UnixMilli())
}
}
}
// The cooldown is enforced by the due query, so a second immediate pass must do
// nothing at all — this is what makes the tick interval independent of it.
func TestRunOnceHonoursCooldownAcrossPasses(t *testing.T) {
s := newTestStore(t)
const url = "https://asurascans.com/comics/x"
seedForCheck(t, s, "asura:x", url, 0)
now := time.UnixMilli(8_000_000)
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
p := newTestPoller(t, s, f, now)
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("first pass fetched %d, want 1", got)
}
// Same instant, and again 59 minutes later: both inside the 1h cooldown.
p.runOnce(context.Background())
p.now = func() time.Time { return now.Add(59 * time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != 1 {
t.Fatalf("fetched %d times inside the cooldown, want 1", got)
}
// Past the cooldown, it is due again.
p.now = func() time.Time { return now.Add(61 * time.Minute) }
p.runOnce(context.Background())
if got := f.callCount(); got != 2 {
t.Fatalf("fetched %d times after the cooldown, want 2", got)
}
}
// A site that retracts a chapter should correct the stored number downward,
// mirroring the userscript's equality check (L427) rather than a >.
func TestRunOnceCorrectsDownward(t *testing.T) {
s := newTestStore(t)
const url = "https://demonicscans.org/manga/Catastrophic-Necromancer"
const key = "demonic:Catastrophic-Necromancer"
high := 400.0
if _, err := s.Upsert(Bookmark{
Key: key, Site: "demonic", SeriesID: "Catastrophic-Necromancer",
SeriesURL: url, LatestChapter: "Chapter 400", LatestChapterNum: &high,
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
f := &fakeFetcher{body: demonicSeriesFixture, status: 200}
newTestPoller(t, s, f, time.UnixMilli(5_000_000)).runOnce(context.Background())
b, _, err := s.Get(key)
if err != nil {
t.Fatalf("Get: %v", err)
}
if b.LatestChapterNum == nil || *b.LatestChapterNum != 296 {
t.Fatalf("LatestChapterNum = %v, want 296", b.LatestChapterNum)
}
}
// series_url is client-supplied via PUT /bookmarks/{key}, so checkOne must
// reject anything that is not a known site with an https URL before spending a
// request on it — the cooldown still gets consumed either way.
func TestCheckOneValidatesSeriesURLBeforeFetching(t *testing.T) {
tests := []struct {
name string
site string
seriesURL string
wantCalls int
}{
{"unknown site", "mangadex", "https://mangadex.org/title/x", 0},
{"http scheme", "asura", "http://asurascans.com/comics/x", 0},
{"unparseable url", "asura", "http://[::1", 0},
{"valid https asura", "asura", "https://asurascans.com/comics/x", 1},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
s := newTestStore(t)
key := tt.site + ":x"
if _, err := s.Upsert(Bookmark{
Key: key, Site: tt.site, SeriesID: "x", SeriesURL: tt.seriesURL,
UpdatedAt: 1000,
}); err != nil {
t.Fatalf("seed: %v", err)
}
now := time.UnixMilli(4_000_000)
f := &fakeFetcher{body: asuraSeriesFixture, status: 200}
newTestPoller(t, s, f, now).checkOne(context.Background(), Bookmark{
Key: key, Site: tt.site, SeriesURL: tt.seriesURL,
})
if got := f.callCount(); got != tt.wantCalls {
t.Fatalf("fetch calls = %d, want %d", got, tt.wantCalls)
}
if got := readLatestCheckedAt(t, s, key); got != now.UnixMilli() {
t.Fatalf("latest_checked_at = %d, want %d (cooldown must be consumed regardless)", got, now.UnixMilli())
}
})
}
}
// A cancelled context must abandon the batch rather than run it to completion.
func TestRunOnceStopsOnCancelledContext(t *testing.T) {
s := newTestStore(t)
for _, k := range []string{"asura:a", "asura:b", "asura:c"} {
seedForCheck(t, s, k, "https://asurascans.com/comics/"+k, 0)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
f := &fakeFetcher{body: "", status: 200}
newTestPoller(t, s, f, time.UnixMilli(5_000_000)).runOnce(ctx)
if got := f.callCount(); got != 0 {
t.Fatalf("fetched %d series with a cancelled context, want 0", got)
}
}
+198 -48
View File
@@ -11,19 +11,50 @@ import (
"strings"
"syscall"
"time"
"bookmarkmanager/backend/internal/api"
"bookmarkmanager/backend/internal/httpmw"
"bookmarkmanager/backend/internal/latest"
"bookmarkmanager/backend/internal/store"
"bookmarkmanager/backend/internal/token"
"bookmarkmanager/backend/internal/userscript"
"bookmarkmanager/backend/internal/web"
)
// Config holds all runtime settings, sourced from environment variables.
type Config struct {
Token string
// TokenKey derives every Reader's userscript credential (internal/token).
// Required: without it no install URL can ever be built.
TokenKey string
AllowedOrigins []string
DBPath string
// DatabaseURL is the Postgres connection URL; required, no default,
// because a wrong guess would silently start on an empty database.
DatabaseURL string
// CoverDir is the filesystem volume for immutable cover bytes. Required:
// serving a stored address without durable bytes would be worse than a
// startup failure.
CoverDir string
// PublicBaseURL is the origin this deployment answers on, e.g.
// "https://bookmarks.example.com". Required: cover URLs go out absolute
// because the userscript renders them on third-party origins, where a
// relative path would resolve against the Site (ADR-0007), and there is
// no way to guess it from a request the poller never sees.
PublicBaseURL string
Port string
// WebPassword gates the browser UI. Empty disables the web routes entirely.
WebPassword string
// OwnerDiscordID identifies the seeded owner Reader (issue #22). Required:
// bookmarks are scoped to a Reader, and a fresh deployment needs one
// before anybody logs in. The owner is also the only Reader who can revoke
// another Reader's sessions.
OwnerDiscordID string
// Discord is the OAuth application the browser UI signs in with.
Discord web.DiscordConfig
// UserscriptPath is the file served at /u/{token}/manga-bookmark.user.js.
// Supplied by a bindmount so the script can be edited without a rebuild.
UserscriptPath string
// NovelUserscriptPath is the file served at
// /u/{token}/novel-bookmark.user.js. Same bindmount, second script: the
// two libraries are separate installs.
NovelUserscriptPath string
// LatestPoll configures the background latest-chapter fetcher.
LatestPoll LatestPoll
}
@@ -37,6 +68,7 @@ type Config struct {
type LatestPoll struct {
Enabled bool
Cooldown time.Duration
BrowserCooldown time.Duration
Interval time.Duration
Stagger time.Duration
Batch int
@@ -44,6 +76,7 @@ type LatestPoll struct {
const (
defaultPollCooldown = time.Hour
defaultBrowserPollCooldown = 6 * time.Hour
defaultPollInterval = 10 * time.Minute
defaultPollStagger = 20 * time.Second
defaultPollBatch = 14
@@ -104,20 +137,27 @@ func envInt(key string, def int) int {
return n
}
func clampPollCooldown(name string, d time.Duration) time.Duration {
if d < minPollCooldown {
log.Printf("config: %s %s is below the %s floor, clamping", name, d, minPollCooldown)
return minPollCooldown
}
return d
}
// loadLatestPoll reads the poller's settings, clamping anything that would make
// it antisocial.
func loadLatestPoll() LatestPoll {
p := LatestPoll{
Enabled: envBool("LATEST_CHAPTER_POLL_ENABLED", true),
Cooldown: envDuration("LATEST_CHAPTER_POLL_COOLDOWN", defaultPollCooldown),
BrowserCooldown: envDuration("LATEST_CHAPTER_POLL_BROWSER_COOLDOWN", defaultBrowserPollCooldown),
Interval: envDuration("LATEST_CHAPTER_POLL_INTERVAL", defaultPollInterval),
Stagger: envDuration("LATEST_CHAPTER_POLL_STAGGER", defaultPollStagger),
Batch: envInt("LATEST_CHAPTER_POLL_BATCH", defaultPollBatch),
}
if p.Cooldown < minPollCooldown {
log.Printf("config: cooldown %s is below the %s floor, clamping", p.Cooldown, minPollCooldown)
p.Cooldown = minPollCooldown
}
p.Cooldown = clampPollCooldown("cooldown", p.Cooldown)
p.BrowserCooldown = clampPollCooldown("browser cooldown", p.BrowserCooldown)
// batch x stagger has to fit inside one tick or a batch is still running
// when the next one is due. Run() serialises them, so this degrades to a
// slower cadence rather than to overlapping fetches — worth a warning, not
@@ -131,13 +171,24 @@ func loadLatestPoll() LatestPoll {
func loadConfig() Config {
c := Config{
Token: os.Getenv("API_TOKEN"),
DBPath: envOr("DB_PATH", "/data/bookmarks.db"),
TokenKey: os.Getenv("TOKEN_KEY"),
DatabaseURL: os.Getenv("DATABASE_URL"),
CoverDir: os.Getenv("COVER_DIR"),
PublicBaseURL: os.Getenv("PUBLIC_BASE_URL"),
Port: envOr("PORT", "8080"),
WebPassword: os.Getenv("WEB_PASSWORD"),
OwnerDiscordID: os.Getenv("OWNER_DISCORD_ID"),
UserscriptPath: envOr("USERSCRIPT_PATH", "/userscript/manga-bookmark.user.js"),
NovelUserscriptPath: envOr("NOVEL_USERSCRIPT_PATH", "/userscript/novel-bookmark.user.js"),
LatestPoll: loadLatestPoll(),
}
c.Discord = web.DiscordConfig{
ClientID: os.Getenv("DISCORD_CLIENT_ID"),
ClientSecret: os.Getenv("DISCORD_CLIENT_SECRET"),
GuildID: os.Getenv("DISCORD_GUILD_ID"),
RequiredRole: os.Getenv("DISCORD_REQUIRED_ROLE"),
APIBase: envOr("DISCORD_API_BASE", "https://discord.com/api/v10"),
RedirectURI: os.Getenv("DISCORD_REDIRECT_URI"),
}
for _, o := range strings.Split(os.Getenv("ALLOWED_ORIGINS"), ",") {
if o = strings.TrimSpace(o); o != "" {
c.AllowedOrigins = append(c.AllowedOrigins, o)
@@ -149,37 +200,44 @@ func loadConfig() Config {
// newRouter wires routes and middleware. CORS is the outermost layer so
// preflight OPTIONS short-circuits before auth; /bookmarks* is auth-protected,
// /healthz is public.
func newRouter(store *Store, cfg Config) http.Handler {
func newRouter(s *store.Store, cfg Config) http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("GET /healthz", healthz)
h := &api.Handler{Store: s}
mux.HandleFunc("GET /healthz", api.Healthz)
// Public: cover bytes are rendered by the userscript on origins that may
// not send our credentials, and the address is the hash of a URL the Site
// already publishes (ADR-0007).
mux.HandleFunc("GET /covers/{address}", h.Cover)
// Outside withAuth (the updater sends no Authorization header) and outside
// the WEB_PASSWORD gate (the script must be installable either way). The
// path segment carries the token instead.
mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", userscriptHandler(cfg.Token, cfg.UserscriptPath))
// Outside httpmw.Auth (the updater sends no Authorization header) and
// outside the web UI's Discord auth (the script must be installable
// without a browser session). The path segment carries the credential
// instead, and the script is rendered with the resolved Reader's
// credential substituted in.
mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js",
userscript.Handler(s, cfg.UserscriptPath))
mux.HandleFunc("GET /u/{token}/novel-bookmark.user.js",
userscript.Handler(s, cfg.NovelUserscriptPath))
h := &bookmarkHandler{store: store}
protected := http.NewServeMux()
protected.HandleFunc("GET /bookmarks", h.list)
protected.HandleFunc("PUT /bookmarks/{key}", h.put)
protected.HandleFunc("DELETE /bookmarks/{key}", h.delete)
protected.HandleFunc("GET /bookmarks", h.List)
protected.HandleFunc("PUT /bookmarks/{key}", h.Put)
protected.HandleFunc("DELETE /bookmarks/{key}", h.Delete)
auth := withAuth(cfg.Token, protected)
auth := httpmw.Auth(s, protected)
mux.Handle("/bookmarks", auth)
mux.Handle("/bookmarks/", auth)
// The browser UI is registered only when a password is configured, so a
// deployment that forgets WEB_PASSWORD exposes nothing rather than
// exposing an unprotected list.
if cfg.WebPassword != "" {
web, err := newWebHandler(store, cfg)
// The browser UI is always registered; signing in is Discord OAuth, so
// there is no password to forget and no gate to leave unset.
wh, err := web.New(s, cfg.Discord, []byte(cfg.TokenKey),
cfg.UserscriptPath, cfg.NovelUserscriptPath)
if err != nil {
log.Fatalf("web handler: %v", err)
}
web.register(mux)
}
wh.Register(mux)
return withCORS(cfg.AllowedOrigins, guardEmptyUserscriptToken(mux))
return httpmw.CORS(cfg.AllowedOrigins, httpmw.Gzip(guardEmptyUserscriptToken(mux)))
}
// guardEmptyUserscriptToken heads off ServeMux's own path-cleaning redirect:
@@ -199,31 +257,106 @@ func guardEmptyUserscriptToken(next http.Handler) http.Handler {
func main() {
cfg := loadConfig()
if cfg.Token == "" {
log.Fatal("API_TOKEN is required")
if cfg.TokenKey == "" {
log.Fatal("TOKEN_KEY is required")
}
if cfg.OwnerDiscordID == "" {
log.Fatal("OWNER_DISCORD_ID is required")
}
if cfg.DatabaseURL == "" {
log.Fatal("DATABASE_URL is required")
}
if cfg.CoverDir == "" {
log.Fatal("COVER_DIR is required")
}
if cfg.PublicBaseURL == "" {
log.Fatal("PUBLIC_BASE_URL is required")
}
// The web UI signs in through Discord, so a deployment without the OAuth
// application is misconfigured rather than passwordless.
for key, v := range map[string]string{
"DISCORD_CLIENT_ID": cfg.Discord.ClientID,
"DISCORD_CLIENT_SECRET": cfg.Discord.ClientSecret,
"DISCORD_GUILD_ID": cfg.Discord.GuildID,
"DISCORD_REDIRECT_URI": cfg.Discord.RedirectURI,
} {
if v == "" {
log.Fatalf("%s is required", key)
}
}
// The owner's userscript credential is derived from TOKEN_KEY at epoch 0
// (internal/token); the readers row carries its SHA-256, not the
// credential itself.
owner := store.Owner{
DiscordID: cfg.OwnerDiscordID,
TokenHash: token.Hash(token.Token([]byte(cfg.TokenKey), cfg.OwnerDiscordID, 0)),
}
store, err := OpenStore(cfg.DBPath)
s, err := store.Open(cfg.DatabaseURL, owner, cfg.CoverDir, cfg.PublicBaseURL)
if err != nil {
log.Fatalf("open store: %v", err)
}
defer store.Close()
defer s.Close()
// The poller is off the request path entirely: if it cannot start, the
// service still serves bookmarks and the userscript still captures latest
// chapters on its own.
//
// One headless browser serves both consumers that need a Cloudflare
// challenge cleared: the poller's kagane/novelfull page fetches and
// kagane's cover bytes. Optional — unset leaves kagane unpolled and its
// Covers blank until the bytes exist.
var browser latest.Fetcher
pollCtx, stopPoll := context.WithCancel(context.Background())
defer stopPoll()
startLatestPoller(pollCtx, store, cfg.LatestPoll)
if ws := strings.TrimSpace(os.Getenv("BROWSER_WS_URL")); ws != "" {
bf, err := latest.NewBrowserFetcher(ws)
if err != nil {
log.Printf("browser fetcher disabled: %v", err)
} else {
browser = bf
context.AfterFunc(pollCtx, bf.Close)
log.Printf("browser fetcher at %s", ws)
}
}
// A Series nobody had bookmarked before gets its Latest Chapter and its
// Cover from one fetch, at creation, instead of waiting out a poll queue
// ordered by Reader count. Off the write path: the hook returns as soon
// as the goroutine is started.
var tlsFetch latest.Fetcher
if f, err := latest.NewTLSFetcher(); err != nil {
log.Printf("creation-time acquisition: plain-TLS Sites disabled, cannot build client: %v", err)
} else {
tlsFetch = f
}
var browserCover latest.BrowserCoverFetcher
if b, ok := browser.(latest.BrowserCoverFetcher); ok {
browserCover = b
}
// The Acquirer must survive a TLS client failure: kagane needs only the
// sidecar, and novelfull degrades to whatever is left.
if tlsFetch != nil || browser != nil {
acq := &latest.Acquirer{
Store: s,
Fetch: tlsFetch,
BrowserFetch: browser,
BrowserCoverFetch: browserCover,
Covers: latest.NewCoverFetcher(),
Ctx: pollCtx,
}
s.OnSeriesCreated = acq.Acquire
}
startLatestPoller(pollCtx, s, cfg.LatestPoll, browser)
srv := &http.Server{
Addr: ":" + cfg.Port,
Handler: newRouter(store, cfg),
Handler: newRouter(s, cfg),
ReadHeaderTimeout: 10 * time.Second,
}
go func() {
log.Printf("listening on :%s (db=%s, origins=%v)", cfg.Port, cfg.DBPath, cfg.AllowedOrigins)
// The connection URL carries a password, so it stays out of the log.
log.Printf("listening on :%s (origins=%v)", cfg.Port, cfg.AllowedOrigins)
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Fatalf("serve: %v", err)
}
@@ -244,28 +377,45 @@ func main() {
}
}
// newLatestPoller wires the configured cooldowns and fetchers into the poller.
func newLatestPoller(s *store.Store, cfg LatestPoll, fetch, browser latest.Fetcher) *latest.Poller {
var covers latest.BrowserCoverFetcher
if f, ok := browser.(latest.BrowserCoverFetcher); ok {
covers = f
}
return &latest.Poller{
Store: s,
Fetch: fetch,
BrowserFetch: browser,
CoverFetch: covers,
CoverBytesFetch: latest.NewCoverFetcher(),
Now: time.Now,
Cooldown: cfg.Cooldown,
BrowserCooldown: cfg.BrowserCooldown,
Interval: cfg.Interval,
Stagger: cfg.Stagger,
Batch: cfg.Batch,
}
}
// startLatestPoller launches the background poller unless it is disabled or its
// HTTP client cannot be built. Any problem here is logged and skipped: this
// feature going missing degrades the service to userscript-only latest-chapter
// tracking, which is exactly how it behaved before.
func startLatestPoller(ctx context.Context, store *Store, cfg LatestPoll) {
func startLatestPoller(ctx context.Context, s *store.Store, cfg LatestPoll, browser latest.Fetcher) {
if !cfg.Enabled {
log.Println("latest-chapter poller: disabled by config")
return
}
f, err := newTLSFetcher()
f, err := latest.NewTLSFetcher()
if err != nil {
log.Printf("latest-chapter poller: disabled, cannot build client: %v", err)
return
}
p := &latestPoller{
store: store,
fetch: f,
now: time.Now,
cooldown: cfg.Cooldown,
interval: cfg.Interval,
stagger: cfg.Stagger,
batch: cfg.Batch,
}
// Nil browser: sites behind a JavaScript challenge are simply not polled,
// and their latest_chapter comes from the userscript alone — which is how
// the service behaved before the sidecar existed.
p := newLatestPoller(s, cfg, f, browser)
go p.Run(ctx)
}
+166 -4
View File
@@ -1,20 +1,24 @@
package main
import (
"compress/gzip"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"bookmarkmanager/backend/internal/store"
)
func TestLoadLatestPollDefaults(t *testing.T) {
for _, k := range []string{
"LATEST_CHAPTER_POLL_ENABLED", "LATEST_CHAPTER_POLL_COOLDOWN",
"LATEST_CHAPTER_POLL_INTERVAL", "LATEST_CHAPTER_POLL_STAGGER",
"LATEST_CHAPTER_POLL_BATCH",
"LATEST_CHAPTER_POLL_BROWSER_COOLDOWN", "LATEST_CHAPTER_POLL_INTERVAL",
"LATEST_CHAPTER_POLL_STAGGER", "LATEST_CHAPTER_POLL_BATCH",
} {
t.Setenv(k, "")
}
@@ -23,6 +27,7 @@ func TestLoadLatestPollDefaults(t *testing.T) {
want := LatestPoll{
Enabled: true,
Cooldown: time.Hour,
BrowserCooldown: 6 * time.Hour,
Interval: 10 * time.Minute,
Stagger: 20 * time.Second,
Batch: 14,
@@ -32,6 +37,13 @@ func TestLoadLatestPollDefaults(t *testing.T) {
}
}
func TestLoadConfigReadsCoverDirectory(t *testing.T) {
t.Setenv("COVER_DIR", "/covers")
if got := loadConfig().CoverDir; got != "/covers" {
t.Fatalf("CoverDir = %q, want /covers", got)
}
}
func TestLoadLatestPollEnabledParsing(t *testing.T) {
tests := []struct {
raw string
@@ -70,6 +82,30 @@ func TestLoadLatestPollClampsAndFallsBack(t *testing.T) {
wantFrom: func(p LatestPoll) any { return p.Cooldown },
want: 15 * time.Minute,
},
{
name: "browser cooldown below the floor is clamped up",
env: map[string]string{"LATEST_CHAPTER_POLL_BROWSER_COOLDOWN": "1m"},
wantFrom: func(p LatestPoll) any { return p.BrowserCooldown },
want: 15 * time.Minute,
},
{
name: "browser cooldown at the floor is kept",
env: map[string]string{"LATEST_CHAPTER_POLL_BROWSER_COOLDOWN": "15m"},
wantFrom: func(p LatestPoll) any { return p.BrowserCooldown },
want: 15 * time.Minute,
},
{
name: "browser cooldown override is honoured",
env: map[string]string{"LATEST_CHAPTER_POLL_BROWSER_COOLDOWN": "8h"},
wantFrom: func(p LatestPoll) any { return p.BrowserCooldown },
want: 8 * time.Hour,
},
{
name: "browser cooldown unparseable value falls back",
env: map[string]string{"LATEST_CHAPTER_POLL_BROWSER_COOLDOWN": "six hours"},
wantFrom: func(p LatestPoll) any { return p.BrowserCooldown },
want: 6 * time.Hour,
},
{
name: "a valid override is honoured",
env: map[string]string{"LATEST_CHAPTER_POLL_INTERVAL": "5m"},
@@ -119,6 +155,16 @@ func TestLoadLatestPollClampsAndFallsBack(t *testing.T) {
}
}
func TestNewLatestPollerWiresCooldowns(t *testing.T) {
p := newLatestPoller(nil, LatestPoll{
Cooldown: time.Hour,
BrowserCooldown: 6 * time.Hour,
}, nil, nil)
if p.Cooldown != time.Hour || p.BrowserCooldown != 6*time.Hour {
t.Fatalf("poller cooldowns = %s/%s, want 1h/6h", p.Cooldown, p.BrowserCooldown)
}
}
func TestPutStatusValidation(t *testing.T) {
cases := []struct {
name string
@@ -146,7 +192,7 @@ func TestPutStatusValidation(t *testing.T) {
if tc.want != http.StatusOK {
return
}
var got Bookmark
var got store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
@@ -185,7 +231,7 @@ func TestPutOmittedStatusPreservesArchivedAndAppliesProgress(t *testing.T) {
t.Fatalf("status = %d, want 200 (body %s)", rr.Code, rr.Body.String())
}
var got Bookmark
var got store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
@@ -196,3 +242,119 @@ func TestPutOmittedStatusPreservesArchivedAndAppliesProgress(t *testing.T) {
t.Fatalf("stored last_chapter_num = %v, want 12", got.LastChapterNum)
}
}
func TestGzipCompressesTextNotFonts(t *testing.T) {
srv, _ := newWebTestServer(t, testConfig())
cases := []struct {
path string
want bool
}{
{"/static/style.css", true},
{"/static/filter.js", true},
{"/static/htmx.min.js", true},
{"/static/fonts/dm-sans-var-latin.woff2", false},
}
for _, tc := range cases {
req := httptest.NewRequest(http.MethodGet, tc.path, nil)
req.Header.Set("Accept-Encoding", "gzip")
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("GET %s = %d, want 200", tc.path, rr.Code)
}
got := rr.Header().Get("Content-Encoding") == "gzip"
if got != tc.want {
t.Errorf("GET %s Content-Encoding gzip = %v, want %v", tc.path, got, tc.want)
}
if got {
zr, err := gzip.NewReader(rr.Body)
if err != nil {
t.Fatalf("GET %s: body is not gzip: %v", tc.path, err)
}
if _, err := io.ReadAll(zr); err != nil {
t.Fatalf("GET %s: gzip body did not decode: %v", tc.path, err)
}
}
}
// A client that does not ask still gets plain bytes.
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/static/style.css", nil))
if enc := rr.Header().Get("Content-Encoding"); enc != "" {
t.Errorf("Content-Encoding without Accept-Encoding = %q, want empty", enc)
}
}
func TestPutKindValidation(t *testing.T) {
cases := []struct {
name string
kind string
want int
}{
{"empty is no opinion", "", http.StatusOK},
{"manga", "manga", http.StatusOK},
{"novel", "novel", http.StatusOK},
{"garbage", "comic", http.StatusBadRequest},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
srv := newTestServer(t)
body := fmt.Sprintf(`{"title":"Solo","kind":%q}`, tc.kind)
req := auth(httptest.NewRequest(http.MethodPut, "/bookmarks/asura:solo",
strings.NewReader(body)))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if rr.Code != tc.want {
t.Fatalf("status = %d, want %d (body %s)", rr.Code, tc.want, rr.Body.String())
}
if tc.want != http.StatusOK {
return
}
var got store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
want := tc.kind
if want == "" {
want = "manga"
}
if got.Kind != want {
t.Fatalf("stored kind = %q, want %q", got.Kind, want)
}
})
}
}
// The preserve path: a novel row re-PUT by a client that omits the field
// entirely must stay a novel and still record the progress it carried.
func TestPutOmittedKindPreservesNovelAndAppliesProgress(t *testing.T) {
srv := newTestServer(t)
const key = "/bookmarks/lightnovelworld:a-will-eternal"
seed := auth(httptest.NewRequest(http.MethodPut, key,
strings.NewReader(`{"title":"A Will Eternal","kind":"novel","last_chapter_num":10}`)))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, seed)
if rr.Code != http.StatusOK {
t.Fatalf("seed status = %d, want 200 (%s)", rr.Code, rr.Body.String())
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, auth(httptest.NewRequest(http.MethodPut, key,
strings.NewReader(`{"title":"A Will Eternal","last_chapter_num":11}`))))
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200 (%s)", rr.Code, rr.Body.String())
}
var got store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
if got.Kind != store.KindNovel {
t.Fatalf("Kind = %q, want novel", got.Kind)
}
if got.LastChapterNum != 11 {
t.Fatalf("LastChapterNum = %v, want 11", got.LastChapterNum)
}
}
-53
View File
@@ -1,53 +0,0 @@
package main
import (
"crypto/subtle"
"net/http"
"strings"
)
const bearerPrefix = "Bearer "
// withAuth guards a handler with a constant-time bearer-token check.
func withAuth(token string, next http.Handler) http.Handler {
want := []byte(token)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
h := r.Header.Get("Authorization")
if !strings.HasPrefix(h, bearerPrefix) {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
got := []byte(strings.TrimPrefix(h, bearerPrefix))
if subtle.ConstantTimeCompare(got, want) != 1 {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
next.ServeHTTP(w, r)
})
}
// withCORS reflects the request Origin only when it is in allowed, answers
// preflight OPTIONS with 204, and passes everything else through. It wraps the
// auth middleware so preflight (which carries no Authorization header) is never
// rejected by auth.
func withCORS(allowed []string, next http.Handler) http.Handler {
set := make(map[string]struct{}, len(allowed))
for _, o := range allowed {
set[o] = struct{}{}
}
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
origin := r.Header.Get("Origin")
if _, ok := set[origin]; ok && origin != "" {
w.Header().Set("Access-Control-Allow-Origin", origin)
w.Header().Add("Vary", "Origin")
w.Header().Set("Access-Control-Allow-Methods", "GET,PUT,DELETE,OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Authorization,Content-Type")
w.Header().Set("Access-Control-Max-Age", "86400")
}
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
next.ServeHTTP(w, r)
})
}
+356
View File
@@ -0,0 +1,356 @@
package main
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"bookmarkmanager/backend/internal/store"
"bookmarkmanager/backend/internal/token"
)
// registerReader creates an extra Reader the way a first login does and
// returns its id. The credential is derived the same way the owner's is, so it
// authenticates through the real router.
func registerReader(t *testing.T, s *store.Store, discordID string) int64 {
t.Helper()
id, err := s.EnsureReader(discordID, token.Hash(readerCredential(discordID)))
if err != nil {
t.Fatalf("register reader %q: %v", discordID, err)
}
return id
}
// credRequest builds a request authenticated as the Reader whose credential
// is passed.
func credRequest(method, target, cred string) *http.Request {
req := httptest.NewRequest(method, target, nil)
req.Header.Set("Authorization", "Bearer "+cred)
return req
}
// readerCredential is the epoch-0 derived credential of an arbitrary Reader.
func readerCredential(discordID string) string {
return token.Token([]byte(testTokenKey), discordID, 0)
}
// withBody attaches a request body, for PUTs that carry a JSON payload.
func withBody(req *http.Request, body string) *http.Request {
req.Body = io.NopCloser(strings.NewReader(body))
req.ContentLength = int64(len(body))
return req
}
// A refused credential is refused however plausible it looks: only a hash the
// readers table holds authenticates anything.
func TestUnknownCredentialRejected(t *testing.T) {
srv := newRouter(newTestStore(t), testConfig())
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", readerCredential("never-registered")))
if rr.Code != http.StatusUnauthorized {
t.Fatalf("unregistered Reader's credential: status = %d, want 401", rr.Code)
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", ownerCredential()))
if rr.Code != http.StatusOK {
t.Fatalf("owner's derived credential: status = %d, want 200", rr.Code)
}
}
// A Reader's credential authenticates exactly that Reader: rows written under
// one credential are invisible to the other, on the same key.
func TestPerReaderIsolation(t *testing.T) {
s := newTestStore(t)
registerReader(t, s, "other-reader")
srv := newRouter(s, testConfig())
ownerKey := "asura:solo"
putBookmark(t, srv, ownerKey, store.Bookmark{
Key: ownerKey, Site: "asura", SeriesID: "solo",
Title: "Solo Leveling", UpdatedAt: 1,
})
// The other Reader's list is empty even though the owner holds the key.
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", readerCredential("other-reader")))
if rr.Code != http.StatusOK {
t.Fatalf("other reader list: status = %d, want 200", rr.Code)
}
var theirs []store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &theirs); err != nil {
t.Fatalf("decode: %v", err)
}
if len(theirs) != 0 {
t.Fatalf("other reader sees %d bookmarks, want 0 (owner's rows leaked)", len(theirs))
}
// The other Reader writes the same key; both rows coexist, each visible
// only to its owner. The series title is shared (ADR-0003) — the
// reader-owned fields are progress and updated_at.
req := credRequest(http.MethodPut, "/bookmarks/"+ownerKey, readerCredential("other-reader"))
req.Header.Set("Content-Type", "application/json")
body := `{"key":"asura:solo","site":"asura","series_id":"solo","title":"Theirs","last_chapter_num":3}`
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, withBody(req, body))
if rr.Code != http.StatusOK {
t.Fatalf("other reader put: status = %d, want 200", rr.Code)
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", readerCredential("other-reader")))
var theirs2 []store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &theirs2); err != nil {
t.Fatalf("decode: %v", err)
}
if len(theirs2) != 1 || theirs2[0].LastChapterNum != 3 {
t.Fatalf("other reader list = %+v, want their own row with their progress", theirs2)
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", ownerCredential()))
var owners []store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &owners); err != nil {
t.Fatalf("decode: %v", err)
}
if len(owners) != 1 || owners[0].Title != "Solo Leveling" || owners[0].LastChapterNum != 0 {
t.Fatalf("owner list = %+v, want their own row at their own progress", owners)
}
// The mirror: the owner's write does not move the other Reader's progress
// either. Without it, isolation is only asserted in one direction.
req = credRequest(http.MethodPut, "/bookmarks/"+ownerKey, ownerCredential())
req.Header.Set("Content-Type", "application/json")
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, withBody(req, `{"key":"asura:solo","site":"asura","series_id":"solo","title":"Solo Leveling","last_chapter_num":9}`))
if rr.Code != http.StatusOK {
t.Fatalf("owner put: status = %d, want 200", rr.Code)
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", readerCredential("other-reader")))
var theirs3 []store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &theirs3); err != nil {
t.Fatalf("decode: %v", err)
}
if len(theirs3) != 1 || theirs3[0].LastChapterNum != 3 {
t.Fatalf("other reader list = %+v, want progress 3 after the owner's write", theirs3)
}
// DELETE is scoped to its caller too, asserted in both directions: each
// Reader's delete on the shared key takes only their own row.
list := func(cred string) []store.Bookmark {
t.Helper()
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", cred))
var got []store.Bookmark
if err := json.Unmarshal(rr.Body.Bytes(), &got); err != nil {
t.Fatalf("decode: %v", err)
}
return got
}
del := func(cred string) {
t.Helper()
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodDelete, "/bookmarks/"+ownerKey, cred))
if rr.Code != http.StatusNoContent {
t.Fatalf("delete: status = %d, want 204", rr.Code)
}
}
del(readerCredential("other-reader"))
if got := list(ownerCredential()); len(got) != 1 {
t.Fatalf("owner's row was deletable by the other Reader: %+v", got)
}
if got := list(readerCredential("other-reader")); len(got) != 0 {
t.Fatalf("other Reader's own delete left %+v behind", got)
}
// The mirror: the other Reader takes the key again, the owner deletes
// theirs, and the other's row is untouched.
req = credRequest(http.MethodPut, "/bookmarks/"+ownerKey, readerCredential("other-reader"))
req.Header.Set("Content-Type", "application/json")
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, withBody(req, body))
if rr.Code != http.StatusOK {
t.Fatalf("other reader re-put: status = %d, want 200", rr.Code)
}
del(ownerCredential())
if got := list(readerCredential("other-reader")); len(got) != 1 {
t.Fatalf("other Reader's row was deletable by the owner: %+v", got)
}
if got := list(ownerCredential()); len(got) != 0 {
t.Fatalf("owner's own delete left %+v behind", got)
}
}
// The install endpoints are session-gated and render the script directly
// with the Reader's credential inside: the credential never appears in the
// address bar, the page markup, or any Location header.
func TestInstallServesScriptWithCredential(t *testing.T) {
cfg := testConfig()
dir := t.TempDir()
path := filepath.Join(dir, "manga-bookmark.user.js")
novelPath := filepath.Join(dir, "novel-bookmark.user.js")
for _, p := range []string{path, novelPath} {
if err := os.WriteFile(p, []byte("const API_TOKEN = \"__API_TOKEN__\";\n"), 0o644); err != nil {
t.Fatalf("write script: %v", err)
}
}
cfg.UserscriptPath = path
cfg.NovelUserscriptPath = novelPath
srv, st := newWebTestServer(t, cfg)
for _, script := range []string{"manga-bookmark.user.js", "novel-bookmark.user.js"} {
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/install/"+script, nil))
if rr.Code != http.StatusUnauthorized {
t.Fatalf("%s without session: status = %d, want 401", script, rr.Code)
}
req := httptest.NewRequest(http.MethodGet, "/install/"+script, nil)
req.AddCookie(sessionCookie(t, st))
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("%s with session: status = %d, want 200", script, rr.Code)
}
body := rr.Body.String()
if strings.Contains(body, "__API_TOKEN__") {
t.Fatalf("%s served with an unsubstituted placeholder", script)
}
// The credential rides inside the served script — nowhere visible in
// the UI — and is the session holder's own.
if !strings.Contains(body, `API_TOKEN = "`+ownerCredential()+`"`) {
t.Fatalf("%s does not carry the owner's credential:\n%s", script, body)
}
if loc := rr.Header().Get("Location"); loc != "" {
t.Fatalf("%s answered with a redirect, credential in Location %q", script, loc)
}
// The plain link must stay inline: Violentmonkey's updater polls the
// /u/ path and an attachment disposition there would break updates.
if cd := rr.Header().Get("Content-Disposition"); cd != "" {
t.Fatalf("%s served as %q, want inline", script, cd)
}
// ?download=1 is the mobile path: Violentmonkey on Chromium ignores a
// .user.js navigation, so the Reader saves the file and adds it by hand.
req = httptest.NewRequest(http.MethodGet, "/install/"+script+"?download=1", nil)
req.AddCookie(sessionCookie(t, st))
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("%s?download=1: status = %d, want 200", script, rr.Code)
}
if got, want := rr.Header().Get("Content-Disposition"), `attachment; filename="`+script+`"`; got != want {
t.Fatalf("%s?download=1: Content-Disposition = %q, want %q", script, got, want)
}
if !strings.Contains(rr.Body.String(), `API_TOKEN = "`+ownerCredential()+`"`) {
t.Fatalf("%s?download=1 does not carry the owner's credential", script)
}
}
}
// Rotation through the web UI invalidates the old credential immediately,
// mints one that authenticates the API and the script path, and warns that
// every device must reinstall.
func TestRotateCredentialViaWebUI(t *testing.T) {
s, _ := newTestStoreURL(t)
path := filepath.Join(t.TempDir(), "manga-bookmark.user.js")
if err := os.WriteFile(path, []byte("const API_TOKEN = \"__API_TOKEN__\";\n"), 0o644); err != nil {
t.Fatalf("write script: %v", err)
}
cfg := testConfig()
cfg.UserscriptPath = path
srv := newRouter(s, cfg)
oldCred := ownerCredential()
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", oldCred))
if rr.Code != http.StatusOK {
t.Fatalf("old credential before rotation: status = %d, want 200", rr.Code)
}
req := httptest.NewRequest(http.MethodPost, "/rotate-token", nil)
req.AddCookie(sessionCookie(t, s))
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("rotate: status = %d, want 200", rr.Code)
}
if !strings.Contains(rr.Body.String(), "Credential rotated") {
t.Fatalf("rotation response does not warn about reinstall:\n%s", rr.Body.String())
}
// The old credential is dead on the API and on the script path.
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", oldCred))
if rr.Code != http.StatusUnauthorized {
t.Fatalf("old credential after rotation: status = %d, want 401", rr.Code)
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/u/"+oldCred+"/manga-bookmark.user.js", nil))
if rr.Code != http.StatusNotFound {
t.Fatalf("old credential script path after rotation: status = %d, want 404", rr.Code)
}
// The new credential authenticates the API and the script path, and is
// substituted into the served script.
newCred := token.Token([]byte(testTokenKey), testDiscordID, 1)
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, credRequest(http.MethodGet, "/bookmarks", newCred))
if rr.Code != http.StatusOK {
t.Fatalf("new credential after rotation: status = %d, want 200", rr.Code)
}
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/u/"+newCred+"/manga-bookmark.user.js", nil))
if rr.Code != http.StatusOK {
t.Fatalf("new credential script path: status = %d, want 200", rr.Code)
}
if got := rr.Body.String(); !strings.Contains(got, `API_TOKEN = "`+newCred+`"`) {
t.Fatalf("served script does not carry the rotated credential:\n%s", got)
}
// The install link now renders the script with the new credential.
req = httptest.NewRequest(http.MethodGet, "/install/manga-bookmark.user.js", nil)
req.AddCookie(sessionCookie(t, s))
rr = httptest.NewRecorder()
srv.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("install after rotation: status = %d, want 200", rr.Code)
}
if got := rr.Body.String(); !strings.Contains(got, `API_TOKEN = "`+newCred+`"`) {
t.Fatalf("install after rotation does not carry the new credential:\n%s", got)
}
}
// The app page offers the install links; the credential never appears in its
// markup.
func TestIndexShowsSetupPanelWithoutCredential(t *testing.T) {
srv, st := newWebTestServer(t, testConfig())
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(sessionCookie(t, st))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
body := rr.Body.String()
for _, want := range []string{
`href="/install/manga-bookmark.user.js"`,
`href="/install/novel-bookmark.user.js"`,
`href="/install/manga-bookmark.user.js?download=1"`,
`href="/install/novel-bookmark.user.js?download=1"`,
"Rotate credential",
} {
if !strings.Contains(body, want) {
t.Errorf("app page lacks %q", want)
}
}
if strings.Contains(body, ownerCredential()) {
t.Fatal("app page leaks the credential")
}
}
-88
View File
@@ -1,88 +0,0 @@
// Title search runs entirely in the browser: the full list is already in the
// DOM, so filtering it needs no request.
(function () {
function applyFilter() {
var box = document.getElementById("search");
if (!box) return;
var needle = box.value.trim().toLowerCase();
document.querySelectorAll(".card").forEach(function (card) {
var title = (card.dataset.title || "").toLowerCase();
card.hidden = needle !== "" && title.indexOf(needle) === -1;
});
}
document.addEventListener("input", function (e) {
if (e.target && e.target.id === "search") applyFilter();
});
// htmx replaces the list on a tab switch, so re-apply to the new cards.
document.body.addEventListener("htmx:afterSwap", applyFilter);
})();
function setActiveTab(el) {
el.parentElement.querySelectorAll("[role=tab]").forEach(function (t) {
t.classList.toggle("active", t === el);
});
}
// The chapter-edit form and the delete confirm row are the two per-card
// disclosure panels; only one makes sense open at a time. The button that
// owns an open panel carries .open, which is how the strip shows which cell
// the panel belongs to.
function closeCardPanels(key) {
var form = document.getElementById("chapter-form-" + key);
var confirm = document.getElementById("confirm-row-" + key);
if (form) form.hidden = true;
if (confirm) confirm.hidden = true;
var card = document.getElementById("card-" + key);
if (card) {
card.querySelectorAll(".actions .open").forEach(function (b) {
b.classList.remove("open");
});
}
}
function togglePanel(key, panelId, buttonSelector) {
var panel = document.getElementById(panelId + key);
if (!panel) return null;
var opening = panel.hidden;
closeCardPanels(key);
panel.hidden = !opening;
var card = document.getElementById("card-" + key);
var button = card && card.querySelector(buttonSelector);
if (button) button.classList.toggle("open", opening);
return panel;
}
function toggleChapterForm(key) {
var form = togglePanel(key, "chapter-form-", ".actions .pencil");
if (form && !form.hidden) form.querySelector("input").focus();
}
function toggleConfirmRow(key) {
togglePanel(key, "confirm-row-", ".actions .remove");
}
// A failed favourite/chapter/delete request leaves the card in place (htmx
// does not swap on a non-2xx response) but otherwise gives no sign anything
// went wrong. Surface it inline instead of leaving the tap looking ignored.
(function () {
function showError(elt, message) {
var card = elt.closest(".card");
var slot = card && card.querySelector(".error-inline");
if (!slot) return;
slot.textContent = message;
slot.hidden = false;
clearTimeout(slot._hideTimer);
slot._hideTimer = setTimeout(function () {
slot.hidden = true;
}, 5000);
}
document.body.addEventListener("htmx:responseError", function (e) {
showError(e.detail.elt, "Couldn't save — try again.");
});
document.body.addEventListener("htmx:sendError", function (e) {
showError(e.detail.elt, "No connection — try again.");
});
})();
-447
View File
@@ -1,447 +0,0 @@
package main
import (
"database/sql"
"errors"
"fmt"
"regexp"
"strings"
_ "modernc.org/sqlite"
)
// Bookmark is one tracked series, keyed "<site>:<series_id>" across both sites.
//
// LastChapter* is the user's read progress; LatestChapter* is the newest
// chapter the site has published, captured opportunistically by the userscript.
type Bookmark struct {
Key string `json:"key"`
Site string `json:"site"`
SeriesID string `json:"series_id"`
Title string `json:"title"`
SeriesURL string `json:"series_url"`
Cover string `json:"cover"`
LastChapter string `json:"last_chapter"`
LastChapterNum float64 `json:"last_chapter_num"`
LastChapterURL string `json:"last_chapter_url"`
Favorite bool `json:"favorite"`
LatestChapter string `json:"latest_chapter"`
LatestChapterNum *float64 `json:"latest_chapter_num"` // nil until first captured
UpdatedAt int64 `json:"updated_at"` // unix ms; see Upsert
// Status is the lifecycle bucket: reading, archived, or finished.
// Archived series stay polled for new chapters; finished ones do not.
// Empty on the way in means "no opinion" — see Upsert.
Status string `json:"status"`
}
// HasNewChapter reports whether the site has published past the read point.
// A nil LatestChapterNum means nothing has been captured yet, which is not the
// same as "nothing new".
func (b Bookmark) HasNewChapter() bool {
return b.LatestChapterNum != nil && *b.LatestChapterNum > b.LastChapterNum
}
// ContinueURL is where the Continue button points: the chapter last read, or
// the series page when no chapter URL was ever captured.
func (b Bookmark) ContinueURL() string {
if b.LastChapterURL != "" {
return b.LastChapterURL
}
return b.SeriesURL
}
// Lifecycle buckets. A bookmark is in exactly one; favorite is orthogonal.
const (
statusReading = "reading"
statusArchived = "archived"
statusFinished = "finished"
)
const schema = `
CREATE TABLE IF NOT EXISTS bookmarks (
key TEXT PRIMARY KEY,
site TEXT NOT NULL,
series_id TEXT NOT NULL,
title TEXT,
series_url TEXT,
cover TEXT,
last_chapter TEXT,
last_chapter_num REAL,
last_chapter_url TEXT,
favorite INTEGER NOT NULL DEFAULT 0,
latest_chapter TEXT NOT NULL DEFAULT '',
latest_chapter_num REAL,
latest_checked_at INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'reading',
updated_at INTEGER NOT NULL
);`
// The columns above that databases created before them will be missing.
// SQLite has no ADD COLUMN IF NOT EXISTS, so each is added only when absent.
var addedColumns = []struct{ name, ddl string }{
{"favorite", `ALTER TABLE bookmarks ADD COLUMN favorite INTEGER NOT NULL DEFAULT 0`},
{"latest_chapter", `ALTER TABLE bookmarks ADD COLUMN latest_chapter TEXT NOT NULL DEFAULT ''`},
{"latest_chapter_num", `ALTER TABLE bookmarks ADD COLUMN latest_chapter_num REAL`},
// When the server last looked at this series, unix ms; 0 means never, and
// sorts first so a new bookmark is picked up on the next tick with no
// special case. Deliberately NOT in bookmarkColumns — see MarkLatestChecked.
{"latest_checked_at", `ALTER TABLE bookmarks ADD COLUMN latest_checked_at INTEGER NOT NULL DEFAULT 0`},
// Lifecycle bucket. The DEFAULT backfills every pre-existing row as
// 'reading', so there is no separate migration step.
{"status", `ALTER TABLE bookmarks ADD COLUMN status TEXT NOT NULL DEFAULT 'reading'`},
}
const bookmarkColumns = `key, site, series_id, title, series_url, cover,
last_chapter, last_chapter_num, last_chapter_url,
favorite, latest_chapter, latest_chapter_num, updated_at, status`
// Store is the SQLite-backed bookmark store.
type Store struct {
db *sql.DB
}
// OpenStore opens (or creates) the SQLite database at path and applies the schema.
func OpenStore(path string) (*Store, error) {
// busy_timeout guards against SQLITE_BUSY under the reverse proxy's
// concurrent requests; a single writer connection keeps writes serialized.
dsn := path
if !strings.Contains(dsn, "?") {
dsn += "?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)"
}
db, err := sql.Open("sqlite", dsn)
if err != nil {
return nil, fmt.Errorf("open sqlite %q: %w", path, err)
}
db.SetMaxOpenConns(1)
if _, err := db.Exec(schema); err != nil {
db.Close()
return nil, fmt.Errorf("apply schema: %w", err)
}
if err := migrateColumns(db); err != nil {
db.Close()
return nil, fmt.Errorf("migrate schema: %w", err)
}
if err := migrateAsuraKeys(db); err != nil {
db.Close()
return nil, fmt.Errorf("migrate asura keys: %w", err)
}
return &Store{db: db}, nil
}
// migrateColumns brings a pre-existing bookmarks table up to the current
// schema. Safe to run on every start: columns already present are skipped.
func migrateColumns(db *sql.DB) error {
have, err := existingColumns(db, "bookmarks")
if err != nil {
return err
}
for _, c := range addedColumns {
if _, ok := have[c.name]; ok {
continue
}
if _, err := db.Exec(c.ddl); err != nil {
return fmt.Errorf("add column %q: %w", c.name, err)
}
}
return nil
}
// asuraBuildHash matches the trailing "-xxxxxxxx" site-wide build ID Asura
// appends to every series slug. It rotates on each site redeploy, so it
// must not be part of series_id. Must stay in sync with stripBuildHash in
// userscript/manga-bookmark.user.js.
var asuraBuildHash = regexp.MustCompile(`-[0-9a-f]{8}$`)
// migrateAsuraKeys rewrites asura bookmarks whose series_id still carries
// the build hash to the stable, hashless ID. Rows keyed with a hash are
// orphaned on every Asura redeploy (old-hash URLs 302 to new-hash ones, so
// detection yields a key that never matches). When two hash-generations of
// one series collide, the row with the newest updated_at wins and the rest
// are deleted. Idempotent: hashless IDs never match the regex.
func migrateAsuraKeys(db *sql.DB) error {
rows, err := db.Query(`SELECT key, series_id, updated_at FROM bookmarks WHERE site = 'asura'`)
if err != nil {
return fmt.Errorf("list asura rows: %w", err)
}
type row struct {
key, id string
updated int64
}
var all []row
for rows.Next() {
var r row
if err := rows.Scan(&r.key, &r.id, &r.updated); err != nil {
rows.Close()
return fmt.Errorf("scan asura row: %w", err)
}
all = append(all, r)
}
if err := rows.Close(); err != nil {
return err
}
groups := map[string][]row{}
for _, r := range all {
stripped := asuraBuildHash.ReplaceAllString(r.id, "")
groups[stripped] = append(groups[stripped], r)
}
for stripped, g := range groups {
winner := 0
for i := range g {
if g[i].updated > g[winner].updated {
winner = i
}
}
// Losers go first: rewriting the winner to the stripped key while a
// pre-existing hashless row still holds it is a primary-key collision.
for i, r := range g {
if i == winner {
continue
}
if _, err := db.Exec(`DELETE FROM bookmarks WHERE key = ?`, r.key); err != nil {
return fmt.Errorf("drop duplicate %q: %w", r.key, err)
}
}
if r := g[winner]; r.id != stripped {
if _, err := db.Exec(
`UPDATE bookmarks SET key = ?, series_id = ? WHERE key = ?`,
"asura:"+stripped, stripped, r.key); err != nil {
return fmt.Errorf("rewrite key %q: %w", r.key, err)
}
}
}
return nil
}
func existingColumns(db *sql.DB, table string) (map[string]struct{}, error) {
rows, err := db.Query(`SELECT name FROM pragma_table_info(?)`, table)
if err != nil {
return nil, fmt.Errorf("read %s columns: %w", table, err)
}
defer rows.Close()
out := map[string]struct{}{}
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
return nil, fmt.Errorf("scan column name: %w", err)
}
out[name] = struct{}{}
}
return out, rows.Err()
}
// scanBookmark reads one row in bookmarkColumns order, translating SQLite's
// integer bool and nullable latest_chapter_num into Go types.
//
// The optional columns are read through Null* types because rows predating
// this code (or written by hand) may hold NULL where the app only ever writes
// zero values. Only latest_chapter_num distinguishes the two: everywhere else
// NULL and the zero value mean the same thing to clients.
func scanBookmark(scan func(...any) error) (Bookmark, error) {
var (
b Bookmark
title, seriesURL, cover sql.NullString
lastChapter, lastChapterURL, latestChapter sql.NullString
status sql.NullString
lastChapterNum, latestChapterNum sql.NullFloat64
favorite sql.NullInt64
)
if err := scan(
&b.Key, &b.Site, &b.SeriesID, &title, &seriesURL, &cover,
&lastChapter, &lastChapterNum, &lastChapterURL,
&favorite, &latestChapter, &latestChapterNum, &b.UpdatedAt, &status,
); err != nil {
return Bookmark{}, err
}
b.Title = title.String
b.SeriesURL = seriesURL.String
b.Cover = cover.String
b.LastChapter = lastChapter.String
b.LastChapterNum = lastChapterNum.Float64
b.LastChapterURL = lastChapterURL.String
b.Favorite = favorite.Int64 != 0
b.LatestChapter = latestChapter.String
if latestChapterNum.Valid {
b.LatestChapterNum = &latestChapterNum.Float64
}
// A NULL, empty, or unrecognised bucket (e.g. a hand-edited row) would
// leave the row in no list at all, so anything outside the three known
// buckets reads as the default rather than being passed through.
b.Status = status.String
if b.Status != statusReading && b.Status != statusArchived && b.Status != statusFinished {
b.Status = statusReading
}
return b, nil
}
// Close releases the underlying database handle.
func (s *Store) Close() error { return s.db.Close() }
// List returns every bookmark, newest activity first.
func (s *Store) List() ([]Bookmark, error) {
rows, err := s.db.Query(`SELECT ` + bookmarkColumns + `
FROM bookmarks
ORDER BY updated_at DESC`)
if err != nil {
return nil, fmt.Errorf("query bookmarks: %w", err)
}
defer rows.Close()
out := []Bookmark{}
for rows.Next() {
b, err := scanBookmark(rows.Scan)
if err != nil {
return nil, fmt.Errorf("scan bookmark: %w", err)
}
out = append(out, b)
}
return out, rows.Err()
}
// Get returns one bookmark by key. A missing key is not an error: ok is false
// and err is nil. UI mutations read-modify-write through this so they preserve
// the fields they do not touch.
func (s *Store) Get(key string) (Bookmark, bool, error) {
b, err := scanBookmark(s.db.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks WHERE key = ?`, key).Scan)
if errors.Is(err, sql.ErrNoRows) {
return Bookmark{}, false, nil
}
if err != nil {
return Bookmark{}, false, fmt.Errorf("get %q: %w", key, err)
}
return b, true, nil
}
// Upsert inserts or replaces a bookmark by key (last-write-wins) and returns
// the row as actually stored.
//
// b.UpdatedAt is only a candidate: it is applied when the row is new or when
// last_chapter_num changes, and otherwise the stored value is kept. Clients
// order their list by updated_at, so favoriting a series or recording a newly
// published chapter must not disturb that order — only real reading progress
// does. Callers must therefore use the returned bookmark, not the argument.
func (s *Store) Upsert(b Bookmark) (Bookmark, error) {
tx, err := s.db.Begin()
if err != nil {
return Bookmark{}, fmt.Errorf("begin %q: %w", b.Key, err)
}
defer tx.Rollback()
var latestNum any
if b.LatestChapterNum != nil {
latestNum = *b.LatestChapterNum
}
// IS NOT is SQLite's null-safe comparison. Within DO UPDATE, a bare column
// is the stored row and excluded.* is the incoming one; a brand-new key
// never reaches this clause, so it keeps the fresh timestamp from VALUES.
//
// The status column resolves on the VALUES side, not in the conflict
// clause: excluded.* is the row *after* these expressions are evaluated,
// so a default applied there would look identical to a real 'reading' and
// would overwrite an archived row on every PUT from a client that knows
// nothing about the column. Resolved once here, an empty incoming status
// means "keep what is stored", and only a brand-new row falls through to
// the literal default. The subquery runs inside this transaction, so it
// sees the row this statement is about to conflict with.
if _, err := tx.Exec(`
INSERT INTO bookmarks (`+bookmarkColumns+`)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
COALESCE(NULLIF(?, ''), (SELECT status FROM bookmarks WHERE key = ?), 'reading'))
ON CONFLICT(key) DO UPDATE SET
site=excluded.site, series_id=excluded.series_id, title=excluded.title,
series_url=excluded.series_url, cover=excluded.cover,
last_chapter=excluded.last_chapter, last_chapter_num=excluded.last_chapter_num,
last_chapter_url=excluded.last_chapter_url,
favorite=excluded.favorite,
latest_chapter=excluded.latest_chapter,
latest_chapter_num=excluded.latest_chapter_num,
status=excluded.status,
updated_at=CASE
WHEN bookmarks.last_chapter_num IS NOT excluded.last_chapter_num
THEN excluded.updated_at
ELSE bookmarks.updated_at
END`,
b.Key, b.Site, b.SeriesID, b.Title, b.SeriesURL, b.Cover,
b.LastChapter, b.LastChapterNum, b.LastChapterURL,
b.Favorite, b.LatestChapter, latestNum, b.UpdatedAt,
b.Status, b.Key); err != nil {
return Bookmark{}, fmt.Errorf("upsert %q: %w", b.Key, err)
}
stored, err := scanBookmark(tx.QueryRow(
`SELECT `+bookmarkColumns+` FROM bookmarks WHERE key = ?`, b.Key).Scan)
if err != nil {
return Bookmark{}, fmt.Errorf("read back %q: %w", b.Key, err)
}
if err := tx.Commit(); err != nil {
return Bookmark{}, fmt.Errorf("commit %q: %w", b.Key, err)
}
return stored, nil
}
// Delete removes a bookmark by key. Deleting a missing key is not an error.
func (s *Store) Delete(key string) error {
if _, err := s.db.Exec(`DELETE FROM bookmarks WHERE key = ?`, key); err != nil {
return fmt.Errorf("delete %q: %w", key, err)
}
return nil
}
// DueForLatestCheck returns bookmarks whose server-side latest-chapter check has
// aged past cutoffMs, least-recently-checked first, at most limit of them.
//
// Oldest-first is what keeps the poller fair when the backlog outgrows its
// throughput: the most neglected series is always next, so a large collection
// refreshes uniformly slower rather than leaving a tail that never refreshes at
// all. The userscript sorts its own queue the same way (L453).
//
// Bookmarks with no series_url are skipped — there is nothing to fetch, which
// is the same filter the userscript applies at L452.
//
// Finished series are excluded: nothing more is coming, so fetching them only
// burns requests. Archived ones are deliberately still polled — knowing what a
// shelved series is up to is the whole reason for archiving instead of deleting.
func (s *Store) DueForLatestCheck(cutoffMs int64, limit int) ([]Bookmark, error) {
rows, err := s.db.Query(`SELECT `+bookmarkColumns+`
FROM bookmarks
WHERE series_url IS NOT NULL AND series_url <> ''
AND status IS NOT 'finished'
AND latest_checked_at <= ?
ORDER BY latest_checked_at ASC
LIMIT ?`, cutoffMs, limit)
if err != nil {
return nil, fmt.Errorf("query due bookmarks: %w", err)
}
defer rows.Close()
out := []Bookmark{}
for rows.Next() {
b, err := scanBookmark(rows.Scan)
if err != nil {
return nil, fmt.Errorf("scan due bookmark: %w", err)
}
out = append(out, b)
}
return out, rows.Err()
}
// MarkLatestChecked records that the server looked at key at ts, whatever the
// look turned up. Marking a missing key is not an error: the row may have been
// deleted while a fetch was in flight.
//
// This is the one write that does not go through Upsert, and the column is kept
// out of bookmarkColumns on purpose. PUT /bookmarks/{key} decodes a whole
// Bookmark from the client and Upsert writes every column it knows about, so a
// userscript PUT — which has no idea this field exists — would write a zero and
// reset the cooldown, making the poller re-fetch that series every tick for as
// long as the user kept reading it.
func (s *Store) MarkLatestChecked(key string, ts int64) error {
if _, err := s.db.Exec(
`UPDATE bookmarks SET latest_checked_at = ? WHERE key = ?`, ts, key); err != nil {
return fmt.Errorf("mark checked %q: %w", key, err)
}
return nil
}
File diff suppressed because it is too large Load Diff
-81
View File
@@ -1,81 +0,0 @@
{{define "app"}}
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
<meta name="color-scheme" content="dark light">
<title>mangaBookmark</title>
<link rel="stylesheet" href="/static/style.css">
<link rel="preload" href="/static/fonts/instrument-serif-400-latin.woff2" as="font" type="font/woff2" crossorigin>
<link rel="preload" href="/static/fonts/ibm-plex-mono-500-latin.woff2" as="font" type="font/woff2" crossorigin>
<script src="/static/htmx.min.js" defer></script>
<script src="/static/filter.js" defer></script>
</head>
<body>
{{template "icons" .}}
<div class="sheet">
<header class="topbar">
<h1 class="brand">manga<em>Bookmark</em></h1>
<form method="post" action="/logout">
<button type="submit" class="ghost">Log out</button>
</form>
</header>
{{/* Search sits above the tabs on a phone and folds into the tab row on a
wider screen — one flex container, order swapped in CSS. */}}
<div class="chrome">
<div class="searchbar">
<svg viewBox="0 0 24 24" aria-hidden="true"><use href="#i-search"/></svg>
<input id="search" class="search" type="search" placeholder="Find a title"
autocomplete="off" aria-label="Search titles">
</div>
<nav class="tabs" role="tablist">
<a role="tab" href="/?tab=all" class="{{if eq .Tab "all"}}active{{end}}"
hx-get="/ui/list?tab=all" hx-target="#list" hx-swap="innerHTML"
hx-push-url="/?tab=all" hx-on::after-request="setActiveTab(this)">All</a>
<a role="tab" href="/?tab=new" class="tab-new {{if eq .Tab "new"}}active{{end}}"
hx-get="/ui/list?tab=new" hx-target="#list" hx-swap="innerHTML"
hx-push-url="/?tab=new" hx-on::after-request="setActiveTab(this)">Updated{{if .NewCount}}
<span class="count">{{.NewCount}}</span>{{end}}</a>
<a role="tab" href="/?tab=fav" class="{{if eq .Tab "fav"}}active{{end}}"
hx-get="/ui/list?tab=fav" hx-target="#list" hx-swap="innerHTML"
hx-push-url="/?tab=fav" hx-on::after-request="setActiveTab(this)">Favourites</a>
<a role="tab" href="/?tab=archived" class="{{if eq .Tab "archived"}}active{{end}}"
hx-get="/ui/list?tab=archived" hx-target="#list" hx-swap="innerHTML"
hx-push-url="/?tab=archived" hx-on::after-request="setActiveTab(this)">Archived</a>
<a role="tab" href="/?tab=finished" class="{{if eq .Tab "finished"}}active{{end}}"
hx-get="/ui/list?tab=finished" hx-target="#list" hx-swap="innerHTML"
hx-push-url="/?tab=finished" hx-on::after-request="setActiveTab(this)">Finished</a>
</nav>
</div>
{{if .Recent}}
<section class="recent">
<h2>Continue reading</h2>
<div class="recent-strip">
{{range .Recent}}
<a class="recent-card {{if .HasNewChapter}}is-new{{end}}" href="{{.ContinueURL}}"
target="_blank" rel="noopener noreferrer">
<span class="recent-cover">
{{if .Cover}}<img src="{{.Cover}}" alt="" loading="lazy">
{{else}}<span class="monogram" aria-hidden="true">{{.Initial}}</span>{{end}}
{{if .HasNewChapter}}<span class="foot-rule"></span>
{{else if .Favorite}}<span class="foot-rule brass"></span>{{end}}
</span>
<span class="recent-title">{{.Title}}</span>
<span class="recent-chapter">Ch {{.LastChapter}}{{if .HasNewChapter}} · New{{end}}</span>
</a>
{{end}}
</div>
</section>
{{end}}
<main id="list" class="list">
{{template "list" .}}
</main>
</div>
</body>
</html>
{{end}}
-15
View File
@@ -1,15 +0,0 @@
{{define "list"}}
{{if .Items}}
{{range .Items}}{{template "card" .}}{{end}}
{{else if eq .Tab "fav"}}
<div class="empty"><strong>No favourites yet.</strong><p>Star a series to pin it here.</p></div>
{{else if eq .Tab "new"}}
<div class="empty hot"><strong>Nothing new.</strong><p>Every series is caught up to its latest chapter.</p></div>
{{else if eq .Tab "archived"}}
<div class="empty"><strong>Nothing archived.</strong><p>Shelve a series to park it here — it keeps getting checked for new chapters.</p></div>
{{else if eq .Tab "finished"}}
<div class="empty"><strong>Nothing finished yet.</strong><p>Mark a series finished and it moves out of your reading list.</p></div>
{{else}}
<div class="empty"><strong>Nothing here yet.</strong><p>Bookmarks appear once the userscript records a chapter.</p></div>
{{end}}
{{end}}
-30
View File
@@ -1,30 +0,0 @@
{{define "login"}}
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
<meta name="color-scheme" content="dark light">
<title>mangaBookmark</title>
<link rel="stylesheet" href="/static/style.css">
<link rel="preload" href="/static/fonts/instrument-serif-400-latin.woff2" as="font" type="font/woff2" crossorigin>
</head>
<body>
<main class="login-card">
<div>
<span class="eyebrow">Private library</span>
<h1>manga<em>Bookmark</em></h1>
</div>
<form method="post" action="/login">
<div>
<label for="password">Password</label>
<input id="password" name="password" type="password"
autocomplete="current-password" autofocus required>
</div>
<p class="error">{{.Error}}</p>
<button type="submit">Sign in</button>
</form>
</main>
</body>
</html>
{{end}}
-60
View File
@@ -1,60 +0,0 @@
package main
import (
"crypto/subtle"
"log"
"net/http"
"os"
"regexp"
"time"
)
// versionLine matches the userscript metadata block's @version directive.
var versionLine = regexp.MustCompile(`(?m)^// @version[ \t]+.*$`)
// stampVersion replaces the served @version with one derived from the file's
// mtime, discarding whatever the file body says.
//
// Violentmonkey only updates when the served version sorts higher than the
// installed one. Deriving it from the body means one accidental downgrade or
// typo freezes updates forever; an mtime-derived version is monotonic by
// construction, so any later write always outranks any earlier one.
//
// A file with no @version line is returned untouched: such a script never
// auto-updates anyway, and inventing a metadata block is not this handler's job.
func stampVersion(src []byte, mod time.Time) []byte {
return versionLine.ReplaceAll(src, []byte("// @version "+mod.UTC().Format("2006.01.02.1504")))
}
// userscriptHandler serves the userscript to Violentmonkey's updater.
//
// The token lives in the path because the update poll sends no Authorization
// header, and the file embeds API_TOKEN in plain text, so an open path would
// hand that token to anyone who guessed the URL. A mismatch answers 404 rather
// than 401: a prober learns nothing about whether the route exists.
//
// The file is read per request — that is what lets a bindmounted copy be edited
// on the host without a restart. It is ~50 KB and polled about once a day.
func userscriptHandler(token, path string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if subtle.ConstantTimeCompare([]byte(r.PathValue("token")), []byte(token)) != 1 {
http.NotFound(w, r)
return
}
info, err := os.Stat(path)
if err != nil {
log.Printf("userscript: stat %s: %v", path, err)
http.NotFound(w, r)
return
}
src, err := os.ReadFile(path)
if err != nil {
log.Printf("userscript: read %s: %v", path, err)
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "text/javascript; charset=utf-8")
w.Header().Set("Cache-Control", "no-cache")
w.Write(stampVersion(src, info.ModTime()))
}
}
-132
View File
@@ -1,132 +0,0 @@
package main
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// sampleScript is a stand-in for the real userscript: a metadata block with a
// @version line, plus a body that must survive the rewrite untouched.
const sampleScript = `// ==UserScript==
// @name Manga Bookmark Sync
// @version 1.5.0
// @match https://asurascans.com/*
// ==/UserScript==
(function () { "use strict"; })();
`
// writeScript drops a userscript in a temp dir with a known mtime and returns
// its path plus the version string the handler is expected to stamp.
func writeScript(t *testing.T, body string) (path, wantVersion string) {
t.Helper()
path = filepath.Join(t.TempDir(), "manga-bookmark.user.js")
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatalf("write script: %v", err)
}
mod := time.Date(2026, 7, 28, 16, 42, 0, 0, time.UTC)
if err := os.Chtimes(path, mod, mod); err != nil {
t.Fatalf("chtimes: %v", err)
}
return path, "2026.07.28.1642"
}
func newUserscriptServer(t *testing.T, path string) http.Handler {
t.Helper()
store, err := OpenStore(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatalf("OpenStore: %v", err)
}
t.Cleanup(func() { store.Close() })
cfg := testConfig()
cfg.UserscriptPath = path
return newRouter(store, cfg)
}
func getScript(t *testing.T, srv http.Handler, token string) *httptest.ResponseRecorder {
t.Helper()
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, httptest.NewRequest(http.MethodGet, "/u/"+token+"/manga-bookmark.user.js", nil))
return rr
}
func TestUserscriptServedWithStampedVersion(t *testing.T) {
path, wantVersion := writeScript(t, sampleScript)
rr := getScript(t, newUserscriptServer(t, path), testToken)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rr.Code)
}
if ct := rr.Header().Get("Content-Type"); !strings.HasPrefix(ct, "text/javascript") {
t.Errorf("Content-Type = %q, want text/javascript", ct)
}
if cc := rr.Header().Get("Cache-Control"); cc != "no-cache" {
t.Errorf("Cache-Control = %q, want no-cache", cc)
}
body := rr.Body.String()
if !strings.Contains(body, "// @version "+wantVersion) {
t.Errorf("body has no stamped version %q:\n%s", wantVersion, body)
}
if strings.Contains(body, "1.5.0") {
t.Errorf("body still carries the file's own version:\n%s", body)
}
// Everything outside the @version line is served verbatim.
if !strings.Contains(body, `(function () { "use strict"; })();`) {
t.Errorf("body was altered beyond the version line:\n%s", body)
}
if !strings.Contains(body, "// @name Manga Bookmark Sync") {
t.Errorf("metadata block was altered:\n%s", body)
}
}
func TestUserscriptWrongTokenIs404(t *testing.T) {
path, _ := writeScript(t, sampleScript)
srv := newUserscriptServer(t, path)
for _, tok := range []string{"wrong", "", testToken + "x", testToken[:3]} {
if got := getScript(t, srv, tok).Code; got != http.StatusNotFound {
t.Errorf("token %q: status = %d, want 404", tok, got)
}
}
}
func TestUserscriptMissingFileIs404(t *testing.T) {
srv := newUserscriptServer(t, filepath.Join(t.TempDir(), "absent.user.js"))
if got := getScript(t, srv, testToken).Code; got != http.StatusNotFound {
t.Fatalf("status = %d, want 404", got)
}
}
func TestUserscriptWithoutVersionLineServedUnmodified(t *testing.T) {
const noVersion = "// ==UserScript==\n// @name x\n// ==/UserScript==\nconsole.log(1);\n"
path, _ := writeScript(t, noVersion)
rr := getScript(t, newUserscriptServer(t, path), testToken)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rr.Code)
}
if rr.Body.String() != noVersion {
t.Fatalf("body = %q, want it unmodified", rr.Body.String())
}
}
// The endpoint must work on a deployment that never set WEB_PASSWORD, since
// the web routes are not registered at all in that case.
func TestUserscriptServedWithWebUIDisabled(t *testing.T) {
path, _ := writeScript(t, sampleScript)
store, err := OpenStore(filepath.Join(t.TempDir(), "nopass.db"))
if err != nil {
t.Fatalf("OpenStore: %v", err)
}
t.Cleanup(func() { store.Close() })
cfg := testConfig()
cfg.WebPassword = ""
cfg.UserscriptPath = path
if got := getScript(t, newRouter(store, cfg), testToken).Code; got != http.StatusOK {
t.Fatalf("status = %d, want 200", got)
}
}
-385
View File
@@ -1,385 +0,0 @@
package main
import (
"crypto/subtle"
"embed"
"html/template"
"io/fs"
"log"
"math"
"mime"
"net/http"
"strconv"
"strings"
"time"
)
//go:embed templates
var templateFS embed.FS
//go:embed static
var staticFS embed.FS
// recentCount is how many series the "Continue reading" strip shows.
const recentCount = 5
// webHandler serves the browser UI: full pages at / and htmx fragments at /ui/.
// It is a separate handler from bookmarkHandler because the two speak different
// representations (HTML versus JSON) to different clients under different auth.
type webHandler struct {
store *Store
tmpl *template.Template
key []byte
password string
limiter *loginLimiter
}
// listView is what every list-rendering template receives.
type listView struct {
Tab string // "all", "fav", or "new"
Recent []Bookmark
Items []Bookmark
// NewCount is the badge on the Updated tab: how many series being read
// have a chapter out that has not been read. It is counted over the whole
// reading set, not the active tab, so the badge does not change meaning as
// the user moves between tabs.
NewCount int
}
// Initial is the monogram the templates show in place of a cover when the
// source site never gave us an og:image. First rune, uppercased; "?" when even
// the title is missing, so the slot is never empty.
func (b Bookmark) Initial() string {
for _, r := range b.Title {
return strings.ToUpper(string(r))
}
return "?"
}
// loginView is what the login template receives.
type loginView struct {
Error string
}
// newWebHandler parses every template up front so a broken one kills the
// process at startup rather than the first request that touches it.
func newWebHandler(store *Store, cfg Config) (*webHandler, error) {
tmpl, err := template.ParseFS(templateFS, "templates/*.html")
if err != nil {
return nil, err
}
return &webHandler{
store: store,
tmpl: tmpl,
key: sessionKey(cfg.Token, cfg.WebPassword),
password: cfg.WebPassword,
limiter: newLoginLimiter(),
}, nil
}
func (h *webHandler) register(mux *http.ServeMux) {
mux.HandleFunc("GET /{$}", h.index)
mux.HandleFunc("POST /login", h.login)
mux.HandleFunc("POST /logout", h.logout)
mux.Handle("GET /static/", staticHandler())
mux.HandleFunc("GET /ui/list", h.requireSession(h.uiList))
mux.HandleFunc("POST /ui/bookmarks/{key}/favorite", h.requireSession(h.uiFavorite))
mux.HandleFunc("POST /ui/bookmarks/{key}/status", h.requireSession(h.uiStatus))
mux.HandleFunc("POST /ui/bookmarks/{key}/chapter", h.requireSession(h.uiChapter))
mux.HandleFunc("DELETE /ui/bookmarks/{key}", h.requireSession(h.uiDelete))
}
// staticHandler serves the embedded assets. An hour, not longer: assets are
// not fingerprinted, and embed.FS reports a zero ModTime, so http.FileServer
// emits no Last-Modified or ETag and a client has no way to revalidate a
// cached copy after a deploy short of waiting out max-age.
func staticHandler() http.Handler {
sub, err := fs.Sub(staticFS, "static")
if err != nil {
panic("embed static: " + err.Error())
}
// Go's built-in table has no .woff2 and the scratch image has no
// /etc/mime.types, so without this the fonts go out as
// application/octet-stream.
if err := mime.AddExtensionType(".woff2", "font/woff2"); err != nil {
panic("woff2 mime: " + err.Error())
}
files := http.FileServer(http.FS(sub))
return http.StripPrefix("/static/", http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Cache-Control", "public, max-age=3600")
files.ServeHTTP(w, r)
}))
}
// authed reports whether the request carries a valid session cookie.
func (h *webHandler) authed(r *http.Request) bool {
c, err := r.Cookie(sessionCookieName)
return err == nil && verifySession(h.key, c.Value, time.Now().UnixMilli())
}
// requireSession guards the fragment endpoints. It answers 401 rather than
// redirecting, because htmx swaps whatever body it receives into the page and a
// redirected login page would be spliced into the card list.
func (h *webHandler) requireSession(next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if !h.authed(r) {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
next(w, r)
}
}
func (h *webHandler) render(w http.ResponseWriter, status int, name string, data any) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(status)
if err := h.tmpl.ExecuteTemplate(w, name, data); err != nil {
// The status line is already sent, so this can only be logged.
log.Printf("render %s: %v", name, err)
}
}
// index renders the list, or the login page when there is no session. The login
// page is served at / with status 200 rather than as a redirect to a separate
// URL: one page, no redirect loop to reason about.
func (h *webHandler) index(w http.ResponseWriter, r *http.Request) {
if !h.authed(r) {
h.render(w, http.StatusOK, "login", loginView{})
return
}
view, err := h.buildListView(r.URL.Query().Get("tab"))
if err != nil {
log.Printf("index: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.render(w, http.StatusOK, "app", view)
}
// filterBookmarks returns the subset keep reports true for, preserving order.
// It always returns a non-nil slice so an empty tab renders its empty state.
func filterBookmarks(all []Bookmark, keep func(Bookmark) bool) []Bookmark {
out := []Bookmark{}
for _, b := range all {
if keep(b) {
out = append(out, b)
}
}
return out
}
// buildListView loads the list once and derives both the tab-filtered items and
// the recent strip from it.
//
// Archived and finished series appear in their own tab and nowhere else — not
// in All, not in Updated, not in Favourites, and not in the recent strip. An
// archived favourite therefore shows only under Archived: Favourites means
// "favourites I am currently reading".
func (h *webHandler) buildListView(tab string) (listView, error) {
all, err := h.store.List() // already ordered updated_at DESC
if err != nil {
return listView{}, err
}
reading := filterBookmarks(all, func(b Bookmark) bool { return b.Status == statusReading })
// The strip reflects overall reading recency, not the active tab.
recent := reading
if len(recent) > recentCount {
recent = recent[:recentCount]
}
withNew := filterBookmarks(reading, func(b Bookmark) bool { return b.HasNewChapter() })
var items []Bookmark
switch tab {
case "fav":
items = filterBookmarks(reading, func(b Bookmark) bool { return b.Favorite })
case "new":
items = withNew
case "archived":
items = filterBookmarks(all, func(b Bookmark) bool { return b.Status == statusArchived })
case "finished":
items = filterBookmarks(all, func(b Bookmark) bool { return b.Status == statusFinished })
default:
tab = "all"
items = reading
}
return listView{Tab: tab, Recent: recent, Items: items, NewCount: len(withNew)}, nil
}
func (h *webHandler) uiList(w http.ResponseWriter, r *http.Request) {
view, err := h.buildListView(r.URL.Query().Get("tab"))
if err != nil {
log.Printf("ui list: %v", err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.render(w, http.StatusOK, "list", view)
}
func (h *webHandler) login(w http.ResponseWriter, r *http.Request) {
ip := clientIP(r)
if wait := h.limiter.retryAfter(ip, time.Now()); wait > 0 {
secs := int(wait.Seconds()) + 1
w.Header().Set("Retry-After", strconv.Itoa(secs))
h.render(w, http.StatusTooManyRequests, "login", loginView{
Error: "Too many attempts. Try again in " +
strconv.Itoa((secs+59)/60) + " min.",
})
return
}
if err := r.ParseForm(); err != nil {
http.Error(w, "invalid form", http.StatusBadRequest)
return
}
got := r.PostFormValue("password")
if subtle.ConstantTimeCompare([]byte(got), []byte(h.password)) != 1 {
h.limiter.fail(ip, time.Now())
h.render(w, http.StatusUnauthorized, "login", loginView{Error: "Wrong password."})
return
}
h.limiter.reset(ip)
setSessionCookie(w, r, h.key)
http.Redirect(w, r, "/", http.StatusSeeOther)
}
func (h *webHandler) logout(w http.ResponseWriter, r *http.Request) {
clearSessionCookie(w, r)
http.Redirect(w, r, "/", http.StatusSeeOther)
}
// loadForMutation fetches the row a mutation targets, writing the error
// response itself when there is nothing to mutate.
func (h *webHandler) loadForMutation(w http.ResponseWriter, r *http.Request) (Bookmark, bool) {
key := r.PathValue("key")
if key == "" {
http.Error(w, "missing key", http.StatusBadRequest)
return Bookmark{}, false
}
b, ok, err := h.store.Get(key)
if err != nil {
log.Printf("ui get %q: %v", key, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return Bookmark{}, false
}
if !ok {
http.Error(w, "not found", http.StatusNotFound)
return Bookmark{}, false
}
return b, true
}
// saveAndRenderCard upserts and renders the row as stored. Upsert decides
// whether updated_at moves, so the argument's timestamp is only a candidate and
// the response must come from the return value.
//
// ponytail: the swapped card stays put even when its new status no longer
// matches the active tab, add an hx-swap-oob list refresh if that reads as a
// bug rather than as feedback. Archiving from the All tab leaves the card on
// screen until the next list load. The alternative costs a full list round
// trip on every toggle, and the card visibly showing its new state is the
// feedback the user needs.
func (h *webHandler) saveAndRenderCard(w http.ResponseWriter, b Bookmark) {
stored, err := h.store.Upsert(b)
if err != nil {
log.Printf("ui upsert %q: %v", b.Key, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
h.render(w, http.StatusOK, "card", stored)
}
// uiFavorite flips the favourite flag. last_chapter_num is untouched, so
// Upsert keeps the stored updated_at and the list does not reorder.
func (h *webHandler) uiFavorite(w http.ResponseWriter, r *http.Request) {
b, ok := h.loadForMutation(w, r)
if !ok {
return
}
b.Favorite = !b.Favorite
b.UpdatedAt = time.Now().UnixMilli()
h.saveAndRenderCard(w, b)
}
// uiStatus moves a bookmark between lifecycle buckets. This is the only place
// a series can be marked finished — the JSON API refuses that value, so the
// userscript cannot set it even by accident.
//
// last_chapter_num is untouched, so Upsert keeps the stored updated_at and the
// list does not reorder.
func (h *webHandler) uiStatus(w http.ResponseWriter, r *http.Request) {
b, ok := h.loadForMutation(w, r)
if !ok {
return
}
if err := r.ParseForm(); err != nil {
http.Error(w, "invalid form", http.StatusBadRequest)
return
}
switch s := r.PostFormValue("status"); s {
case statusReading, statusArchived, statusFinished:
b.Status = s
default:
http.Error(w, "invalid status", http.StatusBadRequest)
return
}
b.UpdatedAt = time.Now().UnixMilli()
h.saveAndRenderCard(w, b)
}
// uiChapter forces the read chapter to a value the user typed.
//
// Writing the number also clears last_chapter_url: that URL points at the
// chapter actually read, and once the number is forced elsewhere it would send
// the reader backwards. ContinueURL then falls back to the series page, which
// is always right.
//
// A submit that does not change the number touches nothing. The form is
// pre-filled, so a bare tap of Save is an easy accidental submit; it must not
// destroy last_chapter_url, nor rewrite the last_chapter display string ("45.0"
// to "45") behind a frozen updated_at.
func (h *webHandler) uiChapter(w http.ResponseWriter, r *http.Request) {
b, ok := h.loadForMutation(w, r)
if !ok {
return
}
if err := r.ParseForm(); err != nil {
http.Error(w, "invalid form", http.StatusBadRequest)
return
}
raw := strings.TrimSpace(r.PostFormValue("chapter"))
num, err := strconv.ParseFloat(raw, 64)
if err != nil || num < 0 || math.IsNaN(num) || math.IsInf(num, 0) {
http.Error(w, "chapter must be a non-negative number", http.StatusBadRequest)
return
}
if num != b.LastChapterNum {
b.LastChapterURL = ""
b.LastChapter = raw
b.LastChapterNum = num
}
b.UpdatedAt = time.Now().UnixMilli()
h.saveAndRenderCard(w, b)
}
// uiDelete removes the row and answers with an empty body, which htmx swaps in
// place of the card — removing it from the page.
func (h *webHandler) uiDelete(w http.ResponseWriter, r *http.Request) {
key := r.PathValue("key")
if key == "" {
http.Error(w, "missing key", http.StatusBadRequest)
return
}
if err := h.store.Delete(key); err != nil {
log.Printf("ui delete %q: %v", key, err)
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(http.StatusOK)
}
+995 -178
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More