mangaBookmark

sulthan/mangaBookmark

Fork 0

Commit Graph

Author	SHA1	Message	Date
sulthan	62772e1eaa	feat: server-side latest-chapter polling (#2 ) Adds a background goroutine to the backend that re-checks each bookmarked series' newest published chapter on its own schedule, so `latest_chapter` stays fresh even when the manga sites are never opened in a browser. This is a second, parallel signal, not a replacement: the userscript keeps its own `maybeCaptureLatestOnSeriesPage` / `backgroundRefreshLatest` logic, unchanged. `userscript/manga-bookmark.user.js` is byte-identical to `main`. ## How it works One ticker goroutine in the same binary. Each wake it asks SQLite for bookmarks whose `latest_checked_at` has aged past a per-bookmark cooldown, fetches those series pages through a Chrome-fingerprinted HTTP client, extracts the max chapter number with a per-site regex, and writes it back through `Store.Get` + `Store.Upsert`. Every failure path logs and moves on. Two independent clocks: - cooldown — how long one bookmark rests between checks, enforced by the `WHERE` clause in `Store.DueForLatestCheck`, not by a timer. - interval — how often the goroutine wakes and looks. Shortening the interval therefore cannot shorten anyone's cooldown; it only makes the poller wake and find nothing due more often. The row is stamped before the fetch, so an error, a timeout, or a shutdown mid-request still consumes the cooldown — a renamed or challenged series waits out a full cooldown instead of being retried every tick. ## Design decisions worth reviewing `latest_checked_at` is deliberately absent from the `Bookmark` struct and from `bookmarkColumns`. `PUT /bookmarks/{key}` decodes a whole `Bookmark` and `Upsert` writes every column it knows about, so a userscript PUT — which has no idea this field exists — would write a zero and reset the cooldown, making the poller re-fetch that series on every tick for as long as the user kept reading it. Two tests guard this: `TestUpsertPreservesLatestCheckedAt` and `TestPutDoesNotClobberLatestCheckedAt`, the latter driving a real router PUT with a userscript-shaped body. `updated_at` never moves on a latest-chapter bump. All chapter writes go through `Store.Get` + `Store.Upsert`, so the existing `CASE` keeps the stored timestamp when only `latest_chapter_num` changes and the bookmark list does not reorder. `TestRunOnceDoesNotReorderList` asserts both the timestamp and the `List()` head position. Fetches use `bogdanfinn/tls-client` with a Chrome profile. Plain `net/http` was verified working against both sites on 2026-07-26, so this is not fixing an observed block — it is deliberate defence-in-depth against a future fingerprint-based one. The library is pure Go, so `CGO_ENABLED=0`, the static binary, and the distroless image are all unaffected. It does require the Go floor to move 1.23 → 1.24. `checkOne` validates before spending a request. `series_url` is entirely client-supplied through `PUT /bookmarks/{key}`, so without a guard the poller would issue GETs from the server's own network position to any URL a token holder writes. The check requires a known site and an `https` URL with a non-empty host, and sits after the cooldown stamp so an unfetchable row is retried at cooldown pace rather than hot-looping. ## Config Five new env vars, all with defaults sized for this deployment, all wired through `docker-compose.yml`: \| Variable \| Default \| Meaning \| \| --- \| --- \| --- \| \| `LATEST_CHAPTER_POLL_ENABLED` \| `1` \| Kill switch \| \| `LATEST_CHAPTER_POLL_COOLDOWN` \| `1h` \| Per series, floored at `15m` \| \| `LATEST_CHAPTER_POLL_INTERVAL` \| `10m` \| How often to wake \| \| `LATEST_CHAPTER_POLL_BATCH` \| `14` \| Series per wake \| \| `LATEST_CHAPTER_POLL_STAGGER` \| `20s` \| Delay between fetches in a batch \| `batch × (cooldown / interval)` = 84 series hold a true cooldown cadence at these defaults. Past that nothing breaks: the cadence stretches uniformly and the oldest-checked-first ordering keeps it fair. Bad values log and fall back rather than failing startup — the poller is an enhancement, and a typo in one of its knobs must not stop bookmark sync. ## Known limitation (accepted, documented) The poller's `Store.Get` + `Store.Upsert` is not wrapped in a single transaction. If a userscript `PUT` commits in the sub-millisecond window between the two, the poller writes back its stale re-read — reverting that progress and, since the stored `last_chapter_num` now differs, tripping the `updated_at` `CASE` and reordering the list. Accepted rather than fixed for a single-user deployment: the window is one SELECT wide, the poller only writes when a chapter number actually changed, and the next read self-heals it. The alternative — a transactional read-modify-write — means moving or duplicating the `updated_at` `CASE` that four tests and the whole list-ordering invariant depend on. Recorded in `CLAUDE.md` next to the poller's architecture bullet so it is not a silent trap. ## Testing - Full suite green, including `-race`; `go vet` clean; `CGO_ENABLED=0` static build and `docker compose build` both pass on the bumped `golang:1.24-alpine`. - No test touches the network: the `fetcher` interface exists so tests inject a fake, and no test imports `tls-client` or reaches either manga site. - Extraction is fixture-driven against markup trimmed from real pages (2026-07-26), including a Cloudflare challenge page, cross-series chapter links, decimal chapters, and both raw `&` and `&` forms. - Poller tests cover the no-reorder invariant, cooldown enforcement across passes, batch limiting, one bad series not stalling a batch, downward correction on a retracted chapter, cancelled contexts, and all four failure shapes still consuming the cooldown. - Migration from a pre-column database has its own test — `newTestStore` takes the `CREATE TABLE` path, so the `ALTER TABLE` path would otherwise be untested. - Live smoke test: real server, real fetch of asurascans.com. Log showed `latest is now Chapter 181` and `due=1 checked=1`; `GET /bookmarks` returned `latest_chapter_num: 181` with `updated_at` byte-identical to the PUT that created the row — the no-reorder invariant confirmed against a live site, not just a fake. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Reviewed-on: #2 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>	2026-07-26 18:54:42 +07:00
sulthan	ebc7a546c5	feat: password-gated web UI on the same backend (#1 ) Adds a password-gated browser UI for the bookmark list, served by the same Go binary and container as the userscript API. ## What - `GET /` — list page, or the login page when there is no session (200, no redirect). - `POST /login`, `POST /logout` — stateless HMAC session cookie, 60-day Max-Age. - `GET /ui/list?tab=all\|fav`, `POST /ui/bookmarks/{key}/favorite`, `POST /ui/bookmarks/{key}/chapter`, `DELETE /ui/bookmarks/{key}` — htmx fragments. - `GET /static/` — embedded `style.css`, `htmx.min.js`, `filter.js`. Mobile-first dark CSS, 2–3 column grid at ≥900px, "Continue reading" strip of the five most recent series, NEW badge, client-side title search, no build step. ## Stack Go `html/template` + htmx 2.0.4 (vendored, 50 KB) + plain CSS. No npm, no bundler. Templates and assets are `go:embed`-ed, so `CGO_ENABLED=0` and the distroless image still hold. ## Auth `WEB_PASSWORD` gates the UI; unset means the web routes are never registered and `/` returns 404. Session cookie is `HttpOnly`, `SameSite=Lax`, `Secure` when the request is HTTPS. The signing key derives from `API_TOKEN` + `WEB_PASSWORD`, so rotating either logs every browser out. Login is rate-limited to 10 failures per 20 minutes per client IP, keyed on the rightmost* `X-Forwarded-For` entry (Traefik appends the observed peer, so the leftmost is client-spoofable). CGNAT lockout is a known, accepted limitation — the window self-heals. ## Invariants preserved - A session cookie never authenticates `/bookmarks*`. That API stays JSON + bearer token, unchanged, as does the userscript. - `Store.Upsert` is byte-for-byte unmodified. Every UI write goes read-modify-write through the new `Store.Get`, so the conditional-`updated_at` rule (favouriting must not reorder the list, a chapter override must) lives in exactly one function. ## Deployment `docker-compose.prod.yml` gains a second Traefik router on `MANGA_WEB_HOST` pointing at the same service — one container, one certificate resolver, no second service. Both `MANGA_API_HOST` and `MANGA_WEB_HOST` are required (`:?`), with no example fallback in `.env.example`: a placeholder there would make Traefik silently publish the UI on a domain you do not own. Needs a DNS A/AAAA record for `manga.<domain>`. See `DEPLOY.md` §1b. ## Docs - Design: `docs/superpowers/specs/2026-07-25-web-ui-design.md` - Plan: `plans/2026-07-25-web-ui-implementation-plan.md` ## Verification `gofmt` clean, `go vet`, `go test -race ./...`, `CGO_ENABLED=0 go build`, a real `docker build` + curl smoke test, and a Playwright pass covering login reject/accept, favourite-without-reorder, chapter edit, delete-with-confirm, search, tab switch + back button, 390px with no horizontal overflow, and zero JS console errors. Reviewed-on: #1 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>	2026-07-26 03:59:58 +07:00
claude	f58d113934	feat: manga bookmark sync backend + Bromite userscript Backend (Go, stdlib net/http + modernc.org/sqlite, CGO-free static binary): - GET/PUT/DELETE /bookmarks{,/key} + /healthz - bearer auth (constant-time), CORS origin reflection + 204 preflight - SQLite store keyed <site>:<series_id>, last-write-wins, server-set updated_at - httptest + temp-sqlite tests (auth, CORS, round-trip); go vet clean - multi-stage Dockerfile (distroless static nonroot) + compose (base + prod proxy override) Userscript (single Bromite-compatible IIFE, no GM_* APIs): - Asura + Demonic adapters, URL-regex ids + og: title/cover - Shadow-DOM floating UI, localStorage cache, optimistic sync - auto-progress (no regress) + manual override; framework-agnostic nav watcher Live-verified adapters (Playwright, 2026-07-24): asurascans.com /comics/<slug-hash>, demonicscans.org /manga/<slug> + /title/<slug>/chapter/<n> — corrects the plan's assumed /series/ paths and asuracomic.net domain. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 16:48:28 +07:00

Author

SHA1

Message

Date

sulthan

62772e1eaa

feat: server-side latest-chapter polling (#2 )

Adds a background goroutine to the backend that re-checks each bookmarked series' newest published chapter on its own schedule, so `latest_chapter` stays fresh even when the manga sites are never opened in a browser.

This is a *second, parallel* signal, not a replacement: the userscript keeps its own `maybeCaptureLatestOnSeriesPage` / `backgroundRefreshLatest` logic, unchanged. `userscript/manga-bookmark.user.js` is byte-identical to `main`.

## How it works

One ticker goroutine in the same binary. Each wake it asks SQLite for bookmarks whose `latest_checked_at` has aged past a per-bookmark cooldown, fetches those series pages through a Chrome-fingerprinted HTTP client, extracts the max chapter number with a per-site regex, and writes it back through `Store.Get` + `Store.Upsert`. Every failure path logs and moves on.

Two independent clocks:

- **cooldown** — how long one bookmark rests between checks, enforced by the `WHERE` clause in `Store.DueForLatestCheck`, not by a timer.
- **interval** — how often the goroutine wakes and looks.

Shortening the interval therefore cannot shorten anyone's cooldown; it only makes the poller wake and find nothing due more often.

The row is stamped **before** the fetch, so an error, a timeout, or a shutdown mid-request still consumes the cooldown — a renamed or challenged series waits out a full cooldown instead of being retried every tick.

## Design decisions worth reviewing

**`latest_checked_at` is deliberately absent from the `Bookmark` struct and from `bookmarkColumns`.** `PUT /bookmarks/{key}` decodes a whole `Bookmark` and `Upsert` writes every column it knows about, so a userscript PUT — which has no idea this field exists — would write a zero and reset the cooldown, making the poller re-fetch that series on every tick for as long as the user kept reading it. Two tests guard this: `TestUpsertPreservesLatestCheckedAt` and `TestPutDoesNotClobberLatestCheckedAt`, the latter driving a real router PUT with a userscript-shaped body.

**`updated_at` never moves on a latest-chapter bump.** All chapter writes go through `Store.Get` + `Store.Upsert`, so the existing `CASE` keeps the stored timestamp when only `latest_chapter_num` changes and the bookmark list does not reorder. `TestRunOnceDoesNotReorderList` asserts both the timestamp and the `List()` head position.

**Fetches use `bogdanfinn/tls-client` with a Chrome profile.** Plain `net/http` was verified working against both sites on 2026-07-26, so this is not fixing an observed block — it is deliberate defence-in-depth against a future fingerprint-based one. The library is pure Go, so `CGO_ENABLED=0`, the static binary, and the distroless image are all unaffected. It does require the Go floor to move 1.23 → 1.24.

**`checkOne` validates before spending a request.** `series_url` is entirely client-supplied through `PUT /bookmarks/{key}`, so without a guard the poller would issue GETs from the server's own network position to any URL a token holder writes. The check requires a known site and an `https` URL with a non-empty host, and sits *after* the cooldown stamp so an unfetchable row is retried at cooldown pace rather than hot-looping.

## Config

Five new env vars, all with defaults sized for this deployment, all wired through `docker-compose.yml`:

| Variable | Default | Meaning |
| --- | --- | --- |
| `LATEST_CHAPTER_POLL_ENABLED` | `1` | Kill switch |
| `LATEST_CHAPTER_POLL_COOLDOWN` | `1h` | Per series, floored at `15m` |
| `LATEST_CHAPTER_POLL_INTERVAL` | `10m` | How often to wake |
| `LATEST_CHAPTER_POLL_BATCH` | `14` | Series per wake |
| `LATEST_CHAPTER_POLL_STAGGER` | `20s` | Delay between fetches in a batch |

`batch × (cooldown / interval)` = 84 series hold a true cooldown cadence at these defaults. Past that nothing breaks: the cadence stretches uniformly and the oldest-checked-first ordering keeps it fair. Bad values log and fall back rather than failing startup — the poller is an enhancement, and a typo in one of its knobs must not stop bookmark sync.

## Known limitation (accepted, documented)

The poller's `Store.Get` + `Store.Upsert` is not wrapped in a single transaction. If a userscript `PUT` commits in the sub-millisecond window between the two, the poller writes back its stale re-read — reverting that progress and, since the stored `last_chapter_num` now differs, tripping the `updated_at` `CASE` and reordering the list.

Accepted rather than fixed for a single-user deployment: the window is one SELECT wide, the poller only writes when a chapter number actually changed, and the next read self-heals it. The alternative — a transactional read-modify-write — means moving or duplicating the `updated_at` `CASE` that four tests and the whole list-ordering invariant depend on. Recorded in `CLAUDE.md` next to the poller's architecture bullet so it is not a silent trap.

## Testing

- Full suite green, including `-race`; `go vet` clean; `CGO_ENABLED=0` static build and `docker compose build` both pass on the bumped `golang:1.24-alpine`.
- No test touches the network: the `fetcher` interface exists so tests inject a fake, and no test imports `tls-client` or reaches either manga site.
- Extraction is fixture-driven against markup trimmed from real pages (2026-07-26), including a Cloudflare challenge page, cross-series chapter links, decimal chapters, and both raw `&` and `&amp;` forms.
- Poller tests cover the no-reorder invariant, cooldown enforcement across passes, batch limiting, one bad series not stalling a batch, downward correction on a retracted chapter, cancelled contexts, and all four failure shapes still consuming the cooldown.
- Migration from a pre-column database has its own test — `newTestStore` takes the `CREATE TABLE` path, so the `ALTER TABLE` path would otherwise be untested.
- **Live smoke test:** real server, real fetch of asurascans.com. Log showed `latest is now Chapter 181` and `due=1 checked=1`; `GET /bookmarks` returned `latest_chapter_num: 181` with `updated_at` byte-identical to the PUT that created the row — the no-reorder invariant confirmed against a live site, not just a fake.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: #2
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>

2026-07-26 18:54:42 +07:00

sulthan

ebc7a546c5

feat: password-gated web UI on the same backend (#1 )

Adds a password-gated browser UI for the bookmark list, served by the same Go
binary and container as the userscript API.

## What

- `GET /` — list page, or the login page when there is no session (200, no redirect).
- `POST /login`, `POST /logout` — stateless HMAC session cookie, 60-day Max-Age.
- `GET /ui/list?tab=all|fav`, `POST /ui/bookmarks/{key}/favorite`,
  `POST /ui/bookmarks/{key}/chapter`, `DELETE /ui/bookmarks/{key}` — htmx fragments.
- `GET /static/*` — embedded `style.css`, `htmx.min.js`, `filter.js`.

Mobile-first dark CSS, 2–3 column grid at ≥900px, "Continue reading" strip of the
five most recent series, NEW badge, client-side title search, no build step.

## Stack

Go `html/template` + htmx 2.0.4 (vendored, 50 KB) + plain CSS. No npm, no bundler.
Templates and assets are `go:embed`-ed, so `CGO_ENABLED=0` and the distroless
image still hold.

## Auth

`WEB_PASSWORD` gates the UI; unset means the web routes are never registered and
`/` returns 404. Session cookie is `HttpOnly`, `SameSite=Lax`, `Secure` when the
request is HTTPS. The signing key derives from `API_TOKEN` + `WEB_PASSWORD`, so
rotating either logs every browser out. Login is rate-limited to 10 failures per
20 minutes per client IP, keyed on the **rightmost** `X-Forwarded-For` entry
(Traefik appends the observed peer, so the leftmost is client-spoofable). CGNAT
lockout is a known, accepted limitation — the window self-heals.

## Invariants preserved

- A session cookie never authenticates `/bookmarks*`. That API stays JSON +
  bearer token, unchanged, as does the userscript.
- `Store.Upsert` is byte-for-byte unmodified. Every UI write goes
  read-modify-write through the new `Store.Get`, so the conditional-`updated_at`
  rule (favouriting must not reorder the list, a chapter override must) lives in
  exactly one function.

## Deployment

`docker-compose.prod.yml` gains a second Traefik router on `MANGA_WEB_HOST`
pointing at the same service — one container, one certificate resolver, no second
service. Both `MANGA_API_HOST` and `MANGA_WEB_HOST` are required (`:?`), with no
example fallback in `.env.example`: a placeholder there would make Traefik
silently publish the UI on a domain you do not own. Needs a DNS A/AAAA record for
`manga.<domain>`. See `DEPLOY.md` §1b.

## Docs

- Design: `docs/superpowers/specs/2026-07-25-web-ui-design.md`
- Plan: `plans/2026-07-25-web-ui-implementation-plan.md`

## Verification

`gofmt` clean, `go vet`, `go test -race ./...`, `CGO_ENABLED=0 go build`, a real
`docker build` + curl smoke test, and a Playwright pass covering login
reject/accept, favourite-without-reorder, chapter edit, delete-with-confirm,
search, tab switch + back button, 390px with no horizontal overflow, and zero JS
console errors.

Reviewed-on: #1
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>

2026-07-26 03:59:58 +07:00

claude

f58d113934

feat: manga bookmark sync backend + Bromite userscript

Backend (Go, stdlib net/http + modernc.org/sqlite, CGO-free static binary):
- GET/PUT/DELETE /bookmarks{,/key} + /healthz
- bearer auth (constant-time), CORS origin reflection + 204 preflight
- SQLite store keyed <site>:<series_id>, last-write-wins, server-set updated_at
- httptest + temp-sqlite tests (auth, CORS, round-trip); go vet clean
- multi-stage Dockerfile (distroless static nonroot) + compose (base + prod proxy override)

Userscript (single Bromite-compatible IIFE, no GM_* APIs):
- Asura + Demonic adapters, URL-regex ids + og: title/cover
- Shadow-DOM floating UI, localStorage cache, optimistic sync
- auto-progress (no regress) + manual override; framework-agnostic nav watcher

Live-verified adapters (Playwright, 2026-07-24): asurascans.com /comics/<slug-hash>,
demonicscans.org /manga/<slug> + /title/<slug>/chapter/<n> — corrects the plan's
assumed /series/ paths and asuracomic.net domain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-24 16:48:28 +07:00

3 Commits