ba679223b29fe9281d3d9ea280de27283d9f5974
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 sulthan
|
2a3bb6922d |
Move the browser off the VPS to its own unit (#46) (#52)
Closes #46 once deployed. The headless browser leaves the API stack and becomes its own compose unit (`chrome/docker-compose.yml`) intended for the home machine, reached over the tailnet. No fallback sidecar is left on the VPS. The backend needs no code change — `BROWSER_WS_URL` was already the only coupling. Its default is now empty rather than a pinned Docker IP, so an unconfigured or unreachable browser degrades exactly as it always has: plain-TLS libraries unaffected, kagane/novelfull logged and skipped, stored covers still served. ### What shipped - `chrome/docker-compose.yml` + `chrome/.env.example` — the browser unit, with the CDP port bound to `${BROWSER_BIND_ADDR}` (no default) and the resource limits from the epic: 512 MiB / 1 GiB memory+swap, `oom_score_adj 800`, halved CPU weight, shm 1 GiB -> 128 MiB. - API stack drops the service, its `depends_on` and the `browser` network. - `bookmark-api` gains the `default` network. Dropping `browser` had left it on `db` alone, which is `internal: true` — no published port and, worse, no egress for the poller at all. Caught by actually bringing the stack up. - ADR-0006 for the topology; `DEPLOY.md` §7 for first-time setup of the browser machine; `REDEPLOY.md` §8 for its independent update cadence; architecture diagrams, config tables and troubleshooting rows across README/AGENTS/env. ### Verified locally - Browser unit builds and runs: Chrome 151, UA carries no `HeadlessChrome`, all limits applied as declared. - **Live smoke passes through the new unit**: `TestSmokeKaganeImage` fetched 56710 bytes of `image/webp`, `TestSmokeKaganeGet` got a 200 with a real chapter list. The challenge cleared under the reduced 128 MiB shm. - Bind isolation proven: refused on the host's non-loopback address, accepted on the configured one. - 321 MiB peak of the 512 MiB cap after a full solve; 0 restarts, no OOM kill. - API stack comes up clean, `/healthz` 200; egress confirmed present on `default` and absent on `db`. - `go test ./...`, `go vet`, `gofmt` clean. ### Left to the operator Provisioning the home machine, the Tailscale ACL, setting `BROWSER_WS_URL` in production, and observing acceptance criteria 5-7 (covers with the machine off, several days of zero OOM/restarts, VPS memory improvement). `DEPLOY.md` §7 now carries the before/after `free -m` reading those need. Reviewed-on: #52 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
08749df050 |
feat(backend)!: run on Postgres with a migration-owned schema (#28)
Swap modernc.org/sqlite for jackc/pgx/v5 with no observable change: same endpoints, same wire format, same updated_at ordering rule. The schema now comes from numbered SQL embedded in the binary and applied on startup, one transaction each, recorded in schema_migrations. That replaces two pieces of SQLite-era machinery, both deleted rather than ported: the column probing (Postgres has ADD COLUMN IF NOT EXISTS, and there is no legacy database left to probe) and the Asura key rewrite, which has run clean on every start for months now that the userscripts strip build hashes before writing. Its regexp survives as latest.asuraBuildHash, where the poller still needs it to scope chapter links to a series whose slug carries a rotating hash. Types get real: favorite is a boolean, chapter numbers double precision, timestamps stay unix-ms bigint. SQLite's null-safe IS NOT becomes IS DISTINCT FROM, which is what implements the rule that only reading progress reorders a list. Inside COALESCE/NULLIF the status and kind parameters need an explicit ::text -- there is no target column to infer from and Postgres refuses to guess. Tests lose their free t.TempDir() database, so Docker is now a hard prerequisite for `go test ./...`: internal/pgtest starts one postgres:17-alpine per test binary and hands each test a database of its own. Also lands CONTEXT.md and the four ADRs written while scoping #18. BREAKING CHANGE: DB_PATH is retired for DATABASE_URL, which is required and has no default. Compose gains a postgres service on an internal network with its own volume; POSTGRES_PASSWORD joins .env. The old bookmarks-data volume is deliberately left undeclared so `docker compose down -v` cannot take the pre-migration database with it. main is not deployable until #25 and #26 land. Closes #20 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
4229c179b0 |
rebrand: MangaBM → BookmarkManager, add novel library support (#15)
Two intertwined changes — the rebrand and the novel library were developed on
the same branch because the novel UI plumbing is part of the new "Bookmark
Manager" wordmark in the web shell.
## What it does
- **Rebrand**: MangaBM → BookmarkManager across the Go module, compose stack,
env vars, Traefik hostnames, container/image names, userscript storage
prefixes (`mangabm:cache` → `bmgr:manga:cache`, `mangabm:queue` → `bmgr:manga:queue`),
and docs.
- **Novel library**: same backend, two libraries. New `kind` column splits
bookmarks into `manga` / `novel`; PUT validates it. Two userscripts:
- `manga-bookmark.user.js` — unchanged behaviour, just stamps its own `kind`.
- `novel-bookmark.user.js` — separate Violentmonkey install with adapters
for **novelfull.com** (polled via headless browser — Cloudflare JS
challenge) and **lightnovelworld.net** (polled via plain TLS).
- **Web UI**: library switch on the app shell. Login art, libswitch, and
novel-site colours from the Cinder design snapshot.
## Plumbing
- `addedColumns` ALTER for `kind` runs on first start after upgrade; every
pre-existing row is backfilled to `'manga'`. No manual SQL, no down-time.
- `ALLOWED_ORIGINS` gains the two novel sites.
- New `NOVEL_USERSCRIPT_PATH` env (default `/userscript/novel-bookmark.user.js`),
bindmounted alongside the manga script.
- Traefik router names `mangabm*` → `bmapi*` / `bmweb*`.
## Test status
- `go test ./...` — green
- `node --test userscript/test/logic.test.js` — 34 pass
- `node --test userscript/test/novel-logic.test.js` — 11 pass
- `node --check` on both userscripts — clean
## Notes for the redeploy
.env keys were renamed (`MANGA_API_HOST` → `BOOKMARK_API_HOST`,
`MANGA_WEB_HOST` → `BOOKMARK_WEB_HOST`). Update DNS / Traefik labels on the
prod override before pulling, otherwise the public hostnames go dark.
See the redeploy instructions I'll post next to this PR.
Reviewed-on: #15
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
|
||
|
sulthan
|
180ee78b1f |
Add comix.to and kagane.to support (#13)
Tracks read progress on comix.to and kagane.to alongside asura and demonic, in both the userscript and the backend. Implements `docs/superpowers/plans/2026-08-03-comix-kagane-support.md`. ## Userscript - `comix` adapter — `/title/<id>-<slug>`; only the id prefix is identity (the slug follows the title). No `og:image`, so the cover is matched by `alt`. - `kagane` adapter — reader URLs are uuids with no chapter number, so it comes out of `og:title`; anchor scanning is structurally impossible, replaced by `latestChapterFromApi` against kagane's same-origin JSON API. - `seriesId` threaded through `latestChapterFromAnchors` so comix can scope its scan to its own series and a recommendation strip cannot win the maximum. - `@match` for both hosts, panel chips, v1.6.0. ## Backend - `latestChapterFrom` cases: comix parses the SSR JSON state blob (`latestChapterUrl`, scoped to the series id); kagane parses API JSON (`chapter_no`). - Poller allowlist extended; `Poller.BrowserFetch` with `fetcherFor(site)` routes kagane to a browser fetcher. Nil means kagane is not polled at all — never a fallback to the TLS fetcher, which would only ever retrieve a challenge page. - `BrowserFetcher`: chromedp against a `headless-shell` sidecar. kagane sits behind a Cloudflare JS challenge that no TLS fingerprint clears, and the request is made inside the page rather than by replaying `cf_clearance`. - `BROWSER_WS_URL` wiring, sidecar in both compose files (no `ports:`, dedicated non-external network), Dockerfile on `golang:1.26-alpine` — chromedp requires go 1.26. - Web UI `--comix` / `--kagane` tokens in both colour branches. ## Notes for review - `series_url` is client-supplied and a headless browser is a strong SSRF primitive, so kagane's host is pinned twice: in `fetchableSeriesURL` and again in `kaganeAPIURL`. - Three chained defects found during verification made the browser path dead under Compose (sidecar flag collision, Chrome's Host-header DNS-rebinding check, the wrong chromedp option). Fixed; the compose comments record the wrong configurations too, so they don't get "simplified" back. - `ALLOWED_ORIGINS` now includes both new origins. Without it every write from comix/kagane silently fails CORS preflight, parks in the retry queue, and drops at the cap. ## Verification 221 backend tests, 32 userscript tests, static `CGO_ENABLED=0` build, both compose configs. Two gaps, both real: 1. The userscript on live pages via Violentmonkey needs a human browser profile — not run. Check: comix series page (title/cover, no chapter), comix chapter page (records the number; an *older* chapter must not regress it), comix SPA navigation without reload, kagane series page (og:image cover), kagane reader (number from `og:title`), both chips opening the right sites. 2. The kagane browser path has not completed end-to-end anywhere. Dial/navigate/fetch is confirmed, but Cloudflare 403'd headless-shell's Chrome on every attempt from the dev sandbox, and comix's poll-through-Docker was blocked by that environment's TLS interception. Both environment-dependent rather than branch defects — the first real deploy is the actual verification. Reviewed-on: #13 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com> |
||
|
sulthan
|
ebc7a546c5 |
feat: password-gated web UI on the same backend (#1)
Adds a password-gated browser UI for the bookmark list, served by the same Go
binary and container as the userscript API.
## What
- `GET /` — list page, or the login page when there is no session (200, no redirect).
- `POST /login`, `POST /logout` — stateless HMAC session cookie, 60-day Max-Age.
- `GET /ui/list?tab=all|fav`, `POST /ui/bookmarks/{key}/favorite`,
`POST /ui/bookmarks/{key}/chapter`, `DELETE /ui/bookmarks/{key}` — htmx fragments.
- `GET /static/*` — embedded `style.css`, `htmx.min.js`, `filter.js`.
Mobile-first dark CSS, 2–3 column grid at ≥900px, "Continue reading" strip of the
five most recent series, NEW badge, client-side title search, no build step.
## Stack
Go `html/template` + htmx 2.0.4 (vendored, 50 KB) + plain CSS. No npm, no bundler.
Templates and assets are `go:embed`-ed, so `CGO_ENABLED=0` and the distroless
image still hold.
## Auth
`WEB_PASSWORD` gates the UI; unset means the web routes are never registered and
`/` returns 404. Session cookie is `HttpOnly`, `SameSite=Lax`, `Secure` when the
request is HTTPS. The signing key derives from `API_TOKEN` + `WEB_PASSWORD`, so
rotating either logs every browser out. Login is rate-limited to 10 failures per
20 minutes per client IP, keyed on the **rightmost** `X-Forwarded-For` entry
(Traefik appends the observed peer, so the leftmost is client-spoofable). CGNAT
lockout is a known, accepted limitation — the window self-heals.
## Invariants preserved
- A session cookie never authenticates `/bookmarks*`. That API stays JSON +
bearer token, unchanged, as does the userscript.
- `Store.Upsert` is byte-for-byte unmodified. Every UI write goes
read-modify-write through the new `Store.Get`, so the conditional-`updated_at`
rule (favouriting must not reorder the list, a chapter override must) lives in
exactly one function.
## Deployment
`docker-compose.prod.yml` gains a second Traefik router on `MANGA_WEB_HOST`
pointing at the same service — one container, one certificate resolver, no second
service. Both `MANGA_API_HOST` and `MANGA_WEB_HOST` are required (`:?`), with no
example fallback in `.env.example`: a placeholder there would make Traefik
silently publish the UI on a domain you do not own. Needs a DNS A/AAAA record for
`manga.<domain>`. See `DEPLOY.md` §1b.
## Docs
- Design: `docs/superpowers/specs/2026-07-25-web-ui-design.md`
- Plan: `plans/2026-07-25-web-ui-implementation-plan.md`
## Verification
`gofmt` clean, `go vet`, `go test -race ./...`, `CGO_ENABLED=0 go build`, a real
`docker build` + curl smoke test, and a Playwright pass covering login
reject/accept, favourite-without-reorder, chapter edit, delete-with-confirm,
search, tab switch + back button, 390px with no horizontal overflow, and zero JS
console errors.
Reviewed-on: #1
Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com>
Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
|
||
 claude
|
894a421a9d |
feat: Traefik router labels for prod deploy
Add traefik.enable + Host/entrypoints/tls/certresolver/service labels to the prod override, driven by MANGA_API_HOST / PROXY_NETWORK / TRAEFIK_ENTRYPOINT / TRAEFIK_CERTRESOLVER env vars (documented in .env.example). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude
|
f58d113934 |
feat: manga bookmark sync backend + Bromite userscript
Backend (Go, stdlib net/http + modernc.org/sqlite, CGO-free static binary):
- GET/PUT/DELETE /bookmarks{,/key} + /healthz
- bearer auth (constant-time), CORS origin reflection + 204 preflight
- SQLite store keyed <site>:<series_id>, last-write-wins, server-set updated_at
- httptest + temp-sqlite tests (auth, CORS, round-trip); go vet clean
- multi-stage Dockerfile (distroless static nonroot) + compose (base + prod proxy override)
Userscript (single Bromite-compatible IIFE, no GM_* APIs):
- Asura + Demonic adapters, URL-regex ids + og: title/cover
- Shadow-DOM floating UI, localStorage cache, optimistic sync
- auto-progress (no regress) + manual override; framework-agnostic nav watcher
Live-verified adapters (Playwright, 2026-07-24): asurascans.com /comics/<slug-hash>,
demonicscans.org /manga/<slug> + /title/<slug>/chapter/<n> — corrects the plan's
assumed /series/ paths and asuracomic.net domain.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|