Compare commits

...

2 Commits

Author SHA1 Message Date
sulthan 5d4d890c34 docs: propagate novel library + new sites into stale AGENTS.md top-matter
The rebrand (4229c17) rewrote root AGENTS.md as a compression pass and
switched Bromite->Violentmonkey, but the description still called the
project a manga-only tracker over two sites. Six sites, two libraries and
two userscripts now exist.

Root AGENTS.md:
- 'What this is' names both scripts and their sites, the kind column, and
  the <site>:<series_id> key shape.
- Origins constraint generalised past Asura/Demonic.
- Records that kagane and novelfull are reliably Cloudflare-challenged and
  browser-polled, which is the standing exception to the 'blocking is
  IP-reputation-based and not reproducible' note directly above it.
- Diagram says two userscripts.

backend/AGENTS.md:
- Store key list gained novelfull|lightnovelworld and the kind column.
- NOVEL_USERSCRIPT_PATH documented; it shipped in main.go:150 undocumented.

Child userscript/AGENTS.md already covered the novel adapters, so it is
untouched.
2026-08-06 18:16:05 +07:00
sulthan 91565252d5 docs: add Go and userscript secure-coding rules to AGENTS.md
Repo had two security invariants (bearer auth, CORS) but no guidance for
code an agent writes. Studies put AI-generated web/backend code at ~40%
vulnerable, concentrated in broken access control, injection, session and
error handling — the exact surfaces here.

Rules are extracted, not pasted from a generic checklist: each one names a
guard that already exists in-tree (fetchableSeriesURL, maxBodyBytes,
ConstantTimeCompare, MaxBytesReader, el({text})), so the instruction is
match-this rather than invent-something. Per OpenSSF guidance, irrelevant
rules make a model generate code compensating for attacks that cannot
happen, so container signing, IaC, PII and memory-safety items are left out.
2026-08-06 18:11:35 +07:00
2 changed files with 46 additions and 6 deletions
+41 -3
View File
@@ -4,21 +4,27 @@ Guidance for OpenCode (and Claude Code) working in this repo.
## What this is ## What this is
Manga read-progress tracker, user read on **asurascans.com** (current domain; asuracomic.net 301s here) and **demonicscans.org** via **Violentmonkey**. Userscript inject on-page UI (floating button + slide-in panel), sync progress to self-hosted Go backend so bookmarks unify across both sites and devices. Read-progress tracker for two libraries — manga and novels — behind one self-hosted Go backend. Two separate Violentmonkey userscripts inject on-page UI (floating button + slide-in panel) and sync progress, so bookmarks unify across sites and devices:
- `manga-bookmark.user.js` — **asurascans.com** (current domain; asuracomic.net 301s here), **demonicscans.org**, **comix.to**, **kagane.to**.
- `novel-bookmark.user.js` — **novelfull.com**, **lightnovelworld.net**.
One backend, one `bookmarks` table: a `kind` column (`manga`|`novel`) splits the libraries and the web UI switches between them. Rows are keyed `<site>:<series_id>`.
## Hard constraints (drive design — don't violate) ## Hard constraints (drive design — don't violate)
Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free where plain web APIs suffice — keeps portability across engines: Userscript targets **Violentmonkey**, so `GM_*` APIs available, but stay GM-free where plain web APIs suffice — keeps portability across engines:
- **Avoid `GM_*` unless needed.** Prefer page `localStorage` over `GM_setValue`/`GM_getValue`, on-page UI over `GM_registerMenuCommand`, plain `fetch()` over `GM_xmlhttpRequest` for cross-origin. - **Avoid `GM_*` unless needed.** Prefer page `localStorage` over `GM_setValue`/`GM_getValue`, on-page UI over `GM_registerMenuCommand`, plain `fetch()` over `GM_xmlhttpRequest` for cross-origin.
- Cross-origin `fetch()` work **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block). - Cross-origin `fetch()` work **only** against CORS-enabled backend. Manga sites `https://`, so backend **must be HTTPS** (else mixed-content block).
- Asura and Demonic are **separate origins with separate `localStorage`** — shared remote store only way to unify bookmarks. Cloud sync required, not optional. - Every site is its **own origin with its own `localStorage`** — a shared remote store is the only way to unify bookmarks. Cloud sync required, not optional.
- Userscript run in **isolated world**, so embedded API token safe from site's JS. - Userscript run in **isolated world**, so embedded API token safe from site's JS.
- Cloudflare's block on manga sites **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — Cloudflare's bot scoring can flip previously-clean IP without notice. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test. - Cloudflare's block on manga sites **IP-reputation-based, not universal — and not reliably reproducible.** Verified 2026-07-26: plain `curl` from both CGNAT dev machine *and* deployed VPS got clean 200s with real HTML on both asurascans.com and demonicscans.org (homepage, series, chapter pages) — no interactive Turnstile challenge from either IP at test time. Contradicts earlier untested assumption CGNAT dev IP blocked; wasn't, at least this date. Treat "does curl work right now" as live, time-varying fact to re-check, not fixed property of machine — Cloudflare's bot scoring can flip previously-clean IP without notice. Backend fetcher still needs graceful-degrade path for when challenged, and adapters should be **verified against live pages** (Playwright MCP, on-device devtools, direct probe) before finalizing, not assumed from single earlier test.
- **kagane.to and novelfull.com are the exception to the above** — both sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the backend polls them over CDP (`BROWSER_WS_URL`) and skips them entirely when that's unset. The four other sites poll fine over plain TLS.
## Architecture ## Architecture
``` ```
Violentmonkey userscript (isolated world, per-site adapters, localStorage cache) Two Violentmonkey userscripts (isolated world, per-site adapters, localStorage cache)
-- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume) -- fetch() HTTPS --> reverse proxy (TLS + CORS) --> Go net/http --> SQLite (volume)
``` ```
@@ -62,9 +68,41 @@ instantly.
## Security invariants ## Security invariants
Existing guarantees — don't regress:
- Auth on `/bookmarks*`: require `Authorization: Bearer <API_TOKEN>`, **constant-time compare**, 401 otherwise. - Auth on `/bookmarks*`: require `Authorization: Bearer <API_TOKEN>`, **constant-time compare**, 401 otherwise.
- CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` with `204`. - CORS: reflect `Origin` only when in `ALLOWED_ORIGINS`; allow `GET,PUT,DELETE,OPTIONS` + headers `Authorization,Content-Type`; answer preflight `OPTIONS` with `204`.
## Secure coding rules (code you write here)
Anchored to OWASP Top 10 / ASVS. Every rule below already has a working example in-tree — match it, don't start a second convention. AI-written backends fail on exactly these: broken access control, injection, weak session/error handling, invented dependencies.
Go backend:
- SQL always parameterized (`?`). Only compile-time constants (`bookmarkColumns`) may be concatenated into query text — never a request value, not even a validated one.
- `html/template` only for anything a browser parses, never `text/template`. Never wrap stored or fetched strings in `template.HTML`/`JS`/`URL`; that switches off the escaping every template depends on.
- Any outbound fetch of a client-supplied URL passes `fetchableSeriesURL` (site + `https` + host check) first. `series_url` arrives in a PUT body, so without the gate the poller will probe arbitrary hosts from the server's own network position. New fetch path reuses the gate rather than re-deriving one.
- Cap every remote body with `io.LimitReader` (`maxBodyBytes`). An unbounded read is an OOM handed to whatever is on the other end.
- Compare secrets with `hmac.Equal` / `subtle.ConstantTimeCompare`, never `==`. Covers API token, web password, session MAC.
- Errors: generic text to the client (`http.Error(w, "internal error", 500)`), detail to `log.Printf`. Never log `API_TOKEN`, `WEB_PASSWORD`, a session cookie value, or a whole `Authorization` header.
- Proxy headers are trusted only where they already are: `X-Forwarded-Proto` for the Secure cookie flag, **rightmost** `X-Forwarded-For` for client IP (leftmost is attacker-supplied). Don't read either anywhere else.
- Session cookies keep `HttpOnly`, `SameSite`, `Secure`-when-HTTPS, and expiry checked before signature.
- Stdlib crypto only. No hand-rolled hashing, no MD5/SHA-1 anywhere security-bearing.
- Validate at the handler boundary before storing: body capped by `http.MaxBytesReader` (64 KB), empty `key` and unknown `status`/`kind` rejected with `400`. A bad value that reaches the store becomes every later reader's problem.
Userscript:
- Site-derived and stored strings render via `el(..., {text})` / `textContent`. `{html}` and `innerHTML` are for author-written literal markup only (`TEMPLATE`, `CSS`) — never a title, chapter label, or API response field. The page DOM belongs to a third-party site; treat it as attacker-controlled.
- Isolated world protects the token from the site's JS. It does not protect anything from an `innerHTML` sink you add yourself.
- The `API_TOKEN` literal sits in both userscripts and must equal backend `API_TOKEN`. Never copy it into logs, docs, commit messages, issues, or a new file. Rotation touches three places: backend env plus both scripts.
- `fetch()` targets `API_BASE` only — no dynamic origin, no site-supplied URL. `authHeaders()` goes nowhere but the backend.
- `localStorage` is shared with the site's own JS: cache and queue live there, credentials never do.
- Wrap every `localStorage` read/write and `JSON.parse` in try/catch (quota, private mode, corrupt entry), as the existing helpers do.
Dependencies: stdlib first; a new module needs a stated reason. Confirm a package actually exists before adding it — a plausible name may be fiction (~20% of LLM-proposed packages don't resolve, which is how slopsquatting lands). Pin exact versions.
Review gate: auth, CORS, session, crypto, and the fetch gate are security-critical. Editing one is not a drive-by change — say which invariant you preserved and run `go test ./...` before calling it done.
## Comments ## Comments
Comment only if code alone can't carry info. Cost per read — must earn spot. Comment only if code alone can't carry info. Cost per read — must earn spot.
+5 -3
View File
@@ -11,7 +11,7 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
packages together into `newRouter`. Root-level `*_test.go` hold packages together into `newRouter`. Root-level `*_test.go` hold
integration tests that exercise the full router; unit tests for a integration tests that exercise the full router; unit tests for a
package live beside it under `internal/`. package live beside it under `internal/`.
- **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`|`comix`|`kagane`). Sync **last-write-wins**. Schema and endpoint list in plan. - **Single-user store.** One `bookmarks` table keyed `<site>:<series_id>` (`asura`|`demonic`|`comix`|`kagane`|`novelfull`|`lightnovelworld`), with a `kind` column (`manga`|`novel`) splitting the two libraries. Sync **last-write-wins**. Schema and endpoint list in plan.
- **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth). - **Endpoints:** `GET /bookmarks`, `PUT /bookmarks/{key}` (upsert; see `updated_at` rule below), `DELETE /bookmarks/{key}`, `GET /healthz` (no auth).
- **Web UI:** same binary serve password-gated browser UI on second - **Web UI:** same binary serve password-gated browser UI on second
hostname — `GET /` (list, or login page when no session), hostname — `GET /` (list, or login page when no session),
@@ -75,8 +75,10 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
(gates browser UI; unset disable it), (gates browser UI; unset disable it),
`LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER` `LATEST_CHAPTER_POLL_ENABLED`/`_COOLDOWN`/`_INTERVAL`/`_BATCH`/`_STAGGER`
(background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`). (background latest-chapter poller; defaults on, `1h`/`10m`/`14`/`20s`).
`USERSCRIPT_PATH` (file served at `/u/{token}/manga-bookmark.user.js`, `USERSCRIPT_PATH` and `NOVEL_USERSCRIPT_PATH` (files served at
default `/userscript/manga-bookmark.user.js`, supplied by bindmount). `/u/{token}/manga-bookmark.user.js` and `/u/{token}/novel-bookmark.user.js`,
defaults `/userscript/manga-bookmark.user.js` and
`/userscript/novel-bookmark.user.js`, both supplied by bindmount).
`BROWSER_WS_URL` (headless-shell CDP endpoint for kagane and novelfull; `BROWSER_WS_URL` (headless-shell CDP endpoint for kagane and novelfull;
unset disables browser polling and leaves those sites to the userscript unset disables browser polling and leaves those sites to the userscript
alone). alone).