Closes #63 Deletes the second way to reach a Cover. Since #62, every Site's cover bytes land in the content-addressed store at creation or on the poll, and the one public route serves them all — nothing needs the kagane proxy anymore. ## What went - **Template-level rewrite:** `Bookmark.CoverURL()` and both templates' use of it. Cards and chrome now render `.Cover` — the wire value — and nothing else. `Bookmark.CoverSource` was dead once `CoverURL` went, so it and its `bookmarkColumns` entry are gone too. - **Kagane-only cover route and its identifier validation:** `GET /img/kagane/{id}`, `web.CoverFetcher`, `coverIDRe`, and the whole `internal/web/cover.go`. - **The proxy's persistence:** `store.KaganeImageID`, `GetKaganeCover`, `PutKaganeCover`, `kaganeCoverSourceURL`, `kaganeCoverRe`. - **The kagane-shaped branch in the byte-fetch routing:** `fetchCoverBytes` no longer takes a `site` argument and no longer names a Site. The URL shape kagane's API publishes is claimed by the browser module itself — `kaganeImageURLRe` + `browserCoverURL` live in `latest/browser.go` with the rest of the per-Site knowledge — and `BrowserFetcher.Image` is now URL-driven (it validates the URL it will navigate to, same SSRF discipline as before). The no-plain-TLS-fallback rule for a claimed URL is preserved: a claimed address with no browser is an error, never a challenge-page fetch. ## What stayed (deliberately) - `BrowserFetcher.Image` and the browser-backed acquisition path: kagane genuinely serves cover bytes behind the challenge + `cross-origin-resource-policy: same-origin`, so the sidecar remains the only fetcher for them — it just routes by URL claim now instead of by Site name. - `fetcherFor`'s per-Site page routing (kagane/novelfull page fetches) — that is the page path, not a cover path. ## Acceptance criteria - [x] Template-level kagane cover rewrite gone - [x] Kagane-only cover route and its identifier validation gone - [x] Tests removed/rewritten against the general route, guarantees kept: unstored + traversal-shaped addresses serve nothing (`TestPublicCoverRejectsUnknownAddress`), non-image content types never echoed (`TestPublicCoverNeverEchoesNonImage` — new; the store-side gate was already pinned by `TestCoverStoreAcceptsAnySourceURL`). Store reopen-persistence and filesystem content-addressing tests rewritten against `PutCover`/`GetCover`, no guarantee lost. - [x] No Site name in a cover code path outside the acquisition module (`grep kagane backend`: store/web/templates/api are clean; remaining hits are `latest/browser.go` + `latest/sites.go`, tests, docs) - [x] Web UI and panel render Covers for all six Sites (templates render the wire address; panel renders `b.cover` — untouched, it never had a kagane path) - [x] `go test ./...` green ## Verification - `go vet ./...` clean - `go test ./...` — all packages pass (root 16.9s, latest 12.7s, store 12.7s, web 0.004s) - `CGO_ENABLED=0 go build` produces the static binary - Cover-path tests run verbosely: `TestPublicCoverServesStoredBytesUnauthenticated`, `TestPublicCoverRejectsUnknownAddress` (unknown/malformed/traversal/empty), `TestPublicCoverNeverEchoesNonImage`, `TestListRendersAcquiredCover`, `TestAcquireKaganeCoverThroughBrowser`, `TestRunOncePrefetchesKaganeCover`, `TestRunOnceRoutesNonKaganeCoverToPublicFetcher` all pass; the three `SMOKE_*` tests skip without the browser sidecar, as designed Live browser verification of the "web UI and panel render Covers for all six Sites" criterion is being run separately with Playwright against real Site pages and a locally mocked backend. Reviewed-on: #73 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #73.
This commit is contained in:
+17
-15
@@ -119,7 +119,9 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
|
||||
browser-fetched page and the bytes go over plain TLS. With no browser
|
||||
configured, kagane Covers are simply absent; novelfull still gets one — at
|
||||
creation and on the poll — when its page body happens to answer a plain
|
||||
request (the challenge is a live time-varying fact).
|
||||
request (the challenge is a live time-varying fact). The old kagane-only
|
||||
serving path (`/img/kagane/{id}`, template rewrite, `CoverFetcher`) is gone
|
||||
(issue #63): the one public route serves every Site.
|
||||
- **`updated_at` drives list order, so moves only on real reading progress:** server apply its timestamp when row new or `last_chapter_num` changes, else keep stored value — favouriting series or recording newly published chapter must not reorder list. `PUT` therefore returns row **as stored**, clients must adopt that response rather than own payload. See `plans/2026-07-25-bookmark-list-favorites-design.md` §4.
|
||||
- **Lifecycle buckets:** `status` on each bookmark is `reading` | `archived` |
|
||||
`finished`, orthogonal to `favorite`. Archived and finished appear only in
|
||||
@@ -156,21 +158,21 @@ Guidance for OpenCode (and Claude Code) working under `backend/`. See root `AGEN
|
||||
Reader's credential at serve time).
|
||||
`BROWSER_WS_URL` (CDP endpoint of the browser, which runs on a **separate
|
||||
machine** and is reached over the tailnet — ADR-0006, `chrome/docker-compose.yml`.
|
||||
Used by the poller for kagane and novelfull *and* by the web UI's kagane
|
||||
cover proxy; unset — the default — disables browser polling and serves 404
|
||||
for covers not already stored, leaving those sites to the userscript alone.
|
||||
Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s
|
||||
Used by the poller for kagane and novelfull page fetches and by the cover
|
||||
pipeline for kagane's image bytes (the browser is the only route that clears
|
||||
the challenge kagane serves its covers behind); unset — the default —
|
||||
disables browser polling and leaves kagane Covers blank until stored bytes
|
||||
exist. Must be a tailnet IP, never a hostname: Chrome's DevTools handler 500s
|
||||
`/json/version` for any Host that isn't an IP or `localhost`).
|
||||
- **kagane covers are proxied, not hot-linked:** kagane serves cover images
|
||||
behind the same challenge as its pages and with
|
||||
`cross-origin-resource-policy: same-origin`, so no `<img>` on the web UI's
|
||||
origin can load one — not even from a browser holding the clearance cookie
|
||||
`og:image` to `/img/kagane/{id}`. `internal/web/cover.go` reads the persistent
|
||||
`covers` table first, then fetches a miss through `latest.BrowserFetcher.Image`.
|
||||
The templates render `.CoverURL`, never `.Cover`. The id is matched against a
|
||||
UUID regex before it reaches the browser: the stored value is client-supplied,
|
||||
so an unchecked one is an SSRF primitive pointed at the deployment's own
|
||||
network.
|
||||
- **No per-Site cover path (issue #63):** every Cover — all six Sites — is
|
||||
served by the one public `GET /covers/{addr}` route from content-addressed
|
||||
bytes. There is no proxy, no per-Site rewrite, no second place that decides
|
||||
a Cover's renderable address: the wire `cover` is it. The only place a Site
|
||||
name still appears in cover code is the extraction module (`latest`), where
|
||||
kagane's image URLs are claimed by `browserOnlyCoverURL` — they answer a
|
||||
plain fetch with a challenge and `cross-origin-resource-policy: same-origin`;
|
||||
every other Site's CDN answers plain TLS. Templates render `.Cover` — the
|
||||
wire value — never anything else.
|
||||
- **Web UI also owns:** session-gated `GET /install/{manga,novel}-bookmark.user.js`
|
||||
(renders the bindmounted script with the acting Reader's derived credential
|
||||
substituted in — the credential never appears in page markup, the address
|
||||
|
||||
+34
-206
@@ -1,40 +1,15 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"errors"
|
||||
"database/sql"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"bookmarkmanager/backend/internal/pgtest"
|
||||
"bookmarkmanager/backend/internal/store"
|
||||
)
|
||||
|
||||
// fakeCovers stands in for the headless browser. It counts calls so the test
|
||||
// can prove the store spares the browser after the first navigation.
|
||||
type fakeCovers struct {
|
||||
body []byte
|
||||
contentType string
|
||||
err error
|
||||
calls atomic.Int32
|
||||
lastID atomic.Value
|
||||
}
|
||||
|
||||
func (f *fakeCovers) Image(_ context.Context, imageID string) ([]byte, string, error) {
|
||||
f.calls.Add(1)
|
||||
f.lastID.Store(imageID)
|
||||
if f.err != nil {
|
||||
return nil, "", f.err
|
||||
}
|
||||
return f.body, f.contentType, nil
|
||||
}
|
||||
|
||||
const testCoverID = "019fe11a-84c3-7fc3-a84b-88787374b617"
|
||||
|
||||
func getCover(t *testing.T, srv http.Handler, path string, cookie *http.Cookie) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
req := httptest.NewRequest(http.MethodGet, path, nil)
|
||||
@@ -46,186 +21,6 @@ func getCover(t *testing.T, srv http.Handler, path string, cookie *http.Cookie)
|
||||
return rr
|
||||
}
|
||||
|
||||
// kagane serves its covers behind a Cloudflare challenge and with
|
||||
// cross-origin-resource-policy: same-origin, so the UI can only show one by
|
||||
// re-serving the bytes from its own origin.
|
||||
func TestKaganeCoverPersistsAndReusesStoredBytes(t *testing.T) {
|
||||
cf := &fakeCovers{body: []byte("\x00webp-bytes"), contentType: "image/webp"}
|
||||
cfg := testConfig()
|
||||
cfg.Covers = cf
|
||||
srv, st := newWebTestServer(t, cfg)
|
||||
cookie := sessionCookie(t, st)
|
||||
|
||||
rr := getCover(t, srv, "/img/kagane/"+testCoverID, cookie)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("first request: status = %d, want 200", rr.Code)
|
||||
}
|
||||
if got := rr.Body.String(); got != string(cf.body) {
|
||||
t.Fatalf("first request: body = %q, want %q", got, cf.body)
|
||||
}
|
||||
|
||||
// A new Handler has no process-local state from the first request. The same
|
||||
// store must still answer without navigating the browser again.
|
||||
srv = newRouter(st, cfg)
|
||||
rr = getCover(t, srv, "/img/kagane/"+testCoverID, cookie)
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("stored request: status = %d, want 200", rr.Code)
|
||||
}
|
||||
if got := rr.Body.String(); got != string(cf.body) {
|
||||
t.Fatalf("stored request: body = %q, want %q", got, cf.body)
|
||||
}
|
||||
if got := cf.calls.Load(); got != 1 {
|
||||
t.Fatalf("fetcher called %d times, want 1 — stored bytes must survive a new handler", got)
|
||||
}
|
||||
if got := cf.lastID.Load(); got != testCoverID {
|
||||
t.Fatalf("fetched image id = %v, want %s", got, testCoverID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKaganeCoverServesStoredBytesWithoutBrowser(t *testing.T) {
|
||||
cf := &fakeCovers{err: errors.New("browser must not be called")}
|
||||
cfg := testConfig()
|
||||
cfg.Covers = cf
|
||||
srv, st := newWebTestServer(t, cfg)
|
||||
if err := st.PutKaganeCover(testCoverID, []byte("already-stored"), "image/png"); err != nil {
|
||||
t.Fatalf("PutKaganeCover: %v", err)
|
||||
}
|
||||
|
||||
rr := getCover(t, srv, "/img/kagane/"+testCoverID, sessionCookie(t, st))
|
||||
if rr.Code != http.StatusOK || rr.Body.String() != "already-stored" {
|
||||
t.Fatalf("stored request = (%d, %q), want (200, already-stored)", rr.Code, rr.Body.String())
|
||||
}
|
||||
if got := cf.calls.Load(); got != 0 {
|
||||
t.Fatalf("fetcher called %d times for a stored cover, want 0", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKaganeCoverServesPersistedBytesAfterRestart(t *testing.T) {
|
||||
url := pgtest.URL(t)
|
||||
coverDir := t.TempDir()
|
||||
owner := store.Owner{
|
||||
DiscordID: "cover-owner",
|
||||
TokenHash: sha256.Sum256([]byte("cover-owner-token")),
|
||||
}
|
||||
first, err := store.Open(url, owner, coverDir, testCoverBaseURL)
|
||||
if err != nil {
|
||||
t.Fatalf("Open: %v", err)
|
||||
}
|
||||
if err := first.PutKaganeCover(testCoverID, []byte("survives-restart"), "image/jpeg"); err != nil {
|
||||
first.Close()
|
||||
t.Fatalf("PutKaganeCover: %v", err)
|
||||
}
|
||||
if err := first.Close(); err != nil {
|
||||
t.Fatalf("close first store: %v", err)
|
||||
}
|
||||
|
||||
second, err := store.Open(url, owner, coverDir, testCoverBaseURL)
|
||||
if err != nil {
|
||||
t.Fatalf("reopen: %v", err)
|
||||
}
|
||||
defer second.Close()
|
||||
cf := &fakeCovers{err: errors.New("browser must not be called after restart")}
|
||||
cfg := testConfig()
|
||||
cfg.Covers = cf
|
||||
rr := getCover(t, newRouter(second, cfg), "/img/kagane/"+testCoverID, sessionCookie(t, second))
|
||||
if rr.Code != http.StatusOK || rr.Body.String() != "survives-restart" {
|
||||
t.Fatalf("restarted request = (%d, %q), want (200, survives-restart)", rr.Code, rr.Body.String())
|
||||
}
|
||||
if got := cf.calls.Load(); got != 0 {
|
||||
t.Fatalf("fetcher called %d times after restart, want 0", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The proxy reaches a headless browser, so it is not open to the internet.
|
||||
func TestKaganeCoverRequiresSession(t *testing.T) {
|
||||
cf := &fakeCovers{body: []byte("x"), contentType: "image/webp"}
|
||||
cfg := testConfig()
|
||||
cfg.Covers = cf
|
||||
srv, _ := newWebTestServer(t, cfg)
|
||||
|
||||
rr := getCover(t, srv, "/img/kagane/"+testCoverID, nil)
|
||||
if rr.Code != http.StatusUnauthorized {
|
||||
t.Fatalf("status = %d, want 401", rr.Code)
|
||||
}
|
||||
if got := cf.calls.Load(); got != 0 {
|
||||
t.Fatalf("fetcher called %d times for an unauthenticated request, want 0", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKaganeCoverRejectsBadInput(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
id string
|
||||
fetch *fakeCovers
|
||||
}{
|
||||
{
|
||||
"an id that is not a uuid never reaches the browser",
|
||||
"solo-leveling",
|
||||
&fakeCovers{body: []byte("x"), contentType: "image/webp"},
|
||||
},
|
||||
{
|
||||
"a uuid-shaped id with a trailing segment is rejected whole",
|
||||
testCoverID + "x",
|
||||
&fakeCovers{body: []byte("x"), contentType: "image/webp"},
|
||||
},
|
||||
{
|
||||
"a challenged fetch is a missing cover",
|
||||
testCoverID,
|
||||
&fakeCovers{err: errors.New("challenge held")},
|
||||
},
|
||||
{
|
||||
"a content type outside the image set is not echoed back",
|
||||
testCoverID,
|
||||
&fakeCovers{body: []byte("<script>"), contentType: "text/html"},
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
cfg := testConfig()
|
||||
cfg.Covers = tc.fetch
|
||||
srv, st := newWebTestServer(t, cfg)
|
||||
rr := getCover(t, srv, "/img/kagane/"+tc.id, sessionCookie(t, st))
|
||||
if rr.Code != http.StatusNotFound {
|
||||
t.Fatalf("status = %d, want 404", rr.Code)
|
||||
}
|
||||
_, _, ok, err := st.GetKaganeCover(tc.id)
|
||||
if err != nil {
|
||||
t.Fatalf("GetKaganeCover after rejection: %v", err)
|
||||
}
|
||||
if ok {
|
||||
t.Fatal("rejected cover was persisted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// ServeMux path-cleans a traversal into a redirect before the handler runs, so
|
||||
// the guarantee to pin down is that no request shaped like one ever gets bytes.
|
||||
func TestKaganeCoverTraversalServesNothing(t *testing.T) {
|
||||
cf := &fakeCovers{body: []byte("secret"), contentType: "image/webp"}
|
||||
cfg := testConfig()
|
||||
cfg.Covers = cf
|
||||
srv, st := newWebTestServer(t, cfg)
|
||||
|
||||
rr := getCover(t, srv, "/img/kagane/../../etc/passwd", sessionCookie(t, st))
|
||||
if rr.Code == http.StatusOK {
|
||||
t.Fatalf("status = 200, want anything but a served body")
|
||||
}
|
||||
if got := cf.calls.Load(); got != 0 {
|
||||
t.Fatalf("fetcher called %d times for a traversal, want 0", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Without BROWSER_WS_URL there is no fetcher, and the endpoint must answer
|
||||
// rather than reach for a nil one.
|
||||
func TestKaganeCoverWithoutFetcher(t *testing.T) {
|
||||
srv, st := newWebTestServer(t, testConfig())
|
||||
rr := getCover(t, srv, "/img/kagane/"+testCoverID, sessionCookie(t, st))
|
||||
if rr.Code != http.StatusNotFound {
|
||||
t.Fatalf("status = %d, want 404", rr.Code)
|
||||
}
|
||||
}
|
||||
|
||||
// The acquired Cover is served from this deployment's own origin, to any
|
||||
// browser rendering a third-party page — no session, no credential (ADR-0007).
|
||||
func TestPublicCoverServesStoredBytesUnauthenticated(t *testing.T) {
|
||||
@@ -276,6 +71,39 @@ func TestPublicCoverRejectsUnknownAddress(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A content type outside the image set is never echoed back. The old kagane
|
||||
// proxy could fetch text/html from a challenged fetch and had to refuse it;
|
||||
// the general route's only input is the store, and the store refuses to
|
||||
// record anything that is not an image — but the guarantee is pinned at the
|
||||
// serving boundary, not the write gate, so a poisoned row (migrated data, a
|
||||
// writer that skips the gate) is also never served.
|
||||
func TestPublicCoverNeverEchoesNonImage(t *testing.T) {
|
||||
const sourceURL = "https://cdn.example/cover"
|
||||
st, dsn := newTestStoreURL(t)
|
||||
// The write gate refuses non-image content types outright.
|
||||
if err := st.PutCover(sourceURL, []byte("<script>"), "text/html"); err == nil {
|
||||
t.Fatal("PutCover accepted a non-image content type")
|
||||
}
|
||||
// A legitimate row, then the content type flipped behind the store's back:
|
||||
// the bytes exist at the address, so only the type is hostile.
|
||||
address := store.CoverAddress(sourceURL)
|
||||
if err := st.SetSeriesCover("asura", "solo", sourceURL, []byte("<script>"), "image/png"); err != nil {
|
||||
t.Fatalf("seed row: %v", err)
|
||||
}
|
||||
db, err := sql.Open("pgx", dsn)
|
||||
if err != nil {
|
||||
t.Fatalf("open %s: %v", dsn, err)
|
||||
}
|
||||
defer db.Close()
|
||||
if _, err := db.Exec(`UPDATE covers SET content_type = 'text/html' WHERE address = $1`, address); err != nil {
|
||||
t.Fatalf("poison row: %v", err)
|
||||
}
|
||||
rr := getCover(t, newRouter(st, testConfig()), "/covers/"+address, nil)
|
||||
if rr.Code == http.StatusOK {
|
||||
t.Fatalf("status = 200, want a refusal for a non-image row (body %q)", rr.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// The whole point of acquiring bytes is that the UI shows them: the card's
|
||||
// <img> must carry the public address, not a third-party URL and not a
|
||||
// placeholder.
|
||||
|
||||
@@ -150,6 +150,14 @@ func (h *Handler) Cover(w http.ResponseWriter, r *http.Request) {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
// Refuse anything the write gate would not have recorded: a poisoned row
|
||||
// (migrated data, a writer that skips the gate) must never be echoed back
|
||||
// as bytes of a type no Cover may have.
|
||||
if _, ok := store.CoverContentType(contentType); !ok {
|
||||
log.Printf("cover %s: refusing non-image content type %q", r.PathValue("address"), contentType)
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", contentType)
|
||||
// Content-addressed, so the bytes at this URL can never change. Public
|
||||
// rather than private: no credential gates the route.
|
||||
|
||||
@@ -42,8 +42,8 @@ type Acquirer struct {
|
||||
// Covers retrieves the cover bytes. Nil leaves the Cover blank and the
|
||||
// chapter half working.
|
||||
Covers CoverBytesFetcher
|
||||
// BrowserCoverFetch retrieves kagane cover bytes through the browser
|
||||
// sidecar. Nil leaves kagane Covers blank; nothing falls back to a plain
|
||||
// BrowserCoverFetch retrieves browser-claimed cover bytes through the
|
||||
// sidecar. Nil leaves those Covers blank; nothing falls back to a plain
|
||||
// fetch, which would only ever retrieve a challenge page.
|
||||
BrowserCoverFetch BrowserCoverFetcher
|
||||
// Ctx cancels in-flight acquisitions at shutdown. A hook signature has
|
||||
@@ -136,7 +136,7 @@ func (a *Acquirer) acquire(ctx context.Context, sr store.Series) {
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
bytes, contentType, err := fetchCoverBytes(ctx, sr.Site, cover, a.BrowserCoverFetch, a.Covers)
|
||||
bytes, contentType, err := fetchCoverBytes(ctx, cover, a.BrowserCoverFetch, a.Covers)
|
||||
if err != nil {
|
||||
log.Printf("acquire %q: fetch cover %s: %v", sr.Key(), cover, err)
|
||||
return
|
||||
|
||||
@@ -312,8 +312,8 @@ func TestAcquireKaganeCoverThroughBrowser(t *testing.T) {
|
||||
if got := covers.callCount(); got != 1 {
|
||||
t.Fatalf("browser cover fetches = %d, want 1", got)
|
||||
}
|
||||
if got := covers.calls[0]; got != kaganeImageID {
|
||||
t.Fatalf("browser cover fetched image id %q, want %q", got, kaganeImageID)
|
||||
if got := covers.calls[0]; got != kaganeCoverSrc {
|
||||
t.Fatalf("browser cover fetched URL %q, want %q", got, kaganeCoverSrc)
|
||||
}
|
||||
got := readBookmark(t, s, kaganeKey)
|
||||
if want := testCoverBaseURL + "/covers/" + store.CoverAddress(kaganeCoverSrc); got.Cover != want {
|
||||
|
||||
@@ -24,12 +24,6 @@ const challengeTimeout = 45 * time.Second
|
||||
|
||||
var kaganeSeriesRe = regexp.MustCompile(`^/series/([0-9a-f-]{36})/?$`)
|
||||
|
||||
// kaganeImageIDRe pins the only path segment Image interpolates into an
|
||||
// outbound URL. The id arrives from a stored cover URL, which a client
|
||||
// supplied, so it is matched rather than trusted: a headless browser is a
|
||||
// strong SSRF primitive.
|
||||
var kaganeImageIDRe = regexp.MustCompile(`^[0-9a-f-]{36}$`)
|
||||
|
||||
// BrowserFetcher retrieves pages through a remote headless Chrome over the
|
||||
// DevTools Protocol.
|
||||
//
|
||||
@@ -134,12 +128,14 @@ func (f *BrowserFetcher) Get(ctx context.Context, seriesURL string) (string, int
|
||||
return body, 200, nil
|
||||
}
|
||||
|
||||
// Image retrieves one kagane cover as raw bytes and its content type.
|
||||
// Image retrieves one cover's bytes through the browser sidecar, and its
|
||||
// content type.
|
||||
//
|
||||
// It exists because kagane serves covers behind the same challenge as its
|
||||
// pages *and* with `cross-origin-resource-policy: same-origin`, so an <img> on
|
||||
// the web UI's origin cannot load one even from a browser that already holds
|
||||
// the clearance cookie (verified 2026-08-08). Proxying is the only route.
|
||||
// pages *and* with `cross-origin-resource-policy: same-origin`, so the bytes
|
||||
// are only reachable from inside a browser that already holds the clearance
|
||||
// cookie (verified 2026-08-08). Acquisition through the sidecar is the only
|
||||
// route.
|
||||
//
|
||||
// The image URL is navigated to rather than fetched from some other kagane
|
||||
// page: the challenge only runs on a top-level navigation, and once it clears
|
||||
@@ -149,12 +145,14 @@ func (f *BrowserFetcher) Get(ctx context.Context, seriesURL string) (string, int
|
||||
// The challenge is not solved by the first read: WaitReady("body") is satisfied
|
||||
// by the interstitial too. run holds the tab open until the in-page fetch
|
||||
// succeeds, which is what gives the challenge script the seconds it needs.
|
||||
func (f *BrowserFetcher) Image(ctx context.Context, imageID string) ([]byte, string, error) {
|
||||
if !kaganeImageIDRe.MatchString(imageID) {
|
||||
return nil, "", fmt.Errorf("not a kagane image id: %q", imageID)
|
||||
func (f *BrowserFetcher) Image(ctx context.Context, imageURL string) ([]byte, string, error) {
|
||||
m := kaganeImageURLRe.FindStringSubmatch(imageURL)
|
||||
if m == nil {
|
||||
return nil, "", fmt.Errorf("not a browser-fetchable cover url: %q", imageURL)
|
||||
}
|
||||
imageID := m[1]
|
||||
var dataURL string
|
||||
err := f.run(ctx, "https://kagane.to/api/v2/image/"+imageID+"/compressed",
|
||||
err := f.run(ctx, imageURL,
|
||||
chromedp.Evaluate(`fetch(location.href).then(r => r.ok
|
||||
? r.blob().then(b => new Promise(res => {
|
||||
const fr = new FileReader();
|
||||
|
||||
@@ -22,22 +22,19 @@ type CoverBytesFetcher interface {
|
||||
Fetch(ctx context.Context, sourceURL string) (body []byte, contentType string, err error)
|
||||
}
|
||||
|
||||
// fetchCoverBytes routes a cover's byte retrieval by Site: only kagane needs
|
||||
// the browser for image bytes — its covers answer a plain fetch with a
|
||||
// challenge and `cross-origin-resource-policy: same-origin` — while every
|
||||
// other Site's CDN answers plain TLS. Missing fetchers degrade to an error the
|
||||
// caller logs, never a fallback onto a path that cannot succeed. One routing
|
||||
// rule for the poll and the acquirer, so the two cannot drift apart.
|
||||
func fetchCoverBytes(ctx context.Context, site, cover string, browser BrowserCoverFetcher, tls CoverBytesFetcher) ([]byte, string, error) {
|
||||
if site == "kagane" {
|
||||
// fetchCoverBytes routes a cover's byte retrieval by URL shape, not by Site
|
||||
// name: the browser fetcher's module claims the addresses only it can fetch
|
||||
// (kagane's image route answers a plain fetch with a challenge and
|
||||
// `cross-origin-resource-policy: same-origin`), and everything else goes over
|
||||
// plain TLS. Missing fetchers degrade to an error the caller logs, never a
|
||||
// fallback onto a path that cannot succeed. One routing rule for the poll and
|
||||
// the acquirer, so the two cannot drift apart.
|
||||
func fetchCoverBytes(ctx context.Context, cover string, browser BrowserCoverFetcher, tls CoverBytesFetcher) ([]byte, string, error) {
|
||||
if browserOnlyCoverURL(cover) {
|
||||
if browser == nil {
|
||||
return nil, "", errors.New("no cover fetcher")
|
||||
}
|
||||
imageID, ok := store.KaganeImageID(cover)
|
||||
if !ok {
|
||||
return nil, "", errors.New("invalid kagane cover URL")
|
||||
}
|
||||
return browser.Image(ctx, imageID)
|
||||
return browser.Image(ctx, cover)
|
||||
}
|
||||
if tls == nil {
|
||||
return nil, "", errors.New("no cover fetcher")
|
||||
|
||||
@@ -16,9 +16,11 @@ type Fetcher interface {
|
||||
Get(ctx context.Context, url string) (body string, status int, err error)
|
||||
}
|
||||
|
||||
// BrowserCoverFetcher retrieves one kagane cover through the browser-backed path.
|
||||
// BrowserCoverFetcher retrieves one cover's bytes through the browser-backed
|
||||
// path — the only route that clears the challenge kagane's image URLs answer
|
||||
// a plain fetch with. Satisfied by BrowserFetcher.
|
||||
type BrowserCoverFetcher interface {
|
||||
Image(ctx context.Context, imageID string) (body []byte, contentType string, err error)
|
||||
Image(ctx context.Context, imageURL string) (body []byte, contentType string, err error)
|
||||
}
|
||||
|
||||
// Poller re-checks each bookmarked series' newest published chapter on a
|
||||
@@ -44,8 +46,8 @@ type Poller struct {
|
||||
BrowserFetch Fetcher
|
||||
// CoverFetch is optional; failures are logged and never affect the chapter poll.
|
||||
CoverFetch BrowserCoverFetcher
|
||||
// CoverBytesFetch is optional; it handles non-kagane sources through the same
|
||||
// failure-isolated prefetch path.
|
||||
// CoverBytesFetch is optional; it handles plain-TLS sources through the
|
||||
// same failure-isolated prefetch path.
|
||||
CoverBytesFetch CoverBytesFetcher
|
||||
Now func() time.Time // injected so tests can freeze it
|
||||
Cooldown time.Duration
|
||||
@@ -81,9 +83,10 @@ func (p *Poller) fillBlankCover(ctx context.Context, sr store.Series, body strin
|
||||
|
||||
// prefetchCover heals Series that already carry a third-party source URL but
|
||||
// no stored address — the state left by client-supplied covers before
|
||||
// acquisition moved server-side. Every Site takes the same path; only the
|
||||
// byte fetcher differs (kagane needs the browser). New blanks have no source
|
||||
// URL and go through fillBlankCover from the series page instead.
|
||||
// acquisition moved server-side. Every Site takes the same path; fetchCoverBytes
|
||||
// routes by URL shape, so browser-claimed URLs still need the sidecar. New
|
||||
// blanks have no source URL and go through fillBlankCover from the series page
|
||||
// instead.
|
||||
func (p *Poller) prefetchCover(ctx context.Context, sr store.Series) {
|
||||
if sr.Cover == "" || sr.CoverAddress != "" {
|
||||
return
|
||||
@@ -106,7 +109,7 @@ func (p *Poller) prefetchCover(ctx context.Context, sr store.Series) {
|
||||
// failure is logged against the Series and swallowed so the chapter poll
|
||||
// cannot see it.
|
||||
func (p *Poller) storeCover(ctx context.Context, sr store.Series, sourceURL string) {
|
||||
bytes, contentType, err := fetchCoverBytes(ctx, sr.Site, sourceURL, p.CoverFetch, p.CoverBytesFetch)
|
||||
bytes, contentType, err := fetchCoverBytes(ctx, sourceURL, p.CoverFetch, p.CoverBytesFetch)
|
||||
if err != nil {
|
||||
log.Printf("latest poll %q: fetch cover %s: %v", sr.Key(), sourceURL, err)
|
||||
return
|
||||
|
||||
@@ -116,9 +116,9 @@ type fakeCoverFetcher struct {
|
||||
err error
|
||||
}
|
||||
|
||||
func (f *fakeCoverFetcher) Image(_ context.Context, imageID string) ([]byte, string, error) {
|
||||
func (f *fakeCoverFetcher) Image(_ context.Context, imageURL string) ([]byte, string, error) {
|
||||
f.mu.Lock()
|
||||
f.calls = append(f.calls, imageID)
|
||||
f.calls = append(f.calls, imageURL)
|
||||
f.mu.Unlock()
|
||||
if f.err != nil {
|
||||
return nil, "", f.err
|
||||
@@ -778,9 +778,9 @@ func TestRunOncePrefetchesKaganeCover(t *testing.T) {
|
||||
}
|
||||
p.runOnce(context.Background())
|
||||
|
||||
body, contentType, ok, err := s.GetKaganeCover("019f84bc-9ba0-7ed9-86f5-8b905ec7c28b")
|
||||
body, contentType, ok, err := s.CoverByAddress(store.CoverAddress(coverURL))
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("GetKaganeCover: %v found=%v", err, ok)
|
||||
t.Fatalf("CoverByAddress: %v found=%v", err, ok)
|
||||
}
|
||||
if string(body) != "cover-bytes" || contentType != "image/webp" {
|
||||
t.Fatalf("stored cover = (%q, %q), want (cover-bytes, image/webp)", body, contentType)
|
||||
@@ -877,7 +877,7 @@ func TestRunOnceRejectsInvalidKaganeCover(t *testing.T) {
|
||||
}
|
||||
p.runOnce(context.Background())
|
||||
|
||||
if _, _, found, err := s.GetKaganeCover("019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"); err != nil || found {
|
||||
if _, _, found, err := s.CoverByAddress(store.CoverAddress(coverURL)); err != nil || found {
|
||||
t.Fatalf("invalid cover persisted = %v, err %v; want missing", found, err)
|
||||
}
|
||||
}
|
||||
@@ -902,7 +902,7 @@ func TestRunOnceWithoutCoverFetcherStillPollsKagane(t *testing.T) {
|
||||
}
|
||||
p.runOnce(context.Background())
|
||||
|
||||
if _, _, found, err := s.GetKaganeCover("019f84bc-9ba0-7ed9-86f5-8b905ec7c28b"); err != nil || found {
|
||||
if _, _, found, err := s.CoverByAddress(store.CoverAddress(coverURL)); err != nil || found {
|
||||
t.Fatalf("cover after nil CoverFetch = found %v, err %v; want missing", found, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -165,6 +165,23 @@ var singleQuotedMetaAttrRe = regexp.MustCompile(`(?is)([a-z][a-z0-9:_-]*)\s*=\s*
|
||||
// the target detail entry avoids matching posters from recommended results.
|
||||
var comixInitialDataRe = regexp.MustCompile(`(?is)<script\b[^>]*\bid\s*=\s*["']initial-data["'][^>]*>(.*?)</script>`)
|
||||
|
||||
// kaganeImageURLRe matches the canonical compressed image route kagane's API
|
||||
// publishes — the only cover URL form the extractor emits and the browser
|
||||
// fetcher accepts. The URL is matched in full (scheme, host, id shape) rather
|
||||
// than trusted: the value a fetcher is pointed at may have been client-
|
||||
// supplied, and a headless browser is a strong SSRF primitive.
|
||||
var kaganeImageURLRe = regexp.MustCompile(`^https://kagane\.to/api/v2/image/([0-9a-f-]{36})/compressed$`)
|
||||
|
||||
// browserOnlyCoverURL reports whether the browser sidecar is the only fetcher
|
||||
// for cover bytes at imageURL. kagane's image route answers a plain fetch with
|
||||
// a challenge and `cross-origin-resource-policy: same-origin`, so a TLS fetch
|
||||
// would only ever retrieve a challenge page and must not be attempted
|
||||
// (ADR-0007). This is the byte-fetch router's per-Site knowledge; it lives in
|
||||
// the extraction module, which owns kagane's URL shapes.
|
||||
func browserOnlyCoverURL(imageURL string) bool {
|
||||
return kaganeImageURLRe.MatchString(imageURL)
|
||||
}
|
||||
|
||||
// kagane's browser-fetched series response publishes cover image IDs under
|
||||
// series_covers. The API's canonical compressed image route is the only URL
|
||||
// form accepted by the store and browser fetcher; no rendition is guessed.
|
||||
@@ -178,8 +195,12 @@ func kaganeCoverURL(body string) string {
|
||||
return ""
|
||||
}
|
||||
for _, cover := range response.SeriesCovers {
|
||||
if kaganeImageIDRe.MatchString(cover.ImageID) {
|
||||
return "https://kagane.to/api/v2/image/" + cover.ImageID + "/compressed"
|
||||
// Validate the assembled URL against the same regex the browser
|
||||
// fetcher enforces, so the extractor can never emit an address the
|
||||
// fetch would refuse.
|
||||
imageURL := "https://kagane.to/api/v2/image/" + cover.ImageID + "/compressed"
|
||||
if kaganeImageURLRe.MatchString(imageURL) {
|
||||
return imageURL
|
||||
}
|
||||
}
|
||||
return ""
|
||||
|
||||
@@ -10,9 +10,10 @@ import (
|
||||
"bookmarkmanager/backend/internal/store"
|
||||
)
|
||||
|
||||
// TestSmokeKaganeImage is the live proof that the cover proxy's fetch actually
|
||||
// clears Cloudflare and returns image bytes. It needs the real browser unit
|
||||
// with outbound network, so it runs only when SMOKE_BROWSER_WS_URL is set:
|
||||
// TestSmokeKaganeImage is the live proof that the acquisition path's browser
|
||||
// fetch actually clears Cloudflare and returns image bytes. It needs the real
|
||||
// browser unit with outbound network, so it runs only when SMOKE_BROWSER_WS_URL
|
||||
// is set:
|
||||
//
|
||||
// cd chrome && BROWSER_BIND_ADDR=127.0.0.1 docker compose up -d --build
|
||||
// SMOKE_BROWSER_WS_URL=ws://127.0.0.1:9222 go test -run TestSmokeKaganeImage ./internal/latest
|
||||
@@ -24,12 +25,11 @@ func TestSmokeKaganeImage(t *testing.T) {
|
||||
if ws == "" {
|
||||
t.Skip("SMOKE_BROWSER_WS_URL unset")
|
||||
}
|
||||
const imageID = "019fe11a-84c3-7fc3-a84b-88787374b617" // SP Baby's cover
|
||||
const imageURL = "https://kagane.to/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed" // SP Baby's cover
|
||||
|
||||
// The same URL through a plain client is what the web UI's <img> gets.
|
||||
// The same URL through a plain client is what any other fetcher would get.
|
||||
// Asserting on it keeps the test honest about why the browser is needed.
|
||||
req, err := http.NewRequest(http.MethodGet,
|
||||
"https://kagane.to/api/v2/image/"+imageID+"/compressed", nil)
|
||||
req, err := http.NewRequest(http.MethodGet, imageURL, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -48,7 +48,7 @@ func TestSmokeKaganeImage(t *testing.T) {
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
body, contentType, err := f.Image(ctx, imageID)
|
||||
body, contentType, err := f.Image(ctx, imageURL)
|
||||
if err != nil {
|
||||
t.Fatalf("Image: %v", err)
|
||||
}
|
||||
@@ -64,8 +64,10 @@ func TestSmokeKaganeImage(t *testing.T) {
|
||||
}
|
||||
t.Logf("fetched %d bytes of %s", len(body), contentType)
|
||||
|
||||
if _, _, err := f.Image(ctx, "not-a-uuid"); err == nil {
|
||||
t.Fatal("Image accepted a non-uuid id")
|
||||
// The browser module claims only the cover URL shape it can clear a
|
||||
// challenge for; anything else must be refused before any navigation.
|
||||
if _, _, err := f.Image(ctx, "https://cdn.example/cover.jpg"); err == nil {
|
||||
t.Fatal("Image accepted a cover URL the browser module does not claim")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -40,10 +40,6 @@ type Bookmark struct {
|
||||
// address and never an address that 404s (ADR-0007). A client may still
|
||||
// send this field and it is discarded on the way in; see Upsert.
|
||||
Cover string `json:"cover"`
|
||||
// CoverSource is the third-party address the bytes were fetched from. It
|
||||
// stays off the wire: it is the acquisition path's dedupe key, and no
|
||||
// client is ever asked to render one.
|
||||
CoverSource string `json:"-"`
|
||||
LastChapter string `json:"last_chapter"`
|
||||
LastChapterNum float64 `json:"last_chapter_num"`
|
||||
LastChapterURL string `json:"last_chapter_url"`
|
||||
@@ -155,20 +151,6 @@ func (b Bookmark) Initial() string {
|
||||
return "?"
|
||||
}
|
||||
|
||||
// kaganeCoverRe matches the cover URL kagane's og:image carries, which is what
|
||||
// the userscript stores for that site.
|
||||
var kaganeCoverRe = regexp.MustCompile(`^https://kagane\.to/api/v2/image/([0-9a-f-]{36})/compressed$`)
|
||||
|
||||
// KaganeImageID extracts the validated image id from the cover URL recorded by
|
||||
// the userscript.
|
||||
func KaganeImageID(cover string) (string, bool) {
|
||||
m := kaganeCoverRe.FindStringSubmatch(cover)
|
||||
if m == nil {
|
||||
return "", false
|
||||
}
|
||||
return m[1], true
|
||||
}
|
||||
|
||||
// CoverContentType canonicalises a fetched response's media type and reports
|
||||
// whether the bytes are safe to store and serve. comix answers "image/jpg",
|
||||
// which no standard lists but browsers accept; it is stored as the real name
|
||||
@@ -184,17 +166,6 @@ func CoverContentType(contentType string) (string, bool) {
|
||||
}
|
||||
}
|
||||
|
||||
// CoverURL is the src the web UI puts in an <img>. Cover already is an address
|
||||
// on this origin, so for every site but kagane it is used as-is. kagane's
|
||||
// bytes still arrive through the browser-backed proxy, which is keyed by image
|
||||
// id rather than by content address until #62 moves it onto the same path.
|
||||
func (b Bookmark) CoverURL() string {
|
||||
if imageID, ok := KaganeImageID(b.CoverSource); ok {
|
||||
return "/img/kagane/" + imageID
|
||||
}
|
||||
return b.Cover
|
||||
}
|
||||
|
||||
// Library buckets. A bookmark is in exactly one. This cannot be derived from
|
||||
// Site: asurascans serves manga and novels from the same /comics/ path, so the
|
||||
// userscript that recorded the page is the only party that knows which.
|
||||
@@ -217,7 +188,7 @@ var migrations embed.FS
|
||||
// compile-time constant; every request value is bound as a parameter. The
|
||||
// series-owned fields are joined in from the series table, in scanBookmark
|
||||
// order, so the flat Bookmark reads back whole despite the split (ADR-0004).
|
||||
const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover, s.cover_address,
|
||||
const bookmarkColumns = `b.site, b.series_id, s.title, s.series_url, s.cover_address,
|
||||
b.last_chapter, b.last_chapter_num, b.last_chapter_url,
|
||||
b.favorite, s.latest_chapter, s.latest_chapter_num, b.updated_at, b.status, s.kind`
|
||||
|
||||
@@ -562,7 +533,7 @@ func (s *Store) scanBookmark(scan func(...any) error) (Bookmark, error) {
|
||||
latestChapterNum sql.NullFloat64
|
||||
)
|
||||
if err := scan(
|
||||
&b.Site, &b.SeriesID, &b.Title, &b.SeriesURL, &b.CoverSource, &coverAddress,
|
||||
&b.Site, &b.SeriesID, &b.Title, &b.SeriesURL, &coverAddress,
|
||||
&b.LastChapter, &b.LastChapterNum, &b.LastChapterURL,
|
||||
&b.Favorite, &b.LatestChapter, &latestChapterNum, &b.UpdatedAt, &b.Status, &b.Kind,
|
||||
); err != nil {
|
||||
@@ -617,10 +588,6 @@ func coverRelativePath(address string) string {
|
||||
return address[:2] + "/" + address[2:4] + "/" + address
|
||||
}
|
||||
|
||||
func kaganeCoverSourceURL(imageID string) string {
|
||||
return "https://kagane.to/api/v2/image/" + imageID + "/compressed"
|
||||
}
|
||||
|
||||
func (s *Store) getCover(sourceURL string) ([]byte, string, bool, error) {
|
||||
return s.getCoverByAddress(coverSourceAddress(sourceURL))
|
||||
}
|
||||
@@ -703,18 +670,6 @@ func (s *Store) PutCover(sourceURL string, body []byte, contentType string) erro
|
||||
return s.putCover(sourceURL, body, contentType)
|
||||
}
|
||||
|
||||
// GetKaganeCover returns one persisted cover. Missing covers are reported with
|
||||
// ok=false rather than as an error so the web handler can fetch them once.
|
||||
func (s *Store) GetKaganeCover(imageID string) ([]byte, string, bool, error) {
|
||||
return s.getCover(kaganeCoverSourceURL(imageID))
|
||||
}
|
||||
|
||||
// PutKaganeCover persists one fetched cover. The source URL's content address
|
||||
// makes each stored object immutable, so later writes for that URL are ignored.
|
||||
func (s *Store) PutKaganeCover(imageID string, body []byte, contentType string) error {
|
||||
return s.putCover(kaganeCoverSourceURL(imageID), body, contentType)
|
||||
}
|
||||
|
||||
// CoverAddress is the content address bytes fetched from sourceURL are stored
|
||||
// under. It is a pure function of the URL, so the acquisition path can name a
|
||||
// Cover before it has the bytes.
|
||||
|
||||
@@ -550,45 +550,6 @@ func TestDisplayChapter(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// CoverURL reads the source address for the kagane branch and the wire value
|
||||
// otherwise, so both are set the way scanBookmark sets them.
|
||||
func TestCoverURL(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
coverSource string
|
||||
cover string
|
||||
want string
|
||||
}{
|
||||
{
|
||||
"kagane routes through the proxy",
|
||||
"https://kagane.to/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed",
|
||||
"https://bookmarks.test/covers/" + CoverAddress("kagane"),
|
||||
"/img/kagane/019fe11a-84c3-7fc3-a84b-88787374b617",
|
||||
},
|
||||
{
|
||||
"another site is served from our own origin",
|
||||
"https://gg.asuracomic.net/storage/media/1/conversions/cover.webp",
|
||||
"https://bookmarks.test/covers/" + CoverAddress("asura"),
|
||||
"https://bookmarks.test/covers/" + CoverAddress("asura"),
|
||||
},
|
||||
{
|
||||
"a lookalike host is not rewritten",
|
||||
"https://evil.example/api/v2/image/019fe11a-84c3-7fc3-a84b-88787374b617/compressed",
|
||||
"https://bookmarks.test/covers/" + CoverAddress("evil"),
|
||||
"https://bookmarks.test/covers/" + CoverAddress("evil"),
|
||||
},
|
||||
{"no cover stays empty", "", "", ""},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
b := Bookmark{CoverSource: tc.coverSource, Cover: tc.cover}
|
||||
if got := b.CoverURL(); got != tc.want {
|
||||
t.Errorf("CoverURL() = %q, want %q", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpsertKindDefaultsToManga(t *testing.T) {
|
||||
store := newTestStore(t)
|
||||
got, err := store.Upsert(store.OwnerID(), Bookmark{
|
||||
@@ -1397,7 +1358,7 @@ func TestTwoReadersShareOneSeriesWithIndependentProgress(t *testing.T) {
|
||||
t.Fatalf("due after one Reader left = %+v, want the series still polled", due)
|
||||
}
|
||||
}
|
||||
func TestKaganeCoverPersistsAcrossReopen(t *testing.T) {
|
||||
func TestCoverPersistsAcrossReopen(t *testing.T) {
|
||||
url := pgtest.URL(t)
|
||||
coverDir := t.TempDir()
|
||||
first, err := Open(url, testOwner, coverDir, testCoverBaseURL)
|
||||
@@ -1405,8 +1366,9 @@ func TestKaganeCoverPersistsAcrossReopen(t *testing.T) {
|
||||
t.Fatalf("Open: %v", err)
|
||||
}
|
||||
body := []byte("stored-cover")
|
||||
if err := first.PutKaganeCover("019fe11a-84c3-7fc3-a84b-88787374b617", body, "image/webp"); err != nil {
|
||||
t.Fatalf("PutKaganeCover: %v", err)
|
||||
const sourceURL = "https://cdn.example/covers/series.jpg"
|
||||
if err := first.PutCover(sourceURL, body, "image/webp"); err != nil {
|
||||
t.Fatalf("PutCover: %v", err)
|
||||
}
|
||||
if err := first.Close(); err != nil {
|
||||
t.Fatalf("close first store: %v", err)
|
||||
@@ -1417,9 +1379,9 @@ func TestKaganeCoverPersistsAcrossReopen(t *testing.T) {
|
||||
t.Fatalf("reopen: %v", err)
|
||||
}
|
||||
defer second.Close()
|
||||
got, contentType, ok, err := second.GetKaganeCover("019fe11a-84c3-7fc3-a84b-88787374b617")
|
||||
got, contentType, ok, err := second.GetCover(sourceURL)
|
||||
if err != nil {
|
||||
t.Fatalf("GetKaganeCover: %v", err)
|
||||
t.Fatalf("GetCover: %v", err)
|
||||
}
|
||||
if !ok || !bytes.Equal(got, body) || contentType != "image/webp" {
|
||||
t.Fatalf("stored cover = (%q, %q, %v), want (%q, image/webp, true)", got, contentType, ok, body)
|
||||
@@ -1443,7 +1405,7 @@ func TestOpenRequiresAbsoluteCoverBaseURL(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestKaganeCoverIsContentAddressedOnFilesystem(t *testing.T) {
|
||||
func TestCoverIsContentAddressedOnFilesystem(t *testing.T) {
|
||||
url := pgtest.URL(t)
|
||||
coverDir := t.TempDir()
|
||||
first, err := Open(url, testOwner, coverDir, testCoverBaseURL)
|
||||
@@ -1451,14 +1413,13 @@ func TestKaganeCoverIsContentAddressedOnFilesystem(t *testing.T) {
|
||||
t.Fatalf("Open: %v", err)
|
||||
}
|
||||
body := []byte("stored-cover")
|
||||
const imageID = "019fe11a-84c3-7fc3-a84b-88787374b617"
|
||||
if err := first.PutKaganeCover(imageID, body, "image/webp"); err != nil {
|
||||
const sourceURL = "https://cdn.example/covers/series.jpg"
|
||||
if err := first.PutCover(sourceURL, body, "image/webp"); err != nil {
|
||||
first.Close()
|
||||
t.Fatalf("PutKaganeCover: %v", err)
|
||||
t.Fatalf("PutCover: %v", err)
|
||||
}
|
||||
defer first.Close()
|
||||
|
||||
sourceURL := "https://kagane.to/api/v2/image/" + imageID + "/compressed"
|
||||
addressBytes := sha256.Sum256([]byte(sourceURL))
|
||||
address := hex.EncodeToString(addressBytes[:])
|
||||
wantPath := filepath.Join(address[:2], address[2:4], address)
|
||||
|
||||
@@ -1,89 +0,0 @@
|
||||
package web
|
||||
|
||||
import (
|
||||
"bookmarkmanager/backend/internal/store"
|
||||
"context"
|
||||
"log"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"time"
|
||||
)
|
||||
|
||||
// CoverFetcher retrieves one kagane cover by image id. Satisfied by
|
||||
// latest.BrowserFetcher. It is nil when BROWSER_WS_URL is unset; uncached
|
||||
// covers are then unavailable, while covers already stored by the backend
|
||||
// remain available without a browser.
|
||||
type CoverFetcher interface {
|
||||
Image(ctx context.Context, imageID string) (body []byte, contentType string, err error)
|
||||
}
|
||||
|
||||
// coverIDRe matches the request path segment that becomes part of an outbound
|
||||
// URL. The proxy is session-gated, but the id still reaches a headless browser,
|
||||
// so it is validated at the boundary rather than passed through.
|
||||
var coverIDRe = regexp.MustCompile(`^[0-9a-f-]{36}$`)
|
||||
|
||||
// coverTimeout bounds one proxied cover. Shorter than the fetcher's own
|
||||
// challenge budget on purpose: a browser page is waiting on this, and a cover
|
||||
// that has not arrived by now is better left as a broken slot than as a request
|
||||
// holding a connection open.
|
||||
const coverTimeout = 20 * time.Second
|
||||
|
||||
// kaganeCover serves a kagane cover from the backend's own origin.
|
||||
//
|
||||
// kagane answers image requests with a Cloudflare challenge and
|
||||
// `cross-origin-resource-policy: same-origin`, so the web UI cannot render one
|
||||
// directly under any combination of referrer policy or crossorigin attribute
|
||||
// (verified 2026-08-08). Fetching it through the headless browser that already
|
||||
// clears the challenge, and re-serving it here, is what puts the bytes on an
|
||||
// origin the page may load from.
|
||||
func (h *Handler) kaganeCover(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
if !coverIDRe.MatchString(id) {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
body, contentType, ok, err := h.store.GetKaganeCover(id)
|
||||
if err != nil {
|
||||
log.Printf("read kagane cover %s: %v", id, err)
|
||||
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
if ok {
|
||||
writeCover(w, body, contentType)
|
||||
return
|
||||
}
|
||||
if h.covers == nil {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(r.Context(), coverTimeout)
|
||||
defer cancel()
|
||||
body, contentType, err = h.covers.Image(ctx, id)
|
||||
if err != nil {
|
||||
log.Printf("kagane cover %s: %v", id, err)
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
canonical, ok := store.CoverContentType(contentType)
|
||||
if !ok {
|
||||
log.Printf("kagane cover %s: unexpected content type %q", id, contentType)
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
contentType = canonical
|
||||
if err := h.store.PutKaganeCover(id, body, contentType); err != nil {
|
||||
log.Printf("persist kagane cover %s: %v", id, err)
|
||||
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
writeCover(w, body, contentType)
|
||||
}
|
||||
|
||||
// writeCover sends the bytes with a long cache life: an image id names one
|
||||
// immutable rendering, so a client that has it never needs to ask again.
|
||||
func writeCover(w http.ResponseWriter, body []byte, contentType string) {
|
||||
w.Header().Set("Content-Type", contentType)
|
||||
w.Header().Set("Cache-Control", "private, max-age=604800, immutable")
|
||||
w.Write(body)
|
||||
}
|
||||
@@ -6,7 +6,7 @@
|
||||
<div class="row">
|
||||
<a class="cover" href="{{.ContinueURL}}" target="_blank" rel="noopener noreferrer"
|
||||
tabindex="-1" aria-hidden="true">
|
||||
{{if .CoverURL}}<img src="{{.CoverURL}}" alt="" loading="lazy">
|
||||
{{if .Cover}}<img src="{{.Cover}}" alt="" loading="lazy">
|
||||
{{/* aria-hidden on the cover link is not enough — Chromium still exposes
|
||||
the letter because the link is programmatically focusable — so the
|
||||
monogram carries its own, same as the recent strip's. */}}
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
<a class="recent-card {{if .HasNewChapter}}is-new{{end}}" href="{{.ContinueURL}}"
|
||||
target="_blank" rel="noopener noreferrer">
|
||||
<span class="recent-cover">
|
||||
{{if .CoverURL}}<img src="{{.CoverURL}}" alt="" loading="lazy">
|
||||
{{if .Cover}}<img src="{{.Cover}}" alt="" loading="lazy">
|
||||
{{else}}<span class="monogram" aria-hidden="true">{{.Initial}}</span>{{end}}
|
||||
{{if .HasNewChapter}}<span class="foot-rule"></span>
|
||||
{{else if .Favorite}}<span class="foot-rule brass"></span>{{end}}
|
||||
|
||||
@@ -50,9 +50,6 @@ type Handler struct {
|
||||
// httpClient is the plain stdlib client that talks to Discord. It is not
|
||||
// an injected interface: tests point APIBase at a stub server instead.
|
||||
httpClient *http.Client
|
||||
// covers proxies kagane cover images, which no browser can load directly.
|
||||
// Nil disables the endpoint — see CoverFetcher.
|
||||
covers CoverFetcher
|
||||
}
|
||||
|
||||
// listView is what every list-rendering template receives.
|
||||
@@ -114,7 +111,7 @@ type loginView struct {
|
||||
|
||||
// New parses every template up front so a broken one kills the process at
|
||||
// startup rather than the first request that touches it.
|
||||
func New(s *store.Store, discord DiscordConfig, tokenKey []byte, mangaPath, novelPath string, covers CoverFetcher) (*Handler, error) {
|
||||
func New(s *store.Store, discord DiscordConfig, tokenKey []byte, mangaPath, novelPath string) (*Handler, error) {
|
||||
tmpl, err := template.ParseFS(templateFS, "templates/*.html")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -129,7 +126,6 @@ func New(s *store.Store, discord DiscordConfig, tokenKey []byte, mangaPath, nove
|
||||
states: newOAuthStates(),
|
||||
limiter: session.NewLoginLimiter(),
|
||||
httpClient: &http.Client{Timeout: discordTimeout},
|
||||
covers: covers,
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -146,10 +142,6 @@ func (h *Handler) Register(mux *http.ServeMux) {
|
||||
mux.HandleFunc("POST /ui/bookmarks/{key}/chapter", h.requireSession(h.uiChapter))
|
||||
mux.HandleFunc("DELETE /ui/bookmarks/{key}", h.requireSession(h.uiDelete))
|
||||
|
||||
// Session-gated like every other UI route: the deployment proxies kagane's
|
||||
// images for its own Readers, not for the internet.
|
||||
mux.HandleFunc("GET /img/kagane/{id}", h.requireSession(h.kaganeCover))
|
||||
|
||||
// Install endpoints render the script directly under the session: the
|
||||
// credential travels inside the served bytes, never in the address bar or
|
||||
// the page markup. Updates after install use the credential-bearing /u/
|
||||
|
||||
+4
-9
@@ -57,10 +57,6 @@ type Config struct {
|
||||
NovelUserscriptPath string
|
||||
// LatestPoll configures the background latest-chapter fetcher.
|
||||
LatestPoll LatestPoll
|
||||
// Covers proxies kagane cover images for the web UI. Not from the
|
||||
// environment: it is the shared headless browser, wired in main once it
|
||||
// connects, and nil in every test router.
|
||||
Covers web.CoverFetcher
|
||||
}
|
||||
|
||||
// LatestPoll configures the background latest-chapter poller.
|
||||
@@ -235,7 +231,7 @@ func newRouter(s *store.Store, cfg Config) http.Handler {
|
||||
// The browser UI is always registered; signing in is Discord OAuth, so
|
||||
// there is no password to forget and no gate to leave unset.
|
||||
wh, err := web.New(s, cfg.Discord, []byte(cfg.TokenKey),
|
||||
cfg.UserscriptPath, cfg.NovelUserscriptPath, cfg.Covers)
|
||||
cfg.UserscriptPath, cfg.NovelUserscriptPath)
|
||||
if err != nil {
|
||||
log.Fatalf("web handler: %v", err)
|
||||
}
|
||||
@@ -307,9 +303,9 @@ func main() {
|
||||
// chapters on its own.
|
||||
//
|
||||
// One headless browser serves both consumers that need a Cloudflare
|
||||
// challenge cleared: the poller's kagane/novelfull fetches and the web
|
||||
// UI's kagane cover proxy. Optional — unset leaves both degraded to what
|
||||
// they were before the sidecar existed.
|
||||
// challenge cleared: the poller's kagane/novelfull page fetches and
|
||||
// kagane's cover bytes. Optional — unset leaves kagane unpolled and its
|
||||
// Covers blank until the bytes exist.
|
||||
var browser latest.Fetcher
|
||||
pollCtx, stopPoll := context.WithCancel(context.Background())
|
||||
defer stopPoll()
|
||||
@@ -319,7 +315,6 @@ func main() {
|
||||
log.Printf("browser fetcher disabled: %v", err)
|
||||
} else {
|
||||
browser = bf
|
||||
cfg.Covers = bf
|
||||
context.AfterFunc(pollCtx, bf.Close)
|
||||
log.Printf("browser fetcher at %s", ws)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user