Add comix.to and kagane.to support (#13)
Tracks read progress on comix.to and kagane.to alongside asura and demonic, in both the userscript and the backend. Implements `docs/superpowers/plans/2026-08-03-comix-kagane-support.md`. ## Userscript - `comix` adapter — `/title/<id>-<slug>`; only the id prefix is identity (the slug follows the title). No `og:image`, so the cover is matched by `alt`. - `kagane` adapter — reader URLs are uuids with no chapter number, so it comes out of `og:title`; anchor scanning is structurally impossible, replaced by `latestChapterFromApi` against kagane's same-origin JSON API. - `seriesId` threaded through `latestChapterFromAnchors` so comix can scope its scan to its own series and a recommendation strip cannot win the maximum. - `@match` for both hosts, panel chips, v1.6.0. ## Backend - `latestChapterFrom` cases: comix parses the SSR JSON state blob (`latestChapterUrl`, scoped to the series id); kagane parses API JSON (`chapter_no`). - Poller allowlist extended; `Poller.BrowserFetch` with `fetcherFor(site)` routes kagane to a browser fetcher. Nil means kagane is not polled at all — never a fallback to the TLS fetcher, which would only ever retrieve a challenge page. - `BrowserFetcher`: chromedp against a `headless-shell` sidecar. kagane sits behind a Cloudflare JS challenge that no TLS fingerprint clears, and the request is made inside the page rather than by replaying `cf_clearance`. - `BROWSER_WS_URL` wiring, sidecar in both compose files (no `ports:`, dedicated non-external network), Dockerfile on `golang:1.26-alpine` — chromedp requires go 1.26. - Web UI `--comix` / `--kagane` tokens in both colour branches. ## Notes for review - `series_url` is client-supplied and a headless browser is a strong SSRF primitive, so kagane's host is pinned twice: in `fetchableSeriesURL` and again in `kaganeAPIURL`. - Three chained defects found during verification made the browser path dead under Compose (sidecar flag collision, Chrome's Host-header DNS-rebinding check, the wrong chromedp option). Fixed; the compose comments record the wrong configurations too, so they don't get "simplified" back. - `ALLOWED_ORIGINS` now includes both new origins. Without it every write from comix/kagane silently fails CORS preflight, parks in the retry queue, and drops at the cap. ## Verification 221 backend tests, 32 userscript tests, static `CGO_ENABLED=0` build, both compose configs. Two gaps, both real: 1. The userscript on live pages via Violentmonkey needs a human browser profile — not run. Check: comix series page (title/cover, no chapter), comix chapter page (records the number; an *older* chapter must not regress it), comix SPA navigation without reload, kagane series page (og:image cover), kagane reader (number from `og:title`), both chips opening the right sites. 2. The kagane browser path has not completed end-to-end anywhere. Dial/navigate/fetch is confirmed, but Cloudflare 403'd headless-shell's Chrome on every attempt from the dev sandbox, and comix's poll-through-Docker was blocked by that environment's TLS interception. Both environment-dependent rather than branch defects — the first real deploy is the actual verification. Reviewed-on: #13 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #13.
This commit is contained in:
+50
-28
@@ -11,6 +11,13 @@ import (
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"mangabm/backend/internal/api"
|
||||
"mangabm/backend/internal/httpmw"
|
||||
"mangabm/backend/internal/latest"
|
||||
"mangabm/backend/internal/store"
|
||||
"mangabm/backend/internal/userscript"
|
||||
"mangabm/backend/internal/web"
|
||||
)
|
||||
|
||||
// Config holds all runtime settings, sourced from environment variables.
|
||||
@@ -149,22 +156,22 @@ func loadConfig() Config {
|
||||
// newRouter wires routes and middleware. CORS is the outermost layer so
|
||||
// preflight OPTIONS short-circuits before auth; /bookmarks* is auth-protected,
|
||||
// /healthz is public.
|
||||
func newRouter(store *Store, cfg Config) http.Handler {
|
||||
func newRouter(s *store.Store, cfg Config) http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("GET /healthz", healthz)
|
||||
mux.HandleFunc("GET /healthz", api.Healthz)
|
||||
|
||||
// Outside withAuth (the updater sends no Authorization header) and outside
|
||||
// the WEB_PASSWORD gate (the script must be installable either way). The
|
||||
// path segment carries the token instead.
|
||||
mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", userscriptHandler(cfg.Token, cfg.UserscriptPath))
|
||||
// Outside httpmw.Auth (the updater sends no Authorization header) and
|
||||
// outside the WEB_PASSWORD gate (the script must be installable either
|
||||
// way). The path segment carries the token instead.
|
||||
mux.HandleFunc("GET /u/{token}/manga-bookmark.user.js", userscript.Handler(cfg.Token, cfg.UserscriptPath))
|
||||
|
||||
h := &bookmarkHandler{store: store}
|
||||
h := &api.Handler{Store: s}
|
||||
protected := http.NewServeMux()
|
||||
protected.HandleFunc("GET /bookmarks", h.list)
|
||||
protected.HandleFunc("PUT /bookmarks/{key}", h.put)
|
||||
protected.HandleFunc("DELETE /bookmarks/{key}", h.delete)
|
||||
protected.HandleFunc("GET /bookmarks", h.List)
|
||||
protected.HandleFunc("PUT /bookmarks/{key}", h.Put)
|
||||
protected.HandleFunc("DELETE /bookmarks/{key}", h.Delete)
|
||||
|
||||
auth := withAuth(cfg.Token, protected)
|
||||
auth := httpmw.Auth(cfg.Token, protected)
|
||||
mux.Handle("/bookmarks", auth)
|
||||
mux.Handle("/bookmarks/", auth)
|
||||
|
||||
@@ -172,14 +179,14 @@ func newRouter(store *Store, cfg Config) http.Handler {
|
||||
// deployment that forgets WEB_PASSWORD exposes nothing rather than
|
||||
// exposing an unprotected list.
|
||||
if cfg.WebPassword != "" {
|
||||
web, err := newWebHandler(store, cfg)
|
||||
wh, err := web.New(s, cfg.Token, cfg.WebPassword)
|
||||
if err != nil {
|
||||
log.Fatalf("web handler: %v", err)
|
||||
}
|
||||
web.register(mux)
|
||||
wh.Register(mux)
|
||||
}
|
||||
|
||||
return withCORS(cfg.AllowedOrigins, withGzip(guardEmptyUserscriptToken(mux)))
|
||||
return httpmw.CORS(cfg.AllowedOrigins, httpmw.Gzip(guardEmptyUserscriptToken(mux)))
|
||||
}
|
||||
|
||||
// guardEmptyUserscriptToken heads off ServeMux's own path-cleaning redirect:
|
||||
@@ -203,22 +210,22 @@ func main() {
|
||||
log.Fatal("API_TOKEN is required")
|
||||
}
|
||||
|
||||
store, err := OpenStore(cfg.DBPath)
|
||||
s, err := store.Open(cfg.DBPath)
|
||||
if err != nil {
|
||||
log.Fatalf("open store: %v", err)
|
||||
}
|
||||
defer store.Close()
|
||||
defer s.Close()
|
||||
|
||||
// The poller is off the request path entirely: if it cannot start, the
|
||||
// service still serves bookmarks and the userscript still captures latest
|
||||
// chapters on its own.
|
||||
pollCtx, stopPoll := context.WithCancel(context.Background())
|
||||
defer stopPoll()
|
||||
startLatestPoller(pollCtx, store, cfg.LatestPoll)
|
||||
startLatestPoller(pollCtx, s, cfg.LatestPoll)
|
||||
|
||||
srv := &http.Server{
|
||||
Addr: ":" + cfg.Port,
|
||||
Handler: newRouter(store, cfg),
|
||||
Handler: newRouter(s, cfg),
|
||||
ReadHeaderTimeout: 10 * time.Second,
|
||||
}
|
||||
|
||||
@@ -248,24 +255,39 @@ func main() {
|
||||
// HTTP client cannot be built. Any problem here is logged and skipped: this
|
||||
// feature going missing degrades the service to userscript-only latest-chapter
|
||||
// tracking, which is exactly how it behaved before.
|
||||
func startLatestPoller(ctx context.Context, store *Store, cfg LatestPoll) {
|
||||
func startLatestPoller(ctx context.Context, s *store.Store, cfg LatestPoll) {
|
||||
if !cfg.Enabled {
|
||||
log.Println("latest-chapter poller: disabled by config")
|
||||
return
|
||||
}
|
||||
f, err := newTLSFetcher()
|
||||
f, err := latest.NewTLSFetcher()
|
||||
if err != nil {
|
||||
log.Printf("latest-chapter poller: disabled, cannot build client: %v", err)
|
||||
return
|
||||
}
|
||||
p := &latestPoller{
|
||||
store: store,
|
||||
fetch: f,
|
||||
now: time.Now,
|
||||
cooldown: cfg.Cooldown,
|
||||
interval: cfg.Interval,
|
||||
stagger: cfg.Stagger,
|
||||
batch: cfg.Batch,
|
||||
p := &latest.Poller{
|
||||
Store: s,
|
||||
Fetch: f,
|
||||
Now: time.Now,
|
||||
Cooldown: cfg.Cooldown,
|
||||
Interval: cfg.Interval,
|
||||
Stagger: cfg.Stagger,
|
||||
Batch: cfg.Batch,
|
||||
}
|
||||
|
||||
// Optional: without it, sites behind a JavaScript challenge are simply not
|
||||
// polled, and their latest_chapter comes from the userscript alone — which
|
||||
// is how the service behaved before the sidecar existed.
|
||||
if ws := strings.TrimSpace(os.Getenv("BROWSER_WS_URL")); ws != "" {
|
||||
bf, err := latest.NewBrowserFetcher(ws)
|
||||
if err != nil {
|
||||
log.Printf("latest-chapter poller: browser fetcher disabled: %v", err)
|
||||
} else {
|
||||
p.BrowserFetch = bf
|
||||
context.AfterFunc(ctx, bf.Close)
|
||||
log.Printf("latest-chapter poller: browser fetcher at %s", ws)
|
||||
}
|
||||
}
|
||||
|
||||
go p.Run(ctx)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user