Address review findings on the cover-path deletion (#63)
- Move the kagane cover URL shape into the extraction module (sites.go):
browserOnlyCoverURL + kaganeImageURLRe now own the claim; the byte-fetch
router and BrowserFetcher.Image reference it. One shape gate for producer
and fetcher (the id regex is folded into the full-URL match), so no Site
name appears in a cover path outside the extraction module and the
producer cannot emit an address the fetch would refuse.
- Restore the serving-boundary guarantee: GET /covers/{addr} re-checks the
stored media type via store.CoverContentType and 404s a poisoned row;
TestPublicCoverNeverEchoesNonImage now seeds one directly behind the
write gate and pins the refusal where bytes leave.
- Restore the SSRF rationale (client-supplied stored URL, headless browser
as a strong primitive) on the URL regex.
This commit is contained in:
@@ -150,6 +150,14 @@ func (h *Handler) Cover(w http.ResponseWriter, r *http.Request) {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
// Refuse anything the write gate would not have recorded: a poisoned row
|
||||
// (migrated data, a writer that skips the gate) must never be echoed back
|
||||
// as bytes of a type no Cover may have.
|
||||
if _, ok := store.CoverContentType(contentType); !ok {
|
||||
log.Printf("cover %s: refusing non-image content type %q", r.PathValue("address"), contentType)
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", contentType)
|
||||
// Content-addressed, so the bytes at this URL can never change. Public
|
||||
// rather than private: no credential gates the route.
|
||||
|
||||
@@ -24,26 +24,6 @@ const challengeTimeout = 45 * time.Second
|
||||
|
||||
var kaganeSeriesRe = regexp.MustCompile(`^/series/([0-9a-f-]{36})/?$`)
|
||||
|
||||
// kaganeImageIDRe pins the only image id the kagane extractor accepts. It
|
||||
// arrives from the browser-fetched API body, so it is matched rather than
|
||||
// trusted.
|
||||
var kaganeImageIDRe = regexp.MustCompile(`^[0-9a-f-]{36}$`)
|
||||
|
||||
// kaganeImageURLRe matches the only cover URL Image fetches: the canonical
|
||||
// compressed image route kagane's API publishes. It doubles as the byte-fetch
|
||||
// router's claim check (fetchCoverBytes) — an address of this shape answers a
|
||||
// plain fetch with a challenge, so the browser is the only route for it.
|
||||
var kaganeImageURLRe = regexp.MustCompile(`^https://kagane\.to/api/v2/image/([0-9a-f-]{36})/compressed$`)
|
||||
|
||||
// browserCoverURL reports whether the browser sidecar is the only fetcher for
|
||||
// cover bytes at imageURL. kagane serves them behind the same challenge as its
|
||||
// pages, so a plain TLS fetch would only ever retrieve a challenge page and
|
||||
// must not be attempted (ADR-0007). Per-Site knowledge, kept in the browser
|
||||
// module with the rest of it.
|
||||
func browserCoverURL(imageURL string) bool {
|
||||
return kaganeImageURLRe.MatchString(imageURL)
|
||||
}
|
||||
|
||||
// BrowserFetcher retrieves pages through a remote headless Chrome over the
|
||||
// DevTools Protocol.
|
||||
//
|
||||
|
||||
@@ -30,7 +30,7 @@ type CoverBytesFetcher interface {
|
||||
// fallback onto a path that cannot succeed. One routing rule for the poll and
|
||||
// the acquirer, so the two cannot drift apart.
|
||||
func fetchCoverBytes(ctx context.Context, cover string, browser BrowserCoverFetcher, tls CoverBytesFetcher) ([]byte, string, error) {
|
||||
if browserCoverURL(cover) {
|
||||
if browserOnlyCoverURL(cover) {
|
||||
if browser == nil {
|
||||
return nil, "", errors.New("no cover fetcher")
|
||||
}
|
||||
|
||||
@@ -165,6 +165,23 @@ var singleQuotedMetaAttrRe = regexp.MustCompile(`(?is)([a-z][a-z0-9:_-]*)\s*=\s*
|
||||
// the target detail entry avoids matching posters from recommended results.
|
||||
var comixInitialDataRe = regexp.MustCompile(`(?is)<script\b[^>]*\bid\s*=\s*["']initial-data["'][^>]*>(.*?)</script>`)
|
||||
|
||||
// kaganeImageURLRe matches the canonical compressed image route kagane's API
|
||||
// publishes — the only cover URL form the extractor emits and the browser
|
||||
// fetcher accepts. The URL is matched in full (scheme, host, id shape) rather
|
||||
// than trusted: the value a fetcher is pointed at may have been client-
|
||||
// supplied, and a headless browser is a strong SSRF primitive.
|
||||
var kaganeImageURLRe = regexp.MustCompile(`^https://kagane\.to/api/v2/image/([0-9a-f-]{36})/compressed$`)
|
||||
|
||||
// browserOnlyCoverURL reports whether the browser sidecar is the only fetcher
|
||||
// for cover bytes at imageURL. kagane's image route answers a plain fetch with
|
||||
// a challenge and `cross-origin-resource-policy: same-origin`, so a TLS fetch
|
||||
// would only ever retrieve a challenge page and must not be attempted
|
||||
// (ADR-0007). This is the byte-fetch router's per-Site knowledge; it lives in
|
||||
// the extraction module, which owns kagane's URL shapes.
|
||||
func browserOnlyCoverURL(imageURL string) bool {
|
||||
return kaganeImageURLRe.MatchString(imageURL)
|
||||
}
|
||||
|
||||
// kagane's browser-fetched series response publishes cover image IDs under
|
||||
// series_covers. The API's canonical compressed image route is the only URL
|
||||
// form accepted by the store and browser fetcher; no rendition is guessed.
|
||||
@@ -178,8 +195,12 @@ func kaganeCoverURL(body string) string {
|
||||
return ""
|
||||
}
|
||||
for _, cover := range response.SeriesCovers {
|
||||
if kaganeImageIDRe.MatchString(cover.ImageID) {
|
||||
return "https://kagane.to/api/v2/image/" + cover.ImageID + "/compressed"
|
||||
// Validate the assembled URL against the same regex the browser
|
||||
// fetcher enforces, so the extractor can never emit an address the
|
||||
// fetch would refuse.
|
||||
imageURL := "https://kagane.to/api/v2/image/" + cover.ImageID + "/compressed"
|
||||
if kaganeImageURLRe.MatchString(imageURL) {
|
||||
return imageURL
|
||||
}
|
||||
}
|
||||
return ""
|
||||
|
||||
Reference in New Issue
Block a user