sulthan
21615be2bd
Closes #94. ## What Two phases per the spec, in three feature commits plus two review-fix commits: **Phase one — one registry entry per Site** (`2d134fb`) The six per-site comparison points that used to live across three files collapse into one `sites` map in `backend/internal/latest/sites.go`: Latest Chapter parse, Cover parse, browser-backed list, fetcher route, host pins, and the browser payload read all become lookups into it. `browserBackedSites()` is derived from the registry (sorted, deterministic); `fetcherFor` and `fetchableSeriesURL` keep their signatures and become lookups; `BrowserFetcher.Get` dispatches through the entries' `Read`/`Done` while the tab lifecycle stays in `BrowserFetcher.run`. **Phase two — one shared Series-page read** (`f215130`) `readSeriesPage` (new `read.go`) performs the read the Poll and the Acquisition have in common: gate, route, fetch, parse Latest Chapter, parse Cover address. It returns facts only — polling and persistence policies (stamp order, cooldowns, cover policy) stay with the callers; `acquire.go` gained the comment naming the deliberate post-fetch stamp order. The poll's legacy cover heal and the no-chapter byte-count diagnostic were restored after review (`d998f87`) so the claims "the Poll keeps its own Cover policy" and "pinning is the only behavioural change" both hold. ## Behaviour - All six Sites now pin their host exactly; asura/demonic/comix previously accepted any https host. For asura this is a strict improvement: its dead old domain redirects deep links to the site root and would parse the wrong document. - Everything else is unchanged: existing parse tables, the challenge-body table and the gate table pass unmodified except the one deliberate exception — the gate table gains the three new pin cases. ## Security invariants preserved - The address gate is recognisably the same rule, now a single registry lookup: `https` + exact hostname match, all callers route through it. No fetch path was widened; asura/demonic/comix were narrowed. - The second host pin inside each browser entry's Read is retained deliberately (browser = strong SSRF primitive, `series_url` is client-supplied) and is not deduplicated against the shared gate. - Review hardening: `fetcherFor` now fails closed for unknown site strings (previously fell through to the TLS fetcher on an unreachable path), and the browser dispatch iterates a sorted list so outcomes cannot depend on map order. - The security review's log-injection finding was checked against Go's `url.Parse` and does not hold: control characters are rejected anywhere in a URL, so a client-supplied value in a log line cannot carry a newline. ## Review Reviewed on three axes (spec, standards, security) by read-only subagents over `672c16f..f1b26f4`. No blocking findings; all minor/nit findings addressed in `d998f87` and `700de20`. Verified end to end with `go test ./...` (Docker Postgres per test package) on every commit. ## Out of scope (tracked separately) - Dropping asuracomic.net (CORS allowlist, userscript match, API fixtures, live env) — separate issue, per spec. Reviewed-on: #95 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
59 lines
2.2 KiB
Go
59 lines
2.2 KiB
Go
package latest
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
)
|
|
|
|
// seriesRead carries the two facts the poll and the acquirer both extract
|
|
// from a series page. Persistence, stamps and scheduling stay with the
|
|
// callers, so the policies that keep the two flows distinct (stamp order,
|
|
// cooldowns) are not swallowed by the module.
|
|
type seriesRead struct {
|
|
Latest latestChapter
|
|
HasLatest bool
|
|
Cover string
|
|
HasCover bool
|
|
// BodyLen is the fetched body's length, surfaced because the no-chapter
|
|
// log uses it to tell a markup change from a body the size cap cut short.
|
|
BodyLen int
|
|
}
|
|
|
|
// errNotFetchable and errNoFetcher separate the gate and the route from fetch
|
|
// failures so each caller keeps its own distinct log line for all three.
|
|
var (
|
|
errNotFetchable = errors.New("series url not fetchable")
|
|
errNoFetcher = errors.New("no fetcher for site")
|
|
)
|
|
|
|
// readSeriesPage performs the series-page read the poll and the acquirer have
|
|
// in common: gate the address, choose the route, fetch the page, extract the
|
|
// Latest Chapter and the Cover address. It persists nothing and stamps
|
|
// nothing.
|
|
//
|
|
// series_url arrives in a client-supplied PUT body (PUT /bookmarks/{key}
|
|
// accepts any string), so the gate is not an optimisation against burning a
|
|
// request on an unknown site: without it, the server would issue a GET from
|
|
// its own network position to whatever URL a token-holder writes, including
|
|
// link-local/internal addresses or non-https schemes.
|
|
func readSeriesPage(ctx context.Context, site, seriesURL string, browser, tls Fetcher) (seriesRead, error) {
|
|
if !fetchableSeriesURL(site, seriesURL) {
|
|
return seriesRead{}, fmt.Errorf("%w: site=%q url=%q", errNotFetchable, site, seriesURL)
|
|
}
|
|
f := fetcherFor(site, browser, tls)
|
|
if f == nil {
|
|
return seriesRead{}, fmt.Errorf("%w: site %q", errNoFetcher, site)
|
|
}
|
|
body, status, err := f.Get(ctx, seriesURL)
|
|
if err != nil {
|
|
return seriesRead{}, fmt.Errorf("fetch %s: %w", seriesURL, err)
|
|
}
|
|
if status != 200 {
|
|
return seriesRead{}, fmt.Errorf("fetch %s: status %d", seriesURL, status)
|
|
}
|
|
latest, hasLatest := latestChapterFrom(site, seriesURL, body)
|
|
cover, hasCover := coverFrom(site, seriesURL, body)
|
|
return seriesRead{Latest: latest, HasLatest: hasLatest, Cover: cover, HasCover: hasCover, BodyLen: len(body)}, nil
|
|
}
|