The Poll fills blank Covers for every Site and both Libraries #70

2026-08-10T09:26:17+07:00

sulthan commented

2026-08-10 09:26:17 +07:00

Closes #61.

Summary

Permanently-blank Series (the half of #47 that creation-time acquisition cannot reach) heal on the next due poll cycle. The cover path is no longer kagane-only: every Site and both Libraries fill a blank Cover from the series page the chapter poll already fetched, and never replace a Cover that already exists.

What changed

`backend/internal/latest/poller.go`

fillBlankCover — when Cover and CoverAddress are both blank, extract a source URL via coverFrom from the series-page body and store bytes through SetSeriesCover. Skips any Series that already has a source URL (owned by prefetch) or a stored address (never overwrite).
prefetchCover — source-URL healing path, now site-uniform. Kagane no longer special-cases into PutKaganeCover alone; every Site lands on SetSeriesCover, so the wire Cover becomes a content-addressed public URL. Reuses already-stored bytes when present.
storeCover / fetchCoverBytes — shared fetch+persist. Only kagane routes image bytes through the browser fetcher; every other Site uses plain TLS CoverBytesFetch. Failures log with the Series key and never return to the chapter path.
checkOne — after a successful series-page fetch, calls fillBlankCover once regardless of whether chapter extraction succeeded (cover fill is independent of the chapter signal).

`backend/internal/latest/poller_test.go`

Extended the existing poller harness (real store, fake fetchers) rather than a new one:

TestRunOnceFillsBlankCoverFromSeriesPage — asura manga, lightnovelworld novel, kagane manga; asserts wire Cover + correct fetcher routing.
TestRunOnceDoesNotReplaceExistingCover — second poll does not refetch.
TestRunOnceRetriesFailedBlankCoverOnNextPoll — failed fill stays blank, next due cycle retries (no separate queue).
TestRunOnceBlankCoverFailureDoesNotBlockChapter — chapter still lands; failure log carries the Series key.
Kagane prefetch test now also asserts the content-addressed wire Cover.

Acceptance criteria (#61)

Criterion	Status
Cover prefetch runs for every Site	done
Cover prefetch runs for both Libraries	done
Poll fills a blank Cover	done
Poll never replaces an existing Cover	done
Failed cover fetch does not fail/block chapter poll	done
Failed cover fetch retried next poll, no separate queue	done
Failures logged with the Series	done
Existing poller tests extended	done
`go test ./...` green	done
Manually verified: blank Series gets Cover after a poll cycle	left for you

Out of scope / not closed

Does not close #47 or #55 (per ticket).
No migration/backfill script — the Poll walks every Series already.
No admin refetch (#54).

Review notes addressed

Removed the kagane-only PutKaganeCover branch from prefetch so source-URL healing also sets CoverAddress (wire Cover).
Guard so fillBlankCover does not double-fetch after prefetchCover healed the same snapshot.
Single fillBlankCover call site after the series-page fetch.

Test plan

go test ./... (backend; needs Docker/Postgres via pgtest)
After deploy: pick a Series that was blank, wait one poll cycle, confirm Cover in web UI and userscript panel

Closes #61. ## Summary Permanently-blank Series (the half of #47 that creation-time acquisition cannot reach) heal on the next due poll cycle. The cover path is no longer kagane-only: every Site and both Libraries fill a blank Cover from the series page the chapter poll already fetched, and never replace a Cover that already exists. ## What changed ### `backend/internal/latest/poller.go` - **`fillBlankCover`** — when `Cover` and `CoverAddress` are both blank, extract a source URL via `coverFrom` from the series-page body and store bytes through `SetSeriesCover`. Skips any Series that already has a source URL (owned by prefetch) or a stored address (never overwrite). - **`prefetchCover`** — source-URL healing path, now site-uniform. Kagane no longer special-cases into `PutKaganeCover` alone; every Site lands on `SetSeriesCover`, so the wire Cover becomes a content-addressed public URL. Reuses already-stored bytes when present. - **`storeCover` / `fetchCoverBytes`** — shared fetch+persist. Only kagane routes image bytes through the browser fetcher; every other Site uses plain TLS `CoverBytesFetch`. Failures log with the Series key and never return to the chapter path. - **`checkOne`** — after a successful series-page fetch, calls `fillBlankCover` once regardless of whether chapter extraction succeeded (cover fill is independent of the chapter signal). ### `backend/internal/latest/poller_test.go` Extended the existing poller harness (real store, fake fetchers) rather than a new one: - `TestRunOnceFillsBlankCoverFromSeriesPage` — asura manga, lightnovelworld novel, kagane manga; asserts wire Cover + correct fetcher routing. - `TestRunOnceDoesNotReplaceExistingCover` — second poll does not refetch. - `TestRunOnceRetriesFailedBlankCoverOnNextPoll` — failed fill stays blank, next due cycle retries (no separate queue). - `TestRunOnceBlankCoverFailureDoesNotBlockChapter` — chapter still lands; failure log carries the Series key. - Kagane prefetch test now also asserts the content-addressed wire Cover. ## Acceptance criteria (#61) | Criterion | Status | |---|---| | Cover prefetch runs for every Site | done | | Cover prefetch runs for both Libraries | done | | Poll fills a blank Cover | done | | Poll never replaces an existing Cover | done | | Failed cover fetch does not fail/block chapter poll | done | | Failed cover fetch retried next poll, no separate queue | done | | Failures logged with the Series | done | | Existing poller tests extended | done | | `go test ./...` green | done | | Manually verified: blank Series gets Cover after a poll cycle | **left for you** | ## Out of scope / not closed - Does **not** close #47 or #55 (per ticket). - No migration/backfill script — the Poll walks every Series already. - No admin refetch (#54). ## Review notes addressed - Removed the kagane-only `PutKaganeCover` branch from prefetch so source-URL healing also sets `CoverAddress` (wire Cover). - Guard so `fillBlankCover` does not double-fetch after `prefetchCover` healed the same snapshot. - Single `fillBlankCover` call site after the series-page fetch. ## Test plan - [x] `go test ./...` (backend; needs Docker/Postgres via `pgtest`) - [x] After deploy: pick a Series that was blank, wait one poll cycle, confirm Cover in web UI and userscript panel

sulthan added 1 commit 2026-08-10 09:26:17 +07:00

The Poll fills blank Covers for every Site and both Libraries (#61 ) 6ca1b6fdf4

Permanently-blank Series rows heal on the next due cycle: the chapter-page
body already in hand is scanned for a Cover, bytes are stored, and the wire
address is set. A non-blank Cover is never replaced. Failures log against the
Series, leave the chapter poll alone, and retry on the next due cycle with no
separate queue. Kagane source-URL prefetch now lands on the same
content-addressed SetSeriesCover path as every other Site.

sulthan referenced this pull request

2026-08-10 09:26:25 +07:00

The Poll fills blank Covers for every Site and both Libraries #61

sulthan merged commit b9220b3dfc into main

2026-08-10 10:14:27 +07:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sulthan/mangaBookmark#70