sulthan/mangaBookmark
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Cover bytes move to a content-addressed filesystem store #56

New Issue
Closed
opened 2026-08-09 23:12:11 +07:00 by sulthan · 1 comment
sulthan commented 2026-08-09 23:12:11 +07:00
Owner
Copy Link
Copy Source

Parent

Spec: #55. Originating bug: #47. Architecture and rejected alternatives: docs/adr/0007-backend-hosts-cover-bytes.md. Domain vocabulary: CONTEXT.md.

Do not close #47 or #55 from this ticket.

What to build

Cover bytes stop living in the database and start living on a filesystem volume, addressed by their content. From a Reader point of view nothing changes at all: kagane Covers still render in the web UI, still behind the session, still fetched through the browser sidecar when they are missing, still absent when no sidecar is configured. This ticket is the foundation every later ticket writes through, and it is deliberately shaped so that it can land without any user-visible behaviour changing — if a Reader can tell this shipped, something is wrong.

Underneath, a stored Cover becomes an immutable object keyed by the SHA-256 of the URL its bytes came from. The bytes are written to a path derived from that hash, sharded two levels deep so no single directory ever accumulates thousands of entries, under a directory given by new configuration. The database keeps the content address, the path and the content type — not the bytes.

Content addressing is the load-bearing choice here, and it is worth understanding before implementing: because the address changes whenever the image changes, a stored object can never be stale, which is what makes the long-lived immutable cache headers correct and removes the need for any invalidation logic anywhere in the system — including for the admin refetch deferred to #54. Do not substitute a Series-keyed path; it reintroduces the invalidation problem this avoids.

The existing byte column is removed. Its rows are dropped, not migrated: the kagane path already re-fetches a missing Cover on demand, so a migration would be careful work to preserve a cache that rebuilds itself for free.

Configuration gains the storage directory, and Compose gains a volume for it beside the database volume. The directory is required — a deployment without it cannot store anything, so startup must fail loudly rather than serve broken addresses later.

Acceptance criteria

  • Cover bytes are written to and read from a filesystem volume; the database row holds content address, path and content type only
  • The content address is the SHA-256 of the source URL, and the on-disk path is sharded two levels deep from it
  • kagane Covers still render in the web UI, still session-gated, with no visible change
  • A Cover already stored is served without invoking the browser sidecar
  • A Cover already stored survives a process restart against the same volume and database, without a second fetch
  • With no browser sidecar configured, a stored Cover is still served and an unstored one is still absent
  • The old byte column is gone; existing rows are dropped rather than migrated
  • Compose declares a volume for the cover directory
  • Startup fails with a clear message when the cover directory is unset
  • The new environment variable is documented alongside the existing backend configuration
  • go test ./... is green

Blocked by

  • None — can start immediately.
## Parent Spec: #55. Originating bug: #47. Architecture and rejected alternatives: `docs/adr/0007-backend-hosts-cover-bytes.md`. Domain vocabulary: `CONTEXT.md`. Do not close #47 or #55 from this ticket. ## What to build Cover bytes stop living in the database and start living on a filesystem volume, addressed by their content. From a Reader point of view **nothing changes at all**: kagane Covers still render in the web UI, still behind the session, still fetched through the browser sidecar when they are missing, still absent when no sidecar is configured. This ticket is the foundation every later ticket writes through, and it is deliberately shaped so that it can land without any user-visible behaviour changing — if a Reader can tell this shipped, something is wrong. Underneath, a stored Cover becomes an immutable object keyed by the SHA-256 of the URL its bytes came from. The bytes are written to a path derived from that hash, sharded two levels deep so no single directory ever accumulates thousands of entries, under a directory given by new configuration. The database keeps the content address, the path and the content type — not the bytes. Content addressing is the load-bearing choice here, and it is worth understanding before implementing: because the address changes whenever the image changes, a stored object can never be stale, which is what makes the long-lived immutable cache headers correct and removes the need for any invalidation logic anywhere in the system — including for the admin refetch deferred to #54. Do not substitute a Series-keyed path; it reintroduces the invalidation problem this avoids. The existing byte column is removed. Its rows are **dropped, not migrated**: the kagane path already re-fetches a missing Cover on demand, so a migration would be careful work to preserve a cache that rebuilds itself for free. Configuration gains the storage directory, and Compose gains a volume for it beside the database volume. The directory is required — a deployment without it cannot store anything, so startup must fail loudly rather than serve broken addresses later. ## Acceptance criteria - [x] Cover bytes are written to and read from a filesystem volume; the database row holds content address, path and content type only - [x] The content address is the SHA-256 of the source URL, and the on-disk path is sharded two levels deep from it - [x] kagane Covers still render in the web UI, still session-gated, with no visible change - [x] A Cover already stored is served without invoking the browser sidecar - [x] A Cover already stored survives a process restart against the same volume and database, without a second fetch - [x] With no browser sidecar configured, a stored Cover is still served and an unstored one is still absent - [x] The old byte column is gone; existing rows are dropped rather than migrated - [x] Compose declares a volume for the cover directory - [x] Startup fails with a clear message when the cover directory is unset - [x] The new environment variable is documented alongside the existing backend configuration - [x] `go test ./...` is green ## Blocked by - None — can start immediately.
sulthan added the ready-for-agent label 2026-08-09 23:12:11 +07:00
sulthan commented 2026-08-09 23:42:04 +07:00
Author
Owner
Copy Link
Copy Source

Implemented on branch issue-56-cover-filesystem.

Acceptance criteria:

  • Cover bytes use filesystem storage; Postgres stores address, path, and content type only.
  • Address is SHA-256(source URL); files use two-level sharding.
  • Kagane Covers remain session-gated and render through existing web flow.
  • Stored Covers avoid browser fetches.
  • Restart test proves same database and volume avoid a second fetch.
  • Stored Covers serve without a browser; missing Covers remain absent without one.
  • Legacy byte column and rows are dropped by migration 0008.
  • Compose declares cover-data and passes required COVER_DIR.
  • Startup fails clearly when COVER_DIR is unset.
  • COVER_DIR documented in backend configuration and deployment docs.
  • go test ./... passes.

Verification: CGO_ENABLED=0 go build ./..., docker build -t manga-bookmark-cover-check ./backend, and docker compose config --services all pass. Parents #47 and #55 remain open as required.

Implemented on branch issue-56-cover-filesystem. Acceptance criteria: - [x] Cover bytes use filesystem storage; Postgres stores address, path, and content type only. - [x] Address is SHA-256(source URL); files use two-level sharding. - [x] Kagane Covers remain session-gated and render through existing web flow. - [x] Stored Covers avoid browser fetches. - [x] Restart test proves same database and volume avoid a second fetch. - [x] Stored Covers serve without a browser; missing Covers remain absent without one. - [x] Legacy byte column and rows are dropped by migration 0008. - [x] Compose declares cover-data and passes required COVER_DIR. - [x] Startup fails clearly when COVER_DIR is unset. - [x] COVER_DIR documented in backend configuration and deployment docs. - [x] go test ./... passes. Verification: CGO_ENABLED=0 go build ./..., docker build -t manga-bookmark-cover-check ./backend, and docker compose config --services all pass. Parents #47 and #55 remain open as required.
sulthan added ready-for-human and removed ready-for-agent labels 2026-08-09 23:42:14 +07:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something is broken
 enhancement
New feature or improvement
 needs-info
Waiting on reporter for more information
 needs-triage
Maintainer needs to evaluate
 ready-for-agent
Fully specified, ready for an AFK agent
 ready-for-human
Needs human implementation
 wayfinder:grilling
wayfinder:map
wayfinder:prototype
wayfinder:research
wontfix
Will not be actioned
No Label ready-for-human
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
sulthan
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sulthan/mangaBookmark#56
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?