e8d1cba6c5
Refs #56 ## Summary Moves Kagane cover bytes out of Postgres bytea storage into an immutable, content-addressed filesystem store. Reader-visible behavior remains unchanged: the existing session-gated route serves stored bytes, missing bytes use the existing browser fetch path, and no browser still returns a missing cover. ## Changes - Added migration 0008, which drops the legacy `covers` table and recreates it with only `address`, `path`, and `content_type`. Existing byte rows are intentionally dropped. - Added SHA-256 source-URL addressing with two-level sharding (`ab/cd/<sha256>`). Writes use a temp file plus atomic link; reads validate the stored relative path before opening it. - Made `COVER_DIR` required in runtime config and Compose. Compose passes it as a Docker build argument and volume target, so custom durable paths keep image ownership, runtime config, and the named `cover-data` volume aligned. - Updated every `store.Open` caller and documented configuration, deployment, backup, and troubleshooting behavior. - Added filesystem, restart, migration-drop, no-browser, and content-addressing coverage. ## Verification - `go test ./...` - `CGO_ENABLED=0 go build ./...` - `docker build --build-arg COVER_DIR=/data/covers -t manga-bookmark-cover-check-custom ./backend` - `docker compose config --format json` confirms custom `COVER_DIR` is the volume target - `git diff --check origin/main` - LSP diagnostics clean for touched Go files Parents #47 and #55 remain open as required by #56. Reviewed-on: #65 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
134 lines
6.9 KiB
YAML
134 lines
6.9 KiB
YAML
# Base stack — works standalone for local smoke testing (`docker compose up`).
|
|
# The service binds 127.0.0.1:8080; a host reverse proxy (nginx/Caddy/Traefik)
|
|
# terminates TLS for bookmark-api.<domain> and forwards to it.
|
|
#
|
|
# If your proxy runs in Docker on its own network, use the prod override which
|
|
# attaches to that network instead of publishing a port:
|
|
# docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
#
|
|
# The browser is not here. It is its own unit on the home machine —
|
|
# chrome/docker-compose.yml — reached over the tailnet via BROWSER_WS_URL.
|
|
|
|
services:
|
|
bookmark-api:
|
|
build:
|
|
context: ./backend
|
|
args:
|
|
COVER_DIR: ${COVER_DIR:?set COVER_DIR in .env}
|
|
image: bookmarkmanager-backend:latest
|
|
container_name: bookmark-api
|
|
restart: unless-stopped
|
|
environment:
|
|
# TOKEN_KEY derives every Reader's userscript credential (issue #24) —
|
|
# compose refuses to start without it.
|
|
TOKEN_KEY: ${TOKEN_KEY:?set TOKEN_KEY in .env}
|
|
# Owner's Discord user ID — required. Seeds the owner Reader (the
|
|
# administrator); every other Reader registers on their first login.
|
|
OWNER_DISCORD_ID: ${OWNER_DISCORD_ID:?set OWNER_DISCORD_ID in .env}
|
|
ALLOWED_ORIGINS: ${ALLOWED_ORIGINS:-https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to,https://novelfull.com,https://lightnovelworld.net}
|
|
# The bookmarks database. Host is the compose service name; the password
|
|
# comes from .env so it is never committed.
|
|
DATABASE_URL: ${DATABASE_URL:-postgres://bookmarks:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@postgres:5432/bookmarks?sslmode=disable}
|
|
# Required path inside the API. The build seeds ownership at this path
|
|
# and the named volume below mounts there.
|
|
COVER_DIR: ${COVER_DIR:?set COVER_DIR in .env}
|
|
PORT: "8080"
|
|
# Log timestamps only. Go's `log` stamps lines in local time, and this
|
|
# service has no other use for a zone: bookmark timestamps are unix ms
|
|
# and the two real time columns are timestamptz, both absolute instants.
|
|
# Purely so these lines read on the same clock as the browser's. Named
|
|
# API_TZ rather than TZ so an operator's exported shell TZ cannot leak
|
|
# in; distroless already carries tzdata, so the name just resolves.
|
|
TZ: ${API_TZ:-Asia/Jakarta}
|
|
# Discord OAuth for the browser UI (ADR-0002). The first four are
|
|
# required; DISCORD_REQUIRED_ROLE is optional and empty by default.
|
|
# Guild membership is the whole gate: any member becomes a Reader.
|
|
DISCORD_CLIENT_ID: ${DISCORD_CLIENT_ID:?set DISCORD_CLIENT_ID in .env}
|
|
DISCORD_CLIENT_SECRET: ${DISCORD_CLIENT_SECRET:?set DISCORD_CLIENT_SECRET in .env}
|
|
DISCORD_GUILD_ID: ${DISCORD_GUILD_ID:?set DISCORD_GUILD_ID in .env}
|
|
DISCORD_REQUIRED_ROLE: ${DISCORD_REQUIRED_ROLE:-}
|
|
DISCORD_API_BASE: ${DISCORD_API_BASE:-https://discord.com/api/v10}
|
|
DISCORD_REDIRECT_URI: ${DISCORD_REDIRECT_URI:?set DISCORD_REDIRECT_URI in .env}
|
|
# Path inside the container; matches the bindmount above.
|
|
USERSCRIPT_PATH: ${USERSCRIPT_PATH:-/userscript/manga-bookmark.user.js}
|
|
# Second script from the same bindmount; the novel library is a separate
|
|
# Violentmonkey install.
|
|
NOVEL_USERSCRIPT_PATH: ${NOVEL_USERSCRIPT_PATH:-/userscript/novel-bookmark.user.js}
|
|
# Latest-chapter poller. LATEST_CHAPTER_POLL_ENABLED=0 in .env is the kill
|
|
# switch; it only takes effect because these are listed here.
|
|
LATEST_CHAPTER_POLL_ENABLED: ${LATEST_CHAPTER_POLL_ENABLED:-1}
|
|
LATEST_CHAPTER_POLL_COOLDOWN: ${LATEST_CHAPTER_POLL_COOLDOWN:-1h}
|
|
LATEST_CHAPTER_POLL_BROWSER_COOLDOWN: ${LATEST_CHAPTER_POLL_BROWSER_COOLDOWN:-6h}
|
|
LATEST_CHAPTER_POLL_INTERVAL: ${LATEST_CHAPTER_POLL_INTERVAL:-10m}
|
|
LATEST_CHAPTER_POLL_BATCH: ${LATEST_CHAPTER_POLL_BATCH:-14}
|
|
LATEST_CHAPTER_POLL_STAGGER: ${LATEST_CHAPTER_POLL_STAGGER:-20s}
|
|
# CDP endpoint for sites behind a JavaScript challenge (kagane,
|
|
# novelfull). The browser is not part of this stack — it runs on the home
|
|
# machine as its own unit (chrome/docker-compose.yml) and is reached over
|
|
# the tailnet. Unset disables browser polling for those sites and serves
|
|
# 404 from the cover proxy for covers not already stored; the userscript
|
|
# still covers them. Set it in .env to ws://<home machine tailnet IP>:9222.
|
|
#
|
|
# Must be an IP, not a MagicDNS hostname: Chrome's DevTools HTTP handler
|
|
# rejects the discovery request (GET /json/version) with a 500 unless the
|
|
# Host header is an IP address or "localhost" — confirmed 2026-08-03,
|
|
# independent of chromedp's own dial logic. The same trap that used to
|
|
# force a pinned Docker IP now forbids the tailnet name.
|
|
BROWSER_WS_URL: ${BROWSER_WS_URL:-}
|
|
depends_on:
|
|
# The migration runner is the first thing the binary does, so a Postgres
|
|
# that is still initialising means a crash-loop until it is not.
|
|
postgres:
|
|
condition: service_healthy
|
|
volumes:
|
|
# The userscript is served from here, read fresh on every request. Editing
|
|
# the file in this checkout takes effect on the next Violentmonkey poll —
|
|
# no rebuild, no restart. `git pull` restores the committed version, which
|
|
# is why a redeploy always ships the repo's script.
|
|
- ./userscript:/userscript:ro
|
|
# Content-addressed cover bytes survive API restarts and redeploys.
|
|
- cover-data:${COVER_DIR:?set COVER_DIR in .env}
|
|
# Bound to loopback only: the proxy (or curl during smoke test) reaches it,
|
|
# the public internet does not.
|
|
ports:
|
|
- "127.0.0.1:8080:8080"
|
|
# `default` is not decoration: `db` is `internal: true`, and a container on
|
|
# nothing but an internal network gets neither a published port nor egress
|
|
# — which would silently kill every poller fetch.
|
|
networks:
|
|
- default
|
|
- db
|
|
|
|
postgres:
|
|
image: postgres:17-alpine
|
|
restart: unless-stopped
|
|
environment:
|
|
POSTGRES_DB: bookmarks
|
|
POSTGRES_USER: bookmarks
|
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U bookmarks -d bookmarks"]
|
|
interval: 5s
|
|
timeout: 3s
|
|
retries: 10
|
|
volumes:
|
|
- postgres-data:/var/lib/postgresql/data
|
|
# Deliberately no `ports:` — only bookmark-api, over the `db` network,
|
|
# reaches it. Use `docker compose exec postgres psql` for a shell.
|
|
networks:
|
|
- db
|
|
|
|
volumes:
|
|
postgres-data:
|
|
cover-data:
|
|
# The pre-Postgres SQLite volume (bookmarks-data) is deliberately no longer
|
|
# declared here: undeclared means `docker compose down -v` cannot take it
|
|
# with the rest, so the old database survives the cutover until someone
|
|
# removes it by hand.
|
|
|
|
networks:
|
|
# Postgres needs no egress and nothing outside bookmark-api needs to reach
|
|
# it, so this one really can be cut off from the outside world.
|
|
db:
|
|
internal: true
|