sulthan
e8d1cba6c5
Refs #56 ## Summary Moves Kagane cover bytes out of Postgres bytea storage into an immutable, content-addressed filesystem store. Reader-visible behavior remains unchanged: the existing session-gated route serves stored bytes, missing bytes use the existing browser fetch path, and no browser still returns a missing cover. ## Changes - Added migration 0008, which drops the legacy `covers` table and recreates it with only `address`, `path`, and `content_type`. Existing byte rows are intentionally dropped. - Added SHA-256 source-URL addressing with two-level sharding (`ab/cd/<sha256>`). Writes use a temp file plus atomic link; reads validate the stored relative path before opening it. - Made `COVER_DIR` required in runtime config and Compose. Compose passes it as a Docker build argument and volume target, so custom durable paths keep image ownership, runtime config, and the named `cover-data` volume aligned. - Updated every `store.Open` caller and documented configuration, deployment, backup, and troubleshooting behavior. - Added filesystem, restart, migration-drop, no-browser, and content-addressing coverage. ## Verification - `go test ./...` - `CGO_ENABLED=0 go build ./...` - `docker build --build-arg COVER_DIR=/data/covers -t manga-bookmark-cover-check-custom ./backend` - `docker compose config --format json` confirms custom `COVER_DIR` is the volume target - `git diff --check origin/main` - LSP diagnostics clean for touched Go files Parents #47 and #55 remain open as required by #56. Reviewed-on: #65 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
117 lines
5.8 KiB
Bash
117 lines
5.8 KiB
Bash
# Copy to .env and fill in. Never commit the real .env.
|
|
|
|
# Secret every Reader's userscript credential is derived from (issue #24):
|
|
# the backend rebuilds install URLs from it, and only SHA-256 hashes of the
|
|
# credentials ever touch the database. Generate one:
|
|
# openssl rand -hex 32
|
|
TOKEN_KEY=changeme-generate-a-long-random-token
|
|
|
|
# The owner's Discord user ID — seeded at startup as the first Reader, the
|
|
# administrator (the only one who can revoke another Reader's sessions), and
|
|
# the owner of every bookmark that predates registration. Discord snowflake,
|
|
# e.g. 1046923170000000000.
|
|
OWNER_DISCORD_ID=changeme-your-discord-user-id
|
|
|
|
# Comma-separated origins allowed to call the API (CORS). Both Asura domains
|
|
# plus Demonic, Comix, Kagane, and the two novel sites. Add/remove as the
|
|
# sites' hostnames change.
|
|
ALLOWED_ORIGINS=https://asuracomic.net,https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to,https://novelfull.com,https://lightnovelworld.net
|
|
|
|
# Password for the bundled Postgres container, and therefore half of the
|
|
# DATABASE_URL compose builds for the backend. Generate one:
|
|
# openssl rand -hex 24
|
|
POSTGRES_PASSWORD=changeme-generate-a-long-random-password
|
|
|
|
# Override only to point the backend at a Postgres compose does not run.
|
|
# DATABASE_URL=postgres://user:pass@host:5432/bookmarks?sslmode=require
|
|
|
|
# Directory inside bookmark-api for immutable, content-addressed Cover bytes.
|
|
# Compose builds the image and mounts its named volume at this path.
|
|
COVER_DIR=/covers
|
|
|
|
# --- Prod override (Traefik) only ---
|
|
# Subdomain Traefik routes to this service (required by the prod override).
|
|
# BOOKMARK_API_HOST=bookmark-api.example.com
|
|
# Traefik's docker network name, if not "proxy".
|
|
# PROXY_NETWORK=proxy
|
|
# Traefik HTTPS entrypoint + cert resolver names, if yours differ from these.
|
|
# TRAEFIK_ENTRYPOINT=websecure
|
|
# TRAEFIK_CERTRESOLVER=le
|
|
|
|
# --- Web UI (Discord OAuth) ---
|
|
# Sign-in is a Discord authorization code grant (ADR-0002), and it is also
|
|
# registration: any member of the configured guild becomes a Reader on their
|
|
# first successful login, with their own empty library. Create the application
|
|
# at https://discord.com/developers/applications and register the exact
|
|
# callback URL ($BOOKMARK_WEB_HOST/auth/discord/callback) as an OAuth2
|
|
# redirect.
|
|
DISCORD_CLIENT_ID=
|
|
DISCORD_CLIENT_SECRET=
|
|
# The guild whose membership gates sign-in (Developer Mode -> right-click the
|
|
# server -> Copy Server ID).
|
|
DISCORD_GUILD_ID=
|
|
# Exact callback URL, e.g. https://bookmark.example.com/auth/discord/callback.
|
|
# Discord matches it verbatim, so it must equal the registered redirect.
|
|
DISCORD_REDIRECT_URI=
|
|
# Optional: a role snowflake members must hold on top of guild membership.
|
|
# Empty (the default) means membership alone suffices.
|
|
# DISCORD_REQUIRED_ROLE=
|
|
|
|
# Subdomain Traefik routes to the browser UI (required by the prod override).
|
|
# Left commented on purpose: an example value here would be a silent
|
|
# wrong-hostname fallback, and Traefik would publish the UI router on a domain
|
|
# you do not own. The same container also answers on BOOKMARK_API_HOST for the
|
|
# userscript's API.
|
|
# BOOKMARK_WEB_HOST=bookmark.example.com
|
|
|
|
# --- Latest-chapter poller ---
|
|
# The backend re-checks each bookmarked series' newest published chapter on its
|
|
# own schedule, so latest_chapter stays fresh even when you never open the manga
|
|
# sites. This runs in parallel with the userscript's own in-browser check.
|
|
# Set to 0 to turn it off entirely.
|
|
# LATEST_CHAPTER_POLL_ENABLED=1
|
|
#
|
|
# Two independent clocks. COOLDOWN is how long a plain-TLS series rests between
|
|
# checks; BROWSER_COOLDOWN is the longer rest for kagane and novelfull. INTERVAL
|
|
# is how often the poller wakes up and looks for series past their cooldowns.
|
|
# Shortening INTERVAL cannot shorten either cooldown.
|
|
LATEST_CHAPTER_POLL_COOLDOWN=1h # plain-TLS per series, floor 15m
|
|
LATEST_CHAPTER_POLL_BROWSER_COOLDOWN=6h # browser-backed per series, floor 15m
|
|
LATEST_CHAPTER_POLL_INTERVAL=10m # how often to wake
|
|
LATEST_CHAPTER_POLL_BATCH=14 # series per wake
|
|
LATEST_CHAPTER_POLL_STAGGER=20s # delay between fetches in a batch
|
|
#
|
|
# Uses a ticker, not an immediate first run: the first poll happens one
|
|
# INTERVAL after startup, not at startup. A container restarting more often
|
|
# than INTERVAL never polls.
|
|
#
|
|
# BATCH x (COOLDOWN / INTERVAL) series hold the cooldown cadence — 84 with these
|
|
# defaults. Beyond that the cadence stretches uniformly rather than breaking;
|
|
# raise BATCH or lower INTERVAL. Keep BATCH x STAGGER under INTERVAL.
|
|
|
|
# CDP endpoint of the browser, used for the two sites behind a Cloudflare
|
|
# JavaScript challenge (kagane, novelfull) and by the web UI's kagane cover
|
|
# proxy. Unset disables browser polling and serves 404 for covers not already
|
|
# stored; those sites then rely on the userscript alone. That is also exactly
|
|
# how an unreachable browser degrades, so a home machine that is off costs
|
|
# chapter freshness and nothing else.
|
|
#
|
|
# The browser does NOT run in this stack. It is its own compose unit on the
|
|
# home machine (chrome/docker-compose.yml, chrome/.env.example) and is reached
|
|
# over the tailnet, so set this to that machine's tailnet address:
|
|
#
|
|
# BROWSER_WS_URL=ws://100.x.y.z:9222
|
|
#
|
|
# It must be the tailnet **IP**, never a MagicDNS hostname and never the old
|
|
# Docker service name: Chrome's DevTools HTTP handler 500s any /json/version
|
|
# request whose Host header isn't an IP or "localhost", which silently breaks
|
|
# every kagane poll. Left unset here on purpose — a wrong default would poll a
|
|
# stranger's address, and "no browser" is a safe, self-announcing state.
|
|
# BROWSER_WS_URL=ws://100.x.y.z:9222
|
|
|
|
# Zone the backend stamps its log lines in. Cosmetic only. Nothing else in
|
|
# the service has a zone: bookmark timestamps are unix ms, and the two real
|
|
# time columns are timestamptz. Defaults to Asia/Jakarta; set to UTC for the
|
|
# conventional server default.
|
|
# API_TZ=Asia/Jakarta
|