mangaBookmark/DEPLOY.md

Deployment

Step-by-step for the backend (Docker + Traefik) and the Bromite userscript. Assumes you already run Traefik in Docker with a working HTTPS entrypoint and an ACME/cert resolver, and control a domain.

---

0. Prerequisites

• Docker + Docker Compose on the server.
• A Traefik instance watching a Docker network (default name assumed: proxy).
• DNS: an A/AAAA record for bookmark-api.<yourdomain> pointing at the server.
• The repo copied to the server, e.g. ~/mangaBookmark/ (needs backend/, docker-compose.yml, docker-compose.prod.yml, .env.example).

Confirm the Traefik network exists (create if not):

docker network ls | grep proxy || docker network create proxy

---

1. Configure .env

cd ~/mangaBookmark
cp .env.example .env

Edit .env:

# Required — secret every Reader's userscript credential is derived from.
# Only SHA-256 hashes of credentials are stored.
TOKEN_KEY=<paste output of: openssl rand -hex 32>

# Required — the owner's Discord user ID. Seeds the first Reader: the
# administrator, and the owner of every bookmark that predates registration.
# The value is the snowflake in your Discord profile (Settings →
# Advanced → Developer Mode → right-click your name → Copy User ID).
OWNER_DISCORD_ID=<discord user id>

# CORS allowlist — leave as-is unless a site changes hostname.
ALLOWED_ORIGINS=https://asurascans.com,https://demonicscans.org,https://comix.to,https://kagane.to,https://novelfull.com,https://lightnovelworld.net

# Required — password for the bundled Postgres container. Compose builds the
# backend's DATABASE_URL out of it and has no fallback for either.
POSTGRES_PASSWORD=<paste output of: openssl rand -hex 24>

# Leave unset. Only set this to point the backend at a Postgres compose does
# not run; it then replaces the URL built from POSTGRES_PASSWORD above.
# DATABASE_URL=postgres://user:pass@host:5432/bookmarks?sslmode=require

# Required path inside bookmark-api. Compose builds the image and mounts the
# named cover-data volume at this path.
COVER_DIR=/covers

# Required — the origin this deployment answers on, no trailing slash. Cover
# URLs on the wire are absolute, because the userscript renders them on a
# Site's own origin (ADR-0007). Same host as BOOKMARK_API_HOST below.
PUBLIC_BASE_URL=https://bookmark-api.violetcrown.my.id

# Required for the Traefik override. Both have no fallback — compose refuses
# to start without them. BOOKMARK_WEB_HOST is required even if the web UI
# were unused; see 1b.
BOOKMARK_API_HOST=bookmark-api.violetcrown.my.id
BOOKMARK_WEB_HOST=bookmark.violetcrown.my.id

# Only if your Traefik setup differs from these defaults:
# PROXY_NETWORK=proxy
# TRAEFIK_ENTRYPOINT=websecure
# TRAEFIK_CERTRESOLVER=le

Generate + insert the two secrets in three lines:

sed -i "s|^TOKEN_KEY=.*|TOKEN_KEY=$(openssl rand -hex 32)|" .env
sed -i "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=$(openssl rand -hex 24)|" .env
grep -E '^TOKEN_KEY=' .env

TOKEN_KEY derives every Reader's userscript credential (issue #24); only SHA-256 hashes of the credentials are stored, so this secret is what a database leak alone cannot recover. Changing it invalidates every installed script at once.

POSTGRES_PASSWORD is read only while the postgres-data volume is empty, which in practice means at first boot. Changing it afterwards changes the URL the backend dials but not the password the database expects, and bookmark-api crash-loops on password authentication failed. Set it before §2 and leave it alone.

An .env written before issue #100 carries the old poll-pace names (LATEST_CHAPTER_POLL_COOLDOWN, _BROWSER_COOLDOWN, _INTERVAL, _BATCH, _STAGGER). All five are dead configuration now — the pace lives in the Site registry (backend/internal/latest/sites.go), so delete those lines and keep only the kill switch LATEST_CHAPTER_POLL_ENABLED. Leaving them behind is harmless (nothing reads them) but silently misleads the next person who edits the file.

Match TRAEFIK_ENTRYPOINT / TRAEFIK_CERTRESOLVER to your Traefik's actual names (check your Traefik static config — common alternatives: https, myresolver, cloudflare). Wrong names = no certificate issued.

---

1b. Web UI

The browser UI is served by the same container on a second hostname. Sign-in is a Discord authorization code grant (ADR-0002): the owner's Discord account, gated by membership in one configured guild.

1. Add a DNS A/AAAA record for bookmark.<yourdomain> pointing at the server — the same address as bookmark-api.<yourdomain>.

2. Create the Discord application at https://discord.com/developers/applications:

   • OAuth2 → Redirects: add the exact callback URL https://bookmark.violetcrown.my.id/auth/discord/callback. Discord matches it verbatim — a trailing slash or different hostname breaks sign-in.
   • OAuth2 → General: note the Client ID, and generate a Client Secret.
   • No scopes or bot setup are needed in the dashboard; the service requests identify and guilds.members.read itself, and checks the user's membership of the guild, not the application's.

3. Set the variables in .env:

   BOOKMARK_WEB_HOST=bookmark.violetcrown.my.id
   DISCORD_CLIENT_ID=<client id>
   DISCORD_CLIENT_SECRET=<client secret>
   DISCORD_GUILD_ID=<guild snowflake>
   DISCORD_REDIRECT_URI=https://bookmark.violetcrown.my.id/auth/discord/callback
   # Optional: only members holding this role may sign in.
   # DISCORD_REQUIRED_ROLE=<role snowflake>
   # Optional: display name for that guild — shown on the login screen so a
   # stranger knows which Discord to ask for an invite.
   # DISCORD_GUILD_NAME=Your Guild Name
   

   The guild id is in Discord's client with Developer Mode on: right-click the server name → Copy Server ID. The four uncommented variables are required — the backend refuses to start without them. Guild membership is registration: any member of DISCORD_GUILD_ID becomes a Reader with their own library on their first sign-in. OWNER_DISCORD_ID from §1 is only the administrator — the Reader who can revoke another Reader's sessions.

4. Redeploy and check:

   docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
   curl -s -o /dev/null -w '%{http_code}\n' https://bookmark.violetcrown.my.id/
   

   Expected 200, serving the login page with the Discord button. Signing in lands on the library; an account outside the guild is refused with a message that names neither the guild nor its id.

Sessions are rows in the database: the cookie carries only an opaque id, and every request looks the row up and checks its expiry. Deleting a session row — or the whole sessions table — logs the browser out immediately; nothing is signed, so rotating a credential does not affect browser sessions. Sessions last 60 days.

---

2. Build + start

docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build

This merges the base file (build/image/env/volume) with the prod override (no host port, Traefik network + router labels). Always pass both -f flags — the prod file is not standalone.

Two services come up: bookmark-api (the backend) and postgres (its database, postgres:17-alpine). Postgres publishes no port — it sits alone with bookmark-api on an internal: true network — and stops everything if it is missing: bookmark-api waits for pg_isready to pass, then applies its embedded migrations, and only then listens. The schema is created that way; there is nothing to import by hand.

There is deliberately no browser here. Kagane and novelfull need one, and it runs on a separate machine over the tailnet — §7. Until you do that step, BROWSER_WS_URL is unset, the poller logs and skips those two sites, and everything else works normally.

Check it's up and healthy:

docker compose -f docker-compose.yml -f docker-compose.prod.yml ps
# bookmark-api Up; postgres Up (healthy)
docker logs bookmark-api --tail 20   # expect: "listening on :8080 ..."

---

3. Verify over HTTPS

Give Traefik a few seconds to issue the cert, then:

# Health (no auth) — must be valid TLS, no cert warning.
curl -s https://bookmark-api.violetcrown.my.id/healthz          # -> ok

# Auth enforced.
curl -s -o /dev/null -w '%{http_code}\n' \
  https://bookmark-api.violetcrown.my.id/bookmarks              # -> 401

# A Reader's own credential. It is derived, never stored in .env — take it from
# the Userscripts panel's install link after signing in, or from an installed
# script's API_TOKEN constant.
TOKEN=<your Reader credential>
curl -s -H "Authorization: Bearer $TOKEN" \
  https://bookmark-api.violetcrown.my.id/bookmarks              # -> []

# CORS preflight from a real site origin.
curl -s -i -X OPTIONS \
  -H 'Origin: https://asurascans.com' \
  -H 'Access-Control-Request-Method: PUT' \
  https://bookmark-api.violetcrown.my.id/bookmarks/x | grep -i access-control
# -> Access-Control-Allow-Origin: https://asurascans.com  (+ Methods/Headers)

All four must pass. Valid TLS is non-negotiable — the manga sites are HTTPS, so a bad cert makes the browser block the userscript's fetch() (mixed content).

---

4. Configure the userscript

The bindmounted userscript/*.user.js files carry __API_TOKEN__ placeholders and the deployment's @downloadURL/@updateURL lines. Check the metadata block — it ships hardcoded to this deployment's domain, so a deployer who copies the repo to another domain must edit the two lines or the script auto-updates from someone else's backend:

// @downloadURL  https://bookmark-api.yourdomain.com/u/__API_TOKEN__/manga-bookmark.user.js
// @updateURL    https://bookmark-api.yourdomain.com/u/__API_TOKEN__/manga-bookmark.user.js

The backend substitutes __API_TOKEN__ with the requesting Reader's derived credential at serve time (issue #24), so no real credential ever sits in the file. Only the API_BASE constant and the metadata hostname are deployer edits; do not put a credential in this file.

---

5. Install on Bromite

1. Bromite → Settings → User scripts → enable (accept the permission prompt).
2. Sign in to the web UI, open the Userscripts panel, and open the install link — Bromite detects .user.js and offers to install. The script already carries your credential; you never see or type one.
3. Confirm install — the @match list covers both sites.
4. Open a series on asurascans.com or demonicscans.org → a 📑 button appears bottom-right → tap → + Bookmark this.

Optional desktop test: the script is GM_*-free, so the same file installs in Tampermonkey/Violentmonkey for quick checks before going mobile.

Rotating the credential in the same web-UI panel invalidates every installed copy immediately — reinstall on all devices, or they silently stop syncing.

---

6. Smoke-test the full loop

1. Bookmark a series on Asura.
2. curl -s -H "Authorization: Bearer $TOKEN" https://bookmark-api.yourdomain.com/bookmarks on the server — the series should appear.
3. Open a chapter of that series — reopen the panel; last-read updates to that chapter (auto, never regresses on older chapters).
4. Open Demonic, open the panel — the Asura bookmark shows there too (shared store, cross-site unified list).

---

7. The browser, on the home machine

Kagane and novelfull sit behind a Cloudflare JavaScript challenge no TLS fingerprint clears, so the poller reaches them through a real Chrome over CDP. That browser does not run on the VPS: it held 471 MiB of a 1974 MiB box with no swap, and a residential IP avoids the cloud-hosting-IP signature Bot Fight Mode challenges anyway (ADR-0006). It is its own compose unit, deployed and updated independently of everything above.

Do this after §2, on the second machine. Both machines must already be on the same tailnet.

First, on the VPS, record what you are reclaiming — this is the whole point of the move and there is no way to measure it afterwards:

free -m | awk '/^Mem:/ {print "available before:", $NF, "MiB"}'

Take it again after §7 is finished and the old sidecar is gone. Expect roughly the sidecar's former footprint back (measured at 471 MiB working set, 595 MiB cgroup).

On the home machine:

git clone <this repo> ~/mangaBookmark && cd ~/mangaBookmark/chrome

tailscale ip -4                    # -> 100.x.y.z, this machine's tailnet IP
cp .env.example .env
echo "BROWSER_BIND_ADDR=$(tailscale ip -4)" >> .env

docker compose up -d --build

The clone is only for chrome/; nothing else on this machine reads the rest of the repo. The unit is its own compose project (bookmark-browser), so it shares no volume, network or lifecycle with an API stack that happens to sit beside it.

BROWSER_BIND_ADDR has no default on purpose. CDP authenticates nothing — whatever reaches port 9222 drives the browser and, through it, this host — so the bind address is the access control, backed by Tailscale device identity. On the VPS that job was done by Docker network membership; this machine has a real LAN, so 0.0.0.0 would be a hole punched into your home network. Compose refuses to start rather than guess.

Narrow it to the one device that needs it. The bind address keeps CDP off your LAN; it still leaves port 9222 open to every device on the tailnet, and CDP has no login — a compromised phone is enough to drive this host. A new tailnet's policy is allow-all, so this is the step that makes "Tailscale identity is the access control" true rather than aspirational.

Tailscale has no deny, so a restriction is expressed by removing the blanket grant and enumerating what is left. That only works if the browser machine can be excluded from a selector that still covers your own devices — which is what tagging buys: a tagged device has no user, so autogroup:member and autogroup:self stop matching it. Tagging is the mechanism, not decoration.

In the admin console, under Access controls, the shipped policy grants {"src": ["*"], "dst": ["*"], "ip": ["*"]}. Replace it:

{
  "tagOwners": {
    // Empty list: implicitly owned by the tailnet Owner/Admins, which is you.
    "tag:bookmark-api":     [],
    "tag:bookmark-browser": [],
  },

  "grants": [
    // The only thing on the tailnet that may drive the browser.
    {
      "src": ["tag:bookmark-api"],
      "dst": ["tag:bookmark-browser"],
      "ip":  ["tcp:9222"],
    },
    // Your own devices reach your own devices, and the VPS, in full.
    {
      "src": ["autogroup:member"],
      "dst": ["autogroup:self", "tag:bookmark-api"],
      "ip":  ["*"],
    },
    // On the browser machine you get SSH and nothing else. Widen this to `*`
    // and the restriction above is void; delete it and you are locked out.
    {
      "src": ["autogroup:member"],
      "dst": ["tag:bookmark-browser"],
      "ip":  ["tcp:22"],
    },
    // Uncomment if you route traffic through an exit node — dropping the
    // blanket grant takes exit-node access with it.
    // {"src": ["autogroup:member"], "dst": ["autogroup:internet"], "ip": ["*"]},
  ],

  // Tagged devices left `autogroup:self`, so Tailscale SSH needs them named.
  // Irrelevant if you reach these boxes with ordinary sshd over the tailnet —
  // that is the `tcp:22` grant above.
  "ssh": [
    {
      "action": "check",
      "src":    ["autogroup:member"],
      "dst":    ["autogroup:self", "tag:bookmark-api", "tag:bookmark-browser"],
      "users":  ["autogroup:nonroot", "root"],
    },
  ],

  // Run on every save, so a later edit that reopens 9222 is rejected outright.
  "tests": [
    { "src": "tag:bookmark-api", "accept": ["tag:bookmark-browser:9222"] },
    {
      "src":    "you@example.com",
      "accept": ["tag:bookmark-browser:22"],
      "deny":   ["tag:bookmark-browser:9222"],
    },
  ],
}

Then apply the tags — on the VPS and the home machine respectively:

sudo tailscale up --advertise-tags=tag:bookmark-api
sudo tailscale up --advertise-tags=tag:bookmark-browser

Each re-authenticates in a browser and issues a new node key; the tailnet IP is unchanged, so BROWSER_WS_URL and BROWSER_BIND_ADDR still hold. Key expiry is disabled once a device is tagged, which is what you want for a server — an expired key would otherwise take the poller down every few months.

Tagging replaces the device's user identity, so do this only to machines that exist to run these services. If your "home machine" is also your daily driver, tag it anyway and reach it through the :22 rule above, or skip the tag and accept that any device of yours can reach CDP.

Enforcement is by the destination's packet filter, so the check below is real, not advisory.

Prove the bind is tight, from the home machine itself:

curl -s -m 3 http://$(tailscale ip -4):9222/json/version   # -> JSON
curl -s -m 3 http://<this machine's LAN IP>:9222/json/version
# -> curl: (7) Failed to connect ... Connection refused

The first call is also what wakes Chrome: it is not running until something connects, and it is reaped again after five idle minutes. A cold first response takes a few seconds; that is the browser starting, not a fault.

That check proves the bind, not the ACL — traffic that starts on the node is not filtered. Prove the ACL from somewhere else: on your laptop or phone the same URL must now time out, and from the VPS it must answer.

# on any other device of yours -> hangs until timeout
curl -s -m 5 http://<home machine tailnet IP>:9222/json/version
# on the VPS -> JSON
curl -s -m 20 http://<home machine tailnet IP>:9222/json/version

On the VPS:

cd ~/mangaBookmark
echo 'BROWSER_WS_URL=ws://100.x.y.z:9222' >> .env      # the home machine's tailnet IP
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

It must be the tailnet IP. A MagicDNS hostname fails: Chrome's DevTools HTTP handler answers /json/version with a 500 for any Host header that is not an IP or localhost, and the failure looks like a broken site rather than a broken hostname.

Prove it end to end. This is the only check that says the challenge actually clears from that machine's egress — it fetches a real kagane cover and a real chapter list:

cd backend
SMOKE_BROWSER_WS_URL=ws://100.x.y.z:9222 go test -run TestSmokeKagane ./internal/latest

A red run means "not clearing from this address right now", which is a live fact to re-check before it is a defect — a Site's Cloudflare settings, and the fingerprint this Chrome presents after an update, both move. Then, from the web UI, open a bookmarked kagane series and confirm the cover renders. Once a cover is stored it is served from Postgres forever after, so the browser being asleep, unreachable, or mid-power-outage costs chapter freshness and nothing visible.

Finally, take the VPS free -m reading again and compare it against the one from the top of this section.

Updating the browser is independent of the API stack and has its own runbook — REDEPLOY.md §8.

---

Updating

Pull new code, then rebuild:

docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build

Data persists in the named volume postgres-data across rebuilds. (If this server predates the Postgres migration, the old SQLite volume bookmarks-data is still on disk and deliberately undeclared in compose so down -v cannot take it; see REDEPLOY.md §1 for when to remove it.)

The browser is a separate unit on a separate machine with its own update command — §7. Nothing above touches it, and it needs no coordination: the API picks up a restarted Chrome's new debugger UUID by itself.

---

Troubleshooting

Symptom	Likely cause / fix
No cert / TLS error at the domain	TRAEFIK_ENTRYPOINT or TRAEFIK_CERTRESOLVER name wrong; or DNS not resolving yet. Check docker logs <traefik>.
404 from Traefik	Service not on the proxy network, or BOOKMARK_API_HOST mismatch. Confirm docker network inspect proxy lists bookmark-api.
fetch fails in the userscript, curl works	Origin missing from ALLOWED_ORIGINS, or mixed content (backend not HTTPS).
401 with the right credential	The script's credential no longer matches the stored hash — most likely a rotation happened and the device was not reinstalled. Reinstall from the web UI.
401 after rotation, even right after reinstalling	TOKEN_KEY changed between the rotation and the reinstall; credentials are derived from it, so changing it invalidates every credential. Keep it stable.
Panel button absent	URL didn't match an adapter, or user scripts disabled in Bromite.
compose ... config errors about TOKEN_KEY, OWNER_DISCORD_ID or POSTGRES_PASSWORD	Run compose from the dir with .env, or export the vars. All three are required and none has a fallback.
bookmark-api restarts in a loop, password authentication failed for user "bookmarks"	POSTGRES_PASSWORD was changed after first boot; Postgres only applies it to an empty postgres-data. Restore the old value, or reset the role (REDEPLOY.md troubleshooting).
bookmark-api never logs listening on :8080	It is blocked on postgres passing pg_isready, or a migration failed. docker compose -f docker-compose.yml -f docker-compose.prod.yml logs postgres.
kagane rows never get a latest_chapter; log says browser fetcher disabled or nothing at all	BROWSER_WS_URL unset. Expected before §7 is done.
kagane polls all fail; log shows a 500 from /json/version	BROWSER_WS_URL names a MagicDNS hostname (or any name). Chrome's DevTools handler only accepts an IP or localhost — use the tailnet IP.
kagane polls fail with a connection error	Home machine off, off the tailnet, or the unit is down. tailscale ping <machine>, then docker compose ps in its chrome/. Costs freshness only; stored covers keep serving.
kagane cover is a placeholder for a newly bookmarked series	Its cover has never been fetched and the browser is unreachable. It fills in on the next successful poll of that series.
compose in chrome/ errors set BROWSER_BIND_ADDR to this machine's tailnet IP	No chrome/.env, or the variable is empty. Deliberate — it has no default so an unset value cannot publish CDP to the LAN.
browser container restarts, or is OOM-killed	docker inspect bookmark-browser --format '{{.RestartCount}} {{.State.OOMKilled}}'. The 512 MiB cap is sized against a measured 645 MiB untuned peak; a real breach is a Chrome regression worth reading docker logs for, not a number to raise reflexively.

Backend config reference and endpoint list: see README.md.

---

Installing / updating the userscript

The backend serves the script itself, so Violentmonkey can auto-update it. Complements §4 above — the @downloadURL/@updateURL lines point at the credential-bearing path, so auto-updates come from the same place as the install.

Install once, on the phone (Cromite + Violentmonkey): sign in to the web UI, open the Userscripts panel, and open the install link for the library — the script is served with your credential already inside it. Its @downloadURL/@updateURL point at the same credential-bearing path for updates:

https://bookmark-api.<your-domain>/u/<your credential>/manga-bookmark.user.js

Violentmonkey offers to install it. The credential is in the path because Violentmonkey's update poll sends no Authorization header, and the script embeds the credential in plain text — an open URL would leak it. A wrong credential answers 404. The credential is derived from TOKEN_KEY and never appears anywhere but this URL and the rendered script.

Updating, without a redeploy:

vi userscript/manga-bookmark.user.js   # on the VPS, in this checkout

./userscript is bindmounted read-only into the container and read fresh on every request, so the edit is live immediately. The served @version is derived from the file's mtime (YYYY.MM.DD.HHMM, UTC), not from the @version in the file, so any edit outranks the installed copy and Violentmonkey pulls it on its next check. The @version in the repo is a human marker only.

Updating via redeploy: git pull overwrites the file with the committed version, which is the intended behaviour — a deploy always ships the repo's script. Note that git pull sets mtime to checkout time, so even a rollback serves a higher version and is adopted.

If the mount is missing, the endpoint answers 404 and logs it; bookmark sync is unaffected.