mangaBookmark/REDEPLOY.md

Redeploy runbook

Shipping new code to a server that is already running. First-time setup (DNS, .env, Traefik, installing the userscript) is DEPLOY.md — this file assumes all of that exists and picks up at "there is a running stack and I want it to run the new commit."

Whole thing is ~5 minutes, most of it waiting on docker build. Order matters: back up before you pull. A backup taken after a bad migration is a backup of the damage.

Paths below assume the checkout is at /opt/mangabm; substitute your own. The one absolute rule about paths: backups live in ../mangabm-backups/, a sibling of the project directory (/opt/mangabm-backups), never inside it. It sits outside the repo so git pull, git clean -fd and a bad rm -rf inside the checkout cannot take the backups with them.

/opt/
├── mangabm/            <- the checkout (this repo)
└── mangabm-backups/    <- bookmarks-YYYYmmdd-HHMMSS.db

---

0. Preflight

cd /opt/mangabm

# Both -f flags, every time. The prod override is not standalone.
COMPOSE="docker compose -f docker-compose.yml -f docker-compose.prod.yml"

$COMPOSE ps                        # manga-api should be Up
git status --short                 # expect empty
git log --oneline -1               # note this hash — it is your rollback target
df -h /var/lib/docker | tail -1    # a build needs room

If git status is dirty, someone edited files on the server. Decide before you pull: git stash to keep it, git checkout -- . to discard. A git pull onto a dirty tree fails halfway and leaves you in a worse spot than either.

Create the backup directory once, and make sure it is a sibling, not a child:

mkdir -p ../mangabm-backups
BACKUP_DIR="$(cd .. && pwd)/mangabm-backups"   # absolute — Docker needs it
echo "$BACKUP_DIR"                             # -> /opt/mangabm-backups

---

1. Back up the database

The database is a single SQLite file in the named Docker volume, at /data/bookmarks.db inside the container. Find the volume's real name — Compose prefixes it with the project directory:

docker volume ls --filter name=bookmarks-data
# -> local   mangabm_bookmarks-data
VOL=$(docker volume ls --filter name=bookmarks-data -q | head -1)

Preferred: hot backup, no downtime

The store runs in WAL mode, so recent writes may still be sitting in bookmarks.db-wal. Copying bookmarks.db alone while the container runs can therefore silently drop the newest bookmarks. VACUUM INTO folds the WAL in and writes one consistent file, safe to run against a live database:

STAMP=$(date -u +%Y%m%d-%H%M%S)      # UTC, sorts chronologically as text
docker run --rm \
  -v "$VOL":/data \
  -v "$BACKUP_DIR":/backup \
  alpine sh -c "apk add -q sqlite &&
    sqlite3 /data/bookmarks.db \"VACUUM INTO '/backup/bookmarks-$STAMP.db'\""

ls -lh "$BACKUP_DIR"/bookmarks-$STAMP.db

$STAMP is the "time in the name" — bookmarks-20260730-014233.db. UTC, so the files sort in real order and never collide across a DST shift.

Note the source volume is mounted read-write, which looks wrong for a backup and is not. Opening a WAL database requires creating the -shm shared-memory file; with :ro the command fails with unable to open database file and no backup is produced. VACUUM INTO never writes to the source itself.

Verify it before you trust it. An unreadable backup is worse than none, because you will act as though you have one:

docker run --rm -v "$BACKUP_DIR":/backup alpine sh -c "apk add -q sqlite &&
  sqlite3 /backup/bookmarks-$STAMP.db 'PRAGMA integrity_check;' &&
  sqlite3 /backup/bookmarks-$STAMP.db 'SELECT count(*) FROM bookmarks;'"
# -> ok
# -> 37

The count should match what the web UI shows. Zero rows on a server you know has bookmarks means you backed up the wrong volume.

Fallback: cold copy (no network for apk add sqlite)

Stop the service first, then copy the database and its sidecars — the -wal is not optional, it is where the newest writes are:

$COMPOSE stop
docker run --rm -v "$VOL":/data:ro -v "$BACKUP_DIR":/backup alpine sh -c "
  cp /data/bookmarks.db /backup/bookmarks-$STAMP.db
  [ -f /data/bookmarks.db-wal ] && cp /data/bookmarks.db-wal /backup/bookmarks-$STAMP.db-wal
  [ -f /data/bookmarks.db-shm ] && cp /data/bookmarks.db-shm /backup/bookmarks-$STAMP.db-shm
  ls -1 /backup"
$COMPOSE start

Costs ~10 seconds of downtime. A clean shutdown usually checkpoints the WAL away, so seeing only the .db file is normal and fine — the [ -f ] guards exist for the case where it did not. Restoring this variant means putting whichever files you got back together, under their original names.

Read-only is safe here precisely because nothing opens the database: it is a file copy, not a SQLite connection.

Retention

Keep a month, drop the rest — a bookmark database this small compresses the decision to "disk is free, but not infinite":

ls -1t "$BACKUP_DIR"/bookmarks-*.db | tail -n +31 | xargs -r rm -v

---

2. Pull the new code

git pull --ff-only
git log --oneline -3

--ff-only so a diverged history fails loudly instead of opening a merge you did not plan on the production box.

Check whether .env needs anything new. New config lands in .env.example, and compose fails at start for a missing required var — better to find out now:

git diff HEAD@{1} HEAD -- .env.example docker-compose.yml docker-compose.prod.yml

If a variable was added there, add it to .env before continuing.

---

3. Rebuild and restart

$COMPOSE up -d --build

A rebuild is mandatory for any UI change. The HTML templates, CSS, JavaScript and fonts are compiled into the binary by //go:embed, so editing them on the server — or pulling them — changes nothing until the image is rebuilt. The one exception is userscript/manga-bookmark.user.js, which is bindmounted read-only and read fresh per request.

$COMPOSE ps                        # Up, and recently (re)created
docker logs manga-api --tail 20    # -> "listening on :8080 ..."

Nothing in the log about the database or the poller failing. The image is tagged mangabm-backend:latest, so the previous image is still on disk untagged — that is what makes the rollback in §6 quick.

---

4. Verify the deploy

Same four API checks as DEPLOY.md §3, plus the web UI. Set the host names once:

API=https://manga-api.violetcrown.my.id
WEB=https://manga.violetcrown.my.id
TOKEN=$(grep -E '^API_TOKEN=' .env | cut -d= -f2)

curl -s $API/healthz                                            # -> ok
curl -s -o /dev/null -w '%{http_code}\n' $API/bookmarks         # -> 401
curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks | head -c 200
                                                                # -> your data, not []
curl -s -i -X OPTIONS -H 'Origin: https://asurascans.com' \
  -H 'Access-Control-Request-Method: PUT' \
  $API/bookmarks/x | grep -i access-control                     # -> allow-origin echoed

[] from the third call is the alarm that matters: the volume is not attached and you are looking at an empty database. Stop and check $COMPOSE config --volumes before touching anything else.

Web UI and its assets:

curl -s -o /dev/null -w '%{http_code}\n' $WEB/                  # -> 200 (login page)
curl -s -o /dev/null -w '%{http_code}\n' $WEB/static/style.css  # -> 200

# The fonts are self-hosted; this is what tells you they shipped and are typed.
curl -s -o /dev/null -w '%{http_code} %{content_type}\n' \
  $WEB/static/fonts/instrument-serif-400-latin.woff2
# -> 200 font/woff2

# The served userscript, whose @version is its mtime — a pull bumps it.
curl -s $API/u/$TOKEN/manga-bookmark.user.js | grep '@version'

404 on the font means static/fonts/ did not make it into the image; the UI will still render, in Georgia, which is easy to miss on a phone. application/ octet-stream instead of font/woff2 means an older binary is running.

Then open $WEB in a browser and confirm, in one glance:

• Serif brand and serif row titles — not the system sans fallback.
• A series with an unread chapter has a crimson title on an ember underline; everything else is cool grey. That single detail exercises the whole design path (template class, CSS, and the poller's latest_chapter).
• Tapping the pencil opens the chapter form; Save closes it and the row keeps its place in the list.

---

5. Smoke-test the full loop

The API answering is not the same as the product working. Do this on the phone, against the real sites — it is the only check that covers the userscript, CORS and the shared store together:

1. Open a series on asurascans.com, open the panel, + Bookmark this.
2. On the server: curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks → the series is in the JSON.
3. Open a chapter of it, reopen the panel → last-read shows that chapter, and opening an older chapter does not move it backwards.
4. Open demonicscans.org, open the panel → the Asura bookmark is listed there too. Different origin, one store — this is the whole point of the backend, and the check that fails first when CORS or TLS regressed.
5. Open $WEB → the same series appears, with the same chapter.
6. Turn airplane mode on, tap ★ on a row, turn it off, reopen the panel → the star stuck. That exercises the retry queue.

If 1–5 pass, the deploy is good.

---

6. Rollback

Two independent things can be wrong, so undo only what broke.

Bad code, database fine — go back to the previous commit and rebuild:

git log --oneline -5
git checkout <previous-hash>
$COMPOSE up -d --build

Database damaged — restore the backup from §1. Stop first: the running process holds the WAL, and dropping a file under a live SQLite connection corrupts what you were trying to save.

$COMPOSE stop

docker run --rm -v "$VOL":/data -v "$BACKUP_DIR":/backup alpine sh -c '
  rm -f /data/bookmarks.db /data/bookmarks.db-wal /data/bookmarks.db-shm &&
  cp /backup/bookmarks-<STAMP>.db /data/bookmarks.db &&
  chown 65532:65532 /data/bookmarks.db &&
  ls -l /data'

$COMPOSE start
docker logs manga-api --tail 20
curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks | head -c 200

Two steps here are easy to skip and both bite:

• Delete the stale -wal and -shm. Leaving them beside a restored database mixes two different histories; SQLite will either refuse to open it or quietly reapply writes you meant to discard.
• chown 65532:65532. The image is distroless/static:nonroot and runs as that uid, while the helper container above writes as root. A root-owned database opens read-only-ish: reads work, so /bookmarks looks fine, and then every write fails. That is the worst possible failure mode — it looks restored.

---

7. The whole thing, as one block

For a routine redeploy where nothing needs deciding:

cd /opt/mangabm
COMPOSE="docker compose -f docker-compose.yml -f docker-compose.prod.yml"
BACKUP_DIR="$(cd .. && pwd)/mangabm-backups"; mkdir -p "$BACKUP_DIR"
VOL=$(docker volume ls --filter name=bookmarks-data -q | head -1)
STAMP=$(date -u +%Y%m%d-%H%M%S)

docker run --rm -v "$VOL":/data -v "$BACKUP_DIR":/backup alpine sh -c \
  "apk add -q sqlite && sqlite3 /data/bookmarks.db \"VACUUM INTO '/backup/bookmarks-$STAMP.db'\" &&
   sqlite3 /backup/bookmarks-$STAMP.db 'PRAGMA integrity_check;'" &&
git pull --ff-only &&
$COMPOSE up -d --build &&
sleep 5 &&
curl -sf https://manga-api.violetcrown.my.id/healthz && echo " deploy ok"

The && chain is deliberate: if the backup or its integrity check fails, nothing is pulled and nothing is rebuilt. Then still do §5 by hand — no shell command can tell you the panel works on the phone.

---

Troubleshooting

Symptom	Cause / fix
/bookmarks returns [] after redeploy	Volume not attached — check $COMPOSE config --volumes and that you passed both -f files. Do not re-bookmark; the data is still in the volume.
UI looks like plain Georgia / system sans	static/fonts/ missing from the image, or the browser cached an old style.css. /static/* is served max-age=3600, so hard-reload or wait an hour.
CSS or template change did not appear	You restarted without --build. Assets are //go:embeded.
Font answers application/octet-stream	Old binary — the .woff2 MIME registration is in web.go. Rebuild.
Everyone logged out of the web UI	API_TOKEN or WEB_PASSWORD changed; sessions are derived from both. Expected, just log in again.
compose errors about MANGA_WEB_HOST	Run from the directory holding .env. Both host vars are required even when the web UI is unused.
Userscript did not update on the phone	Violentmonkey polls on its own schedule; force a check. @version comes from the file's mtime, so confirm the pull actually touched it.
apk add sqlite fails (no network)	Use the cold-copy fallback in §1 — and copy bookmarks.db-wal too.
Reads work but every write fails after a restore	Restored file is root-owned; the container is uid 65532. chown 65532:65532 it (§6).
Backup command: unable to open database file	Source volume mounted :ro. WAL needs to create -shm; mount it read-write (§1).

Full first-time setup: DEPLOY.md. Config reference and endpoints: README.md. UI conventions: docs/design-system.md.