From 01de8903b456532ae0fe1c585cab186cca7dcb47 Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Sat, 8 Aug 2026 15:46:05 +0700 Subject: [PATCH] docs: correct cutover and redeploy runbooks against the real deployment (#26) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dry-running CUTOVER.md against production surfaced four things that would have failed mid-cutover: - The volume is named after the compose project, which is the lowercased directory name (mangabookmark), not the repo name. Both runbooks hardcoded bookmarkmanager_bookmarks-data. Derive it from docker instead. - §1 asserted a clean stop leaves no -wal. compose stop SIGKILLs after 10s, and a surviving -wal holds writes bookmarks.db alone does not, so the export would silently lose them. Check rather than assume. - jq is not installed on the server; §5's read-path check now uses python3, which the runbook already requires. - REDEPLOY §1 still listed the pre-split table set, omitting readers and sessions from both the \dt output and the pg_restore contents. Also records the git-pull failure the redeploy hits on a checkout whose remote is the HTTPS clone URL. --- CUTOVER.md | 38 ++++++++++++++++++++++++++------------ REDEPLOY.md | 15 ++++++++++----- 2 files changed, 36 insertions(+), 17 deletions(-) diff --git a/CUTOVER.md b/CUTOVER.md index a255475..86c5236 100644 --- a/CUTOVER.md +++ b/CUTOVER.md @@ -1,7 +1,7 @@ # SQLite → Postgres cutover runbook One-way, one-time. Moves the owner's reading history out of the retired SQLite -volume (`bookmarkmanager_bookmarks-data`, holding `/data/bookmarks.db`) and into +volume (`_bookmarks-data`, holding `/data/bookmarks.db`) and into the Postgres schema the migration runner builds. There is no dual-write period: the old database is read once, at cutover, from a **fresh export** — anything written to SQLite after the export is lost, so the old API must already be down. @@ -32,10 +32,10 @@ is dropped) — writing it from the spec below costs less than maintaining it would. Beyond `DEPLOY.md`'s prerequisites (Docker and Compose), this runbook needs -`python3` on the machine running §3 — its stdlib `sqlite3` module is the whole -SQLite dependency — and `jq` for the one read-path check in §5. Neither has to -be the server: §3 only reads the snapshot copy, so it can run on a laptop and -the resulting `import.sql` be copied over. +`python3` — its stdlib `sqlite3` module is the whole SQLite dependency, and it +stands in for `jq` in §5, which is not installed on the server. It does not have +to run on the server: §3 only reads the snapshot copy, so it can run on a laptop +and the resulting `import.sql` be copied over. --- @@ -45,21 +45,34 @@ the resulting `import.sql` be copied over. already stale. ```bash -cd /opt/bookmarkmanager +cd ~/mangaBookmark # wherever the checkout lives COMPOSE="docker compose -f docker-compose.yml -f docker-compose.prod.yml" -BACKUP_DIR="$(cd .. && pwd)/bookmarkmanager-backups"; mkdir -p "$BACKUP_DIR" +BACKUP_DIR="$(cd .. && pwd)/$(basename "$PWD")-backups"; mkdir -p "$BACKUP_DIR" STAMP=$(date -u +%Y%m%d-%H%M%S) +# The volume is _bookmarks-data, and the project name defaults +# to the lowercased *directory* name, not the repo name — here that makes it +# mangabookmark_bookmarks-data. Ask Docker instead of typing it out. +VOL=$(docker volume ls -q --filter name=_bookmarks-data); echo "$VOL" + $COMPOSE stop bookmark-api -# Copy the file straight out of the retired volume. Nothing is writing to it, -# so a plain copy is consistent — no -wal to worry about after a clean stop. -docker run --rm -v bookmarkmanager_bookmarks-data:/from:ro -v "$BACKUP_DIR":/to \ +# A clean SIGTERM closes the store, which checkpoints and unlinks the -wal, so +# bookmarks.db alone is then the whole database. But `compose stop` SIGKILLs +# after 10s, and a surviving -wal holds writes the main file does not — assert +# it is gone rather than assuming the shutdown was clean. +docker run --rm -v "$VOL":/d:ro alpine ls -l /d # -> bookmarks.db, alone + +docker run --rm -v "$VOL":/from:ro -v "$BACKUP_DIR":/to \ alpine cp /from/bookmarks.db "/to/bookmarks-$STAMP.db" ls -lh "$BACKUP_DIR/bookmarks-$STAMP.db" ``` +If `-wal` and `-shm` are still there, the container was killed mid-write. Copy +all three under the same basename and let SQLite replay the log when §3 opens +it — copying only `bookmarks.db` silently drops whatever the log still holds. + Work on a **copy** of that file for the rest of this runbook. The export is the last line of retreat; nothing below should be able to write to it. @@ -256,7 +269,8 @@ would catch a correct import behind a broken join: ```bash API=https://bookmark-api.violetcrown.my.id TOKEN=$(grep -E '^API_TOKEN=' .env | cut -d= -f2) # grace-window credential -curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks | jq 'length' # -> 29 +curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks | + python3 -c 'import json,sys; print(len(json.load(sys.stdin)))' # -> 29 ``` --- @@ -266,7 +280,7 @@ curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks | jq 'length' # -> 29 - **Keep the old SQLite volume for a month.** It is already undeclared in compose, so `docker compose down -v` cannot take it. Remove it by hand once the Postgres data has been trusted for a while: - `docker volume rm bookmarkmanager_bookmarks-data` (see `REDEPLOY.md` §1). + `docker volume rm "$VOL"` (see `REDEPLOY.md` §1). - **Delete the generator and the working copies:** `rm -rf /tmp/cutover`. The timestamped export in `$BACKUP_DIR` is the copy that is kept. - **Take the first Postgres dump immediately** — `REDEPLOY.md` §1. Until that diff --git a/REDEPLOY.md b/REDEPLOY.md index 0b2965a..dca70a5 100644 --- a/REDEPLOY.md +++ b/REDEPLOY.md @@ -59,7 +59,7 @@ network can reach it — so every command below goes in through the container: ```bash $COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -c '\dt' -# -> bookmarks, schema_migrations, series +# -> bookmarks, readers, schema_migrations, series, sessions ``` Inside the container that connects over the local socket as the `bookmarks` @@ -99,8 +99,10 @@ you will act as though you have one: docker run --rm -v "$BACKUP_DIR":/backup postgres:17-alpine \ pg_restore --list "/backup/bookmarks-$STAMP.dump" | grep 'TABLE DATA' # -> 1234; 0 0 TABLE DATA public bookmarks bookmarks -# -> 1235; 0 0 TABLE DATA public schema_migrations bookmarks -# -> 1236; 0 0 TABLE DATA public series series +# -> 1235; 0 0 TABLE DATA public readers bookmarks +# -> 1236; 0 0 TABLE DATA public schema_migrations bookmarks +# -> 1237; 0 0 TABLE DATA public series bookmarks +# -> 1238; 0 0 TABLE DATA public sessions bookmarks # 2. Sanity-check the live row count you just captured. $COMPOSE exec -T postgres psql -U bookmarks -d bookmarks \ @@ -161,10 +163,12 @@ ls -1t "$BACKUP_DIR"/bookmarks-*.dump | tail -n +31 | xargs -r rm -v declared in `docker-compose.yml` any more, which is what keeps `docker compose down -v` from taking it with the rest of the stack. It is not the live database and nothing reads it — the one-way move out of it is `CUTOVER.md`. Once the -Postgres data has been trusted for a while, remove it by hand — nothing else will: +Postgres data has been trusted for a while, remove it by hand — nothing else will. +Its full name is `_bookmarks-data`, and the project name is the +lowercased directory name of the checkout, so ask Docker rather than typing it: ```bash -docker volume rm bookmarkmanager_bookmarks-data +docker volume rm "$(docker volume ls -q --filter name=_bookmarks-data)" ``` --- @@ -395,6 +399,7 @@ panel works on the phone. | `postgres` never leaves `starting`; `bookmark-api` never starts either | The healthcheck (`pg_isready`) is failing and `bookmark-api` waits on it. `$COMPOSE logs postgres` — usually `postgres-data` was initialised by a different major version ("database files are incompatible with server"), or the disk is full. | | `pg_restore`: `cannot drop … other objects depend on it` / `being accessed by other users` | Live connections block `--clean`. `$COMPOSE stop bookmark-api` first (§6). If they persist: `$COMPOSE exec -T postgres psql -U bookmarks -d postgres -c "select pg_terminate_backend(pid) from pg_stat_activity where datname='bookmarks' and pid <> pg_backend_pid()"`. | | Dump is 0 bytes, or `pg_restore`: `did not find magic string in file header` | You ran `exec` without `-T`. The allocated TTY rewrites newlines in the binary stream and corrupts the archive in flight (§1). | +| `git pull`: `could not read Username for 'https://…'` | The checkout's remote is the HTTPS clone URL and the server has no credential helper, so the pull prompts into a closed stdin. Switch it to SSH once — `git remote set-url origin ssh://git@gitea.violetcrown.my.id:2222/sulthan/mangaBookmark.git`. Gitea's SSH listens on **2222**, not 22; port 22 is the host's own sshd and answers `Permission denied (publickey)` no matter which key is registered. | Full first-time setup: `DEPLOY.md`. The one-off SQLite→Postgres move: `CUTOVER.md`. Config reference and endpoints: `README.md`.