CUTOVER.md

@@ -1,7 +1,7 @@
 # SQLite → Postgres cutover runbook
 
 One-way, one-time. Moves the owner's reading history out of the retired SQLite
-volume (`bookmarkmanager_bookmarks-data`, holding `/data/bookmarks.db`) and into
+volume (`<compose project>_bookmarks-data`, holding `/data/bookmarks.db`) and into
 the Postgres schema the migration runner builds. There is no dual-write period:
 the old database is read once, at cutover, from a **fresh export** — anything
 written to SQLite after the export is lost, so the old API must already be down.
@@ -32,10 +32,10 @@ is dropped) — writing it from the spec below costs less than maintaining it
 would.
 
 Beyond `DEPLOY.md`'s prerequisites (Docker and Compose), this runbook needs
-`python3` on the machine running §3 — its stdlib `sqlite3` module is the whole
+`python3`: its stdlib `sqlite3` module is the whole SQLite dependency, and §5's
-SQLite dependency — and `jq` for the one read-path check in §5. Neither has to
+read-path check uses it in place of `jq`, which the server does not have. It
-be the server: §3 only reads the snapshot copy, so it can run on a laptop and
+does not have to run on the server — §3 only reads the snapshot copy, so it can
-the resulting `import.sql` be copied over.
+run on a laptop and the resulting `import.sql` be copied over.
 
 ---
 
@@ -45,21 +45,37 @@ the resulting `import.sql` be copied over.
 already stale.
 
 ```bash
-cd /opt/bookmarkmanager
+cd ~/mangaBookmark                 # wherever the checkout lives
 COMPOSE="docker compose -f docker-compose.yml -f docker-compose.prod.yml"
-BACKUP_DIR="$(cd .. && pwd)/bookmarkmanager-backups"; mkdir -p "$BACKUP_DIR"
+BACKUP_DIR="$(cd .. && pwd)/$(basename "$PWD")-backups"; mkdir -p "$BACKUP_DIR"
 STAMP=$(date -u +%Y%m%d-%H%M%S)
 
+# The volume is <compose project>_bookmarks-data, and the project name defaults
+# to the lowercased *directory* name, not the repo name — on this host the
+# checkout is ~/mangaBookmark, so the volume is mangabookmark_bookmarks-data.
+# Derive it exactly rather than with a `--filter name=` substring match, which
+# would return every volume whose name merely contains the string.
+VOL="$(basename "$PWD" | tr '[:upper:]' '[:lower:]')_bookmarks-data"
+docker volume inspect "$VOL" >/dev/null && echo "$VOL"
+
 $COMPOSE stop bookmark-api
 
-# Copy the file straight out of the retired volume. Nothing is writing to it,
+# A clean SIGTERM closes the store, which checkpoints and unlinks the -wal, so
-# so a plain copy is consistent — no -wal to worry about after a clean stop.
+# bookmarks.db alone is then the whole database. But `compose stop` SIGKILLs
-docker run --rm -v bookmarkmanager_bookmarks-data:/from:ro -v "$BACKUP_DIR":/to \
+# after 10s, and a surviving -wal holds writes the main file does not — assert
+# it is gone rather than assuming the shutdown was clean.
+docker run --rm -v "$VOL":/d:ro alpine ls -l /d      # -> bookmarks.db, alone
+
+docker run --rm -v "$VOL":/from:ro -v "$BACKUP_DIR":/to \
   alpine cp /from/bookmarks.db "/to/bookmarks-$STAMP.db"
 
 ls -lh "$BACKUP_DIR/bookmarks-$STAMP.db"
 ```
 
+If `-wal` and `-shm` are still there, the container was killed mid-write. Copy
+all three under the same basename and let SQLite replay the log when §3 opens
+it — copying only `bookmarks.db` silently drops whatever the log still holds.
+
 Work on a **copy** of that file for the rest of this runbook. The export is the
 last line of retreat; nothing below should be able to write to it.
 
@@ -256,7 +272,8 @@ would catch a correct import behind a broken join:
 ```bash
 API=https://bookmark-api.violetcrown.my.id
 TOKEN=$(grep -E '^API_TOKEN=' .env | cut -d= -f2)   # grace-window credential
-curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks | jq 'length'   # -> 29
+curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks |
+  python3 -c 'import json,sys; print(len(json.load(sys.stdin)))'   # -> 29
 ```
 
 ---
@@ -265,8 +282,10 @@ curl -s -H "Authorization: Bearer $TOKEN" $API/bookmarks | jq 'length'   # -> 29
 
 - **Keep the old SQLite volume for a month.** It is already undeclared in
   compose, so `docker compose down -v` cannot take it. Remove it by hand once
-  the Postgres data has been trusted for a while:
+  the Postgres data has been trusted for a while. That happens in a shell where
-  `docker volume rm bookmarkmanager_bookmarks-data` (see `REDEPLOY.md` §1).
+  `$VOL` from §1 is long gone, so re-derive it:
+  `docker volume rm "$(basename ~/mangaBookmark | tr '[:upper:]' '[:lower:]')_bookmarks-data"`
+  (see `REDEPLOY.md` §1).
 - **Delete the generator and the working copies:** `rm -rf /tmp/cutover`. The
   timestamped export in `$BACKUP_DIR` is the copy that is kept.
 - **Take the first Postgres dump immediately** — `REDEPLOY.md` §1. Until that

DEPLOY.md

@@ -11,7 +11,7 @@ ACME/cert resolver, and control a domain.
 - Docker + Docker Compose on the server.
 - A Traefik instance watching a Docker network (default name assumed: `proxy`).
 - DNS: an `A`/`AAAA` record for `bookmark-api.<yourdomain>` pointing at the server.
-- The repo copied to the server, e.g. `/opt/bookmarkmanager/` (needs `backend/`,
+- The repo copied to the server, e.g. `~/mangaBookmark/` (needs `backend/`,
   `docker-compose.yml`, `docker-compose.prod.yml`, `.env.example`).
 
 Confirm the Traefik network exists (create if not):
@@ -25,7 +25,7 @@ docker network ls | grep proxy || docker network create proxy
 ## 1. Configure `.env`
 
 ```bash
-cd /opt/bookmarkmanager
+cd ~/mangaBookmark
 cp .env.example .env
 ```
 

README.md

@@ -33,6 +33,26 @@ Bromite userscript (isolated world, Shadow DOM UI, localStorage cache)
 | `DATABASE_URL` | *(required)* | Postgres connection URL, e.g. `postgres://bookmarks:…@postgres:5432/bookmarks?sslmode=disable`. Compose builds it from `POSTGRES_PASSWORD`. |
 | `PORT` | `8080` | Plain HTTP; TLS terminated by the proxy. |
 | `BROWSER_WS_URL` | `ws://172.28.0.10:9222` | Headless-shell CDP endpoint used to poll Kagane past its JS challenge. Must be an IP or `localhost` — Chrome's DevTools handler 500s any other Host header. |
+| `DISCORD_CLIENT_ID` | *(required)* | Discord application credentials for the browser sign-in (ADR-0002). |
+| `DISCORD_CLIENT_SECRET` | *(required)* | As above. Never logged, never echoed in an error. |
+| `DISCORD_GUILD_ID` | *(required)* | The one guild whose membership gates sign-in, checked at login only. |
+| `DISCORD_REDIRECT_URI` | *(required)* | Exact callback URL; Discord matches it verbatim against the registered redirect. |
+| `DISCORD_REQUIRED_ROLE` | empty | Role snowflake a member must additionally hold. Empty means guild membership alone suffices. |
+| `DISCORD_API_BASE` | `https://discord.com/api/v10` | Test seam — tests point it at a local stub so the real token exchange runs. |
+| `USERSCRIPT_PATH` | `/userscript/manga-bookmark.user.js` | Bindmounted file served at `/u/{token}/manga-bookmark.user.js`. |
+| `NOVEL_USERSCRIPT_PATH` | `/userscript/novel-bookmark.user.js` | Same, for the novel library. |
+| `LATEST_CHAPTER_POLL_ENABLED` | `1` | `0` turns the poller off entirely. |
+| `LATEST_CHAPTER_POLL_COOLDOWN` | `1h` | Rest between checks of one series; floor `15m`. |
+| `LATEST_CHAPTER_POLL_INTERVAL` | `10m` | How often the poller wakes. Cannot shorten a cooldown. |
+| `LATEST_CHAPTER_POLL_BATCH` | `14` | Series per wake. Keep `BATCH × STAGGER` under `INTERVAL`. |
+| `LATEST_CHAPTER_POLL_STAGGER` | `20s` | Delay between fetches in a batch — this is the outbound request rate. |
+
+Compose reads a few more from the same `.env` that the backend never sees:
+`POSTGRES_PASSWORD` (required — `DATABASE_URL` is built from it, and Postgres
+only applies it while `postgres-data` is empty), `BOOKMARK_API_HOST` and
+`BOOKMARK_WEB_HOST` (required by the prod override), and the optional
+`PROXY_NETWORK` / `TRAEFIK_ENTRYPOINT` / `TRAEFIK_CERTRESOLVER`. Full commentary
+is in `.env.example`; deployment order is `DEPLOY.md`.
 
 ### Endpoints
 

REDEPLOY.md

@@ -9,16 +9,16 @@ Whole thing is ~5 minutes, most of it waiting on `docker build`. Order matters:
 **back up before you pull.** A backup taken after a bad migration is a backup of
 the damage.
 
-Paths below assume the checkout is at `/opt/bookmarkmanager`; substitute your own. The
+Paths below assume the checkout is at `~/mangaBookmark`, which is where it lives
-one absolute rule about paths: **backups live in `../bookmarkmanager-backups/`**, a
+on this deployment; substitute your own. The one absolute rule about paths:
-sibling of the project directory (`/opt/bookmarkmanager-backups`), never inside it. It
+**backups live in a `-backups` sibling of the checkout**, never inside it. It
 sits outside the repo so `git pull`, `git clean -fd` and a bad `rm -rf` inside
 the checkout cannot take the backups with them.
 
 ```
-/opt/
+~/
-├── bookmarkmanager/            <- the checkout (this repo)
+├── mangaBookmark/            <- the checkout (this repo)
-└── bookmarkmanager-backups/    <- bookmarks-YYYYmmdd-HHMMSS.dump
+└── mangaBookmark-backups/    <- bookmarks-YYYYmmdd-HHMMSS.dump
 ```
 
 ---
@@ -26,7 +26,7 @@ the checkout cannot take the backups with them.
 ## 0. Preflight
 
 ```bash
-cd /opt/bookmarkmanager
+cd ~/mangaBookmark
 
 # Both -f flags, every time. The prod override is not standalone.
 COMPOSE="docker compose -f docker-compose.yml -f docker-compose.prod.yml"
@@ -44,9 +44,9 @@ dirty tree fails halfway and leaves you in a worse spot than either.
 Create the backup directory once, and make sure it is a sibling, not a child:
 
 ```bash
-mkdir -p ../bookmarkmanager-backups
+BACKUP_DIR="$(cd .. && pwd)/$(basename "$PWD")-backups"   # absolute — Docker needs it
-BACKUP_DIR="$(cd .. && pwd)/bookmarkmanager-backups"   # absolute — Docker needs it
+mkdir -p "$BACKUP_DIR"
-echo "$BACKUP_DIR"                             # -> /opt/bookmarkmanager-backups
+echo "$BACKUP_DIR"                        # -> /home/sulthan/mangaBookmark-backups
 ```
 
 ---
@@ -59,7 +59,7 @@ network can reach it — so every command below goes in through the container:
 
 ```bash
 $COMPOSE exec -T postgres psql -U bookmarks -d bookmarks -c '\dt'
-# -> bookmarks, schema_migrations, series
+# -> bookmarks, readers, schema_migrations, series, sessions
 ```
 
 Inside the container that connects over the local socket as the `bookmarks`
@@ -99,8 +99,10 @@ you will act as though you have one:
 docker run --rm -v "$BACKUP_DIR":/backup postgres:17-alpine \
   pg_restore --list "/backup/bookmarks-$STAMP.dump" | grep 'TABLE DATA'
 # -> 1234; 0 0 TABLE DATA public bookmarks bookmarks
-# -> 1235; 0 0 TABLE DATA public schema_migrations bookmarks
+# -> 1235; 0 0 TABLE DATA public readers bookmarks
-# -> 1236; 0 0 TABLE DATA public series series
+# -> 1236; 0 0 TABLE DATA public schema_migrations bookmarks
+# -> 1237; 0 0 TABLE DATA public series bookmarks
+# -> 1238; 0 0 TABLE DATA public sessions bookmarks
 
 # 2. Sanity-check the live row count you just captured.
 $COMPOSE exec -T postgres psql -U bookmarks -d bookmarks \
@@ -129,8 +131,11 @@ container stopped the shutdown checkpoint has already flushed everything and a
 plain archive of the volume is consistent.
 
 ```bash
-VOL=$(docker volume ls --filter name=postgres-data -q | head -1)
+# Derived exactly, not with a `--filter name=` substring match plus `head -1`:
-echo "$VOL"                          # -> bookmarkmanager_postgres-data
+# that quietly picks the first of however many volumes happen to contain the
+# string, and archiving the wrong data directory is not a visible failure.
+VOL="$(basename "$PWD" | tr '[:upper:]' '[:lower:]')_postgres-data"
+docker volume inspect "$VOL" >/dev/null && echo "$VOL"   # -> mangabookmark_postgres-data
 
 $COMPOSE stop
 docker run --rm -v "$VOL":/from:ro -v "$BACKUP_DIR":/to alpine \
@@ -161,10 +166,12 @@ ls -1t "$BACKUP_DIR"/bookmarks-*.dump | tail -n +31 | xargs -r rm -v
 declared in `docker-compose.yml` any more, which is what keeps `docker compose
 down -v` from taking it with the rest of the stack. It is not the live database
 and nothing reads it — the one-way move out of it is `CUTOVER.md`. Once the
-Postgres data has been trusted for a while, remove it by hand — nothing else will:
+Postgres data has been trusted for a while, remove it by hand — nothing else will.
+Its full name is `<compose project>_bookmarks-data`, and the project name is the
+lowercased directory name of the checkout:
 
 ```bash
-docker volume rm bookmarkmanager_bookmarks-data
+docker volume rm "$(basename "$PWD" | tr '[:upper:]' '[:lower:]')_bookmarks-data"
 ```
 
 ---
@@ -395,6 +402,7 @@ panel works on the phone.
 | `postgres` never leaves `starting`; `bookmark-api` never starts either | The healthcheck (`pg_isready`) is failing and `bookmark-api` waits on it. `$COMPOSE logs postgres` — usually `postgres-data` was initialised by a different major version ("database files are incompatible with server"), or the disk is full. |
 | `pg_restore`: `cannot drop … other objects depend on it` / `being accessed by other users` | Live connections block `--clean`. `$COMPOSE stop bookmark-api` first (§6). If they persist: `$COMPOSE exec -T postgres psql -U bookmarks -d postgres -c "select pg_terminate_backend(pid) from pg_stat_activity where datname='bookmarks' and pid <> pg_backend_pid()"`. |
 | Dump is 0 bytes, or `pg_restore`: `did not find magic string in file header` | You ran `exec` without `-T`. The allocated TTY rewrites newlines in the binary stream and corrupts the archive in flight (§1). |
+| `git pull`: `could not read Username for 'https://…'` | The checkout's remote is the HTTPS clone URL and the server has no credential helper, so the pull prompts into a closed stdin. Switch it to SSH once — `git remote set-url origin ssh://git@gitea.violetcrown.my.id:2222/sulthan/mangaBookmark.git`. Gitea's SSH listens on **2222**, not 22; port 22 is the host's own sshd and answers `Permission denied (publickey)` no matter which key is registered. |
 
 Full first-time setup: `DEPLOY.md`. The one-off SQLite→Postgres move:
 `CUTOVER.md`. Config reference and endpoints: `README.md`.

backend/internal/web/discord.go

@@ -174,7 +174,7 @@ func (h *Handler) discordCallback(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	member, isMember, err := h.discordMember(r.Context(), tok.AccessToken, userID)
+	member, isMember, err := h.discordMember(r.Context(), tok.AccessToken)
 	if err != nil {
 		h.limiter.Fail(ip, time.Now())
 		log.Printf("discord member check: %v", err)
@@ -272,13 +272,20 @@ type discordMember struct {
 	Roles []string `json:"roles"`
 }
 
-// discordMember fetches the user's membership in the configured guild — the
+// discordMember fetches the current user's membership in the configured guild.
-// single-guild endpoint, not the list of every guild the user is in, so the
+//
-// gate asks exactly the question it names. A 404 or 403 (not in the guild, or
+// This is the OAuth endpoint (Get Current User Guild Member), the one the
-// the token lacks the scope) is a non-member, not an error.
+// guilds.members.read scope grants. Its bot-side twin, GET /guilds/{id}/
-func (h *Handler) discordMember(ctx context.Context, accessToken, userID string) (discordMember, bool, error) {
+// members/{user}, reads almost identically and is the wrong one: it wants a
-	u := h.discord.APIBase + "/guilds/" + url.PathEscape(h.discord.GuildID) +
+// Bot token and the application present in the guild, and answers a user
-		"/members/" + url.PathEscape(userID)
+// Bearer token with 401 — which fails as an outage rather than a refusal, so
+// nobody could sign in at all.
+//
+// A 404 or 403 (not in the guild, or the token lacks the scope) is a
+// non-member, not an error.
+func (h *Handler) discordMember(ctx context.Context, accessToken string) (discordMember, bool, error) {
+	u := h.discord.APIBase + "/users/@me/guilds/" +
+		url.PathEscape(h.discord.GuildID) + "/member"
 	req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
 	if err != nil {
 		return discordMember{}, false, err

backend/web_test.go

@@ -103,7 +103,13 @@ func newDiscordStub(t *testing.T) (*discordStub, *httptest.Server) {
 			if status == http.StatusOK {
 				fmt.Fprintf(w, `{"id":%q,"username":"owner"}`, st.ownerID)
 			}
+		// Discord answers the bot endpoint with 401 for a user Bearer token.
+		// Standing in for that keeps a regression onto it loud: without this
+		// the request would fall through to 404 and read as "not a member",
+		// which is a refusal the caller treats as ordinary.
 		case strings.HasPrefix(r.URL.Path, "/guilds/"):
+			w.WriteHeader(http.StatusUnauthorized)
+		case strings.HasPrefix(r.URL.Path, "/users/@me/guilds/"):
 			st.memberPaths = append(st.memberPaths, r.URL.Path)
 			st.memberAuth = append(st.memberAuth, r.Header.Get("Authorization"))
 			status := st.memberStatus
@@ -299,12 +305,13 @@ func TestDiscordLoginFullFlow(t *testing.T) {
 	}
 
 	// Identity and membership were fetched with the exchanged token, and the
-	// membership check used the single-guild endpoint.
+	// membership check used the OAuth single-guild endpoint — the one
+	// guilds.members.read grants, not its bot-token twin.
 	if len(stub.userAuth) != 1 || stub.userAuth[0] != "Bearer tok-1" {
 		t.Fatalf("users/@me Authorization = %v, want [Bearer tok-1]", stub.userAuth)
 	}
-	if len(stub.memberPaths) != 1 || stub.memberPaths[0] != "/guilds/guild-1/members/owner-snowflake" {
+	if len(stub.memberPaths) != 1 || stub.memberPaths[0] != "/users/@me/guilds/guild-1/member" {
-		t.Fatalf("member requests = %v, want the single-guild endpoint", stub.memberPaths)
+		t.Fatalf("member requests = %v, want the OAuth single-guild endpoint", stub.memberPaths)
 	}
 	if len(stub.memberAuth) != 1 || stub.memberAuth[0] != "Bearer tok-1" {
 		t.Fatalf("member Authorization = %v, want [Bearer tok-1]", stub.memberAuth)