feat(latest): split browser poll cooldown (#48)
## Summary Split latest-chapter polling cooldowns by fetch cost. Browser-backed kagane and novelfull series now rest longer without changing the cadence of plain-TLS sites. ## Behavior - Plain-TLS series keep the 1h default cooldown. - Browser-backed series use `LATEST_CHAPTER_POLL_BROWSER_COOLDOWN`, defaulting to 6h. - Both cooldowns share the existing 15m minimum floor; invalid values retain the existing fallback behavior. - The poller still selects both classes in one due query per cycle. - Existing ordering and exclusions remain unchanged: reader-count precedence, least-recently-checked ordering, finished exclusion, archived polling, and orphan exclusion. ## Implementation - Added the browser cooldown to backend configuration and passed it through production poller construction. - Added the browser-site list as the single routing source used for both due-query cutoff selection and fetcher choice. - Kept all query values parameterized; the site list is passed as a bound PostgreSQL array parameter. - Updated startup logging to report interval, plain cooldown, browser cooldown, batch, and stagger. - Documented the variable, default, and floor in `README.md`, `.env.example`, `backend/AGENTS.md`, and `docker-compose.yml`. ## Review findings addressed The first review found that configuration parsing was correct but `startLatestPoller` did not pass `BrowserCooldown` into `latest.Poller`; every browser-backed row would therefore have been due immediately. Production construction now goes through `newLatestPoller`, with a regression test covering both cooldown fields. The review also identified duplicated browser-site knowledge in fetch routing. `slices.Contains(browserBackedSites, site)` now reuses the same list already supplied to the store query. ## Verification - Focused backend tests pass: `go test ./internal/latest ./internal/store .`. - Full suite passes: `go test ./...`. - `graphify update .` completed. - Issue #42 was updated and closed. Reviewed-on: #48 Co-authored-by: Sulthan Zaki <sultankiki05@gmail.com> Co-committed-by: Sulthan Zaki <sultankiki05@gmail.com>
This commit was merged in pull request #48.
This commit is contained in:
+9
-7
@@ -67,13 +67,15 @@ DISCORD_REDIRECT_URI=
|
||||
# Set to 0 to turn it off entirely.
|
||||
# LATEST_CHAPTER_POLL_ENABLED=1
|
||||
#
|
||||
# Two independent clocks. COOLDOWN is how long one series rests between checks;
|
||||
# INTERVAL is how often the poller wakes up and looks for series past that
|
||||
# cooldown. Shortening INTERVAL cannot shorten a COOLDOWN.
|
||||
# LATEST_CHAPTER_POLL_COOLDOWN=1h # per series, floor 15m
|
||||
# LATEST_CHAPTER_POLL_INTERVAL=10m # how often to wake
|
||||
# LATEST_CHAPTER_POLL_BATCH=14 # series per wake
|
||||
# LATEST_CHAPTER_POLL_STAGGER=20s # delay between fetches in a batch
|
||||
# Two independent clocks. COOLDOWN is how long a plain-TLS series rests between
|
||||
# checks; BROWSER_COOLDOWN is the longer rest for kagane and novelfull. INTERVAL
|
||||
# is how often the poller wakes up and looks for series past their cooldowns.
|
||||
# Shortening INTERVAL cannot shorten either cooldown.
|
||||
LATEST_CHAPTER_POLL_COOLDOWN=1h # plain-TLS per series, floor 15m
|
||||
LATEST_CHAPTER_POLL_BROWSER_COOLDOWN=6h # browser-backed per series, floor 15m
|
||||
LATEST_CHAPTER_POLL_INTERVAL=10m # how often to wake
|
||||
LATEST_CHAPTER_POLL_BATCH=14 # series per wake
|
||||
LATEST_CHAPTER_POLL_STAGGER=20s # delay between fetches in a batch
|
||||
#
|
||||
# Uses a ticker, not an immediate first run: the first poll happens one
|
||||
# INTERVAL after startup, not at startup. A container restarting more often
|
||||
|
||||
Reference in New Issue
Block a user