From 3f680b58603118298407a4fabbe224f67ab917fe Mon Sep 17 00:00:00 2001 From: Sulthan Zaki Date: Wed, 19 Aug 2026 19:19:51 +0700 Subject: [PATCH] Research: completion markers on the six Sites' Series pages (#129) --- docs/research/completion-markers.md | 375 ++++++++++++++++++++++++++++ 1 file changed, 375 insertions(+) create mode 100644 docs/research/completion-markers.md diff --git a/docs/research/completion-markers.md b/docs/research/completion-markers.md new file mode 100644 index 0000000..a308cd3 --- /dev/null +++ b/docs/research/completion-markers.md @@ -0,0 +1,375 @@ +# Machine-readable completion markers on the six Sites' Series pages + +Research note for Gitea issue #129 (part of #114, "dashboard suggests *potentially +finished* Series"): for each of the six Sites, does the Series page publish a +completion marker a Poll could read — an explicit status field, not a +heuristic — and how trustworthy is it? The dashboard side (storage, +presentation, containment) is issue #130's job, not this note's. + +All facts fetched live on **2026-08-19**. Sources are the live pages (URLs +below, fetched 2026-08-19) and the in-tree adapter code +(`backend/internal/latest/…`, branch `main` @ `f5d3fe5`). Every claim carries a +URL or a `file:line`. Interpretation rather than observation is marked +`[INFERENCE]`. Raw fetches archived in `/tmp/cm/` (throwaway). + +Fetch method per Site: plain TLS (`curl`) where the site serves real HTML; +real Chrome via the repo's CDP sidecar pattern (Playwright) where a Cloudflare +challenge blocks plain fetch — comix.to and kagane.to always, novelfull.com +time-varying (today it 403'd plain fetch and cleared instantly in Chrome). + +--- + +## 1. Summary verdict table + +| Site | Status signal | Fixed vocabulary | Hiatus/dropped ≠ completed | Where the marker lives | Adapter cost | +|---|---|---|---|---|---| +| asurascans.com | **Yes** — visible block + JSON in page state | `{ongoing, completed, hiatus, dropped}` (fixed, verified via browse filter) | **Yes** — four distinct values | Series page Status block; JSON in astro-island props; badge on listing cards; no chapter-list END row | **Low–Medium** | +| demonicscans.org | **Yes** — info-block label/value pair | exactly `{Ongoing, Completed}` | **No** — the site cannot express hiatus/dropped | Series page info block only; search filter; no badges on listings, no END rows | **Medium** | +| comix.to | **Yes** — `status` key in `initial-data` JSON (same blob the adapter parses) + header badge | `{releasing, finished, on_hiatus}` observed; 5 values in browse filter incl. `discontinued`, `not yet released` | **Yes** — `on_hiatus` distinct; `discontinued` a separate value | Series page header badge; `initial-data` JSON; browse-card chips | **Low** | +| kagane.to | **Yes** — **two** explicit fields in the API JSON the adapter already fetches | `{Ongoing, Completed, Hiatus}` observed on both fields (99-series sample); no Dropped seen | **Yes for hiatus** (`upload_status`), **dropped unobservable** | Series API JSON (`publication_status`, `upload_status`); two badges on the page; no status filter in search | **Low** | +| novelfull.com | **Yes** — info-panel status link into `/status/` taxonomy | exactly `{Ongoing, Completed}` (`OnGoing`/`Ongoing` alias); `Hiatus`/`Dropped` 404 | **No** — not expressible | Series page info panel; `/status/` taxonomy pages; no chapter-list END row | **Medium** | +| lightnovelworld.net | **Yes** — visible class-named span **and** JSON-LD `creativeWorkStatus` | `{Ongoing, Completed, Hiatus}` (search filter radios; no dropped) | **Yes** — `PotentialActionStatus` (hiatus) ≠ `CompletedActionStatus` | Series page `.sertostat` span; `` JSON +(`comixInitialDataRe`, `sites.go:248`); the Series detail entry +`["manga","detail",""]` contains `"status"` and the chapter list. + +Observed values (three sampled series, all 2026-08-19): + +| Series | URL | `status` | Header badge | +|---|---|---|---| +| Dungeons & Crayons (ongoing) | `https://comix.to/title/n8we-dungeons-and-crayons` | `"releasing"` | `RELEASING` | +| Countach (completed) | `https://comix.to/title/q77m-countach` | `"finished"` | `FINISHED` | +| How to Survive as the Daughter… (hiatus) | `https://comix.to/title/d0n08-how-to-survive-as-the-daughter-of-the-emperor-who-killed-me` | `"on_hiatus"` | `ON_HIATUS` | + +The visible header badge (hiatus sample): + +```html +ON_HIATUS +``` + +The class encodes the value (`--status-on_hiatus`), the text is the raw +uppercase snake_case label. + +**Vocabulary: five fixed values** from the browse advanced filter ("RELEASE +STATUS"): `Releasing`, `Finished`, `On hiatus`, `Discontinued`, +`Not yet released`. The filter URL encodes snake_case: +`https://comix.to/browse?statuses=on_hiatus` (plural `statuses`; a plain +`?status=finished` probe did *not* filter the list — titles of other statuses +remained). `discontinued` and `not yet released` were not observed on series +pages but are site-selectable filter values `[INFERENCE: they map to +`discontinued`/`not_yet_released` keys]`. + +**Hiatus/dropped ≠ completed: yes** — `on_hiatus` is a distinct JSON value and +`discontinued` is a separate filter option. + +**Other marker locations.** Browse cards render status chips (FINISHED / +RELEASING seen); the header badge above; the browse filter itself. No +chapter-list END row (chapter entries are per-chapter rows). + +**Adapter touchpoints.** `comixLatestChapter` (`sites.go:185`) already parses +this exact `["manga","detail",""]` entry for the chapter URL — `status` +is a sibling key of the entry, so the read cost is near-zero. Cover +`comixCoverEntry` (`sites.go:344`). Browser-backed read `comixRead` +(`browser.go:154`), `Done` excludes the interstitial (`sites.go:471`). + +### 2.4 kagane.to — two explicit status fields in the API JSON the adapter already fetches + +Fetch: Cloudflare JS challenge; cleared through a real Chrome (12–32 s, flaky; +the challenge occasionally re-fires). The adapter's payload shape was fetched +in-tab: the Series page itself requests +`https://kagane.to/api/v2/series/` (`kaganeSeriesRe`, `browser.go:25`) and +the poller consumes that JSON (`kaganeRead`, `browser.go:128`). Series URLs are +`/series/`, e.g. `https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b` +(Infinite Decryption: The Strongest Level 0). + +The API JSON carries **two explicit status fields** plus year bounds: + +```json +"publication_status": "Ongoing", +"upload_status": "Ongoing", +"start_year": 2025, +"end_year": null, +``` + +- `publication_status` — the work's status (Ongoing / Completed / Hiatus). +- `upload_status` — the site's own release status. + +**They diverge — proven.** "'Cause Calypso Can" +(`https://kagane.to/series/019c2034-f431-73ac-9350-11bc498f8c40`): +`"publication_status": "Ongoing"` but `"upload_status": "Hiatus"` — the work is +ongoing, the site's release is paused. This is precisely the +translation-stopped-vs-work-ended distinction the dashboard's false-positive +question needs, and kagane is the **only** Site that separates the two. + +**Vocabulary.** Across a 99-series sample (search page 1, in-tab re-POST of the +page's own `POST /api/v2/search/series?page=0&size=99` request with body +`{"content_rating":["Safe","Suggestive"]}` — the page's own request returned +38 results, a direct re-POST 403'd even with Referer), both fields take only +`{Ongoing, Completed, Hiatus}`. **No Dropped value observed** on either field +in the sample; a dropped release presumably stays `Hiatus` or `Ongoing` +`[INFERENCE]`. The search page exposes no status filter (filters: Format, +Genres, Tags, People, Sources, Content Rating, Language, Presets). + +**Page renders two badges** (value in text only, class carries no value), the +hiatus one yellow: + +```html +Hiatus +``` + +**Adapter touchpoints.** `kaganeLatestChapter` (`sites.go:197`) scans +`"chapter_no":""` in the API body (`kaganeChapterRe`, `sites.go:98`); +cover `kaganeCoverEntry` (`sites.go:349`). `publication_status`/`upload_status` +are top-level keys of the same body — read cost near-zero. + +### 2.5 novelfull.com — binary status, taxonomy-linked + +Fetch: time-varying challenge. Plain TLS today returned HTTP 403 with a +"Just a moment…" interstitial (~5.7 KB); the same URL cleared instantly through +a real Chrome, and the cleared DOM is the adapter's payload shape (`novelfullRead`, +`browser.go:141`; `Done` excludes the interstitial, `sites.go:501`). + +**Series page — info panel** (completed sample, +`https://novelfull.com/reverend-insanity.html`): + +```html +

Status:

Completed
+``` + +Ongoing sample (`https://novelfull.com/a-cunning-pervert-in-the-cultivation-world.html`): +`Ongoing` in the same block. + +**Vocabulary: exactly `{Ongoing, Completed}`**, proven by the taxonomy pages: +`https://novelfull.com/status/Ongoing` and `https://novelfull.com/status/OnGoing` +both return 200 with identical listings (the two spellings alias; the page title +normalizes to "Status: Ongoing"), while `https://novelfull.com/status/Hiatus` and +`https://novelfull.com/status/Dropped` return 404. No hiatus/dropped state +exists on the site. + +**Hiatus/dropped ≠ completed: no** — not expressible; a stalled novel keeps +reading `Ongoing` indefinitely. Safe direction for the Poll (nothing falsely +"completed"), but the site cannot surface stalled releases. + +**Other marker locations.** The `/status/` taxonomy listing pages; the +series-page info panel only. Homepage latest-updates cards carry no status +(2026-08-19). **No chapter-list END row** (chapter rows are plain anchors; +checked `reverend-insanity.html`'s cleared DOM). + +**Adapter touchpoints.** `novelfullLatestChapter` (`sites.go:204`) scopes +chapter anchors by `novelfullSlugRe` (`sites.go:103`); cover `novelfullCoverEntry` +(`sites.go:339`). The status `` in the info panel is a new, +independent selector on the same body. + +### 2.6 lightnovelworld.net — class-named span + schema.org JSON-LD + +Fetch: plain TLS, 200 (2026-08-19). Series URLs `/novel//`. + +**Visible marker — `.sertostat` span, class = value** (completed sample, +`https://lightnovelworld.net/novel/a-will-eternal/`): + +```html +
+ Completed +
+``` + +Ongoing sample (`https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/`): +`Ongoing`; hiatus sample +(`https://lightnovelworld.net/novel/rebuild-world-wn/`): `Hiatus`. + +**Machine-readable marker — JSON-LD in the head.** The same pages carry a +`