Files
mangaBookmark/docs/research/completion-markers.md
T

376 lines
21 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Machine-readable completion markers on the six Sites' Series pages
Research note for Gitea issue #129 (part of #114, "dashboard suggests *potentially
finished* Series"): for each of the six Sites, does the Series page publish a
completion marker a Poll could read — an explicit status field, not a
heuristic — and how trustworthy is it? The dashboard side (storage,
presentation, containment) is issue #130's job, not this note's.
All facts fetched live on **2026-08-19**. Sources are the live pages (URLs
below, fetched 2026-08-19) and the in-tree adapter code
(`backend/internal/latest/…`, branch `main` @ `f5d3fe5`). Every claim carries a
URL or a `file:line`. Interpretation rather than observation is marked
`[INFERENCE]`. Raw fetches archived in `/tmp/cm/` (throwaway).
Fetch method per Site: plain TLS (`curl`) where the site serves real HTML;
real Chrome via the repo's CDP sidecar pattern (Playwright) where a Cloudflare
challenge blocks plain fetch — comix.to and kagane.to always, novelfull.com
time-varying (today it 403'd plain fetch and cleared instantly in Chrome).
---
## 1. Summary verdict table
| Site | Status signal | Fixed vocabulary | Hiatus/dropped ≠ completed | Where the marker lives | Adapter cost |
|---|---|---|---|---|---|
| asurascans.com | **Yes** — visible block + JSON in page state | `{ongoing, completed, hiatus, dropped}` (fixed, verified via browse filter) | **Yes** — four distinct values | Series page Status block; JSON in astro-island props; badge on listing cards; no chapter-list END row | **Low–Medium** |
| demonicscans.org | **Yes** — info-block label/value pair | exactly `{Ongoing, Completed}` | **No** — the site cannot express hiatus/dropped | Series page info block only; search filter; no badges on listings, no END rows | **Medium** |
| comix.to | **Yes** — `status` key in `initial-data` JSON (same blob the adapter parses) + header badge | `{releasing, finished, on_hiatus}` observed; 5 values in browse filter incl. `discontinued`, `not yet released` | **Yes** — `on_hiatus` distinct; `discontinued` a separate value | Series page header badge; `initial-data` JSON; browse-card chips | **Low** |
| kagane.to | **Yes** — **two** explicit fields in the API JSON the adapter already fetches | `{Ongoing, Completed, Hiatus}` observed on both fields (99-series sample); no Dropped seen | **Yes for hiatus** (`upload_status`), **dropped unobservable** | Series API JSON (`publication_status`, `upload_status`); two badges on the page; no status filter in search | **Low** |
| novelfull.com | **Yes** — info-panel status link into `/status/<value>` taxonomy | exactly `{Ongoing, Completed}` (`OnGoing`/`Ongoing` alias); `Hiatus`/`Dropped` 404 | **No** — not expressible | Series page info panel; `/status/<value>` taxonomy pages; no chapter-list END row | **Medium** |
| lightnovelworld.net | **Yes** — visible class-named span **and** JSON-LD `creativeWorkStatus` | `{Ongoing, Completed, Hiatus}` (search filter radios; no dropped) | **Yes** — `PotentialActionStatus` (hiatus) ≠ `CompletedActionStatus` | Series page `.sertostat` span; `<script type="application/ld+json">` in the head; no badges on listings, no END rows | **Low–Medium** |
Bottom line: **all six Sites publish an explicit, machine-readable completion
marker on the Series page.** None requires inferring completion from absence of
updates. Four of six (asura, comix, kagane, lnw) express hiatus as a distinct
value; demonic and novelfull have a binary ongoing/completed vocabulary, which
is *safe* for the Poll (a stalled-but-unlabelled series reads "ongoing") but
cannot surface dropped/hiatus. Only kagane separates *work* status from
*translation/upload* status — the exact distinction the false-positive
question cares about. Details and quotes per Site in §2; the false-positive
verdict in §3.
---
## 2. Per-Site findings
### 2.1 asurascans.com — four-value status, JSON in page state
Fetch: plain TLS, 200, no challenge (all pages below, 2026-08-19). Series URLs
are `/comics/<slug>` where `<slug>` ends in a build hash the site appends
(adapter strips it: `asuraBuildHash`, `sites.go:73`).
**Series page — visible Status block** (ongoing sample,
`https://asurascans.com/comics/chronicles-of-the-demon-faction-f886a8af`):
```html
<div class="text-xs text-white/50 mb-1">Status</div>
<div class="flex items-center gap-2">
<span class="w-2.5 h-2.5 rounded-full bg-[#A78BFA]"></span>
<span class="text-base font-bold text-[#A78BFA] capitalize"> ongoing </span>
</div>
```
The completed sample (`https://asurascans.com/comics/solo-leveling`) renders the
same block with the value `completed`. The label is a plain-text span — the
value must be extracted from the text node, not a class.
**Same value in machine-readable JSON.** The series page state (serialized into
the astro-island props, HTML-escaped in the served document) carries
`&quot;status&quot;:[0,&quot;completed&quot;]` (Solo Leveling) /
`[0,&quot;ongoing&quot;]` (ongoing samples) — one key inside the same props
object the adapter already scans for the chapter list.
**Vocabulary: fixed, four values.** Homepage cards and browse pages embed one
status per card; the browse filter enumerates the set:
`https://asurascans.com/browse?status=completed`, `…?status=hiatus`,
`…?status=dropped` each return 20-card lists whose blobs read
`&quot;status&quot;:[0,&quot;completed&quot;]`, `[0,&quot;hiatus&quot;]`,
`[0,&quot;dropped&quot;]`. Homepage: 72 `ongoing` + 4 `hiatus` card blobs.
**Semantics of `dropped`** — scanlation-editorial, not work-ended
`[INFERENCE, evidence below]`. `https://asurascans.com/comics/demon-king-b60d532c`
is labelled `dropped` and its chapter links stop at `…/chapter/14`; the work
continues elsewhere. So the Poll must treat **only `completed`** as
potentially-finished; `dropped`/`hiatus` mean the site stopped releasing.
**Other marker locations.** Listing cards carry a badge:
`<span class="text-xs font-medium px-2 py-1 rounded capitalize bg-[#913FE2]/20 text-[#A78BFA]"> ongoing </span>`
for ongoing, and `… bg-[#86EFAC]/20 text-[#86EFAC]"> completed </span>` for
completed — the *color* does not discriminate (completed, hiatus and dropped
all share the green `#86EFAC` badge; ongoing is purple `#913FE2`), so a badge
reader must read the text. **No structured chapter-list END row**: Solo
Leveling's final chapter is titled `Side Story 21 { THE END }` — the marker is
prose inside a chapter title, unusable as a signal.
**Adapter touchpoints.** `asuraLatestChapter` (`sites.go:158`) scopes chapter
anchors by `asuraSlugRe` (`sites.go:67`); cover via `ogImageCover`
(`sites.go:334`, wired `sites.go:450`). The `status` key lives in the same
escaped props blob the chapter-list scan already walks — a new regex on the
same body, or a parse of the visible Status block.
### 2.2 demonicscans.org — binary vocabulary, nothing else expressible
Fetch: plain TLS, 200 (2026-08-19). Series URLs `/manga/<title>`.
**Series page — info block** (ongoing sample,
`https://demonicscans.org/manga/Catastrophic-Necromancer`):
```html
<div class="flex flex-row">
<li style="width:150px;color:#b2b2b2;">Status</li>
<li>Ongoing</li>
</div>
```
Completed sample (`https://demonicscans.org/manga/Solo-Leveling`): same markup,
`<li>Completed</li>`. The row sits inside the same info-block `<div>` as
Author / Rating / Last Update — a sibling `<li>` pair per field.
**Vocabulary: exactly `{Ongoing, Completed}`.** The advanced-search filter
(`https://demonicscans.org/advanced.php`) proves the set — the status `<select>`
has two options:
```html
<select class="form-control" name="status">
<option value="all">All</option>
<option value="ongoing">Ongoing</option>
<option value="completed">Completed</option>
</select>
```
There is no hiatus/dropped value anywhere on the site (`[INFERENCE]`: no such
string in the filter or any series page sampled). A scanlation that stalls
silently keeps reading `Ongoing`; the site cannot label a series
dropped/on-hiatus.
**Other marker locations.** None: homepage and `/lastupdates.php` show no
status text or badges on their cards (2026-08-19). No chapter-list END rows
(the last-chapter anchor is a plain `chaptered.php` link).
**Adapter touchpoints.** `demonicLatestChapter` (`sites.go:177`) scans
`chaptered.php?manga=<id>&chapter=<n>` anchors via `demonicChapterRe`
(`sites.go:77`), unscoped; cover `ogImageCover` (`sites.go:334`, wired
`sites.go:457`). A status reader adds a selector over the info-block `<li>`
pair; the block is sloppy HTML (bare `<li>` inside `<div>`s) but stable across
the two sampled themes/versions.
### 2.3 comix.to — `status` key in the exact JSON the adapter already parses
Fetch: Cloudflare JS challenge; cleared through a real Chrome (~12 s), then the
adapter's own payload shape was fetched in-tab (2026-08-19). The payload is
server-rendered HTML carrying `<script id="initial-data">…</script>` JSON
(`comixInitialDataRe`, `sites.go:248`); the Series detail entry
`["manga","detail","<slug>"]` contains `"status"` and the chapter list.
Observed values (three sampled series, all 2026-08-19):
| Series | URL | `status` | Header badge |
|---|---|---|---|
| Dungeons & Crayons (ongoing) | `https://comix.to/title/n8we-dungeons-and-crayons` | `"releasing"` | `RELEASING` |
| Countach (completed) | `https://comix.to/title/q77m-countach` | `"finished"` | `FINISHED` |
| How to Survive as the Daughter… (hiatus) | `https://comix.to/title/d0n08-how-to-survive-as-the-daughter-of-the-emperor-who-killed-me` | `"on_hiatus"` | `ON_HIATUS` |
The visible header badge (hiatus sample):
```html
<span class="mpage__badge mpage__badge--status mpage__badge--status-on_hiatus">ON_HIATUS</span>
```
The class encodes the value (`--status-on_hiatus`), the text is the raw
uppercase snake_case label.
**Vocabulary: five fixed values** from the browse advanced filter ("RELEASE
STATUS"): `Releasing`, `Finished`, `On hiatus`, `Discontinued`,
`Not yet released`. The filter URL encodes snake_case:
`https://comix.to/browse?statuses=on_hiatus` (plural `statuses`; a plain
`?status=finished` probe did *not* filter the list — titles of other statuses
remained). `discontinued` and `not yet released` were not observed on series
pages but are site-selectable filter values `[INFERENCE: they map to
`discontinued`/`not_yet_released` keys]`.
**Hiatus/dropped ≠ completed: yes** — `on_hiatus` is a distinct JSON value and
`discontinued` is a separate filter option.
**Other marker locations.** Browse cards render status chips (FINISHED /
RELEASING seen); the header badge above; the browse filter itself. No
chapter-list END row (chapter entries are per-chapter rows).
**Adapter touchpoints.** `comixLatestChapter` (`sites.go:185`) already parses
this exact `["manga","detail","<slug>"]` entry for the chapter URL — `status`
is a sibling key of the entry, so the read cost is near-zero. Cover
`comixCoverEntry` (`sites.go:344`). Browser-backed read `comixRead`
(`browser.go:154`), `Done` excludes the interstitial (`sites.go:471`).
### 2.4 kagane.to — two explicit status fields in the API JSON the adapter already fetches
Fetch: Cloudflare JS challenge; cleared through a real Chrome (12–32 s, flaky;
the challenge occasionally re-fires). The adapter's payload shape was fetched
in-tab: the Series page itself requests
`https://kagane.to/api/v2/series/<uuid>` (`kaganeSeriesRe`, `browser.go:25`) and
the poller consumes that JSON (`kaganeRead`, `browser.go:128`). Series URLs are
`/series/<uuid>`, e.g. `https://kagane.to/series/019f84bc-9ba0-7ed9-86f5-8b905ec7c28b`
(Infinite Decryption: The Strongest Level 0).
The API JSON carries **two explicit status fields** plus year bounds:
```json
"publication_status": "Ongoing",
"upload_status": "Ongoing",
"start_year": 2025,
"end_year": null,
```
- `publication_status` — the work's status (Ongoing / Completed / Hiatus).
- `upload_status` — the site's own release status.
**They diverge — proven.** "'Cause Calypso Can"
(`https://kagane.to/series/019c2034-f431-73ac-9350-11bc498f8c40`):
`"publication_status": "Ongoing"` but `"upload_status": "Hiatus"` — the work is
ongoing, the site's release is paused. This is precisely the
translation-stopped-vs-work-ended distinction the dashboard's false-positive
question needs, and kagane is the **only** Site that separates the two.
**Vocabulary.** Across a 99-series sample (search page 1, in-tab re-POST of the
page's own `POST /api/v2/search/series?page=0&size=99` request with body
`{"content_rating":["Safe","Suggestive"]}` — the page's own request returned
38 results, a direct re-POST 403'd even with Referer), both fields take only
`{Ongoing, Completed, Hiatus}`. **No Dropped value observed** on either field
in the sample; a dropped release presumably stays `Hiatus` or `Ongoing`
`[INFERENCE]`. The search page exposes no status filter (filters: Format,
Genres, Tags, People, Sources, Content Rating, Language, Presets).
**Page renders two badges** (value in text only, class carries no value), the
hiatus one yellow:
```html
<span class="inline-flex items-center rounded-full border px-2.5 py-0.5 … bg-yellow-500/10 text-yellow-600 dark:text-yellow-400">Hiatus</span>
```
**Adapter touchpoints.** `kaganeLatestChapter` (`sites.go:197`) scans
`"chapter_no":"<n>"` in the API body (`kaganeChapterRe`, `sites.go:98`);
cover `kaganeCoverEntry` (`sites.go:349`). `publication_status`/`upload_status`
are top-level keys of the same body — read cost near-zero.
### 2.5 novelfull.com — binary status, taxonomy-linked
Fetch: time-varying challenge. Plain TLS today returned HTTP 403 with a
"Just a moment…" interstitial (~5.7 KB); the same URL cleared instantly through
a real Chrome, and the cleared DOM is the adapter's payload shape (`novelfullRead`,
`browser.go:141`; `Done` excludes the interstitial, `sites.go:501`).
**Series page — info panel** (completed sample,
`https://novelfull.com/reverend-insanity.html`):
```html
<div><h3>Status:</h3><a href="/status/Completed">Completed</a></div>
```
Ongoing sample (`https://novelfull.com/a-cunning-pervert-in-the-cultivation-world.html`):
`<a href="/status/Ongoing">Ongoing</a>` in the same block.
**Vocabulary: exactly `{Ongoing, Completed}`**, proven by the taxonomy pages:
`https://novelfull.com/status/Ongoing` and `https://novelfull.com/status/OnGoing`
both return 200 with identical listings (the two spellings alias; the page title
normalizes to "Status: Ongoing"), while `https://novelfull.com/status/Hiatus` and
`https://novelfull.com/status/Dropped` return 404. No hiatus/dropped state
exists on the site.
**Hiatus/dropped ≠ completed: no** — not expressible; a stalled novel keeps
reading `Ongoing` indefinitely. Safe direction for the Poll (nothing falsely
"completed"), but the site cannot surface stalled releases.
**Other marker locations.** The `/status/<value>` taxonomy listing pages; the
series-page info panel only. Homepage latest-updates cards carry no status
(2026-08-19). **No chapter-list END row** (chapter rows are plain anchors;
checked `reverend-insanity.html`'s cleared DOM).
**Adapter touchpoints.** `novelfullLatestChapter` (`sites.go:204`) scopes
chapter anchors by `novelfullSlugRe` (`sites.go:103`); cover `novelfullCoverEntry`
(`sites.go:339`). The status `<a href="/status/…">` in the info panel is a new,
independent selector on the same body.
### 2.6 lightnovelworld.net — class-named span + schema.org JSON-LD
Fetch: plain TLS, 200 (2026-08-19). Series URLs `/novel/<slug>/`.
**Visible marker — `.sertostat` span, class = value** (completed sample,
`https://lightnovelworld.net/novel/a-will-eternal/`):
```html
<div class="sertostat">
<span class="Completed">Completed</span>
</div>
```
Ongoing sample (`https://lightnovelworld.net/novel/all-jobs-and-classes-i-just-wanted-one-skill-not-them-all/`):
`<span class="Ongoing">Ongoing</span>`; hiatus sample
(`https://lightnovelworld.net/novel/rebuild-world-wn/`): `<span class="Hiatus">Hiatus</span>`.
**Machine-readable marker — JSON-LD in the head.** The same pages carry a
`<script type="application/ld+json">` block (line 469 of the a-will-eternal
fetch) with a schema.org `Book` entry whose `creativeWorkStatus` is the
**completion signal, mapped to the ActionStatus vocabulary**:
| Sample | `creativeWorkStatus` |
|---|---|
| A Will Eternal (completed) | `"https://schema.org/CompletedActionStatus"` |
| All Jobs and Classes… (ongoing) | `"https://schema.org/ActiveActionStatus"` |
| Rebuild World (hiatus) | `"https://schema.org/PotentialActionStatus"` |
**Vocabulary: `{Ongoing, Completed, Hiatus}`.** The homepage search filter
enumerates it: radio inputs `id="anime_status-ongoing"`,
`id="anime_status-completed"`, `id="anime_status-hiatus"` — no dropped option.
**Hiatus ≠ completed: yes** — `PotentialActionStatus` (hiatus) is a distinct
JSON-LD value from `CompletedActionStatus`; the Poll can treat only the latter
as potentially-finished.
**Other marker locations.** The `.sertostat` span only. Homepage/listing cards
carry **no** status badges (2026-08-19 — earlier word hits on the homepage were
the filter radios and a series title containing "Dropped"). **No chapter-list
END row**: the chapter list is `epl-num`/`epl-title` rows with no end marker on
the completed A Will Eternal.
**Adapter touchpoints.** `lnwLatestChapter` (`sites.go:222`) scopes chapter
anchors by `lnwChapterRe` (`sites.go:115`) and truncates at the
`lnwCommentMarker` ("wpd-threads", `sites.go:124`); cover `ogImageCover`
(`sites.go:334`, wired `sites.go:508`). The JSON-LD block is a separate script
in the head — one regex or JSON parse away; the `.sertostat` span is in the
article body.
---
## 3. The false-positive question — cross-site verdict
The dashboard wants to *suggest* "potentially finished" without false
positives (issue #129 context). Per Site, what would a Poll have to believe to
mark a Series finished, and can that belief be wrong?
| Site | Poll reads | False-positive exposure |
|---|---|---|
| asurascans.com | `status == completed` only | `dropped`/`hiatus` are distinct values — reading them as anything but completed is a false *negative*, not a false positive. `completed` is the site's own editorial call on a finished work (Solo Leveling). Safe. |
| demonicscans.org | `Status == Completed` | Binary vocabulary: `Completed` is the site's label and hiatus/dropped are unexpressible, so a label never lies *about completion* — but "Completed" is site-editorial and could mark a translation ended early. Poll a labelled `Completed` series as a *suggestion*, not a fact. |
| comix.to | `status == finished` | `on_hiatus` and `discontinued` are distinct values; only `finished` triggers. Same editorial caveat. Safe. |
| kagane.to | `publication_status == Completed` **or** `upload_status == Completed` | **The only Site where the two questions are answered separately.** Work-finished should read `publication_status`; `upload_status == Completed` is the site's release-complete flag and can coexist with `publication_status == Ongoing` (inverse of the Calypso Can case `[INFERENCE]`). The Poll should decide which field drives "potentially finished". |
| novelfull.com | `Status == Completed` | Binary vocabulary; hiatus/dropped unexpressible (taxonomy 404s). A labelled `Completed` (e.g. Reverend Insanity, The Legendary Mechanic) is the site's call. No hiatus false-positive path exists. |
| lightnovelworld.net | `creativeWorkStatus == CompletedActionStatus` | `PotentialActionStatus` (hiatus) is distinct; only `CompletedActionStatus` triggers. Safest single-value reading of the six. |
Cross-site verdict: **no Site's status field is owner-written
`series.finished_at` data** (that stays the owner's prerogative, #114/#124
context) — every value here is the site's editorial status. But **none of the
six forces the Poll to guess**: all publish explicit markers, and only asura
and kagane introduce *translation-state* vocabulary (`dropped` / `upload_status`)
that a naive "status == completed" reader could misinterpret — in both cases
the danger is a false *negative* (treating a dropped/hiatus series as still
ongoing), not a false positive, **provided the Poll reads only the
completed/finished value** and treats anything else as not-completed.
---
## 4. Adapter touchpoints (for #130's containment decision)
| Site | Marker source | Existing parse this piggybacks on | New parse needed |
|---|---|---|---|
| asura | escaped `&quot;status&quot;:[0,&quot;…&quot;]` in astro-island props, or visible Status block | `asuraLatestChapter` walks the same props blob (`sites.go:158`) | one regex on the same body |
| demonic | info-block `<li>` pair | — (`demonicLatestChapter` scans chapter anchors only, `sites.go:177`) | selector over the info block |
| comix | `["manga","detail",…]."status"` in `initial-data` | `comixLatestChapter` parses that exact entry (`sites.go:185`, `sites.go:248`) | sibling key read |
| kagane | `publication_status` / `upload_status` top-level keys | `kaganeLatestChapter` reads the same API body (`sites.go:197`) | sibling key read (×2) |
| novelfull | info-panel `<a href="/status/…">` | — (`novelfullLatestChapter` scopes chapter anchors, `sites.go:204`) | selector over the info panel |
| lightnovelworld | head JSON-LD `creativeWorkStatus`, or `.sertostat` span | — (`lnwLatestChapter` truncates at `wpd-threads`, `sites.go:222`) | one JSON-LD regex / parse |
Challenge status for a future Poll: asura, demonic, lnw are plain-fetch; comix
and kagane require the browser sidecar on every poll (Cloudflare challenge
present today, 2026-08-19); novelfull requires the sidecar *whenever its
challenge is up* (it was up today, 2026-08-19).