[davidschachterADFA]

ADFA-6589

The documentation side of ADFA-6589: keep documentation.db's pages from referring to content on the internet. Code On The Go #2161 already stops the app from connecting; this PR changes the content, so pages don't show blanks where remote content used to be, and searches for remote addresses don't find false ones.

Four new steps, run before anything else, in this order (see Run order):

Step	Commits	What it does on the 2026-10-13 database
remove_skin_font_comment.py	7715805	Removes a commented-out Google Fonts block from 362 pages
bundle_remote_images.py	37d7c5b, 237e110	Stores 19 API diagrams (203,413 bytes); points 21 <img> on 11 pages at them or a stored WebP
remove_remote_loads.py	f9b6513	Drops 14 remote srcsets and 7 dark-theme icons; 4 badges become words; 44 YouTube players become links
check_remote_loads.py	cd647c2, a0c3d04	Reads every page and fails the build on an off-device load in any shape it lists: 90 on 35 pages before the steps, 0 after

Result: 0 off-device loads remain, down from 90 on 35 pages, and a new remote load in any of the shapes listed under Step 4 now fails the build.

Step 1: Help+Manual's commented-out Google Fonts block

Every Help+Manual page (all 362 text/html rows under i/ in the 2026-10-13 database) carries the same 1,005-byte HTML comment from the Help+Manual skin. It's author guidance ("Insert Google web fonts references here. ...") with an example fonts.googleapis.com stylesheet link.

• It loads nothing. But every search for internet addresses counts it, which is how ADFA-6589 first reported 362 Google Fonts loads that don't exist.
• The lasting fix is in the skin: ADFA-6609, assigned to the Help+Manual author. Until that lands, this step removes the block when the database is built.

New scripts/remove_skin_font_comment.py DB [--dry-run] (7715805):

• What it removes: a comment only if its content's sha256 matches the known block. It cuts only the <!--...--> bytes. All other comments stay: the ZOOMSTOP/ZOOMRESTART markers that search highlighting needs, and the skin's own markers.
• What it reports: any other comment, or text, carrying the block's opening sentence is left in place, reported, and makes the run exit 1. So a changed skin note is never removed or kept without notice.
• Writes: only changed rows are rewritten, with their own compression, including the shared Brotli dictionary. Chunked rows are handled.
• Run shape: one transaction, with a backup and VACUUM only when something changes. A second run changes nothing. --dry-run writes nothing.

Step 2: markup that loads files from the internet

New scripts/remove_remote_loads.py DB [--dry-run] (f9b6513) changes four kinds of element. A page works without any of them, but each fetches a file from the internet:

• Remote srcset: 14 <img> on three a/devsite/ pages have a stored src and a srcset on developer.android.com. A WebView prefers the srcset, so offline the image shows broken. The srcset goes, with the sizes that only qualifies it.
• Dark-theme icons: 7 <source media="(prefers-color-scheme: dark)"> on the stylus page point at icons the database doesn't hold. They go, and dark mode shows the light icon.
• kotl.in badges: 4 stability badges become their level as text, for example <code>Stable</code>. Only the four known levels match, and the alt text must name the badge's own level.
• YouTube players: 44 players on 20 Kotlin pages become links, "TITLE (video on YouTube)". In the app, tapping one asks before opening the browser (CodeOnTheGo #2161).

How it edits:

• The Kotlin pages are JSON. They are edited as bytes inside their JSON strings, and parsed before and after the edit.
• An element that resembles these but differs is left in place, named, and makes the run exit 1.

Step 3: the API diagrams pages load from developer.android.com

21 <img> on 11 API reference pages (MediaPlayer, Activity, Media3's Timeline and others) load 20 diagrams. The 19 the database doesn't hold are now in scripts/remote_media/adfa-6589/, with a files.tsv that records each file's source URL, database path, content type, size, sha256 and license:

• License: developer.android.com/license gives Apache 2.0 for documentation and CC BY 2.5 for other content. Both allow redistribution with attribution.
• Fetching: each of the 20 was fetched once, and every size matches the server's.
• Contents: the 14 SVGs have no script and no external references. Their <font-face> elements name local fonts only.

New scripts/bundle_remote_images.py DB [--dry-run] (37d7c5b):

• Checks first: it checks every file against files.tsv. On any mismatch it writes nothing and exits 1.
• Storing: each file becomes a Content row at a/devsite/media/ + the URL path with / turned into _, as the other devsite media are stored. The content type and compression come from the database.
• Rewriting: only an exact src="URL" on an <img> is rewritten, to a relative path, as all 726 existing references to a/devsite/media/ are.
• Guards: a file is stored only while a page still loads it. A path already holding different bytes is reported, and its pages are left alone.

Activity's lifecycle diagram (237e110, ADFA-6614):

• Already stored: the database holds it as a/devsite/media/guide_components_images_activity_lifecycle.webp, which the lifecycle guide page shows. Side by side the two are the same picture: the PNG is 513x663 and the WebP 500x646.
• How it's used: a new existing.tsv beside files.tsv lists that row for the PNG's URL. So Activity.html points at the WebP, and nothing is stored, saving 45,666 bytes.
• Checked first: the row must hold exactly the recorded bytes, under the recorded type. Otherwise the page is left loading the URL and the run exits 1.

Step 4: fail the build on any off-device load left

The first three steps fix the shapes they were written for, and each reports only near-misses of its own shapes. An <iframe>, a stylesheet <link> or a CSS url() on another host would pass all three with exit 0. New scripts/check_remote_loads.py DB (cd647c2, from the review's R41-01) only reads.

What counts as a load:

• Addresses: absolute http(s) or protocol-relative (//host), on any host but localhost, 127.x and [::1].
• Places:
  • src, srcset, any data-*src or data-*srcset, poster and background on any element, and <object data>;
  • <link> unless its rel loads nothing, such as canonical or alternate;
  • <base href>, <meta http-equiv=refresh>;
  • SVG href;
  • <a ping> and form actions;
  • CSS url(), @import and image-set() in style attributes, <style> blocks and CSS rows, with CSS escapes decoded;
  • <iframe srcdoc>;
  • <?xml-stylesheet?>;
  • in JSON pages, every src and the markup inside strings.

It exits 1 naming each row, the URL and the shape.

What doesn't count:

• <a href>, since the app asks before opening a link;
• HTML comments and XML namespaces.

JavaScript: decoded, so an unreadable script fails the run, but not searched. 27 of the 412 script rows name an http(s) URL, and all are comments, namespaces, URL-parsing bases or share buttons. A remote <script src> is still caught.

Shared by the three steps that write:

• One transaction, with a backup and VACUUM only when something changes.
• Rows that don't change are never recompressed.
• A second run changes nothing, and --dry-run writes nothing.

Run order

These four steps run first, in this order: remove_skin_font_comment.py, bundle_remote_images.py, remove_remote_loads.py, check_remote_loads.py. In particular they run before the cp DB migrated.db snapshot.

• Why bundle comes before remove (R41-05, b24e9f0): bundle has to point an <img src> at its stored file before remove_remote_loads drops that <img>'s remote srcset. On this database both orders give identical results. update_media_references.py's bundle identifies pages by the digest of their decoded text, so running this step after it would make a restore skip the pages it changed. i/hmftsearch.html is a real case. The lineage check and the media bundle are unaffected.

Results on the 2026-10-13 database

Step 1:

Check	Result
Rows changed	362, all text/html under i/; no variants, nothing skipped
Changed rows	each equals the original with exactly the one block removed (byte comparison, decoded independently)
All other rows, tables, schema	byte-identical
Stored bytes	Content -129,888; database file 175,982,592 -> 175,799,296
Second run / dry run	0 changes; file unchanged
Remaining fonts.googleapis mentions	0 of 29,253 text rows
PRAGMA integrity_check	ok

Steps 2 and 3:

Check	Result
Elements changed	A 14 <img> on 3 pages; B 7 on 1; D 4 on 1; YouTube 44 on 20; C 21 <img> on 11
Rows	35 pages changed (21 of them JSON), 19 added, 0 removed; no id, language, type or template changes
Byte comparison	an independent prediction from the original pages matches all 35, byte for byte
All other rows, schema	byte-identical; only sqlite_sequence moved (19 inserts)
JSON pages	all 21 parse
New rows	all 19 decode to the committed files
check_remote_loads.py	0 loads after, exit 0; 90 loads on 35 pages before, exit 1
Second run / integrity_check	0 changes / ok
#38/#40 pipeline afterwards (at 37d7c5b)	exit 0; the new rows and 21 references intact and resolving; scan still 0

How these numbers were measured:

• They were measured at 37d7c5b and then compared again at 237e110, which stores 19 files and rewrites the same 21 references.
• Against the 37d7c5b run, only the PNG row is missing, and Activity.html differs only in that one src, which now names the WebP. Every other row is identical, the WebP row is untouched, and a second run changes nothing.
• The ADFA-3305: Convert animated GIFs to VP9 WebM and pack documentation.db #38/ADFA-6596: Convert the documentation videos to VP9 WebM from their originals #40 pipeline wasn't rerun at 237e110.
• The new order (b24e9f0) gives a database identical to the old order's: every Content row and every other table.
• What the 90 loads are: 21 <img src>, 14 <img srcset>, 7 <source srcset>, 4 badges and 44 YouTube players. They are the same 151 page-and-URL pairs the earlier scan found. Its 149 counted distinct URLs, and the 14 srcsets alone hold 75 of them.

Testing

• New scripts/test_remove_skin_font_comment.py, 12 tests:

  • the exact span;
  • every other byte kept;
  • pages without the block left byte for byte;
  • a changed note reported with exit 1;
  • idempotency and dry run;
  • non-HTML rows untouched;
  • a dictionary database;
  • a chunked page;
  • an unreadable row.

• Mutations: each test fails when the behaviour it is named for is removed (17 mutations). One was rerun independently: dropping the digest check fails the changed-note test.

• New scripts/test_remove_remote_loads.py, 22 tests, and scripts/test_bundle_remote_images.py, 22 tests:

  • plain and dictionary Brotli;
  • JSON and HTML pages;
  • chunked pages and unreadable rows;
  • idempotency and dry run;
  • unexpected shapes reported with exit 1;
  • all other bytes kept;
  • a tampered file refused;
  • the committed files checked against files.tsv;
  • a URL in existing.tsv pointed at its row, and refused if the row is missing, holds other bytes or has another type.

• Mutations: 58 were run against the two new scripts at 37d7c5b, and 9 against 237e110's guards. Each was killed by the test named for it.

• New scripts/test_check_remote_loads.py, 24 tests:

  • the review's 13 shapes, each reported;
  • each case that doesn't count, not reported;
  • JSON pages, a dictionary database, a chunked page, an unreadable row;
  • data-*src attributes, <!-->, CSS escapes, /* in a CSS string, and a tooltip that opens with { (a0c3d04).

  76 of 77 mutations are killed. The 77th opens the database read-write, which no test can tell apart, because the step writes nothing either way.

• Review fixes:

  • test_bundle_remote_images.py grew to 26 tests (R41-02, R41-03, R41-07);
  • .gitattributes keeps scripts/remote_media/** from line-ending conversion (R41-04, 57293fd), checked with a core.autocrlf=true export;
  • the second review's three findings (R41-09 to R41-11), fixed in a0c3d04, with 10 mutants each killed by the test named for it.

• Neighbouring suites: the four new suites pass, 84 tests. Across all of scripts/, 171 pass, and the same 4 failures and 5 collection errors occur before and after this PR (missing scour, faker, openpyxl, tqdm_loggable, and the older test_android_html_page.py). test_media_bundle.py gives 21 passed; its other 3 tests need scour, which isn't installed here, and this PR doesn't change that suite or the optimizer.

• Python 3.11: the scripts compile under it.

• python-tests.yaml doesn't run scripts/ tests, so these ran only locally (ADFA-6615).

Corrections to pushed commit messages

These commits are pushed, so they stay as they are:

• 7715805: "all 362 text/html rows under i/" should read "362 of the 366". The other four are Help+Manual template files that lack the note. The script's docstring is fixed (f11bbb2).
• 37d7c5b:
  • "(149 before)" counts distinct URLs, with each srcset candidate counted separately. The count of loads is 90 on 35 pages.
  • "With both ADFA-6589 steps" should read "all three".
• 237e110: "every other row is identical" should read "every other row's content is identical". The 19 new rows' ids are each one lower, because the PNG is no longer stored.

🤖 Generated with Claude Code

https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3

[davidschachterADFA]

Independent review

An independent Claude Code review covered all four commits (7715805, f9b6513, 37d7c5b, 237e110).

No critical or high findings: one medium, three low and four nits.

• On the 2026-10-13 database every step came out right, and the pages and new rows are exact.
• The medium finding is about future databases: no step catches a remote load in a shape it wasn't written for.

All are open. File:line references are to 237e110.

ID	Sev	File:line	Defect
R41-01	medium	bundle_remote_images.py:272	Other remote-load shapes pass all three steps with exit 0
R41-02	low	test_bundle_remote_images.py:509	No test pins existing.tsv's sha256 check
R41-03	low	bundle_remote_images.py:159,168	Four guards have no test that fails without them
R41-04	low	remote_media/adfa-6589/*.svg	No .gitattributes: a CRLF checkout breaks 14 SVG digests
R41-05	nit	remove_remote_loads.py:73	"Their order does not matter" is not always true
R41-06	nit	remove_skin_font_comment.py:13	Counts in a docstring and commit messages are off
R41-07	nit	bundle_remote_images.py:246,249	Two messages say the wrong thing
R41-08	nit	PR #40 body, Run order	The run order lists only one of the three new steps

R41-01 (medium). Each tool reports only near-misses of its own shapes.

• Test page: a fixture page with 13 other remote loads, run through both tools:
  • an <img src> on another host, and an unlisted developer.android.com URL;
  • a listed file as http:// and as //developer.android.com/;
  • a srcset on another host;
  • a YouTube <iframe> and youtube-nocookie;
  • CSS url();
  • <source src> in a <video>;
  • data-src;
  • a Google Fonts <link rel=stylesheet>;
  • a JSON image block with a remote src, and a JSON html block with an iframe.
• Result: both tools exited 0 and reported nothing.
• A docstring claim it breaks: bundle's docstring says any other appearance of a listed URL is named, but the http:// and // forms are not.
• Why it matters: none of these shapes is in this database, so nothing is wrong today. But the pipeline reruns for each release, and the ticket's aim is "never".
• Suggested fix: a check step that runs last before the snapshot and exits 1 on any off-device load.

R41-02. The "changed" case (WEBP + b"x") also changes the size. A size-only check passes every test (reproduced). Suggested fix: a case with the same size and one byte flipped, as test_a_tampered_file_is_refused does for files.tsv.

R41-03. These mutants survive:

• the URL query/fragment check (:159);
• the image/ content-type check (:168), since the "bad digest" case gets the same message without it;
• find_sites' b">" not in page[tag:start] guard;
• the "lists no files" check when only existing.tsv has entries.

R41-04.

• Reproduced: with core.autocrlf=true, all 14 SVGs come out CRLF and fail their sha256. The step then refuses to run. It fails safe, but it can't run on such a checkout (no .gitattributes, reproduced).
• Suggested fix: scripts/remote_media/** -text.

R41-05.

• Mixed tag: take an <img> with a listed remote src and a remote srcset.
  • remove_remote_loads first: it exits 1, then bundle fixes the src, and the srcset stays.
  • bundle first: both are fixed.
• This database: both orders give identical results.
• Suggested fix: name the order, bundle before remove_remote_loads.

R41-06.

• Skin step: the docstring and 7715805 say "all 362 text/html rows under i/". There are 366: four Help+Manual template files lack the note.
• "149 before" (37d7c5b): this counts distinct URLs, with each srcset candidate counted separately, not loads.
• "Both ADFA-6589 steps" (37d7c5b): there are three.
• "Every other row is identical" (237e110): the 19 new rows' contents are identical, but their ids shift by one.

R41-07.

• The "N URL(s) in existing.tsv name images already in the database" line prints before the database is opened.
• A problem in existing.tsv is reported as "do not match files.tsv".

Checked and found correct

• Step 1: 362 rows each lose exactly the one comment, and the comment's sha256 matches. No fonts.googleapis mention remains. The search highlighter does use the ZOOMSTOP and ZOOMRESTART markers that are kept.
• Step 2:
  • 24 rows and 69 elements, and all 24 JSON pages parse.
  • All 43 distinct YouTube ids and their titles carry over.
  • Each badge level matches its src and alt.
  • The dark-theme icons aren't stored.
  • The HTML stays well-formed.
• Step 3:
  • The new rows have the right ids, types and compression: PNG and GIF raw, SVG compressed against the dictionary. Each decodes to its committed file.
  • Every relative src resolves.
  • At 237e110: 19 files stored and the same 21 references. Only the PNG row is gone, Activity.html differs in that one src, and the WebP decodes to existing.tsv's sha256.
• Everything else:
  • All other rows and tables are byte-identical apart from sqlite_sequence, and integrity_check is ok.
  • A second run writes nothing, and dry runs write nothing.
  • Both step orders give identical results on this database.
• Sibling sweep: 0 off-device loads remain. This covered every load attribute, iframes, CSS url() and @import, JSON image blocks, and CSS, SVG and JS files.
• The 20 files:
  • Sizes and sha256 match files.tsv.
  • The SVGs have no script, external href or font file.
  • files.tsv is the only license record; the repo has none for the 682 existing devsite media.
• ADFA-3305: Convert animated GIFs to VP9 WebM and pack documentation.db #38/ADFA-6596: Convert the documentation videos to VP9 WebM from their originals #40 pipeline afterwards: the new rows and references survive. With PNG added to --types, 6 references move to WebP correctly, the existing.tsv WebP isn't renamed, and --restore-originals works.
• Tests and mutants:
  • All suites pass at each commit.
  • 47 mutations were run; apart from those in R41-02 and R41-03, each was caught by the test named for it.

Not verified:

• the license wording, since there was no network;
• rendering on a device;
• the video part of the pipeline.

🤖 Generated with Claude Code

[davidschachterADFA]

Review answered

All seven findings on this PR are fixed in 57293fd..cd647c2. R41-08 belongs to #40's description and is fixed there.

ID	Sev	Status	Commit
R41-01	medium	Fixed: new check_remote_loads.py fails the build on any off-device load, in any shape	cd647c2
R41-02	low	Fixed: a same-size, one-byte-flipped existing.tsv case; the size-only mutant now fails	65f6476
R41-03	low	Fixed: a case for each of the four guards, plus two more gaps found in a sweep of all 33 checks	65f6476
R41-04	low	Fixed: .gitattributes marks scripts/remote_media/** as -text	57293fd
R41-05	nit	Fixed: the order is named (bundle before remove), with a test of the mixed tag	b24e9f0
R41-06	nit	Docstring fixed; the pushed commit messages are corrected in the description	f11bbb2
R41-07	nit	Fixed: messages name the list at fault and claim only what was checked	ac04c2d
R41-08	nit	To be fixed in #40's description, with the full run order

R41-01:

• What it checks: the step reads every page, stylesheet, SVG and JSON page, and every Tooltips row. Any address that would load from off the device is reported with its row, URL and shape, and the run exits 1. The description lists what counts and what doesn't.
• JavaScript is decoded but not searched. 27 script rows name URLs, all of them comments, namespaces, URL-parsing bases or share buttons; counting them would fail every build. A remote <script src> is still caught.
• On the 2026-10-13 database: 90 loads on 35 pages before the steps, and 0 after. Those are the same 151 page-and-URL pairs the earlier scan found; its 149 counted distinct URLs.
• Independent probe: a separate check, apart from the step's 21 tests, used 11 more shapes. These included //host, an upper-case scheme, backslashes, a two-word rel, poster, CSS background-image, <embed>, a mixed srcset, an SVG <image>, a CSS @import and a JSON image block. All 11 were reported. A link, a commented-out <img> and localhost addresses were not.

How it was checked at cd647c2:

• Suites: the four new suites pass (81 tests). Across all of scripts/, the failures and collection errors are the same before and after; they come from packages not installed here.
• Mutations:
  • 76 of 77 against the check step are killed. The 77th opens the database read-write, which no test can see.
  • 12 more against bundle_remote_images.py are killed.
  • The R41-02 size-only mutant was rerun independently and fails test_an_existing_row_that_is_missing_changed_or_retyped_is_reported.
• Real data, on a copy, in the new order:
  • the same 35 pages and 362 i/ pages change, and 19 rows are added;
  • the database is identical to the old order's;
  • the check reports 0, and a second run of each step changes nothing.
• Python 3.11: every changed file compiles there.

Not fixed:

• relative_reference's resolve check: no test pins it. Only an existing.tsv path with . or .. segments could reach it, and it raises before anything is written.
• Two other bundle_remote_images.py mutants survive. The check_files size comparison and find_sites' tag >= 0 change nothing a test can see, because the sha256 and the <img match already do their work.

🤖 Generated with Claude Code

[davidschachterADFA]

Independent review of the fix commits

An independent Claude Code review covered 57293fd, f11bbb2, 65f6476, ac04c2d, b24e9f0 and cd647c2, which answer the first independent review (R41-01 to R41-08). It has 1 low and 2 nits. Every finding is fixed as claimed, every real-data figure reproduces, and all 31 mutants I ran against check_remote_loads.py are killed. The new check still misses a few shapes, so "any shape" claims too much. All are open.

ID	Sev	File:line	Finding
R41-09	low	check_remote_loads.py:134	data-nav-src, which the shipped sidebar.js fetches, is not counted
R41-10	nit	check_remote_loads.py:143,306	<!-->, CSS escapes and /* in a CSS string hide a load
R41-11	nit	check_remote_loads.py:402	A tooltip starting with { fails the build as "does not parse as JSON"

R41-09.

• Reproduced: a page holding <aside id="sidebar" data-nav-src="https://e.com/nav.html"></aside>, run through check_remote_loads.run: exit 0, nothing reported.
• Cause: URL_ATTRIBUTES counts data-src because lazy loaders fetch it. page.peb writes data-nav-src="/k/html/_nav.html", and the database's own assets/sidebar.js does fetch(sidebar.getAttribute("data-nav-src")). A remote value in a page, a JSON page's markup or the template would be a load the step passes.
• Real data: the only data-nav-src is the local one in page.peb.
• Suggested fix: add data-nav-src to URL_ATTRIBUTES, or count every data-*src attribute, failing safe.

R41-10.

• Reproduced with check_text(..., "markup") on Python 3.14.4; each reports nothing:
  • <!--><img src="https://e.com/x.png">-->, and the same with <!--->. In HTML5 both are complete empty comments, so a browser loads the <img>; html.parser reads them as opening a comment that runs to the next -->. (--!> is handled.)
  • <div style="background:url(htt\70://e.com/x.png)"> and url(http\3a //e.com/x.png). CSS decodes escapes, so a browser fetches http://e.com/x.png.
  • <style>a{content:"/*"} b{background:url(http://e.com/x.png)} /* */</style>. _CSS_COMMENT starts a comment inside the string and removes the url().
• Why it matters: the PR body says "a new remote load in any shape now fails the build", and the response says "in any shape". None of these shapes is in the 2026-10-13 database; without comment stripping the check still finds 0 loads after the steps, so no remote url() sits only in a CSS comment there.
• Suggested fix: replace <!--> and <!---> with <!----> before parsing; decode CSS escapes before matching; skip strings when stripping CSS comments. Or say "every shape listed" instead of "any shape".

R41-11.

• Reproduced: a database whose one tooltip has summary { x -> x * 2 } is a lambda: exit 1, "Tooltips 1 (kotlin.Function) summary: does not parse as JSON, so what its template renders cannot be checked".
• Cause: tooltip text is read with the "markup" reader, which takes any text starting with { for a JSON page. Templates got their own "template" reader for the same reason; tooltips did not.
• Real data: 0 tooltips start with {, so today's run passes. A Kotlin lambda tooltip would stop the build with a misleading message.
• Suggested fix: read tooltips with the "template" reader (markup, never JSON).

Earlier findings

ID	Verdict	Evidence
R41-01	Fixed, with R41-09 and R41-10 left	90 loads on 35 rows before the steps and 0 after, on a copy. All 31 mutants killed, among them: no backslash or protocol-relative handling, loopback counted, every <link> counted, JSON src or markup ignored, Templates or Tooltips unread, no chunk reassembly, unreadable rows passing.
R41-02	Fixed	A size-only check of existing.tsv rows fails test_an_existing_row_that_is_missing_changed_or_retyped_is_reported.
R41-03	Fixed	Removing the query/fragment check, the prefix check, the image/ check, the > guard, the "lists no files" check, the src=" check or the missing-file check each fails a test.
R41-04	Fixed	A core.autocrlf=true export of scripts/remote_media gives 14 of 19 digests wrong at 237e110 and 0 at cd647c2. Emptying .gitattributes fails test_git_never_converts_the_committed_files.
R41-05	Fixed	Both orders give identical databases on a copy: all 21 tables row for row, and the schema. The docstrings and CLAUDE.md name the order.
R41-06	Fixed	366 text/html rows under i/; 362 hold the note once; the four without it are the files the docstring names.
R41-07	Fixed	Restoring the old "do not match files.tsv" ending for a list problem fails test_an_existing_tsv_not_as_written_is_refused.
R41-08	Fixed	#38's and #40's run orders list the four steps, in this order.

Checked and found correct

• Real data at cd647c2, on copies of the 2026-10-13 database, in the new order:
  • the skin step changes 362 rows (129,888 stored bytes); bundle stores 19 files and rewrites 21 references on 11 pages; remove changes 24 rows, 69 elements;
  • the check finds 90 loads on 35 rows before (21 <img src>, 14 <img srcset>, 7 <source srcset>, 4 JSON <img src>, 44 JSON <video src>) and 0 after, exit 0;
  • a second run of each step changes nothing, and the file's sha256 is unchanged; integrity_check is ok.
• Script rows: 27 of 412 name an http(s) address, on 15 hosts plus two template placeholders, as the docstring says.
• Shapes the check does catch, beyond its tests: <img/src=...>, an entity-encoded scheme, a comment closed by --!>, <track src>, <input type=image src>, rel="preload" and upper-case STYLESHEET, a quoted meta refresh URL, a tab inside a srcset, @import in a style attribute and in an SVG <style>, @font-face sources. SVG data: URIs used as images load nothing and are rightly not counted.
• Tests at cd647c2 (Python 3.14): 12, 22, 26 and 21 pass, 81 in all, under pytest and as plain scripts.

Not verified:

• How the WebView renders the changed pages.
• The license wording of the stored diagrams (no network).

🤖 Generated with Claude Code

[davidschachterADFA]

Response to the review of the fix commits

ID	Status	What changed
R41-09	Fixed in a0c3d04	Every data-*src and data-*srcset attribute counts, data-nav-src among them
R41-10	Fixed in a0c3d04	<!--> and <!---> read as empty comments; CSS escapes decoded; /* in a CSS string opens no comment
R41-11	Fixed in a0c3d04	Tooltips are read with the "template" reader: markup, never a JSON page

Decisions to check:

• R41-09: every data-*src counts rather than adding data-nav-src alone, failing safe as the reviewer offered. data-source and other names not ending in src or srcset still do not count.
• R41-10: all three suggested fixes. Comments are removed before escapes are decoded, so an escaped /\2a opens no comment, as in a browser. The description now says "any of the shapes listed under Step 4" instead of "any shape": a script can still build a URL that no pattern sees.

Checks:

• Python 3.14: 12, 26, 22 and 24 pass, 84 in all, under pytest and as plain scripts. Across scripts/, 171 pass, with the same 4 failures (scour missing, and test_android_html_page.py) and 5 collection errors as before. Both changed files compile under Python 3.11.16.
• 10 mutants, each applied alone; each fails its test:
  • R41-09 revert (only data-src and data-srcset), or data-*src narrowed to data-src -> test_every_data_src_and_data_srcset_attribute_is_reported;
  • data-*srcset not read -> that test and test_every_element_and_attribute_that_loads_is_reported;
  • R41-10: no comment rewrite, <!--> rewritten but not <!--->, escapes not decoded, a hex escape's trailing space kept, the old comment regex that ignores strings, and escapes decoded before comments are removed -> test_markup_and_css_a_parser_misreads_are_read_as_a_browser_reads_them, each at its own assertion;
  • R41-11 revert (tooltips read as "markup") -> test_a_tooltip_opening_with_a_brace_is_markup_not_a_json_page.
• On Python 3.14 and 3.11, html.parser reads <!--> and <!---> as opening a comment that hides the <img> after it, and reads <!----> as a complete one. Inside a real comment, the rewrite ends the comment where a browser ends it.
• Real data, a copy of the 2026-10-13 database, with the check at cd647c2 and at a0c3d04:
  • before the steps, both exit 1 with the same 90 loads on 35 rows, line for line;
  • the three steps in the documented order change 362 rows (129,888 stored bytes), store 19 files and rewrite 21 references on 11 pages, then change 24 rows and 69 elements;
  • after them, both exit 0 with "Nothing loads from off the device", and their --verbose output is identical.

🤖 Generated with Claude Code