Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
12b9920
ADFA-3305: Store the a/ reference pages as JSON plus shared templates
davidschachterADFA Oct 19, 2026
66249e1
ADFA-3305: Refuse a database that already declares version 3
davidschachterADFA Oct 19, 2026
621e78e
ADFA-3305: Test a block's footer and a render that fails
davidschachterADFA Oct 19, 2026
94b9d29
ADFA-3305: Write the line-break and space characters as escapes
davidschachterADFA Oct 19, 2026
1fc3fb9
ADFA-3305: Say that ADFA-5548's pipeline converts the same pages
davidschachterADFA Oct 19, 2026
48a3734
ADFA-3305: Say why this step may follow the row drops
davidschachterADFA Oct 19, 2026
0765f83
ADFA-3305: Retire ADFA-5548's Android docs workflows
davidschachterADFA Oct 19, 2026
032829e
ADFA-3305: Store texts that five or more a/ pages repeat once, in bun…
davidschachterADFA Oct 20, 2026
f3f84b0
ADFA-3305: Give the shared texts' figures from this branch's ship-tim…
davidschachterADFA Oct 20, 2026
8c3ffc8
ADFA-3305: Group shared texts into bundles by the pages that use them
davidschachterADFA Oct 20, 2026
713ab3e
ADFA-3305: Give the shared texts' figures with bundles grouped by page
davidschachterADFA Oct 20, 2026
af27f50
ADFA-3305: Let a merged group merge again with a lower-numbered one (…
davidschachterADFA Oct 20, 2026
9388896
ADFA-3305: Let a merge fill a bundle exactly (R56-02)
davidschachterADFA Oct 20, 2026
a20f1e3
ADFA-3305: Test the shared texts' boundaries (R56-02)
davidschachterADFA Oct 20, 2026
1151c74
ADFA-3305: Test shared texts holding Pebble syntax and line breaks (R…
davidschachterADFA Oct 20, 2026
592a1f6
ADFA-3305: Give the shared texts' figures for the fixed packing; say …
davidschachterADFA Oct 20, 2026
05db3ae
Merge pull request #56 from appdevforall/fix/ADFA-3305-android-interning
davidschachterADFA Oct 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
333 changes: 0 additions & 333 deletions .github/workflows/build-android-docs-local.yaml

This file was deleted.

382 changes: 0 additions & 382 deletions .github/workflows/build-android-docs.yaml

This file was deleted.

2 changes: 0 additions & 2 deletions .github/workflows/build-kotlin-docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,6 @@ permissions:

# This workflow overwrites a single shared Drive file - never let two runs
# race to upload against each other.
# Shared with build-android-docs.yaml: both rewrite the same documentation.db on Drive, so they
# have to serialise against each other, not merely against a second run of themselves.
concurrency:
group: documentation-db
cancel-in-progress: false
Expand Down
21 changes: 21 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,27 @@ uses. Any future Android/Java tooltip work should target the normalized `Tooltip
(ADFA-6589); this step ships after both. A media rollback (`--restore-originals`) cannot undo its
pages: start the rollback from this step's pre-run backup, keep that backup with the rollback
bundle, and run this step again afterwards.
- **`scripts/android_pages_to_templates.py`** - stores the Android reference pages under `a/`
(except `a/devsite/`) as JSON plus `android-reference.peb` and one shared include per
inherited-member block header (`android-inherited-<n>.peb`), with every text of 32 or more
characters that 5 or more pages hold (link holes excepted) stored once in a shared-text bundle
(`android-text-<n>.peb`, at most 8,000 bytes of source each, texts grouped by the pages that use
them), after rendering every page back
byte for byte with the app's engine (`scripts/hm_render_check/`, as above); a page that does not
render back, or is over the app's render limit, stays HTML (ADFA-3305). Run it on the ship-time
copy only, never on the master: after the steps that write `a/` pages, `drop_unread_rows.py` and
the Kotlin trim, before the 3.0.0 migration (which compresses every template; the app reads
compressed templates from documentation schema 3.0.0, CodeOnTheGo stacked on #2163) and the final
dictionary re-mint.
It refuses a database that already holds its templates, or that declares documentation version 3
or later (whose templates are compressed, so its plain ones would not render).
Following `drop_unread_rows.py` is an exception to that step's "after every page-writing step"
rule, and safe: every converted page renders back byte for byte, so it links to nothing the row
drops did not already check.
ADFA-5548's `ProcessDocs/AndroidDocs/AndroidDocsToJson/load_android_json_db.py` converts the same
`a/android/` and `a/androidx/` pages its own way. This step is the one that ships: the workflows
that ran that pipeline on the master (`build-android-docs.yaml`, `build-android-docs-local.yaml`)
were retired in ADFA-3305, never having run, since after it this step would convert almost nothing.
- **`scripts/myServer.py`** — a minimal Python `http.server` reference implementation that predates
`WebServer.kt`. It queries a differently-cased `Documentation.db`, doesn't implement Brotli
decompression (there's a literal `TODO: Replace this function with Brotli decompression`), and
Expand Down
38 changes: 8 additions & 30 deletions ProcessDocs/AndroidDocs/AndroidDocsToJson/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,36 +331,14 @@ a row whose type and template are not changing) but not here, where a scraped pa
0 and has to come out templated. So the columns that say how to serve a row are set afterwards, on
the base row and on any continuation row.

## Running it in CI

Two workflows, mirroring the Kotlin and Java pairs:

| | |
|---|---|
| `.github/workflows/build-android-docs.yaml` | reads and writes the real database on Google Drive |
| `.github/workflows/build-android-docs-local.yaml` | the same five steps against a path on disk |
| `run-build-android-docs-with-act.sh` | drives the local one under [act](https://github.com/nektos/act) |

The steps between the two are identical -- convert, load, verify, re-mint,
verify the re-mint -- and only the ends differ: Drive download/upload behind
Workload Identity Federation, or `cp` from and to `db_path`. The local workflow
exists because WIF validates the OIDC token's issuer against GitHub's own token
endpoint, so act can never get past the Drive workflow's auth step no matter
what secrets it is given.

Unlike the Kotlin workflow (which clones kotlin-web-site) and the Java one
(which unpacks a JDK's `lib/src.zip`), there is no external source to fetch: the
12,106 derived HTML pages are committed in this repository, so a run is
reproducible from a commit alone and there is no ref to pin. The 9 GB raw scrape
they came from stays on Drive and is not needed.

Two inputs are worth knowing about. `only` converts a substring of the corpus
and turns a twelve-minute run into seconds, which is how to smoke-test a change;
it leaves the rest of the rows as they were, so it needs `verify_complete=false`
beside it. `skip_remint` drops the two slowest steps, which is right for a
structural check and for a re-run against a database already minted for this
corpus -- re-minting is not cumulative, and a second pass over an already-minted
database measured -0.7%.
## Not run in CI

The two workflows that ran these five steps against the master (`build-android-docs.yaml` on
Google Drive, `build-android-docs-local.yaml` on disk) and `run-build-android-docs-with-act.sh`
were retired in ADFA-3305, never having run. The `a/` pages ship converted on the ship-time copy
by `scripts/android_pages_to_templates.py` instead, which leaves a row already in another template
alone, so running this pipeline on the master first would leave it almost nothing to convert. The
scripts and their tests stay; the workflows are in git history (61e62469, 0ae218f8).

## The renderer

Expand Down
228 changes: 0 additions & 228 deletions run-build-android-docs-with-act.sh

This file was deleted.

Loading
Loading