From 1e1d6c347876e7b562a8b7ed4e82a0c0c44575b8 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sat, 17 Oct 2026 14:23:14 -0700 Subject: [PATCH 1/9] ADFA-6597: Find a tooltip through Tooltips' (categoryId, tag) index Written as a join of Tooltips and TooltipCategories, the tooltip query made SQLite scan all of Tooltips - 46,128 rows, 13 MB - for every tooltip shown, because documentation.db carries no ANALYZE statistics (EXPLAIN QUERY PLAN: "SCAN T"). Looking the category's id up first lets the UNIQUE (categoryId, tag) index find the row: 6.2 ms -> 0.004 ms per lookup on a desktop against the real database, same rows. IN rather than =, so two categories sharing a name still come back as duplicates, which getTooltip reports as before. The buttons query still scans TooltipButtons (5.4 MB); that needs an index in the database, which this change cannot add. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3 --- docs/documentation-database.md | 2 +- idetooltips/build.gradle.kts | 1 + .../androidide/idetooltips/ToolTipManager.kt | 13 +-- .../idetooltips/TooltipQueryTest.kt | 85 +++++++++++++++++++ 4 files changed, 95 insertions(+), 6 deletions(-) create mode 100644 idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt diff --git a/docs/documentation-database.md b/docs/documentation-database.md index 8fb2303bcc..1bd58218cd 100644 --- a/docs/documentation-database.md +++ b/docs/documentation-database.md @@ -76,7 +76,7 @@ Of the five sites below, only `DocumentationContentSource` and `ToolTipManager` - **`common/.../documentation/DocumentationContentSource.kt`** — the one Tier 3 pipeline that reads this database: row lookup (a request target is matched raw first, since stored `Content.path` values are percent-encoded, with a percent-decoded fallback on a miss — shared by both transports so they agree on which pages exist), chunked-row reassembly, dictionary-aware Brotli decode gated on the declared documentation version, and the database swaps — to a newer sdcard debug database, and reopening the installed file when an asset install rewrites it in place under the cached handle — under a read/write lock so several threads can read while a swap cannot close the handle under them. It also renders the rows that are Pebble template contexts (`templateId > 0`, the Kotlin doc set's pages) — resolving each template's references to other templates against `Templates.name` as it goes — so both transports below serve finished pages and neither needs the template engine itself. All of that logic exists once (ADFA-5176). - **`common/.../documentation/DocumentationRequestInterceptor.kt`** — serves Tier 3 *in-process* for the app's WebViews (`HelpActivity`, the tooltip fragment, `FAQActivity`), through `WebViewClient.shouldInterceptRequest`, so a page's assets cost a database read instead of a TCP connection each (ADFA-5176). It matches the same `http://localhost:6174/...` URL space, so the strings.xml entries, `ToolTipManager`'s link builder and the `DocumentationExtension` contract need no changes; anything it declines — a `/pr/` endpoint, an unknown path, a failed read — falls through to `WebServer` unchanged. Both transports get their `Content-Type` charset from the same place, `ContentTypeHeaders` (ADFA-5241), so a row does not describe itself differently depending on which one served it. Set `/sdcard/Download/CodeOnTheGo.nointercept` to force documentation back onto the server. - **`app/.../localWebServer/WebServer.kt`** — serves Tier 3 over HTTP on port 6174, for WebViews that are not wired to the interceptor above and for the `/pr/` developer endpoints. It reads through `DocumentationContentSource`, so it holds no database, template engine or decode logic of its own; what remains here is HTTP: request parsing, the `/pr/` pages, error responses, and the CSS/asset shortcuts. Also serves a Dynamic Bookshelf JSON payload (joining `Content`/`Bookshelf`/`BookCategories`, rendered through the `bookshelf` template) and debug-only HTML dumps at `/pr/db` (`LastChange`, last 20 rows) and `/pr/pr` (recent projects, from a *different* database). -- **`idetooltips/.../ToolTipManager.kt`** — serves Tier 1/2. Looks up `Tooltips` joined to `TooltipCategories` by `(category, tag)`, then `TooltipButtons` for the Tier 3 links shown at the bottom. +- **`idetooltips/.../ToolTipManager.kt`** — serves Tier 1/2. Looks up a tooltip by `(category, tag)`: the category's id from `TooltipCategories` first, so the row is found through `Tooltips`' `UNIQUE (categoryId, tag)` index rather than a scan of the whole table (ADFA-6597). Then `TooltipButtons` for the Tier 3 links shown at the bottom, which still scans that table: it has no index on `tooltipId`. - **`plugin-manager/.../documentation/PluginDocumentationManager.kt`** (with `Tier3AssetWalker.kt`, and the `DocumentationExtension` contract in `plugin-api`) — lets plugins contribute their own help content into the same lookup paths, under the reserved `plugin//...` prefix. Brotli assets are compressed against the same `CompressionDictionary` the rest of the table uses, read from the database being written. When that dictionary can't be read the install is abandoned rather than written plain, and retried on the plugin's next activation. ## Editing the database diff --git a/idetooltips/build.gradle.kts b/idetooltips/build.gradle.kts index 442393178a..dd544aa7fc 100644 --- a/idetooltips/build.gradle.kts +++ b/idetooltips/build.gradle.kts @@ -28,4 +28,5 @@ dependencies { implementation(projects.common) testImplementation(libs.tests.junit) + testImplementation(libs.tests.robolectric) } diff --git a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt index 6e2e4de7b3..2abc0cc6c5 100644 --- a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt +++ b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt @@ -53,12 +53,15 @@ object TooltipManager { private val debugDatabaseFile: File = File(android.os.Environment.getExternalStorageDirectory().toString() + "/Download/documentation.db") - private const val QUERY_TOOLTIP = """ + // Looks the category up first, so the tag is found through Tooltips' UNIQUE (categoryId, tag) + // index. Written as a join, it made SQLite's planner scan all of Tooltips (46,128 rows, 13 MB) + // for every tooltip shown, because documentation.db carries no ANALYZE statistics (ADFA-6597). + // IN rather than =, so two categories sharing a name still come back as duplicates, as before. + internal const val QUERY_TOOLTIP = """ SELECT T.rowid, T.id, T.summary, T.detail - FROM Tooltips AS T, TooltipCategories AS TC - WHERE T.categoryId = TC.id + FROM Tooltips AS T + WHERE T.categoryId IN (SELECT id FROM TooltipCategories WHERE category = ?) AND T.tag = ? - AND TC.category = ? """ @@ -93,7 +96,7 @@ object TooltipManager { "n/a" } - database.rawQuery(QUERY_TOOLTIP, arrayOf(tag, category)).use { c -> + database.rawQuery(QUERY_TOOLTIP, arrayOf(category, tag)).use { c -> when (c.count) { // A missing tag is a normal outcome (e.g. a Gradle task with no // authored tooltip). Honor the nullable contract so the caller diff --git a/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt b/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt new file mode 100644 index 0000000000..0c6ad30180 --- /dev/null +++ b/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt @@ -0,0 +1,85 @@ +package com.itsaky.androidide.idetooltips + +import android.database.sqlite.SQLiteDatabase +import org.junit.After +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.annotation.Config +import org.robolectric.annotation.ConscryptMode + +/** + * Runs [TooltipManager.QUERY_TOOLTIP] against documentation.db's own Tooltips schema. The query + * plan matters as much as the rows: the join it replaced scanned the whole 13 MB table for every + * tooltip shown (ADFA-6597). + */ +@RunWith(RobolectricTestRunner::class) +@Config(manifest = Config.NONE, sdk = [28]) +@ConscryptMode(ConscryptMode.Mode.OFF) +class TooltipQueryTest { + private lateinit var db: SQLiteDatabase + + @Before + fun setUp() { + db = SQLiteDatabase.create(null) + db.execSQL("CREATE TABLE TooltipCategories (id INTEGER PRIMARY KEY, category TEXT NOT NULL)") + db.execSQL( + "CREATE TABLE Tooltips (id INTEGER PRIMARY KEY AUTOINCREMENT, categoryId INTEGER NOT NULL, " + + "tag TEXT NOT NULL, summary TEXT NOT NULL, detail TEXT NOT NULL, UNIQUE (categoryId, tag), " + + "FOREIGN KEY(categoryId) REFERENCES TooltipCategories(id))", + ) + listOf("ide", "xml", "java", "kotlin").forEachIndexed { i, category -> + db.execSQL("INSERT INTO TooltipCategories VALUES (?, ?)", arrayOf(i + 1, category)) + } + db.beginTransaction() + for (categoryId in 1..4) { + for (n in 0 until 500) { + db.execSQL( + "INSERT INTO Tooltips (categoryId, tag, summary, detail) VALUES (?, ?, ?, ?)", + arrayOf(categoryId, "tag$n", "summary $categoryId/$n", "detail"), + ) + } + } + db.setTransactionSuccessful() + db.endTransaction() + } + + @After + fun tearDown() { + db.close() + } + + @Test + fun `finds the one tooltip for a category and tag`() { + db.rawQuery(TooltipManager.QUERY_TOOLTIP, arrayOf("java", "tag7")).use { c -> + assertEquals(1, c.count) + c.moveToFirst() + assertEquals("summary 3/7", c.getString(2)) + } + db.rawQuery(TooltipManager.QUERY_TOOLTIP, arrayOf("java", "no-such-tag")).use { assertEquals(0, it.count) } + db.rawQuery(TooltipManager.QUERY_TOOLTIP, arrayOf("no-such-category", "tag7")).use { assertEquals(0, it.count) } + } + + @Test + fun `two categories with the same name still come back as two rows`() { + db.execSQL("INSERT INTO TooltipCategories VALUES (5, 'java')") + db.execSQL("INSERT INTO Tooltips (categoryId, tag, summary, detail) VALUES (5, 'tag7', 'duplicate', 'detail')") + + db.rawQuery(TooltipManager.QUERY_TOOLTIP, arrayOf("java", "tag7")).use { assertEquals(2, it.count) } + } + + @Test + fun `finds the tag through the (categoryId, tag) index, not a scan of Tooltips`() { + val plan = + db.rawQuery("EXPLAIN QUERY PLAN " + TooltipManager.QUERY_TOOLTIP, arrayOf("java", "tag7")).use { c -> + buildList { while (c.moveToNext()) add(c.getString(3)) } + } + + assertTrue(plan.toString(), plan.any { "sqlite_autoindex_Tooltips_1" in it }) + assertFalse(plan.toString(), plan.any { Regex("""\bSCAN (TABLE )?(T|Tooltips)\b""").containsMatchIn(it) }) + } +} From e5f44a844be365d3c6552ce815cea5f06ea4b10b Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sat, 17 Oct 2026 15:14:03 -0700 Subject: [PATCH 2/9] ADFA-6597: Test the tooltip query's arguments in the order the app binds them From review of PR #2160: - TooltipQueryTest bound the query's arguments itself, so it would pass whatever order getTooltip used, and this PR reversed that order. The SQL and its binding now live in queryTooltip, in TooltipQuery.kt, which getTooltip calls with named arguments and the test calls too. Swapping the binding fails two of the three tests. - QUERY_TOOLTIP is marked @VisibleForTesting: only the plan test reads it. It is still a public static field in bytecode, as it was inside TooltipManager, and so still in the plugin-api jar. - Drops @ConscryptMode(OFF). The Conscrypt leak it guarded against broke a TLS test in common; idetooltips has none. A file of its own rather than top-level code in ToolTipManager.kt: ktlint leaves ToolTipManager.kt unformatted while its only top-level declaration, object TooltipManager, does not match the file name. A second declaration there makes Spotless reformat all of the file's ~380 changed lines, burying this change. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3 --- docs/documentation-database.md | 2 +- .../androidide/idetooltips/ToolTipManager.kt | 14 +---------- .../androidide/idetooltips/TooltipQuery.kt | 25 +++++++++++++++++++ .../idetooltips/TooltipQueryTest.kt | 18 ++++++------- 4 files changed, 35 insertions(+), 24 deletions(-) create mode 100644 idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt diff --git a/docs/documentation-database.md b/docs/documentation-database.md index 1bd58218cd..d85a0d0b20 100644 --- a/docs/documentation-database.md +++ b/docs/documentation-database.md @@ -76,7 +76,7 @@ Of the five sites below, only `DocumentationContentSource` and `ToolTipManager` - **`common/.../documentation/DocumentationContentSource.kt`** — the one Tier 3 pipeline that reads this database: row lookup (a request target is matched raw first, since stored `Content.path` values are percent-encoded, with a percent-decoded fallback on a miss — shared by both transports so they agree on which pages exist), chunked-row reassembly, dictionary-aware Brotli decode gated on the declared documentation version, and the database swaps — to a newer sdcard debug database, and reopening the installed file when an asset install rewrites it in place under the cached handle — under a read/write lock so several threads can read while a swap cannot close the handle under them. It also renders the rows that are Pebble template contexts (`templateId > 0`, the Kotlin doc set's pages) — resolving each template's references to other templates against `Templates.name` as it goes — so both transports below serve finished pages and neither needs the template engine itself. All of that logic exists once (ADFA-5176). - **`common/.../documentation/DocumentationRequestInterceptor.kt`** — serves Tier 3 *in-process* for the app's WebViews (`HelpActivity`, the tooltip fragment, `FAQActivity`), through `WebViewClient.shouldInterceptRequest`, so a page's assets cost a database read instead of a TCP connection each (ADFA-5176). It matches the same `http://localhost:6174/...` URL space, so the strings.xml entries, `ToolTipManager`'s link builder and the `DocumentationExtension` contract need no changes; anything it declines — a `/pr/` endpoint, an unknown path, a failed read — falls through to `WebServer` unchanged. Both transports get their `Content-Type` charset from the same place, `ContentTypeHeaders` (ADFA-5241), so a row does not describe itself differently depending on which one served it. Set `/sdcard/Download/CodeOnTheGo.nointercept` to force documentation back onto the server. - **`app/.../localWebServer/WebServer.kt`** — serves Tier 3 over HTTP on port 6174, for WebViews that are not wired to the interceptor above and for the `/pr/` developer endpoints. It reads through `DocumentationContentSource`, so it holds no database, template engine or decode logic of its own; what remains here is HTTP: request parsing, the `/pr/` pages, error responses, and the CSS/asset shortcuts. Also serves a Dynamic Bookshelf JSON payload (joining `Content`/`Bookshelf`/`BookCategories`, rendered through the `bookshelf` template) and debug-only HTML dumps at `/pr/db` (`LastChange`, last 20 rows) and `/pr/pr` (recent projects, from a *different* database). -- **`idetooltips/.../ToolTipManager.kt`** — serves Tier 1/2. Looks up a tooltip by `(category, tag)`: the category's id from `TooltipCategories` first, so the row is found through `Tooltips`' `UNIQUE (categoryId, tag)` index rather than a scan of the whole table (ADFA-6597). Then `TooltipButtons` for the Tier 3 links shown at the bottom, which still scans that table: it has no index on `tooltipId`. +- **`idetooltips/.../ToolTipManager.kt`** — serves Tier 1/2. Looks up a tooltip by `(category, tag)` through `queryTooltip` in `TooltipQuery.kt`: the category's id from `TooltipCategories` first, so the row is found through `Tooltips`' `UNIQUE (categoryId, tag)` index rather than a scan of the whole table (ADFA-6597). Then `TooltipButtons` for the Tier 3 links shown at the bottom, which still scans that table: it has no index on `tooltipId`. - **`plugin-manager/.../documentation/PluginDocumentationManager.kt`** (with `Tier3AssetWalker.kt`, and the `DocumentationExtension` contract in `plugin-api`) — lets plugins contribute their own help content into the same lookup paths, under the reserved `plugin//...` prefix. Brotli assets are compressed against the same `CompressionDictionary` the rest of the table uses, read from the database being written. When that dictionary can't be read the install is abandoned rather than written plain, and retried on the plugin's next activation. ## Editing the database diff --git a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt index 2abc0cc6c5..9bf8dd6259 100644 --- a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt +++ b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt @@ -53,18 +53,6 @@ object TooltipManager { private val debugDatabaseFile: File = File(android.os.Environment.getExternalStorageDirectory().toString() + "/Download/documentation.db") - // Looks the category up first, so the tag is found through Tooltips' UNIQUE (categoryId, tag) - // index. Written as a join, it made SQLite's planner scan all of Tooltips (46,128 rows, 13 MB) - // for every tooltip shown, because documentation.db carries no ANALYZE statistics (ADFA-6597). - // IN rather than =, so two categories sharing a name still come back as duplicates, as before. - internal const val QUERY_TOOLTIP = """ - SELECT T.rowid, T.id, T.summary, T.detail - FROM Tooltips AS T - WHERE T.categoryId IN (SELECT id FROM TooltipCategories WHERE category = ?) - AND T.tag = ? - """ - - private const val QUERY_TOOLTIP_BUTTONS = """ SELECT description, uri FROM TooltipButtons @@ -96,7 +84,7 @@ object TooltipManager { "n/a" } - database.rawQuery(QUERY_TOOLTIP, arrayOf(category, tag)).use { c -> + queryTooltip(database, category = category, tag = tag).use { c -> when (c.count) { // A missing tag is a normal outcome (e.g. a Gradle task with no // authored tooltip). Honor the nullable contract so the caller diff --git a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt new file mode 100644 index 0000000000..71354eedc0 --- /dev/null +++ b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt @@ -0,0 +1,25 @@ +package com.itsaky.androidide.idetooltips + +import android.database.Cursor +import android.database.sqlite.SQLiteDatabase +import androidx.annotation.VisibleForTesting + +// Looks the category up first, so the tag is found through Tooltips' UNIQUE (categoryId, tag) +// index. Written as a join, it made SQLite's planner scan all of Tooltips (46,128 rows, 13 MB) +// for every tooltip shown, because documentation.db carries no ANALYZE statistics (ADFA-6597). +// IN rather than =, so two categories sharing a name still come back as duplicates, as before. +@VisibleForTesting +internal const val QUERY_TOOLTIP = """ + SELECT T.rowid, T.id, T.summary, T.detail + FROM Tooltips AS T + WHERE T.categoryId IN (SELECT id FROM TooltipCategories WHERE category = ?) + AND T.tag = ? +""" + +// Outside TooltipManager, so tests can run it without initializing the object, which reads the +// installed app's Environment. Binds the arguments in the order QUERY_TOOLTIP expects them. +internal fun queryTooltip( + database: SQLiteDatabase, + category: String, + tag: String, +): Cursor = database.rawQuery(QUERY_TOOLTIP, arrayOf(category, tag)) diff --git a/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt b/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt index 0c6ad30180..6af7b78ba1 100644 --- a/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt +++ b/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt @@ -10,16 +10,14 @@ import org.junit.Test import org.junit.runner.RunWith import org.robolectric.RobolectricTestRunner import org.robolectric.annotation.Config -import org.robolectric.annotation.ConscryptMode /** - * Runs [TooltipManager.QUERY_TOOLTIP] against documentation.db's own Tooltips schema. The query - * plan matters as much as the rows: the join it replaced scanned the whole 13 MB table for every - * tooltip shown (ADFA-6597). + * Runs [queryTooltip], the lookup TooltipManager.getTooltip makes, against documentation.db's own + * Tooltips schema. The query plan matters as much as the rows: the join it replaced scanned the + * whole 13 MB table for every tooltip shown (ADFA-6597). */ @RunWith(RobolectricTestRunner::class) @Config(manifest = Config.NONE, sdk = [28]) -@ConscryptMode(ConscryptMode.Mode.OFF) class TooltipQueryTest { private lateinit var db: SQLiteDatabase @@ -55,13 +53,13 @@ class TooltipQueryTest { @Test fun `finds the one tooltip for a category and tag`() { - db.rawQuery(TooltipManager.QUERY_TOOLTIP, arrayOf("java", "tag7")).use { c -> + queryTooltip(db, category = "java", tag = "tag7").use { c -> assertEquals(1, c.count) c.moveToFirst() assertEquals("summary 3/7", c.getString(2)) } - db.rawQuery(TooltipManager.QUERY_TOOLTIP, arrayOf("java", "no-such-tag")).use { assertEquals(0, it.count) } - db.rawQuery(TooltipManager.QUERY_TOOLTIP, arrayOf("no-such-category", "tag7")).use { assertEquals(0, it.count) } + queryTooltip(db, category = "java", tag = "no-such-tag").use { assertEquals(0, it.count) } + queryTooltip(db, category = "no-such-category", tag = "tag7").use { assertEquals(0, it.count) } } @Test @@ -69,13 +67,13 @@ class TooltipQueryTest { db.execSQL("INSERT INTO TooltipCategories VALUES (5, 'java')") db.execSQL("INSERT INTO Tooltips (categoryId, tag, summary, detail) VALUES (5, 'tag7', 'duplicate', 'detail')") - db.rawQuery(TooltipManager.QUERY_TOOLTIP, arrayOf("java", "tag7")).use { assertEquals(2, it.count) } + queryTooltip(db, category = "java", tag = "tag7").use { assertEquals(2, it.count) } } @Test fun `finds the tag through the (categoryId, tag) index, not a scan of Tooltips`() { val plan = - db.rawQuery("EXPLAIN QUERY PLAN " + TooltipManager.QUERY_TOOLTIP, arrayOf("java", "tag7")).use { c -> + db.rawQuery("EXPLAIN QUERY PLAN " + QUERY_TOOLTIP, arrayOf("java", "tag7")).use { c -> buildList { while (c.moveToNext()) add(c.getString(3)) } } From b900e2ed79ee038a02dd2501a698f1668d3d5467 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 18 Oct 2026 13:10:30 -0700 Subject: [PATCH 3/9] ADFA-3305: Move plugin tooltip SQL out of PluginDocumentationManager No behavior change. The four helpers that write and delete a plugin's tooltips move, unchanged, to PluginTooltipTables.kt behind two internal functions, writePluginTooltips and removePluginTooltips, so tests can run them against an in-memory database without a Context. The manager now maps each PluginTooltipEntry to its final text and resolved button URIs before handing them over, inside the same transaction as before. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3 --- .../PluginDocumentationManager.kt | 151 ++--------------- .../documentation/PluginTooltipTables.kt | 158 ++++++++++++++++++ 2 files changed, 176 insertions(+), 133 deletions(-) create mode 100644 plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTables.kt diff --git a/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginDocumentationManager.kt b/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginDocumentationManager.kt index b60f15ed32..f13adaa3ee 100644 --- a/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginDocumentationManager.kt +++ b/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginDocumentationManager.kt @@ -110,17 +110,8 @@ class PluginDocumentationManager( db.beginTransaction() try { - removePluginDocumentationInternal(db, pluginId) - - val categoryId = insertOrGetCategoryId(db, pluginCategory(pluginId)) - - for (entry in entries) { - val tooltipId = insertTooltip(db, categoryId, entry) - entry.buttons.sortedBy { it.order }.forEachIndexed { index, button -> - val resolvedUri = resolvePluginButtonUri(pluginId, button.uri, button.directPath) - insertTooltipButton(db, tooltipId, button.description, resolvedUri, index) - } - } + val disclaimer = context.getString(R.string.plugin_documentation_third_party_disclaimer) + writePluginTooltips(db, pluginCategory(pluginId), entries.map { tooltipRow(pluginId, it, disclaimer) }) db.setTransactionSuccessful() Log.d(TAG, "Successfully installed documentation for plugin $pluginId") @@ -158,7 +149,7 @@ class PluginDocumentationManager( db.beginTransaction() try { - removePluginDocumentationInternal(db, pluginId) + removePluginTooltips(db, pluginCategory(pluginId)) db.setTransactionSuccessful() Log.d(TAG, "Successfully removed documentation for plugin $pluginId") true @@ -566,110 +557,6 @@ class PluginDocumentationManager( db.insertOrThrow("Content", null, values) } - private fun removePluginDocumentationInternal( - db: SQLiteDatabase, - pluginId: String, - ) { - val category = pluginCategory(pluginId) - - val cursor = - db.rawQuery( - """ - SELECT T.id FROM Tooltips AS T - INNER JOIN TooltipCategories AS TC ON T.categoryId = TC.id - WHERE TC.category = ? - """.trimIndent(), - arrayOf(category), - ) - - val tooltipIds = mutableListOf() - while (cursor.moveToNext()) { - tooltipIds.add(cursor.getLong(0)) - } - cursor.close() - - if (tooltipIds.isNotEmpty()) { - val placeholders = tooltipIds.joinToString(",") { "?" } - val args = tooltipIds.map { it.toString() }.toTypedArray() - db.delete("TooltipButtons", "tooltipId IN ($placeholders)", args) - db.delete("Tooltips", "id IN ($placeholders)", args) - } - - db.delete("TooltipCategories", "category = ?", arrayOf(category)) - } - - private fun insertOrGetCategoryId( - db: SQLiteDatabase, - category: String, - ): Long { - val cursor = - db.query( - "TooltipCategories", - arrayOf("id"), - "category = ?", - arrayOf(category), - null, - null, - null, - ) - - if (cursor.moveToFirst()) { - val id = cursor.getLong(0) - cursor.close() - return id - } - cursor.close() - - val values = - ContentValues().apply { - put("category", category) - } - return db.insert("TooltipCategories", null, values) - } - - private fun insertTooltip( - db: SQLiteDatabase, - categoryId: Long, - entry: PluginTooltipEntry, - ): Long { - val disclaimer = context.getString(R.string.plugin_documentation_third_party_disclaimer) - - val existingCursor = - db.query( - "Tooltips", - arrayOf("id"), - "categoryId = ? AND tag = ?", - arrayOf(categoryId.toString(), entry.tag), - null, - null, - null, - ) - - if (existingCursor.moveToFirst()) { - val existingId = existingCursor.getLong(0) - existingCursor.close() - - val updateValues = - ContentValues().apply { - put("summary", entry.summary + disclaimer) - put("detail", if (entry.detail.isNotBlank()) entry.detail + disclaimer else "") - } - db.update("Tooltips", updateValues, "id = ?", arrayOf(existingId.toString())) - db.delete("TooltipButtons", "tooltipId = ?", arrayOf(existingId.toString())) - return existingId - } - existingCursor.close() - - val values = - ContentValues().apply { - put("categoryId", categoryId) - put("tag", entry.tag) - put("summary", entry.summary + disclaimer) - put("detail", if (entry.detail.isNotBlank()) entry.detail + disclaimer else "") - } - return db.insert("Tooltips", null, values) - } - private fun escapeLike(value: String): String = value .replace("\\", "\\\\") @@ -684,6 +571,21 @@ class PluginDocumentationManager( return segments.joinToString("/") } + private fun tooltipRow( + pluginId: String, + entry: PluginTooltipEntry, + disclaimer: String, + ): PluginTooltipRow = + PluginTooltipRow( + tag = entry.tag, + summary = entry.summary + disclaimer, + detail = if (entry.detail.isNotBlank()) entry.detail + disclaimer else "", + buttons = + entry.buttons.sortedBy { it.order }.map { button -> + PluginTooltipButtonRow(button.description, resolvePluginButtonUri(pluginId, button.uri, button.directPath)) + }, + ) + private fun resolvePluginButtonUri( pluginId: String, rawUri: String, @@ -696,23 +598,6 @@ class PluginDocumentationManager( return if (absolute) normalized else "plugin/$pluginId/$normalized" } - private fun insertTooltipButton( - db: SQLiteDatabase, - tooltipId: Long, - description: String, - uri: String, - order: Int, - ) { - val values = - ContentValues().apply { - put("tooltipId", tooltipId) - put("description", description) - put("uri", uri) - put("buttonNumberId", order) - } - db.insert("TooltipButtons", null, values) - } - /** * Check if the plugin documentation database is accessible. */ diff --git a/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTables.kt b/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTables.kt new file mode 100644 index 0000000000..5b9e3b2164 --- /dev/null +++ b/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTables.kt @@ -0,0 +1,158 @@ +package com.itsaky.androidide.plugins.manager.documentation + +import android.content.ContentValues +import android.database.sqlite.SQLiteDatabase + +/** One plugin tooltip as it is written: summary and detail final, buttons in display order. */ +internal class PluginTooltipRow( + val tag: String, + val summary: String, + val detail: String, + val buttons: List, +) + +/** One button of a [PluginTooltipRow], its uri already resolved. */ +internal class PluginTooltipButtonRow( + val description: String, + val uri: String, +) + +/** + * Replaces every tooltip in [category] with [tooltips], inside the caller's transaction. + */ +internal fun writePluginTooltips( + db: SQLiteDatabase, + category: String, + tooltips: List, +) { + removePluginTooltips(db, category) + + val categoryId = insertOrGetCategoryId(db, category) + + for (tooltip in tooltips) { + val tooltipId = insertTooltip(db, categoryId, tooltip) + tooltip.buttons.forEachIndexed { index, button -> + insertTooltipButton(db, tooltipId, button.description, button.uri, index) + } + } +} + +/** + * Removes [category] and every tooltip in it, inside the caller's transaction. + */ +internal fun removePluginTooltips( + db: SQLiteDatabase, + category: String, +) { + val cursor = + db.rawQuery( + """ + SELECT T.id FROM Tooltips AS T + INNER JOIN TooltipCategories AS TC ON T.categoryId = TC.id + WHERE TC.category = ? + """.trimIndent(), + arrayOf(category), + ) + + val tooltipIds = mutableListOf() + while (cursor.moveToNext()) { + tooltipIds.add(cursor.getLong(0)) + } + cursor.close() + + if (tooltipIds.isNotEmpty()) { + val placeholders = tooltipIds.joinToString(",") { "?" } + val args = tooltipIds.map { it.toString() }.toTypedArray() + db.delete("TooltipButtons", "tooltipId IN ($placeholders)", args) + db.delete("Tooltips", "id IN ($placeholders)", args) + } + + db.delete("TooltipCategories", "category = ?", arrayOf(category)) +} + +private fun insertOrGetCategoryId( + db: SQLiteDatabase, + category: String, +): Long { + val cursor = + db.query( + "TooltipCategories", + arrayOf("id"), + "category = ?", + arrayOf(category), + null, + null, + null, + ) + + if (cursor.moveToFirst()) { + val id = cursor.getLong(0) + cursor.close() + return id + } + cursor.close() + + val values = + ContentValues().apply { + put("category", category) + } + return db.insert("TooltipCategories", null, values) +} + +private fun insertTooltip( + db: SQLiteDatabase, + categoryId: Long, + tooltip: PluginTooltipRow, +): Long { + val existingCursor = + db.query( + "Tooltips", + arrayOf("id"), + "categoryId = ? AND tag = ?", + arrayOf(categoryId.toString(), tooltip.tag), + null, + null, + null, + ) + + if (existingCursor.moveToFirst()) { + val existingId = existingCursor.getLong(0) + existingCursor.close() + + val updateValues = + ContentValues().apply { + put("summary", tooltip.summary) + put("detail", tooltip.detail) + } + db.update("Tooltips", updateValues, "id = ?", arrayOf(existingId.toString())) + db.delete("TooltipButtons", "tooltipId = ?", arrayOf(existingId.toString())) + return existingId + } + existingCursor.close() + + val values = + ContentValues().apply { + put("categoryId", categoryId) + put("tag", tooltip.tag) + put("summary", tooltip.summary) + put("detail", tooltip.detail) + } + return db.insert("Tooltips", null, values) +} + +private fun insertTooltipButton( + db: SQLiteDatabase, + tooltipId: Long, + description: String, + uri: String, + order: Int, +) { + val values = + ContentValues().apply { + put("tooltipId", tooltipId) + put("description", description) + put("uri", uri) + put("buttonNumberId", order) + } + db.insert("TooltipButtons", null, values) +} From e480d2ac43bf174478b4b0f87ffa119aeda412b2 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 18 Oct 2026 13:21:04 -0700 Subject: [PATCH 4/9] ADFA-3305: Read tooltips from documentation version 3's keyed tables documentation.db version 3 (OfflineDocumentationTools' migrate_tooltip_schema.py) stores the same tooltips in 5.2 MB less: - Tooltips is a WITHOUT ROWID table keyed on (categoryId, tag) holding only the id, so the separate UNIQUE (categoryId, tag) index, a second copy of every tag, is gone. The text is in TooltipTexts, by id. - TooltipButtons is keyed on (tooltipId, buttonNumberId), so a tooltip's buttons are one key range in display order instead of a scan and sort of the whole table. Each distinct description and uri is stored once, in TooltipButtonDescriptions and TooltipButtonUris. queryTooltip joins TooltipTexts by id, and the new queryTooltipButtons joins the two lookup tables. Tooltips has no rowid any more; it always equalled id, so IDETooltipItem.rowId, which the debug dialog and the feedback email show, now carries the id. Both readers refuse a database declaring a version below 3, through DatabaseVersionResolver.requireKeyedTooltips, with a message naming the version, rather than fail on whichever old column a query meets. Tests run both queries against the version 3 DDL, including their query plans (key searches, no scan, no sort), and the version check for versions 2, 4 and none. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3 --- .../utils/DatabaseVersionResolver.kt | 20 +++ docs/documentation-database.md | 39 ++++-- .../androidide/idetooltips/ToolTipManager.kt | 30 ++-- .../androidide/idetooltips/TooltipQuery.kt | 39 +++++- .../idetooltips/TooltipQueryTest.kt | 129 +++++++++++++++--- 5 files changed, 199 insertions(+), 58 deletions(-) diff --git a/common/src/main/java/com/itsaky/androidide/utils/DatabaseVersionResolver.kt b/common/src/main/java/com/itsaky/androidide/utils/DatabaseVersionResolver.kt index 6dc0439354..46f7d908ab 100644 --- a/common/src/main/java/com/itsaky/androidide/utils/DatabaseVersionResolver.kt +++ b/common/src/main/java/com/itsaky/androidide/utils/DatabaseVersionResolver.kt @@ -21,6 +21,12 @@ object DatabaseVersionResolver { // rows are plain Brotli. const val MAJOR_VERSION_WITH_COMPRESSION_DICTIONARY = 2 + // From this MAJOR version on (ADFA-3305), Tooltips is a WITHOUT ROWID table keyed by + // (categoryId, tag) whose text is in TooltipTexts, and TooltipButtons is keyed by (tooltipId, + // buttonNumberId) with its text in TooltipButtonDescriptions and TooltipButtonUris. Below it the + // tooltip tables have the earlier shape, which this build neither reads nor writes. + const val MAJOR_VERSION_WITH_KEYED_TOOLTIPS = 3 + private const val QUERY_VERSION_TABLE_EXISTS = """ SELECT 1 FROM sqlite_master @@ -135,6 +141,20 @@ object DatabaseVersionResolver { } } + /** + * Throws [IllegalStateException] unless [db] declares at least [MAJOR_VERSION_WITH_KEYED_TOOLTIPS], + * naming the version it does declare. Tooltip readers and writers call it first, so a database + * with the earlier tooltip tables fails with that reason rather than on a missing table or + * column partway through. + */ + fun requireKeyedTooltips(db: SQLiteDatabase) { + val major = resolveMajorVersion(db) + check(major != null && major >= MAJOR_VERSION_WITH_KEYED_TOOLTIPS) { + "documentation.db declares documentation version ${major ?: "none"}; its tooltip tables need " + + "$MAJOR_VERSION_WITH_KEYED_TOOLTIPS or later" + } + } + /** * Formats database change metadata into a readable version string. * diff --git a/docs/documentation-database.md b/docs/documentation-database.md index d85a0d0b20..56d7b4410b 100644 --- a/docs/documentation-database.md +++ b/docs/documentation-database.md @@ -42,27 +42,46 @@ Dimensions: `Languages(id, value)` (4-letter codes, e.g. `EN-us`); `ContentTypes ### Tier 1/2: `Tooltips` (the other fact table) +The shape below is documentation version 3 (ADFA-3305), written by `OfflineDocumentationTools`' `migrate_tooltip_schema.py`. Version 2 kept the text in `Tooltips` (an `AUTOINCREMENT` table with a separate `UNIQUE(categoryId, tag)` index) and `description`/`uri` in every `TooltipButtons` row; this build reads and writes neither, and refuses a database declaring less than 3 (`DatabaseVersionResolver.requireKeyedTooltips`). + ```sql -CREATE TABLE Tooltips ( +CREATE TABLE TooltipTexts ( id INTEGER PRIMARY KEY AUTOINCREMENT, - categoryId INTEGER NOT NULL, - tag TEXT NOT NULL, summary TEXT NOT NULL, - detail TEXT NOT NULL, - UNIQUE(categoryId, tag), - FOREIGN KEY(categoryId) REFERENCES TooltipCategories(id) + detail TEXT NOT NULL ); +CREATE TABLE Tooltips ( + id INTEGER NOT NULL, + categoryId INTEGER NOT NULL, + tag TEXT NOT NULL, + PRIMARY KEY (categoryId, tag), + FOREIGN KEY(categoryId) REFERENCES TooltipCategories(id), + FOREIGN KEY(id) REFERENCES TooltipTexts(id) +) WITHOUT ROWID; +CREATE TABLE TooltipButtons ( + tooltipId INTEGER NOT NULL, + buttonNumberId INTEGER NOT NULL, + descriptionId INTEGER NOT NULL, + uriId INTEGER NOT NULL, + PRIMARY KEY (tooltipId, buttonNumberId), + FOREIGN KEY(tooltipId) REFERENCES TooltipTexts(id), + FOREIGN KEY(buttonNumberId) REFERENCES TooltipButtonNumbers(id), + FOREIGN KEY(descriptionId) REFERENCES TooltipButtonDescriptions(id), + FOREIGN KEY(uriId) REFERENCES TooltipButtonUris(id) +) WITHOUT ROWID; +-- plus TooltipButtonDescriptions(id INTEGER PRIMARY KEY, description) and TooltipButtonUris(id INTEGER PRIMARY KEY, uri) ``` - `summary` is Tier 1 (the initial popup), `detail` is Tier 2 (after "See more"); both may contain HTML. -- Looked up by `(categoryId, tag)`, which the IDE stamps on the UI widget that owns the tooltip. The `UNIQUE` constraint gives this lookup an index, so it's fast. +- Looked up by `(categoryId, tag)`, which the IDE stamps on the UI widget that owns the tooltip. `Tooltips` is keyed on exactly that, so the lookup is a search of the table itself, and the text is one more lookup by id in `TooltipTexts`. The text lives apart because a `WITHOUT ROWID` table is an index B-tree: whole rows in its interior pages and any row over 1,002 bytes (at 4 KB pages) spilled to a mostly empty overflow page, which cost 1.18 MB on the 2026-10-18 database. +- A new tooltip takes its id from `TooltipTexts` (`AUTOINCREMENT`, so never one a deleted tooltip had); `Tooltips` has no rowid, so an insert into it returns none. - `TooltipCategories(id, category)` is a tiny dimension table (four categories at the time of writing). -- `TooltipButtons(tooltipId, buttonNumberId, description, uri)` holds the Tier 3 links shown at the bottom of a Tier 2 tooltip — `tooltipId` -> `Tooltips.id`, `buttonNumberId` -> `TooltipButtonNumbers.id`. `uri` should resolve to a `Content.path` (after stripping `?query`/`#fragment`). +- `TooltipButtons` holds the Tier 3 links shown at the bottom of a Tier 2 tooltip, keyed on `(tooltipId, buttonNumberId)`, so a tooltip's buttons are one range of the key, already in display order. Each distinct description and uri is stored once, in `TooltipButtonDescriptions` and `TooltipButtonUris` ("View full documentation" alone used to be stored 43,703 times). A uri should resolve to a `Content.path` (after stripping `?query`/`#fragment`). Neither lookup table has a `UNIQUE` index, which would cost about as much again; writers look a string up before inserting it. - `TooltipButtonNumbers(id)` exists only to pin a fixed, manually-assigned display order when a tooltip has multiple Tier 3 links. (Flagged in the source design doc as something worth redoing without a whole extra table.) ### Supporting tables -- **`DocumentationDatabaseVersion(major, minor, patch, who, comment, changeTime)`** — the database's own semver (ADFA-5220), replacing the heuristics that used to infer the format from which tables happened to exist. **Exactly one row**: the version the file *is*, which `populate_db.py` replaces rather than appends. Nothing in the DDL enforces that, so both readers (`DatabaseVersionResolver.resolveMajorVersion` here, `database_major_version` in docdb-studio) take the newest row rather than the one with the highest `major` (the app orders by `changeTime DESC, rowid DESC`, so the greatest `changeTime` wins and `rowid` only breaks ties) — a rebuild from an older content set is a downgrade and has to read as one, which `MAX(major)` would get wrong. The app's reader additionally logs a warning when it finds more than one row; docdb-studio's does not. `populate_db.py` collapses a file that somehow accumulated several back to one, and `resolveMajorVersion` returns null for a database predating the table. `MAJOR >= 2` is what tells the app its brotli `Content` rows are dictionary-compressed; below that, `DocumentationContentSource` neither reads nor attaches `CompressionDictionary`. Gating on the declared version rather than on the table's presence matters in both directions: a database can carry the dictionary table while its content is still plain brotli (every brotli row would then be decoded with a dictionary attached and fail outright, since ADFA-5240 removed the plain-decode retry that used to mask this), and a migrated database that lost the table fails loudly instead of quietly. +- **`DocumentationDatabaseVersion(major, minor, patch, who, comment, changeTime)`** — the database's own semver (ADFA-5220), replacing the heuristics that used to infer the format from which tables happened to exist. **Exactly one row**: the version the file *is*, which `populate_db.py` replaces rather than appends. Nothing in the DDL enforces that, so both readers (`DatabaseVersionResolver.resolveMajorVersion` here, `database_major_version` in docdb-studio) take the newest row rather than the one with the highest `major` (the app orders by `changeTime DESC, rowid DESC`, so the greatest `changeTime` wins and `rowid` only breaks ties) — a rebuild from an older content set is a downgrade and has to read as one, which `MAX(major)` would get wrong. The app's reader additionally logs a warning when it finds more than one row; docdb-studio's does not. `populate_db.py` collapses a file that somehow accumulated several back to one, and `resolveMajorVersion` returns null for a database predating the table. `MAJOR >= 3` says the tooltip tables have the shape above. `MAJOR >= 2` is what tells the app its brotli `Content` rows are dictionary-compressed; below that, `DocumentationContentSource` neither reads nor attaches `CompressionDictionary`. Gating on the declared version rather than on the table's presence matters in both directions: a database can carry the dictionary table while its content is still plain brotli (every brotli row would then be decoded with a dictionary attached and fail outright, since ADFA-5240 removed the plain-decode retry that used to mask this), and a migrated database that lost the table fails loudly instead of quietly. - **`CompressionDictionary(id, data)`** — single-row table (`id INTEGER PRIMARY KEY CHECK (id = 1)`) holding the raw Brotli dictionary every ADFA-5153-migrated `compression = 'brotli'` `Content` row is compressed against. Trained once, from a representative sample across the whole `Content` table, by `OfflineDocumentationTools`' `migrate_content_to_dictionary_brotli.py` / `populate_db.py` (never retrained after that — a dictionary-compressed row is only decodable against the exact dictionary it was compressed with, so replacing it would silently orphan every already-migrated row). Shipping the dictionary inside `documentation.db` itself, rather than as a separate bundled asset, keeps it version-locked to the content compressed against it. Both the reader and the writer load it through `loadCompressionDictionary` in `:common`, so they cannot disagree about whether a given database's brotli rows carry a dictionary — a row compressed against one the reader won't attach is undecodable, and vice versa. `DocumentationContentSource` loads it lazily -- not at database open or swap time, but on the first content lookup after the active database changes, whatever that row's own compression, and only when `DocumentationDatabaseVersion` declares `MAJOR >= 2` (see above) -- and caches it from then on, reloading again only on the next database change (a swap can bring in a database with a different dictionary or none, so it can't stay cached across one). Each row is then decoded exactly once, with the dictionary attached or not according to that version; there is no retry, because there is no longer a second way a row might have been written. - **`Templates(id, name, content)`** — Pebble template source. The DDL is `id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, content BLOB NOT NULL, UNIQUE('name')` — worth stating, because the single-quoted `UNIQUE('name')` reads like a constant rather than a column and has twice been taken for an absent constraint. SQLite resolves the quoted string to the column, so a duplicate name, a null name and a null body are all rejected by the database, and readers here do not re-check them. `Content.templateId` names a page's outermost template by id; every reference *between* templates is by `name` (ADFA-5405), which is also how `WebServer` reaches well-known templates like `bookshelf`. So a template can `extends`, `include`, `import` or `embed` another one by writing its `Templates.name` verbatim (`{% include "nav.peb" %}`) — the table is a flat namespace, with no directories, prefixes or suffixes applied. A reference to a name with no row fails the request rather than rendering the name; before ADFA-5405 the latter is exactly what happened, silently, because the engine's loader treated the name it was handed as the template body. - **`Bookshelf(contentID, bookCategoryID, title, description)`** / **`BookCategories(id, category, description)`** — the Dynamic Bookshelf: one row per "book" (PDF or similar), linked to its Tier 3 page via `contentID` -> `Content.id`. Two DB triggers keep `Bookshelf` in sync when a PDF row is inserted/deleted from `Content`; `title`/`description` don't come from those triggers and must be set by hand. Non-PDF books need a separate ingestion path (plugin-provided, e.g. via `PluginDocumentationManager`). @@ -76,7 +95,7 @@ Of the five sites below, only `DocumentationContentSource` and `ToolTipManager` - **`common/.../documentation/DocumentationContentSource.kt`** — the one Tier 3 pipeline that reads this database: row lookup (a request target is matched raw first, since stored `Content.path` values are percent-encoded, with a percent-decoded fallback on a miss — shared by both transports so they agree on which pages exist), chunked-row reassembly, dictionary-aware Brotli decode gated on the declared documentation version, and the database swaps — to a newer sdcard debug database, and reopening the installed file when an asset install rewrites it in place under the cached handle — under a read/write lock so several threads can read while a swap cannot close the handle under them. It also renders the rows that are Pebble template contexts (`templateId > 0`, the Kotlin doc set's pages) — resolving each template's references to other templates against `Templates.name` as it goes — so both transports below serve finished pages and neither needs the template engine itself. All of that logic exists once (ADFA-5176). - **`common/.../documentation/DocumentationRequestInterceptor.kt`** — serves Tier 3 *in-process* for the app's WebViews (`HelpActivity`, the tooltip fragment, `FAQActivity`), through `WebViewClient.shouldInterceptRequest`, so a page's assets cost a database read instead of a TCP connection each (ADFA-5176). It matches the same `http://localhost:6174/...` URL space, so the strings.xml entries, `ToolTipManager`'s link builder and the `DocumentationExtension` contract need no changes; anything it declines — a `/pr/` endpoint, an unknown path, a failed read — falls through to `WebServer` unchanged. Both transports get their `Content-Type` charset from the same place, `ContentTypeHeaders` (ADFA-5241), so a row does not describe itself differently depending on which one served it. Set `/sdcard/Download/CodeOnTheGo.nointercept` to force documentation back onto the server. - **`app/.../localWebServer/WebServer.kt`** — serves Tier 3 over HTTP on port 6174, for WebViews that are not wired to the interceptor above and for the `/pr/` developer endpoints. It reads through `DocumentationContentSource`, so it holds no database, template engine or decode logic of its own; what remains here is HTTP: request parsing, the `/pr/` pages, error responses, and the CSS/asset shortcuts. Also serves a Dynamic Bookshelf JSON payload (joining `Content`/`Bookshelf`/`BookCategories`, rendered through the `bookshelf` template) and debug-only HTML dumps at `/pr/db` (`LastChange`, last 20 rows) and `/pr/pr` (recent projects, from a *different* database). -- **`idetooltips/.../ToolTipManager.kt`** — serves Tier 1/2. Looks up a tooltip by `(category, tag)` through `queryTooltip` in `TooltipQuery.kt`: the category's id from `TooltipCategories` first, so the row is found through `Tooltips`' `UNIQUE (categoryId, tag)` index rather than a scan of the whole table (ADFA-6597). Then `TooltipButtons` for the Tier 3 links shown at the bottom, which still scans that table: it has no index on `tooltipId`. +- **`idetooltips/.../ToolTipManager.kt`** — serves Tier 1/2, through `queryTooltip` and `queryTooltipButtons` in `TooltipQuery.kt`. Looks up a tooltip by `(category, tag)`: the category's id from `TooltipCategories` first, so the row is found by `Tooltips`' key rather than a scan of the whole table (ADFA-6597), then its text by id. Then its buttons for the Tier 3 links shown at the bottom, by `TooltipButtons`' key, with their text by id. Refuses a database declaring a version below 3. - **`plugin-manager/.../documentation/PluginDocumentationManager.kt`** (with `Tier3AssetWalker.kt`, and the `DocumentationExtension` contract in `plugin-api`) — lets plugins contribute their own help content into the same lookup paths, under the reserved `plugin//...` prefix. Brotli assets are compressed against the same `CompressionDictionary` the rest of the table uses, read from the database being written. When that dictionary can't be read the install is abandoned rather than written plain, and retried on the plugin's next activation. ## Editing the database diff --git a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt index 9bf8dd6259..173b3ba82f 100644 --- a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt +++ b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt @@ -53,13 +53,6 @@ object TooltipManager { private val debugDatabaseFile: File = File(android.os.Environment.getExternalStorageDirectory().toString() + "/Download/documentation.db") - private const val QUERY_TOOLTIP_BUTTONS = """ - SELECT description, uri - FROM TooltipButtons - WHERE tooltipId = ? - ORDER BY buttonNumberId - """ - suspend fun getTooltip(context: Context, category: String, tag: String): IDETooltipItem? { Log.d(TAG, "In getTooltip() for category='$category', tag='$tag'.") @@ -100,24 +93,19 @@ object TooltipManager { } c.moveToFirst() - val rowId = c.getInt(0) - val tooltipId = c.getInt(1) - val summary = c.getString(2) - val detail = c.getString(3) + val tooltipId = c.getInt(0) + val summary = c.getString(1) + val detail = c.getString(2) val buttons = ArrayList>() - database.rawQuery(QUERY_TOOLTIP_BUTTONS, arrayOf(tooltipId.toString())).use { bc -> - while (bc.moveToNext()) { - buttons.add( - Pair( - bc.getString(0), - "http://localhost:6174/" + bc.getString(1) - ) - ) - } + for ((description, uri) in queryTooltipButtons(database, tooltipId)) { + buttons.add(Pair(description, "http://localhost:6174/$uri")) } - IDETooltipItem(rowId, tooltipId, category, tag, summary, detail, buttons, lastChange) + // Tooltips has no rowid from documentation version 3 on. It always equalled + // id before (id was the INTEGER PRIMARY KEY), so the debug dialog's Row is + // still the id. + IDETooltipItem(tooltipId, tooltipId, category, tag, summary, detail, buttons, lastChange) } } } catch (e: NoTooltipFoundException) { diff --git a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt index 71354eedc0..99a1542cf2 100644 --- a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt +++ b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt @@ -3,23 +3,52 @@ package com.itsaky.androidide.idetooltips import android.database.Cursor import android.database.sqlite.SQLiteDatabase import androidx.annotation.VisibleForTesting +import com.itsaky.androidide.utils.DatabaseVersionResolver -// Looks the category up first, so the tag is found through Tooltips' UNIQUE (categoryId, tag) -// index. Written as a join, it made SQLite's planner scan all of Tooltips (46,128 rows, 13 MB) -// for every tooltip shown, because documentation.db carries no ANALYZE statistics (ADFA-6597). +// From documentation version 3 on (ADFA-3305), Tooltips is keyed by (categoryId, tag) and holds +// only the id; the text is in TooltipTexts, one row per id. Looks the category up first, so the +// tag is found by that key. Written as a join, it made SQLite's planner scan all of Tooltips for +// every tooltip shown, because documentation.db carries no ANALYZE statistics (ADFA-6597). // IN rather than =, so two categories sharing a name still come back as duplicates, as before. @VisibleForTesting internal const val QUERY_TOOLTIP = """ - SELECT T.rowid, T.id, T.summary, T.detail + SELECT T.id, X.summary, X.detail FROM Tooltips AS T + JOIN TooltipTexts AS X ON X.id = T.id WHERE T.categoryId IN (SELECT id FROM TooltipCategories WHERE category = ?) AND T.tag = ? """ +// TooltipButtons is keyed by (tooltipId, buttonNumberId), so a tooltip's buttons are one range of +// that key, already in display order. Each distinct description and uri is stored once. +@VisibleForTesting +internal const val QUERY_TOOLTIP_BUTTONS = """ + SELECT D.description, U.uri + FROM TooltipButtons AS B + JOIN TooltipButtonDescriptions AS D ON D.id = B.descriptionId + JOIN TooltipButtonUris AS U ON U.id = B.uriId + WHERE B.tooltipId = ? + ORDER BY B.buttonNumberId +""" + // Outside TooltipManager, so tests can run it without initializing the object, which reads the // installed app's Environment. Binds the arguments in the order QUERY_TOOLTIP expects them. +// Refuses a database older than version 3 first, rather than fail on whichever old table or +// column the query meets. internal fun queryTooltip( database: SQLiteDatabase, category: String, tag: String, -): Cursor = database.rawQuery(QUERY_TOOLTIP, arrayOf(category, tag)) +): Cursor { + DatabaseVersionResolver.requireKeyedTooltips(database) + return database.rawQuery(QUERY_TOOLTIP, arrayOf(category, tag)) +} + +/** The (description, uri) of each of [tooltipId]'s buttons, in display order. */ +internal fun queryTooltipButtons( + database: SQLiteDatabase, + tooltipId: Int, +): List> = + database.rawQuery(QUERY_TOOLTIP_BUTTONS, arrayOf(tooltipId.toString())).use { c -> + buildList { while (c.moveToNext()) add(Pair(c.getString(0), c.getString(1))) } + } diff --git a/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt b/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt index 6af7b78ba1..fb15914b09 100644 --- a/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt +++ b/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt @@ -4,6 +4,7 @@ import android.database.sqlite.SQLiteDatabase import org.junit.After import org.junit.Assert.assertEquals import org.junit.Assert.assertFalse +import org.junit.Assert.assertThrows import org.junit.Assert.assertTrue import org.junit.Before import org.junit.Test @@ -12,9 +13,11 @@ import org.robolectric.RobolectricTestRunner import org.robolectric.annotation.Config /** - * Runs [queryTooltip], the lookup TooltipManager.getTooltip makes, against documentation.db's own - * Tooltips schema. The query plan matters as much as the rows: the join it replaced scanned the - * whole 13 MB table for every tooltip shown (ADFA-6597). + * Runs [queryTooltip] and [queryTooltipButtons], the lookups TooltipManager.getTooltip makes, + * against documentation.db's version 3 tooltip schema (ADFA-3305), as migrate_tooltip_schema.py in + * OfflineDocumentationTools writes it. The query plans matter as much as the rows: the join + * QUERY_TOOLTIP replaced scanned the whole 13 MB table for every tooltip shown (ADFA-6597), and the + * buttons query used to scan and sort all of TooltipButtons. */ @RunWith(RobolectricTestRunner::class) @Config(manifest = Config.NONE, sdk = [28]) @@ -24,22 +27,23 @@ class TooltipQueryTest { @Before fun setUp() { db = SQLiteDatabase.create(null) - db.execSQL("CREATE TABLE TooltipCategories (id INTEGER PRIMARY KEY, category TEXT NOT NULL)") - db.execSQL( - "CREATE TABLE Tooltips (id INTEGER PRIMARY KEY AUTOINCREMENT, categoryId INTEGER NOT NULL, " + - "tag TEXT NOT NULL, summary TEXT NOT NULL, detail TEXT NOT NULL, UNIQUE (categoryId, tag), " + - "FOREIGN KEY(categoryId) REFERENCES TooltipCategories(id))", - ) + for (statement in SCHEMA) db.execSQL(statement) + setVersion(3) listOf("ide", "xml", "java", "kotlin").forEachIndexed { i, category -> db.execSQL("INSERT INTO TooltipCategories VALUES (?, ?)", arrayOf(i + 1, category)) } + db.execSQL("INSERT INTO TooltipButtonDescriptions VALUES (1, 'View full documentation'), (2, 'Examples')") db.beginTransaction() + var id = 1000 for (categoryId in 1..4) { for (n in 0 until 500) { - db.execSQL( - "INSERT INTO Tooltips (categoryId, tag, summary, detail) VALUES (?, ?, ?, ?)", - arrayOf(categoryId, "tag$n", "summary $categoryId/$n", "detail"), - ) + id++ + db.execSQL("INSERT INTO TooltipTexts VALUES (?, ?, ?)", arrayOf(id, "summary $categoryId/$n", "detail")) + db.execSQL("INSERT INTO Tooltips VALUES (?, ?, ?)", arrayOf(id, categoryId, "tag$n")) + db.execSQL("INSERT INTO TooltipButtonUris VALUES (?, ?)", arrayOf(id, "page$id.html")) + // Button 2 first, so the order has to come from the key, not from insertion. + db.execSQL("INSERT INTO TooltipButtons VALUES (?, 2, 2, ?)", arrayOf(id, id)) + db.execSQL("INSERT INTO TooltipButtons VALUES (?, 1, 1, ?)", arrayOf(id, id)) } } db.setTransactionSuccessful() @@ -51,12 +55,30 @@ class TooltipQueryTest { db.close() } + private fun setVersion(major: Int) { + db.execSQL("DELETE FROM DocumentationDatabaseVersion") + db.execSQL( + "INSERT INTO DocumentationDatabaseVersion (major, minor, patch, who, comment) VALUES (?, 0, 0, 'test', 'test')", + arrayOf(major), + ) + } + + private fun plan( + query: String, + vararg args: String, + ): List = + db.rawQuery("EXPLAIN QUERY PLAN $query", args).use { c -> + buildList { while (c.moveToNext()) add(c.getString(3)) } + } + @Test fun `finds the one tooltip for a category and tag`() { queryTooltip(db, category = "java", tag = "tag7").use { c -> assertEquals(1, c.count) c.moveToFirst() - assertEquals("summary 3/7", c.getString(2)) + assertEquals(1000 + 2 * 500 + 8, c.getInt(0)) + assertEquals("summary 3/7", c.getString(1)) + assertEquals("detail", c.getString(2)) } queryTooltip(db, category = "java", tag = "no-such-tag").use { assertEquals(0, it.count) } queryTooltip(db, category = "no-such-category", tag = "tag7").use { assertEquals(0, it.count) } @@ -65,19 +87,82 @@ class TooltipQueryTest { @Test fun `two categories with the same name still come back as two rows`() { db.execSQL("INSERT INTO TooltipCategories VALUES (5, 'java')") - db.execSQL("INSERT INTO Tooltips (categoryId, tag, summary, detail) VALUES (5, 'tag7', 'duplicate', 'detail')") + db.execSQL("INSERT INTO TooltipTexts VALUES (9999, 'duplicate', 'detail')") + db.execSQL("INSERT INTO Tooltips VALUES (9999, 5, 'tag7')") queryTooltip(db, category = "java", tag = "tag7").use { assertEquals(2, it.count) } } @Test - fun `finds the tag through the (categoryId, tag) index, not a scan of Tooltips`() { - val plan = - db.rawQuery("EXPLAIN QUERY PLAN " + QUERY_TOOLTIP, arrayOf("java", "tag7")).use { c -> - buildList { while (c.moveToNext()) add(c.getString(3)) } - } + fun `finds the tag by Tooltips' key and the text by TooltipTexts' id, scanning neither`() { + val plan = plan(QUERY_TOOLTIP, "java", "tag7") + + assertTrue(plan.toString(), plan.any { Regex("""\bUSING PRIMARY KEY \(categoryId=\? AND tag=\?\)""").containsMatchIn(it) }) + assertTrue(plan.toString(), plan.any { Regex("""\b(X|TooltipTexts)\b.*USING INTEGER PRIMARY KEY""").containsMatchIn(it) }) + assertFalse(plan.toString(), plan.any { Regex("""\bSCAN (TABLE )?(T|X|Tooltips|TooltipTexts)\b""").containsMatchIn(it) }) + } + + @Test + fun `returns a tooltip's buttons in button order with their text`() { + val id = 1000 + 2 * 500 + 8 + + assertEquals( + listOf("View full documentation" to "page$id.html", "Examples" to "page$id.html"), + queryTooltipButtons(db, id), + ) + assertEquals(emptyList>(), queryTooltipButtons(db, 1)) + } + + @Test + fun `finds the buttons by TooltipButtons' key, already in order, and their text by id`() { + val plan = plan(QUERY_TOOLTIP_BUTTONS, "1009") + + assertTrue(plan.toString(), plan.any { Regex("""\bUSING PRIMARY KEY \(tooltipId=\?\)""").containsMatchIn(it) }) + assertFalse(plan.toString(), plan.any { "TEMP B-TREE" in it }) + assertFalse(plan.toString(), plan.any { Regex("""\bSCAN (TABLE )?\w+""").containsMatchIn(it) }) + } + + @Test + fun `refuses a database that declares a version before 3`() { + setVersion(2) + + val e = assertThrows(IllegalStateException::class.java) { queryTooltip(db, category = "java", tag = "tag7") } + assertTrue(e.message, "version 2" in e.message.orEmpty()) + } + + @Test + fun `refuses a database that declares no version`() { + db.execSQL("DROP TABLE DocumentationDatabaseVersion") + + val e = assertThrows(IllegalStateException::class.java) { queryTooltip(db, category = "java", tag = "tag7") } + assertTrue(e.message, "version none" in e.message.orEmpty()) + } + + @Test + fun `reads a database that declares a later version`() { + setVersion(4) + + queryTooltip(db, category = "java", tag = "tag7").use { assertEquals(1, it.count) } + } - assertTrue(plan.toString(), plan.any { "sqlite_autoindex_Tooltips_1" in it }) - assertFalse(plan.toString(), plan.any { Regex("""\bSCAN (TABLE )?(T|Tooltips)\b""").containsMatchIn(it) }) + private companion object { + // documentation.db's version 3 tooltip tables, as migrate_tooltip_schema.py creates them. + val SCHEMA = + listOf( + "CREATE TABLE TooltipCategories (id INTEGER PRIMARY KEY, category TEXT NOT NULL)", + "CREATE TABLE TooltipTexts (id INTEGER PRIMARY KEY AUTOINCREMENT, summary TEXT NOT NULL, detail TEXT NOT NULL)", + "CREATE TABLE Tooltips (id INTEGER NOT NULL, categoryId INTEGER NOT NULL, tag TEXT NOT NULL, " + + "PRIMARY KEY (categoryId, tag), FOREIGN KEY(categoryId) REFERENCES TooltipCategories(id), " + + "FOREIGN KEY(id) REFERENCES TooltipTexts(id)) WITHOUT ROWID", + "CREATE TABLE TooltipButtonDescriptions (id INTEGER PRIMARY KEY, description TEXT NOT NULL)", + "CREATE TABLE TooltipButtonUris (id INTEGER PRIMARY KEY, uri TEXT NOT NULL)", + "CREATE TABLE TooltipButtons (tooltipId INTEGER NOT NULL, buttonNumberId INTEGER NOT NULL, " + + "descriptionId INTEGER NOT NULL, uriId INTEGER NOT NULL, PRIMARY KEY (tooltipId, buttonNumberId), " + + "FOREIGN KEY(tooltipId) REFERENCES TooltipTexts(id), " + + "FOREIGN KEY(descriptionId) REFERENCES TooltipButtonDescriptions(id), " + + "FOREIGN KEY(uriId) REFERENCES TooltipButtonUris(id)) WITHOUT ROWID", + "CREATE TABLE DocumentationDatabaseVersion (major INT NOT NULL, minor INT NOT NULL, patch INT NOT NULL, " + + "who TEXT NOT NULL, comment TEXT NOT NULL, changeTime TIMESTAMP DEFAULT CURRENT_TIMESTAMP)", + ) } } From 5cc8d2bc94f9c7ba6c06feffb4dbfa8160f7b98a Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 18 Oct 2026 13:21:18 -0700 Subject: [PATCH 5/9] ADFA-3305: Write plugin tooltips into documentation version 3's tables PluginTooltipTables.kt now writes the version 3 tooltip tables: - A tooltip's text goes into TooltipTexts first, whose AUTOINCREMENT id then keys the Tooltips row. The id is the one that insert returns: Tooltips is WITHOUT ROWID, and SQLite does not record its inserts as a rowid, so an insert into it returns the previous insert's rowid, not anything of its own. - Each button's description and uri are looked up in, or added to, TooltipButtonDescriptions and TooltipButtonUris, 500 strings per query, so text a built-in button already uses is shared. - A plugin's rows are deleted by category through subqueries, which also removes the old one-variable-per-tooltip IN list, and text no button names any more is deleted after every write and removal. - Every insert throws on failure. db.insert returned -1 instead, so a failed insert left a tooltip without its buttons, or buttons on tooltip -1, and the transaction committed anyway. - Both entry points refuse a database declaring a version below 3 before writing anything. Tests run against the version 3 DDL with one built-in tooltip: what a plugin writes reads back through the app's queries; new ids are never reused; a repeated tag keeps the later entry; text is shared and cleaned up; removing every plugin leaves the tables exactly as they were; 1,200 tooltips write and remove; a version 2 database is refused untouched; and a failed insert into any of the six tables throws and rolls back. plugin-manager's tests now use Robolectric, already in the version catalog, for SQLiteDatabase. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3 --- docs/documentation-database.md | 2 +- plugin-manager/build.gradle.kts | 1 + .../PluginDocumentationManager.kt | 2 +- .../documentation/PluginTooltipTables.kt | 165 ++++++---- .../documentation/PluginTooltipTablesTest.kt | 306 ++++++++++++++++++ 5 files changed, 407 insertions(+), 69 deletions(-) create mode 100644 plugin-manager/src/test/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTablesTest.kt diff --git a/docs/documentation-database.md b/docs/documentation-database.md index 56d7b4410b..844c0fdb39 100644 --- a/docs/documentation-database.md +++ b/docs/documentation-database.md @@ -96,7 +96,7 @@ Of the five sites below, only `DocumentationContentSource` and `ToolTipManager` - **`common/.../documentation/DocumentationRequestInterceptor.kt`** — serves Tier 3 *in-process* for the app's WebViews (`HelpActivity`, the tooltip fragment, `FAQActivity`), through `WebViewClient.shouldInterceptRequest`, so a page's assets cost a database read instead of a TCP connection each (ADFA-5176). It matches the same `http://localhost:6174/...` URL space, so the strings.xml entries, `ToolTipManager`'s link builder and the `DocumentationExtension` contract need no changes; anything it declines — a `/pr/` endpoint, an unknown path, a failed read — falls through to `WebServer` unchanged. Both transports get their `Content-Type` charset from the same place, `ContentTypeHeaders` (ADFA-5241), so a row does not describe itself differently depending on which one served it. Set `/sdcard/Download/CodeOnTheGo.nointercept` to force documentation back onto the server. - **`app/.../localWebServer/WebServer.kt`** — serves Tier 3 over HTTP on port 6174, for WebViews that are not wired to the interceptor above and for the `/pr/` developer endpoints. It reads through `DocumentationContentSource`, so it holds no database, template engine or decode logic of its own; what remains here is HTTP: request parsing, the `/pr/` pages, error responses, and the CSS/asset shortcuts. Also serves a Dynamic Bookshelf JSON payload (joining `Content`/`Bookshelf`/`BookCategories`, rendered through the `bookshelf` template) and debug-only HTML dumps at `/pr/db` (`LastChange`, last 20 rows) and `/pr/pr` (recent projects, from a *different* database). - **`idetooltips/.../ToolTipManager.kt`** — serves Tier 1/2, through `queryTooltip` and `queryTooltipButtons` in `TooltipQuery.kt`. Looks up a tooltip by `(category, tag)`: the category's id from `TooltipCategories` first, so the row is found by `Tooltips`' key rather than a scan of the whole table (ADFA-6597), then its text by id. Then its buttons for the Tier 3 links shown at the bottom, by `TooltipButtons`' key, with their text by id. Refuses a database declaring a version below 3. -- **`plugin-manager/.../documentation/PluginDocumentationManager.kt`** (with `Tier3AssetWalker.kt`, and the `DocumentationExtension` contract in `plugin-api`) — lets plugins contribute their own help content into the same lookup paths, under the reserved `plugin//...` prefix. Brotli assets are compressed against the same `CompressionDictionary` the rest of the table uses, read from the database being written. When that dictionary can't be read the install is abandoned rather than written plain, and retried on the plugin's next activation. +- **`plugin-manager/.../documentation/PluginDocumentationManager.kt`** (with `Tier3AssetWalker.kt`, and the `DocumentationExtension` contract in `plugin-api`) — lets plugins contribute their own help content into the same lookup paths, under the reserved `plugin//...` prefix. Its tooltip SQL is in `PluginTooltipTables.kt`: a plugin's tooltips are replaced or removed by category inside one transaction, each button's text is looked up in or added to the lookup tables, and text no button uses any more is deleted afterwards. Brotli assets are compressed against the same `CompressionDictionary` the rest of the table uses, read from the database being written. When that dictionary can't be read the install is abandoned rather than written plain, and retried on the plugin's next activation. ## Editing the database diff --git a/plugin-manager/build.gradle.kts b/plugin-manager/build.gradle.kts index e75526fe2d..5e5bf01abb 100644 --- a/plugin-manager/build.gradle.kts +++ b/plugin-manager/build.gradle.kts @@ -39,4 +39,5 @@ dependencies { testImplementation(libs.tests.junit) testImplementation(libs.tests.google.truth) + testImplementation(libs.tests.robolectric) } diff --git a/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginDocumentationManager.kt b/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginDocumentationManager.kt index f13adaa3ee..4b42ae1a39 100644 --- a/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginDocumentationManager.kt +++ b/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginDocumentationManager.kt @@ -20,7 +20,7 @@ import kotlin.coroutines.cancellation.CancellationException /** * Manages plugin documentation by writing into the main documentation.db. - * Plugin entries are stored in the existing Tooltips/TooltipCategories/TooltipButtons tables, + * Plugin entries are stored in the existing tooltip tables (see PluginTooltipTables.kt), * differentiated by a "plugin_" category prefix so they never conflict with * built-in documentation. */ diff --git a/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTables.kt b/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTables.kt index 5b9e3b2164..877bbc696b 100644 --- a/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTables.kt +++ b/plugin-manager/src/main/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTables.kt @@ -2,6 +2,7 @@ package com.itsaky.androidide.plugins.manager.documentation import android.content.ContentValues import android.database.sqlite.SQLiteDatabase +import com.itsaky.androidide.utils.DatabaseVersionResolver /** One plugin tooltip as it is written: summary and detail final, buttons in display order. */ internal class PluginTooltipRow( @@ -17,57 +18,106 @@ internal class PluginTooltipButtonRow( val uri: String, ) +// A subquery rather than a join, so Tooltips is searched by its (categoryId, tag) key. +private const val CATEGORY_IDS = "SELECT id FROM TooltipCategories WHERE category = ?" +private const val CATEGORY_TOOLTIP_IDS = "SELECT id FROM Tooltips WHERE categoryId IN ($CATEGORY_IDS)" + +// IN takes one variable per string; 500 stays under SQLite's pre-3.32 default limit of 999. +private const val LOOKUP_CHUNK = 500 + /** * Replaces every tooltip in [category] with [tooltips], inside the caller's transaction. + * + * Writes documentation version 3's tooltip tables (ADFA-3305): the key in Tooltips, the text in + * TooltipTexts, and each button's text as ids into TooltipButtonDescriptions and TooltipButtonUris. + * Refuses an older database before writing anything. Every insert throws on failure, so a failed + * one rolls the transaction back rather than commit a tooltip without its buttons. */ internal fun writePluginTooltips( db: SQLiteDatabase, category: String, tooltips: List, ) { - removePluginTooltips(db, category) + DatabaseVersionResolver.requireKeyedTooltips(db) + deleteCategoryTooltips(db, category) val categoryId = insertOrGetCategoryId(db, category) + val buttons = tooltips.flatMap { it.buttons } + val descriptionIds = textIds(db, "TooltipButtonDescriptions", "description", buttons.map { it.description }) + val uriIds = textIds(db, "TooltipButtonUris", "uri", buttons.map { it.uri }) for (tooltip in tooltips) { val tooltipId = insertTooltip(db, categoryId, tooltip) tooltip.buttons.forEachIndexed { index, button -> - insertTooltipButton(db, tooltipId, button.description, button.uri, index) + val values = + ContentValues().apply { + put("tooltipId", tooltipId) + put("buttonNumberId", index) + put("descriptionId", descriptionIds.getValue(button.description)) + put("uriId", uriIds.getValue(button.uri)) + } + db.insertOrThrow("TooltipButtons", null, values) } } + deleteUnusedButtonText(db) } /** - * Removes [category] and every tooltip in it, inside the caller's transaction. + * Removes [category] and every tooltip in it, inside the caller's transaction, with the button + * text no other button uses. Refuses a database older than documentation version 3. */ internal fun removePluginTooltips( db: SQLiteDatabase, category: String, ) { - val cursor = - db.rawQuery( - """ - SELECT T.id FROM Tooltips AS T - INNER JOIN TooltipCategories AS TC ON T.categoryId = TC.id - WHERE TC.category = ? - """.trimIndent(), - arrayOf(category), - ) + DatabaseVersionResolver.requireKeyedTooltips(db) + deleteCategoryTooltips(db, category) + deleteUnusedButtonText(db) +} - val tooltipIds = mutableListOf() - while (cursor.moveToNext()) { - tooltipIds.add(cursor.getLong(0)) - } - cursor.close() +private fun deleteCategoryTooltips( + db: SQLiteDatabase, + category: String, +) { + // Buttons and text first: both are found through the Tooltips rows deleted after them. + db.execSQL("DELETE FROM TooltipButtons WHERE tooltipId IN ($CATEGORY_TOOLTIP_IDS)", arrayOf(category)) + db.execSQL("DELETE FROM TooltipTexts WHERE id IN ($CATEGORY_TOOLTIP_IDS)", arrayOf(category)) + db.execSQL("DELETE FROM Tooltips WHERE categoryId IN ($CATEGORY_IDS)", arrayOf(category)) + db.delete("TooltipCategories", "category = ?", arrayOf(category)) +} - if (tooltipIds.isNotEmpty()) { - val placeholders = tooltipIds.joinToString(",") { "?" } - val args = tooltipIds.map { it.toString() }.toTypedArray() - db.delete("TooltipButtons", "tooltipId IN ($placeholders)", args) - db.delete("Tooltips", "id IN ($placeholders)", args) - } +// A description or uri no button names any more: one only a removed tooltip used. Text shared with +// a built-in button, or another plugin's, stays. +private fun deleteUnusedButtonText(db: SQLiteDatabase) { + db.execSQL("DELETE FROM TooltipButtonDescriptions WHERE id NOT IN (SELECT descriptionId FROM TooltipButtons)") + db.execSQL("DELETE FROM TooltipButtonUris WHERE id NOT IN (SELECT uriId FROM TooltipButtons)") +} - db.delete("TooltipCategories", "category = ?", arrayOf(category)) +/** + * The id of each of [values] in [table], inserting the ones it lacks. [column] has no index -- a + * UNIQUE one would cost about as much as the table -- so the strings are looked up in chunks, each + * one read of the table, rather than one read per string. + */ +private fun textIds( + db: SQLiteDatabase, + table: String, + column: String, + values: List, +): Map { + val distinct = values.distinct() + val ids = HashMap() + for (chunk in distinct.chunked(LOOKUP_CHUNK)) { + val placeholders = chunk.joinToString(",") { "?" } + db.rawQuery("SELECT $column, id FROM $table WHERE $column IN ($placeholders)", chunk.toTypedArray()).use { c -> + while (c.moveToNext()) ids.putIfAbsent(c.getString(0), c.getLong(1)) + } + } + for (value in distinct) { + if (value !in ids) { + ids[value] = db.insertOrThrow(table, null, ContentValues().apply { put(column, value) }) + } + } + return ids } private fun insertOrGetCategoryId( @@ -96,7 +146,7 @@ private fun insertOrGetCategoryId( ContentValues().apply { put("category", category) } - return db.insert("TooltipCategories", null, values) + return db.insertOrThrow("TooltipCategories", null, values) } private fun insertTooltip( @@ -104,55 +154,36 @@ private fun insertTooltip( categoryId: Long, tooltip: PluginTooltipRow, ): Long { - val existingCursor = - db.query( - "Tooltips", - arrayOf("id"), - "categoryId = ? AND tag = ?", - arrayOf(categoryId.toString(), tooltip.tag), - null, - null, - null, - ) - - if (existingCursor.moveToFirst()) { - val existingId = existingCursor.getLong(0) - existingCursor.close() + val existingId = + db + .rawQuery( + "SELECT id FROM Tooltips WHERE categoryId = ? AND tag = ?", + arrayOf(categoryId.toString(), tooltip.tag), + ).use { if (it.moveToFirst()) it.getLong(0) else null } + + val text = + ContentValues().apply { + put("summary", tooltip.summary) + put("detail", tooltip.detail) + } - val updateValues = - ContentValues().apply { - put("summary", tooltip.summary) - put("detail", tooltip.detail) - } - db.update("Tooltips", updateValues, "id = ?", arrayOf(existingId.toString())) + if (existingId != null) { + // Only a tag the plugin lists twice, since the category was just emptied; the later wins. + db.update("TooltipTexts", text, "id = ?", arrayOf(existingId.toString())) db.delete("TooltipButtons", "tooltipId = ?", arrayOf(existingId.toString())) return existingId } - existingCursor.close() - val values = + // TooltipTexts hands out the id: AUTOINCREMENT, so never one a deleted tooltip had. Tooltips is + // WITHOUT ROWID, and SQLite does not record its inserts as a rowid, so what insertOrThrow + // returns for it is some earlier insert's. + val id = db.insertOrThrow("TooltipTexts", null, text) + val key = ContentValues().apply { + put("id", id) put("categoryId", categoryId) put("tag", tooltip.tag) - put("summary", tooltip.summary) - put("detail", tooltip.detail) - } - return db.insert("Tooltips", null, values) -} - -private fun insertTooltipButton( - db: SQLiteDatabase, - tooltipId: Long, - description: String, - uri: String, - order: Int, -) { - val values = - ContentValues().apply { - put("tooltipId", tooltipId) - put("description", description) - put("uri", uri) - put("buttonNumberId", order) } - db.insert("TooltipButtons", null, values) + db.insertOrThrow("Tooltips", null, key) + return id } diff --git a/plugin-manager/src/test/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTablesTest.kt b/plugin-manager/src/test/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTablesTest.kt new file mode 100644 index 0000000000..07575e8d39 --- /dev/null +++ b/plugin-manager/src/test/kotlin/com/itsaky/androidide/plugins/manager/documentation/PluginTooltipTablesTest.kt @@ -0,0 +1,306 @@ +package com.itsaky.androidide.plugins.manager.documentation + +import android.database.SQLException +import android.database.sqlite.SQLiteDatabase +import org.junit.After +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNotEquals +import org.junit.Assert.assertThrows +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.annotation.Config + +/** + * Runs [writePluginTooltips] and [removePluginTooltips] against documentation.db's version 3 + * tooltip schema (ADFA-3305), as migrate_tooltip_schema.py in OfflineDocumentationTools writes it, + * holding one built-in tooltip that plugins must never disturb. What a plugin writes is read back + * with the queries the app runs (idetooltips' TooltipQuery.kt). + */ +@RunWith(RobolectricTestRunner::class) +@Config(manifest = Config.NONE, sdk = [28]) +class PluginTooltipTablesTest { + private lateinit var db: SQLiteDatabase + + @Before + fun setUp() { + db = SQLiteDatabase.create(null) + for (statement in SCHEMA) db.execSQL(statement) + setVersion(3) + db.execSQL("INSERT INTO TooltipCategories VALUES (1, 'ide')") + db.execSQL("INSERT INTO TooltipTexts VALUES ($BUILT_IN_ID, 'built-in summary', 'built-in detail')") + db.execSQL("INSERT INTO Tooltips VALUES ($BUILT_IN_ID, 1, 'editor')") + db.execSQL("INSERT INTO TooltipButtonDescriptions VALUES (1, '$SHARED_DESCRIPTION')") + db.execSQL("INSERT INTO TooltipButtonUris VALUES (1, 'a/editor.html')") + db.execSQL("INSERT INTO TooltipButtons VALUES ($BUILT_IN_ID, 1, 1, 1)") + } + + @After + fun tearDown() { + db.close() + } + + private fun setVersion(major: Int) { + db.execSQL("DELETE FROM DocumentationDatabaseVersion") + db.execSQL( + "INSERT INTO DocumentationDatabaseVersion (major, minor, patch, who, comment) VALUES (?, 0, 0, 'test', 'test')", + arrayOf(major), + ) + } + + private fun inTransaction(block: () -> Unit) { + db.beginTransaction() + try { + block() + db.setTransactionSuccessful() + } finally { + db.endTransaction() + } + } + + private fun write( + category: String, + vararg tooltips: PluginTooltipRow, + ) = inTransaction { writePluginTooltips(db, category, tooltips.toList()) } + + private fun remove(category: String) = inTransaction { removePluginTooltips(db, category) } + + /** What the app shows for (category, tag): id, summary, detail and buttons, or null if nothing. */ + private fun shown( + category: String, + tag: String, + ): Shown? { + val tooltip = + db.rawQuery(APP_TOOLTIP_QUERY, arrayOf(category, tag)).use { c -> + assertTrue("more than one row for $category/$tag", c.count <= 1) + if (c.moveToFirst()) Triple(c.getLong(0), c.getString(1), c.getString(2)) else null + } ?: return null + val buttons = + db.rawQuery(APP_BUTTONS_QUERY, arrayOf(tooltip.first.toString())).use { c -> + buildList { while (c.moveToNext()) add(c.getString(0) to c.getString(1)) } + } + return Shown(tooltip.first, tooltip.second, tooltip.third, buttons) + } + + private data class Shown( + val id: Long, + val summary: String, + val detail: String, + val buttons: List>, + ) + + private fun count(table: String): Long = + db.rawQuery("SELECT COUNT(*) FROM $table", null).use { + it.moveToFirst() + it.getLong(0) + } + + private fun contents(): Map>> = + TABLES.associateWith { table -> + db.rawQuery("SELECT * FROM $table ORDER BY 1, 2", null).use { c -> + buildList { while (c.moveToNext()) add((0 until c.columnCount).map { c.getString(it) }) } + } + } + + private val builtIn = Shown(BUILT_IN_ID, "built-in summary", "built-in detail", listOf(SHARED_DESCRIPTION to "a/editor.html")) + + private fun row( + tag: String, + summary: String = "summary of $tag", + detail: String = "detail of $tag", + vararg buttons: Pair, + ) = PluginTooltipRow(tag, summary, detail, buttons.map { PluginTooltipButtonRow(it.first, it.second) }) + + @Test + fun `writes tooltips the app reads back with their buttons in order`() { + write( + "plugin_x", + row("x.one", buttons = arrayOf("Read more" to "plugin/x/index.html", "Overview" to "i/plugins-adfa.html")), + row("x.two", detail = ""), + ) + + val one = shown("plugin_x", "x.one")!! + assertEquals("summary of x.one", one.summary) + assertEquals("detail of x.one", one.detail) + assertEquals(listOf("Read more" to "plugin/x/index.html", "Overview" to "i/plugins-adfa.html"), one.buttons) + val two = shown("plugin_x", "x.two")!! + assertEquals("", two.detail) + assertEquals(emptyList>(), two.buttons) + assertNotEquals(one.id, two.id) + assertEquals(builtIn, shown("ide", "editor")) + } + + @Test + fun `gives each new tooltip an id no tooltip has had`() { + write("plugin_x", row("x.one"), row("x.two")) + val first = listOf(shown("plugin_x", "x.one")!!.id, shown("plugin_x", "x.two")!!.id) + remove("plugin_x") + write("plugin_x", row("x.one")) + + val again = shown("plugin_x", "x.one")!!.id + assertTrue("$first, then $again", first.all { it > BUILT_IN_ID } && again > first.max()) + } + + @Test + fun `keeps the later entry for a tag listed twice`() { + write( + "plugin_x", + row("x.one", summary = "first", buttons = arrayOf("First" to "plugin/x/1.html")), + row("x.one", summary = "second", buttons = arrayOf("Second" to "plugin/x/2.html")), + ) + + val one = shown("plugin_x", "x.one")!! + assertEquals("second", one.summary) + assertEquals(listOf("Second" to "plugin/x/2.html"), one.buttons) + assertEquals(2L, count("Tooltips")) + assertEquals(2L, count("TooltipTexts")) + // "First" and plugin/x/1.html were looked up for the first entry; nothing uses them now. + assertEquals(listOf("Second", SHARED_DESCRIPTION), column("TooltipButtonDescriptions", "description")) + assertEquals(listOf("a/editor.html", "plugin/x/2.html"), column("TooltipButtonUris", "uri")) + } + + private fun column( + table: String, + column: String, + ): List = + db.rawQuery("SELECT $column FROM $table ORDER BY $column", null).use { c -> + buildList { while (c.moveToNext()) add(c.getString(0)) } + } + + @Test + fun `stores each button text once, sharing what built-in buttons use`() { + write( + "plugin_x", + row("x.one", buttons = arrayOf(SHARED_DESCRIPTION to "a/editor.html", "Read more" to "plugin/x/a.html")), + row("x.two", buttons = arrayOf("Read more" to "plugin/x/a.html")), + ) + + assertEquals(listOf("Read more", SHARED_DESCRIPTION), column("TooltipButtonDescriptions", "description")) + assertEquals(listOf("a/editor.html", "plugin/x/a.html"), column("TooltipButtonUris", "uri")) + assertEquals(listOf(SHARED_DESCRIPTION to "a/editor.html", "Read more" to "plugin/x/a.html"), shown("plugin_x", "x.one")!!.buttons) + } + + @Test + fun `removing a plugin leaves the database as it was before the plugin wrote`() { + val before = contents() + write( + "plugin_x", + row("x.one", buttons = arrayOf(SHARED_DESCRIPTION to "plugin/x/a.html", "Read more" to "a/editor.html")), + ) + write("plugin_y", row("y.one", buttons = arrayOf("Read more" to "plugin/y/a.html"))) + remove("plugin_x") + + assertEquals(null, shown("plugin_x", "x.one")) + assertEquals(listOf("Read more" to "plugin/y/a.html"), shown("plugin_y", "y.one")!!.buttons) + remove("plugin_y") + + assertEquals(before, contents()) + assertEquals(builtIn, shown("ide", "editor")) + } + + @Test + fun `writing again replaces the plugin's tooltips rather than adding to them`() { + write("plugin_x", row("x.one", buttons = arrayOf("Read more" to "plugin/x/a.html")), row("x.two")) + write("plugin_x", row("x.one", summary = "new", buttons = arrayOf("Overview" to "plugin/x/b.html"))) + + assertEquals(null, shown("plugin_x", "x.two")) + assertEquals(listOf("Overview" to "plugin/x/b.html"), shown("plugin_x", "x.one")!!.buttons) + assertEquals(2L, count("Tooltips")) + assertEquals(2L, count("TooltipTexts")) + assertEquals(2L, count("TooltipButtons")) + assertEquals(listOf("Overview", SHARED_DESCRIPTION), column("TooltipButtonDescriptions", "description")) + assertEquals(listOf("a/editor.html", "plugin/x/b.html"), column("TooltipButtonUris", "uri")) + } + + // Three lookup chunks. Robolectric's SQLite binds more than 33,000 variables, so this cannot show + // that the chunks keep under SQLite's pre-3.32 default limit of 999; only a device's SQLite can. + @Test + fun `writes and removes 1,200 tooltips at once`() { + val many = (1..1200).map { row("x.$it", buttons = arrayOf("Read $it" to "plugin/x/$it.html")) } + write("plugin_x", *many.toTypedArray()) + + assertEquals(listOf("Read 1100" to "plugin/x/1100.html"), shown("plugin_x", "x.1100")!!.buttons) + assertEquals(1201L, count("TooltipButtonUris")) + remove("plugin_x") + assertEquals(1L, count("Tooltips")) + assertEquals(1L, count("TooltipButtonUris")) + } + + @Test + fun `refuses a database before version 3 and writes nothing`() { + setVersion(2) + val before = contents() + + val written = assertThrows(IllegalStateException::class.java) { write("plugin_x", row("x.one")) } + assertTrue(written.message, "version 2" in written.message.orEmpty()) + val removed = assertThrows(IllegalStateException::class.java) { remove("ide") } + assertTrue(removed.message, "version 2" in removed.message.orEmpty()) + assertEquals(before, contents()) + } + + @Test + fun `throws when any insert fails rather than carry on without the row`() { + for (table in listOf( + "TooltipCategories", + "TooltipTexts", + "Tooltips", + "TooltipButtonDescriptions", + "TooltipButtonUris", + "TooltipButtons", + )) { + db.execSQL("CREATE TRIGGER refuse BEFORE INSERT ON $table BEGIN SELECT RAISE(ABORT, 'refused'); END") + val before = contents() + + assertThrows(table, SQLException::class.java) { + write("plugin_x", row("x.one", buttons = arrayOf("Read more" to "plugin/x/a.html"))) + } + assertEquals(table, before, contents()) + db.execSQL("DROP TRIGGER refuse") + } + } + + private companion object { + const val BUILT_IN_ID = 500L + const val SHARED_DESCRIPTION = "View full documentation" + + val TABLES = + listOf( + "TooltipCategories", + "TooltipTexts", + "Tooltips", + "TooltipButtonDescriptions", + "TooltipButtonUris", + "TooltipButtons", + ) + + // What idetooltips' TooltipQuery.kt runs. + const val APP_TOOLTIP_QUERY = + "SELECT T.id, X.summary, X.detail FROM Tooltips AS T JOIN TooltipTexts AS X ON X.id = T.id " + + "WHERE T.categoryId IN (SELECT id FROM TooltipCategories WHERE category = ?) AND T.tag = ?" + const val APP_BUTTONS_QUERY = + "SELECT D.description, U.uri FROM TooltipButtons AS B " + + "JOIN TooltipButtonDescriptions AS D ON D.id = B.descriptionId " + + "JOIN TooltipButtonUris AS U ON U.id = B.uriId WHERE B.tooltipId = ? ORDER BY B.buttonNumberId" + + // documentation.db's version 3 tooltip tables, as migrate_tooltip_schema.py creates them. + val SCHEMA = + listOf( + "CREATE TABLE TooltipCategories (id INTEGER PRIMARY KEY, category TEXT NOT NULL)", + "CREATE TABLE TooltipTexts (id INTEGER PRIMARY KEY AUTOINCREMENT, summary TEXT NOT NULL, detail TEXT NOT NULL)", + "CREATE TABLE Tooltips (id INTEGER NOT NULL, categoryId INTEGER NOT NULL, tag TEXT NOT NULL, " + + "PRIMARY KEY (categoryId, tag), FOREIGN KEY(categoryId) REFERENCES TooltipCategories(id), " + + "FOREIGN KEY(id) REFERENCES TooltipTexts(id)) WITHOUT ROWID", + "CREATE TABLE TooltipButtonDescriptions (id INTEGER PRIMARY KEY, description TEXT NOT NULL)", + "CREATE TABLE TooltipButtonUris (id INTEGER PRIMARY KEY, uri TEXT NOT NULL)", + "CREATE TABLE TooltipButtons (tooltipId INTEGER NOT NULL, buttonNumberId INTEGER NOT NULL, " + + "descriptionId INTEGER NOT NULL, uriId INTEGER NOT NULL, PRIMARY KEY (tooltipId, buttonNumberId), " + + "FOREIGN KEY(tooltipId) REFERENCES TooltipTexts(id), " + + "FOREIGN KEY(descriptionId) REFERENCES TooltipButtonDescriptions(id), " + + "FOREIGN KEY(uriId) REFERENCES TooltipButtonUris(id)) WITHOUT ROWID", + "CREATE TABLE DocumentationDatabaseVersion (major INT NOT NULL, minor INT NOT NULL, patch INT NOT NULL, " + + "who TEXT NOT NULL, comment TEXT NOT NULL, changeTime TIMESTAMP DEFAULT CURRENT_TIMESTAMP)", + ) + } +} From e22ade6cfb7b8e15cd5370d9af4d992d81aa024f Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 18 Oct 2026 23:47:28 -0700 Subject: [PATCH 6/9] ADFA-3305: Move getTooltip's row mapping into readTooltip No behaviour change. The query, the row-count check and the mapping of the row and its buttons to an IDETooltipItem move out of TooltipManager.getTooltip into readTooltip beside queryTooltip, so tests can run them without initializing TooltipManager, which reads the installed app's Environment. Spotless reformatted the moved `when`. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3 --- .../androidide/idetooltips/ToolTipManager.kt | 31 +----------- .../androidide/idetooltips/TooltipQuery.kt | 47 +++++++++++++++++++ 2 files changed, 48 insertions(+), 30 deletions(-) diff --git a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt index 173b3ba82f..ca280f43b7 100644 --- a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt +++ b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt @@ -77,36 +77,7 @@ object TooltipManager { "n/a" } - queryTooltip(database, category = category, tag = tag).use { c -> - when (c.count) { - // A missing tag is a normal outcome (e.g. a Gradle task with no - // authored tooltip). Honor the nullable contract so the caller - // shows nothing rather than an "n/a" popup. - 0 -> throw NoTooltipFoundException(category, tag) - 1 -> { /* Expected case, continue processing */ - } - - else -> throw DatabaseCorruptionException( - "Multiple tooltips found for category='$category', tag='$tag' (found ${c.count} rows). " + - "Each category/tag combination should be unique." - ) - } - - c.moveToFirst() - val tooltipId = c.getInt(0) - val summary = c.getString(1) - val detail = c.getString(2) - - val buttons = ArrayList>() - for ((description, uri) in queryTooltipButtons(database, tooltipId)) { - buttons.add(Pair(description, "http://localhost:6174/$uri")) - } - - // Tooltips has no rowid from documentation version 3 on. It always equalled - // id before (id was the INTEGER PRIMARY KEY), so the debug dialog's Row is - // still the id. - IDETooltipItem(tooltipId, tooltipId, category, tag, summary, detail, buttons, lastChange) - } + readTooltip(database, category = category, tag = tag, lastChange = lastChange) } } catch (e: NoTooltipFoundException) { Log.d(TAG, "No tooltip found for category='$category', tag='$tag'") diff --git a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt index 99a1542cf2..6e407e87aa 100644 --- a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt +++ b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/TooltipQuery.kt @@ -44,6 +44,53 @@ internal fun queryTooltip( return database.rawQuery(QUERY_TOOLTIP, arrayOf(category, tag)) } +/** + * The tooltip for [category] and [tag] as TooltipManager shows it, its buttons pointed at the local + * documentation server, stamped with [lastChange]. Throws [NoTooltipFoundException] when the + * database has none and [DatabaseCorruptionException] when it has more than one. + */ +internal fun readTooltip( + database: SQLiteDatabase, + category: String, + tag: String, + lastChange: String, +): IDETooltipItem = + queryTooltip(database, category = category, tag = tag).use { c -> + when (c.count) { + // A missing tag is a normal outcome (e.g. a Gradle task with no + // authored tooltip). Honor the nullable contract so the caller + // shows nothing rather than an "n/a" popup. + 0 -> { + throw NoTooltipFoundException(category, tag) + } + + 1 -> { // Expected case, continue processing + } + + else -> { + throw DatabaseCorruptionException( + "Multiple tooltips found for category='$category', tag='$tag' (found ${c.count} rows). " + + "Each category/tag combination should be unique.", + ) + } + } + + c.moveToFirst() + val tooltipId = c.getInt(0) + val summary = c.getString(1) + val detail = c.getString(2) + + val buttons = ArrayList>() + for ((description, uri) in queryTooltipButtons(database, tooltipId)) { + buttons.add(Pair(description, "http://localhost:6174/$uri")) + } + + // Tooltips has no rowid from documentation version 3 on. It always equalled + // id before (id was the INTEGER PRIMARY KEY), so the debug dialog's Row is + // still the id. + IDETooltipItem(tooltipId, tooltipId, category, tag, summary, detail, buttons, lastChange) + } + /** The (description, uri) of each of [tooltipId]'s buttons, in display order. */ internal fun queryTooltipButtons( database: SQLiteDatabase, From f9bd830cfa94267ae137729292d11479322d4da2 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 18 Oct 2026 23:51:22 -0700 Subject: [PATCH 7/9] ADFA-3305: Name the refused database file in the tooltip error log getTooltip reads the installed documentation.db or, when it is newer, the /sdcard/Download debug override, and the log said neither: a "declares documentation version 2" line left a developer guessing which file to replace. The log now names the source and its path. Review finding R2163-03. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3 --- .../java/com/itsaky/androidide/idetooltips/ToolTipManager.kt | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt index ca280f43b7..86ca589fa1 100644 --- a/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt +++ b/idetooltips/src/main/java/com/itsaky/androidide/idetooltips/ToolTipManager.kt @@ -58,6 +58,8 @@ object TooltipManager { return withContext(Dispatchers.IO) { var dbPath = Environment.DOC_DB.absolutePath + // Named in the error log: a refused database could be either file, and only the log says which. + var dbSource = "installed database" // TODO: The debug database code should only exist in a debug build. --DS, 30-Jul-2025 val debugDatabaseTimestamp = @@ -66,6 +68,7 @@ object TooltipManager { if (debugDatabaseTimestamp > databaseTimestamp) { // Switch to the debug database. dbPath = debugDatabaseFile.absolutePath + dbSource = "debug database" } try { @@ -88,7 +91,7 @@ object TooltipManager { } catch (e: Exception) { Log.e( TAG, - "Error getting tooltip for category='$category', tag='$tag': ${e.message}" + "Error getting tooltip for category='$category', tag='$tag' from the $dbSource '$dbPath': ${e.message}" ) null } From 49257e05630140db58913edecd786a497a1fffc8 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 18 Oct 2026 23:51:22 -0700 Subject: [PATCH 8/9] ADFA-3305: Test the tooltip item readTooltip builds from a row No test ran getTooltip's row mapping, though this branch renumbered its columns and made rowId the id: with the summary read from the detail column, all 12 idetooltips tests still passed. A tooltip whose id, summary, detail and buttons all differ is now read back into the exact item, so that mutant, a swapped detail and a wrong rowId each fail it. Also covers the missing-tag and duplicate-category exceptions. Review finding R2163-04. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3 --- .../idetooltips/TooltipQueryTest.kt | 50 ++++++++++++++++++- 1 file changed, 49 insertions(+), 1 deletion(-) diff --git a/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt b/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt index fb15914b09..2f669cf444 100644 --- a/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt +++ b/idetooltips/src/test/java/com/itsaky/androidide/idetooltips/TooltipQueryTest.kt @@ -13,7 +13,8 @@ import org.robolectric.RobolectricTestRunner import org.robolectric.annotation.Config /** - * Runs [queryTooltip] and [queryTooltipButtons], the lookups TooltipManager.getTooltip makes, + * Runs [queryTooltip] and [queryTooltipButtons], the lookups TooltipManager.getTooltip makes, and + * [readTooltip], which turns their rows into the item it shows, * against documentation.db's version 3 tooltip schema (ADFA-3305), as migrate_tooltip_schema.py in * OfflineDocumentationTools writes it. The query plans matter as much as the rows: the join * QUERY_TOOLTIP replaced scanned the whole 13 MB table for every tooltip shown (ADFA-6597), and the @@ -122,6 +123,53 @@ class TooltipQueryTest { assertFalse(plan.toString(), plan.any { Regex("""\bSCAN (TABLE )?\w+""").containsMatchIn(it) }) } + @Test + fun `reads a tooltip's id, text and buttons into the item TooltipManager shows`() { + db.execSQL("INSERT INTO TooltipTexts VALUES (4242, 'Declares a read-only property.', 'Longer detail.')") + db.execSQL("INSERT INTO Tooltips VALUES (4242, 4, 'val')") + db.execSQL("INSERT INTO TooltipButtonDescriptions VALUES (3, 'Learn more about properties')") + db.execSQL("INSERT INTO TooltipButtonUris VALUES (9001, 'k/html/properties.html'), (9002, 'k/html/basic-syntax.html')") + db.execSQL("INSERT INTO TooltipButtons VALUES (4242, 2, 2, 9002), (4242, 1, 3, 9001)") + + val item = readTooltip(db, category = "kotlin", tag = "val", lastChange = "2026-10-18 (test)") + + assertEquals( + IDETooltipItem( + rowId = 4242, + id = 4242, + category = "kotlin", + tag = "val", + summary = "Declares a read-only property.", + detail = "Longer detail.", + buttons = + arrayListOf( + "Learn more about properties" to "http://localhost:6174/k/html/properties.html", + "Examples" to "http://localhost:6174/k/html/basic-syntax.html", + ), + lastChange = "2026-10-18 (test)", + ), + item, + ) + } + + @Test + fun `reading a tag with no tooltip throws NoTooltipFoundException`() { + assertThrows(NoTooltipFoundException::class.java) { + readTooltip(db, category = "java", tag = "no-such-tag", lastChange = "test") + } + } + + @Test + fun `reading a tag two same-named categories hold throws DatabaseCorruptionException`() { + db.execSQL("INSERT INTO TooltipCategories VALUES (5, 'java')") + db.execSQL("INSERT INTO TooltipTexts VALUES (9999, 'duplicate', 'detail')") + db.execSQL("INSERT INTO Tooltips VALUES (9999, 5, 'tag7')") + + assertThrows(DatabaseCorruptionException::class.java) { + readTooltip(db, category = "java", tag = "tag7", lastChange = "test") + } + } + @Test fun `refuses a database that declares a version before 3`() { setVersion(2) From ae7b08f5e33614e9cac0c1fb5d61de7098b26d82 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 18 Oct 2026 23:52:11 -0700 Subject: [PATCH 9/9] ADFA-3305: Say what an update keeps and where version 2 still lives docs/documentation-database.md said the database is extracted on install and update; InstallationViewModel extracts it at first-time setup only, so an update keeps the installed file, and a version 2 one then shows no tooltips (accepted for ADFA-3305). It also said docdb-studio edits "this same documentation.db" and validates TooltipButtons.uri: the master it edits stays in version 2, and migrate_tooltip_schema.py converts a copy for shipping, so docdb-studio and the docs/docdb scripts use the version 2 columns. The debug-override notes now say a version 2 file shows no tooltips, and the ToolTipManager entry names readTooltip. idetooltips/README.md cited a Room IDETooltipDatabase that does not exist; the module reads documentation.db with raw SQLite. Review finding R2163-05. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018CMZQuXT3GT7nEct2w4tg3 --- docs/documentation-database.md | 12 ++++++------ idetooltips/README.md | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/documentation-database.md b/docs/documentation-database.md index 844c0fdb39..dab0804000 100644 --- a/docs/documentation-database.md +++ b/docs/documentation-database.md @@ -7,8 +7,8 @@ This is a **prebuilt** database — CoGo never creates or migrates its schema at ## Where it lives - Installed path: `context.getDatabasePath("documentation.db")` (`Environment.DOC_DB` in `common/.../utils/Environment.java`), i.e. the app's private `databases/` dir. -- Bundled as an asset and extracted on install/update by `BundledAssetsInstaller` / `SplitAssetsInstaller`. -- **Debug override:** if `/sdcard/Download/documentation.db` exists and is newer than the installed copy, `DocumentationContentSource` (behind both Tier 3 transports) and `ToolTipManager` swap to it at request time (timestamp-compared per request, not just at startup) — a fast way to test a new database on-device without reinstalling. **The comparison is on modification time, and `adb push` preserves the *source* file's mtime** -- so pushing a database saved earlier than the one already on the device silently does not swap, and the app keeps serving the old one with no error anywhere. Follow a push with `adb shell touch /sdcard/Download/documentation.db` (this cost real debugging time on ADFA-5153). `WebServer`'s debug logging and experiment flags are also file-flag-gated under `/sdcard/Download/` (`CodeOnTheGo.webserver.debug`, `CodeOnTheGo.exp`, `CodeOnTheGo.webserver.cs0`). +- Bundled as an asset and extracted by `BundledAssetsInstaller` / `SplitAssetsInstaller` at first-time setup only: `InstallationViewModel` runs the asset install while no JDK or Android SDK is installed, and returns early once both are. An app update keeps the installed file whatever the new APK bundles, so an install first set up with a version 2 database keeps it, and from documentation version 3 on (ADFA-3305) shows no tooltips; clearing the app's data is what gets it the bundled file. That is accepted for ADFA-3305. +- **Debug override:** if `/sdcard/Download/documentation.db` exists and is newer than the installed copy, `DocumentationContentSource` (behind both Tier 3 transports) and `ToolTipManager` swap to it at request time (timestamp-compared per request, not just at startup) — a fast way to test a new database on-device without reinstalling. **The comparison is on modification time, and `adb push` preserves the *source* file's mtime** -- so pushing a database saved earlier than the one already on the device silently does not swap, and the app keeps serving the old one with no error anywhere. Follow a push with `adb shell touch /sdcard/Download/documentation.db` (this cost real debugging time on ADFA-5153). **A version 2 file pushed there now shows no tooltips:** this build refuses tooltip tables below version 3, and the log line names the debug database and its path. Convert a copy with `OfflineDocumentationTools`' `migrate_tooltip_schema.py` before pushing it. `WebServer`'s debug logging and experiment flags are also file-flag-gated under `/sdcard/Download/` (`CodeOnTheGo.webserver.debug`, `CodeOnTheGo.exp`, `CodeOnTheGo.webserver.cs0`). - **Don't trust a local copy's on-disk schema or row content as ground truth without checking freshness first.** Any manually downloaded or debug-override copy is independent of git history — a stale one can have a different schema (e.g. missing `UNIQUE(path)` or `templateId`) or be missing rows that already exist in the current, maintained database. A stale copy caused a real near-miss in ADFA-5088: a SQL script validated against it would have silently overwritten curated production tooltip content for several tags. Diff or re-download before authoring SQL against a local copy's state, not just before shipping it. ## Schema @@ -95,20 +95,20 @@ Of the five sites below, only `DocumentationContentSource` and `ToolTipManager` - **`common/.../documentation/DocumentationContentSource.kt`** — the one Tier 3 pipeline that reads this database: row lookup (a request target is matched raw first, since stored `Content.path` values are percent-encoded, with a percent-decoded fallback on a miss — shared by both transports so they agree on which pages exist), chunked-row reassembly, dictionary-aware Brotli decode gated on the declared documentation version, and the database swaps — to a newer sdcard debug database, and reopening the installed file when an asset install rewrites it in place under the cached handle — under a read/write lock so several threads can read while a swap cannot close the handle under them. It also renders the rows that are Pebble template contexts (`templateId > 0`, the Kotlin doc set's pages) — resolving each template's references to other templates against `Templates.name` as it goes — so both transports below serve finished pages and neither needs the template engine itself. All of that logic exists once (ADFA-5176). - **`common/.../documentation/DocumentationRequestInterceptor.kt`** — serves Tier 3 *in-process* for the app's WebViews (`HelpActivity`, the tooltip fragment, `FAQActivity`), through `WebViewClient.shouldInterceptRequest`, so a page's assets cost a database read instead of a TCP connection each (ADFA-5176). It matches the same `http://localhost:6174/...` URL space, so the strings.xml entries, `ToolTipManager`'s link builder and the `DocumentationExtension` contract need no changes; anything it declines — a `/pr/` endpoint, an unknown path, a failed read — falls through to `WebServer` unchanged. Both transports get their `Content-Type` charset from the same place, `ContentTypeHeaders` (ADFA-5241), so a row does not describe itself differently depending on which one served it. Set `/sdcard/Download/CodeOnTheGo.nointercept` to force documentation back onto the server. - **`app/.../localWebServer/WebServer.kt`** — serves Tier 3 over HTTP on port 6174, for WebViews that are not wired to the interceptor above and for the `/pr/` developer endpoints. It reads through `DocumentationContentSource`, so it holds no database, template engine or decode logic of its own; what remains here is HTTP: request parsing, the `/pr/` pages, error responses, and the CSS/asset shortcuts. Also serves a Dynamic Bookshelf JSON payload (joining `Content`/`Bookshelf`/`BookCategories`, rendered through the `bookshelf` template) and debug-only HTML dumps at `/pr/db` (`LastChange`, last 20 rows) and `/pr/pr` (recent projects, from a *different* database). -- **`idetooltips/.../ToolTipManager.kt`** — serves Tier 1/2, through `queryTooltip` and `queryTooltipButtons` in `TooltipQuery.kt`. Looks up a tooltip by `(category, tag)`: the category's id from `TooltipCategories` first, so the row is found by `Tooltips`' key rather than a scan of the whole table (ADFA-6597), then its text by id. Then its buttons for the Tier 3 links shown at the bottom, by `TooltipButtons`' key, with their text by id. Refuses a database declaring a version below 3. +- **`idetooltips/.../ToolTipManager.kt`** — serves Tier 1/2, through `readTooltip` in `TooltipQuery.kt`, which runs `queryTooltip` and `queryTooltipButtons` and builds the item shown. Looks up a tooltip by `(category, tag)`: the category's id from `TooltipCategories` first, so the row is found by `Tooltips`' key rather than a scan of the whole table (ADFA-6597), then its text by id. Then its buttons for the Tier 3 links shown at the bottom, by `TooltipButtons`' key, with their text by id. Refuses a database declaring a version below 3. - **`plugin-manager/.../documentation/PluginDocumentationManager.kt`** (with `Tier3AssetWalker.kt`, and the `DocumentationExtension` contract in `plugin-api`) — lets plugins contribute their own help content into the same lookup paths, under the reserved `plugin//...` prefix. Its tooltip SQL is in `PluginTooltipTables.kt`: a plugin's tooltips are replaced or removed by category inside one transaction, each button's text is looked up in or added to the lookup tables, and text no button uses any more is deleted afterwards. Brotli assets are compressed against the same `CompressionDictionary` the rest of the table uses, read from the database being written. When that dictionary can't be read the install is abandoned rather than written plain, and retried on the plugin's next activation. ## Editing the database -Schema changes and data edits happen **outside this repo**, in `OfflineDocumentationTools/docdb-studio` (a Flet GUI over this same `documentation.db`). Conventions enforced there that matter if you're reasoning about data correctness here: +Schema changes and data edits happen **outside this repo**, in `OfflineDocumentationTools/docdb-studio` (a Flet GUI over the master `documentation.db`). The master stays in documentation version 2: `docdb-studio` and the other `OfflineDocumentationTools` scripts read and write the version 2 tooltip tables, and `migrate_tooltip_schema.py` converts a copy of the master to version 3 for shipping. Conventions enforced there that matter if you're reasoning about data correctness here: - The schema is locked — `docdb-studio`'s own `AGENTS.md` says never change it. If a new column/table is genuinely needed, it's a cross-repo change coordinated with that project, not something to route around in CoGo. -- Tooltip uniqueness is `(categoryId, tag)`; `TooltipButtons.uri` values are validated there against `Content.path` (post `?query`/`#fragment` stripping) before being allowed into the database. +- Tooltip uniqueness is `(categoryId, tag)`; button uris (`TooltipButtons.uri` in the version 2 master, `TooltipButtonUris.uri` once converted) are validated there against `Content.path` (post `?query`/`#fragment` stripping) before being allowed into the database. - Every edit made through the tool updates `LastChange` for the affected documentation set, which is how `DatabaseVersionResolver`'s debug logging can say what build of the docs is loaded. ### Writing one-off SQL scripts against this database -Some tickets (e.g. ADFA-5088) ship a one-off `.sql` script under `docs/docdb/` for a `docdb-studio` maintainer to run against the real database, rather than editing it directly through the tool. Gotchas found writing those scripts: +Some tickets (e.g. ADFA-5088) ship a one-off `.sql` script under `docs/docdb/` for a `docdb-studio` maintainer to run against the real database, rather than editing it directly through the tool. They run against the version 2 master, so they use its tooltip columns (`Tooltips.summary`, `TooltipButtons.uri`); never run one against a converted version 3 file. Gotchas found writing those scripts: - **The system SQLite has no JSON1 on some devices, and your desktop does.** `JSON_OBJECT`, `JSON_GROUP_ARRAY` and friends compile fine under the `sqlite3` CLI (3.44 ships JSON1 built in) and then fail at runtime with `no such function: JSON_OBJECT` on real hardware — reproduced on a Galaxy Note 20 Ultra, where the Dynamic Bookshelf was an HTTP 500 until the query was rewritten (ADFA-5179). Nothing on the desktop side of the fence will warn you. Write plain relational SQL and assemble the JSON in Kotlin, and treat any device-only 500 from a new query as this until proven otherwise. Applies to every reader of this database, not just `WebServer`: a `docdb-studio` script, a tooltip query in `ToolTipManager`, and `PluginDocumentationManager` are all equally exposed. - **Keep each `.system` line simple.** The sqlite3 CLI's `.system` dot-command can hit a content-dependent shell-parsing failure when a line chains multiple operators (`;`, `&&`, `||`, parentheses) — it reproduces for some input strings and not others, so it won't necessarily show up in a quick test. Stick to one plain `command | pipe > file` per `.system` line. diff --git a/idetooltips/README.md b/idetooltips/README.md index 5f9ac508ce..7f45aa5ad7 100644 --- a/idetooltips/README.md +++ b/idetooltips/README.md @@ -13,7 +13,7 @@ This module provides functionalities such as: * **Dynamic Tooltip Display:** Show tooltips anchored to specific UI elements or coordinates. * **Rich Content:** Supports HTML content within tooltips for flexible formatting, including links. -* **Database Integration:** Fetches tooltip content and metadata from the Room database - `IDETooltipDatabase`. +* **Database Integration:** Reads tooltip content read-only from `documentation.db` with raw SQLite (`TooltipQuery.kt`), the database described in [docs/documentation-database.md](../docs/documentation-database.md). * **"See More" Functionality:** Handles expanding content or navigating to detailed documentation. * **Custom callbacks:** Customizable action callbacks for link clicks.