diff --git a/KDOC_GUIDELINES.md b/KDOC_GUIDELINES.md index 08ffb5ba97..c48b1922e3 100644 --- a/KDOC_GUIDELINES.md +++ b/KDOC_GUIDELINES.md @@ -405,6 +405,17 @@ and then use add it using `@include`: * For more information: {@include [DocumentationUrls.Move]} ``` +Keep it to **exactly one** such paragraph per KDoc, always the colon form — never +`For more information,` followed by the link. If a shared KDoc-snippet you `@include` already carries the +paragraph, don't add a second one in the declaration: the rendered KDoc then says it twice. When an +operation needs several links, they go on that one line, space-separated: + +``` + * For more information: {@include [DocumentationUrls.Count]} {@include [DocumentationUrls.DataRow.RowFunctions]} +``` + +(see `count.kt`, `concat.kt`, `distinct.kt`). + #### Columns selection information For any method with columns selection, add a section with information about the columns selection. diff --git a/MODULES.md b/MODULES.md new file mode 100644 index 0000000000..ab23635244 --- /dev/null +++ b/MODULES.md @@ -0,0 +1,198 @@ +# Kotlin DataFrame Modules + +An overview of every module in the repository: what it publishes and what it depends on. Module-internal +structure, patterns and gotchas live in each module's own `README.md` and `AGENTS.md` — this file does not +repeat them. +The built set is defined by `settings.gradle.kts`; anything not listed there is not compiled or tested. +All published modules use the group `org.jetbrains.kotlinx`. + +## Aggregate Artifact + +### dataframe (root project) +**Description**: What a user normally depends on — `core` plus the I/O modules, re-exported +- **Location**: `./` +- **Artifact**: `dataframe` +- **Dependencies**: `api` on core, dataframe-arrow, dataframe-csv, dataframe-excel, dataframe-jdbc, dataframe-json +- **Note**: `dataframe-openapi` is deliberately left out of the aggregate — still experimental. + +## Core Module + +### core +**Description**: DataFrame/DataColumn/DataRow types, the operations API, aggregations, math, schema handling, codegen +- **Location**: `./core/` +- **Artifact**: `dataframe-core` +- **Dependencies**: none (base module) +- **Note**: I/O was split off into the modules below; core still holds the deprecated csv/tsv and HTML integrations. + +## I/O and Data Format Modules + +Each depends on `core` and publishes under its own directory name. + +### dataframe-arrow +**Description**: Apache Arrow IPC read/write — the streaming format (`readArrowIPC`/`writeArrowIPC`) and the +random-access/Feather format (`readArrowFeather`/`writeArrowFeather`) — plus Parquet reading via `readParquet` +- **Location**: `./dataframe-arrow/` +- **Dependencies**: core, Apache Arrow (vector/format/memory/dataset), commons-compress + +### dataframe-csv +**Description**: CSV/TSV reading and writing (supersedes the deprecated implementation in core) +- **Location**: `./dataframe-csv/` +- **Dependencies**: core, dataframe-json, Deephaven CSV, commons-csv + +### dataframe-excel +**Description**: Excel read/write, `.xlsx` and legacy `.xls` +- **Location**: `./dataframe-excel/` +- **Dependencies**: core, dataframe-json, Apache POI + +### dataframe-json +**Description**: JSON reading, writing and schema inference +- **Location**: `./dataframe-json/` +- **Dependencies**: core, kotlinx.serialization + +### dataframe-jdbc +**Description**: Reading from SQL databases over JDBC +- **Location**: `./dataframe-jdbc/` +- **Dependencies**: core; JDBC drivers are `compileOnly` or test-only, never shipped +- **Note**: built-in `DbType` support for H2, MariaDB, MySQL, MS SQL Server, PostgreSQL, SQLite and DuckDB; any + other JDBC database works through a user-supplied `DbType`. See `dataframe-jdbc/AGENTS.md`. + +## Integration Modules + +### dataframe-jupyter +**Description**: Kotlin Jupyter integration — rendering and kernel setup +- **Location**: `./dataframe-jupyter/` +- **Dependencies**: the root `dataframe` aggregate (not `core` alone), Log4j + +### dataframe-geo +**Description**: Geospatial column types and operations. **Experimental** +- **Location**: `./dataframe-geo/` +- **Dependencies**: core, GeoTools, JTS, Ktor client +- **Note**: targets Java 11, a restriction inherited from `org.jetbrains.kotlin.jupyter`. + +### dataframe-geo-jupyter +**Description**: Jupyter rendering for geospatial data +- **Location**: `./dataframe-geo-jupyter/` +- **Dependencies**: dataframe-geo, dataframe-jupyter + +### dataframe-openapi +**Description**: Runtime support for data schemas generated from OpenAPI 3.0.0 specifications. **Experimental** +- **Location**: `./dataframe-openapi/` +- **Dependencies**: core + +### dataframe-openapi-generator +**Description**: Generates data schemas from OpenAPI 3.0.0 specifications; the sister module of +`dataframe-openapi`, consumed by the Gradle and Jupyter integrations. **Experimental** +- **Location**: `./dataframe-openapi-generator/` +- **Dependencies**: core, dataframe-openapi, Swagger parser, KotlinPoet + +## Compiler and Build Plugins + +### dataframe-compiler-plugin-core +**Description**: A shaded subset of `:core`, bundled *inside* the Kotlin DataFrame compiler plugin and IntelliJ to +run compile-time interpreters of operations. Not itself a compiler plugin. +- **Location**: `./dataframe-compiler-plugin-core/` +- **Artifact**: `dataframe-compiler-plugin-core` +- **Dependencies**: core, with most transitive dependencies excluded +- **Note**: the **compiler plugin itself is not in this repository** — it is developed in the Kotlin repository at + `github.com/JetBrains/kotlin/tree/master/plugins/kotlin-dataframe`. See `dataframe-compiler-plugin-core/AGENTS.md`. + +### plugins/expressions-converter +**Description**: Kotlin compiler plugin that extracts the intermediate DataFrame expressions out of +`@TransformDataFrameExpressions`-annotated functions, so the website's "explainer dataframe" iframes can be +generated from real sample runs +- **Location**: `./plugins/expressions-converter/` +- **Used by**: `:core`, on the `kotlinCompilerPluginClasspathSamples` configuration + +### plugins/public-api-modifier +**Description**: Exploratory compiler plugin that turns `@AccessApiOverload`-annotated functions internal, to +allow a full-API and a reduced-API artifact from one source tree (PR #959) +- **Location**: `./plugins/public-api-modifier/` +- **Note**: built, but **not enabled anywhere** — no module puts it on a compiler-plugin classpath. + +### Present in the tree but not built +None of these is in `settings.gradle.kts`, so nothing here is compiled or tested — but they are *not* all the +same kind of dead: two are deprecated products, one moved upstream, one moved into the build. Each has its own +`README.md`; read it and `plugins/AGENTS.md` before touching anything. + +**`plugins/dataframe-gradle-plugin`** — the Gradle plugin, published as `org.jetbrains.kotlinx.dataframe` on the +Gradle Plugin Portal. Generated data schemas from a data sample through the `dataframes {}` task, and pulled in +the symbol processor automatically to produce column accessors. + +**`plugins/symbol-processor`** — the KSP plugin, published as `symbol-processor-all`. Generated schemas from +`@file:ImportDataSchema`, and the extension properties for `@DataSchema` classes and interfaces. + +Both are **deprecated as products**, not merely switched off in this build: their last release is +`1.0.0-Beta4` and there will be no further ones, because KSP1 is no longer compatible with Kotlin 2.3+. Schema +generation now goes through dedicated runtime methods and the compiler plugin — see +`docs/StardustDocs/topics/schemas/Migration-From-Plugins.md`, +`docs/StardustDocs/topics/schemas/DataSchemaGenerationMethods.md` and +`docs/StardustDocs/topics/schemas/gradle/Gradle-Plugin.md`; the next iteration of the feature is issue #1844. + +**`plugins/kotlin-dataframe`** — the Kotlin 2.x compiler plugin, which is the **recommended** way to get +compile-time extension properties. It is not deprecated: development simply moved to the Kotlin repository +(`github.com/JetBrains/kotlin/tree/master/plugins/kotlin-dataframe`), and the copy here is an out-of-date +snapshot with its tests turned off (issue #1290). See `docs/StardustDocs/topics/compiler-plugin.md`. + +**`plugins/keywords-generator`** — not in the repository at all (`git ls-files` returns nothing); it is now the +`dfbuild.keywordsGenerator` convention plugin in `build-logic`. An untracked `build/` directory may still sit in +your checkout. + +## Development and Support Modules + +### samples +**Description**: Runnable documentation samples; korro executes them and injects code and output into the website +- **Location**: `./samples/` +- **Dependencies**: the root `dataframe` aggregate (`runtimeOnly`, so the compiler plugin sees real jars) +- **Note**: not published. See `samples/AGENTS.md`. + +### common-test-utils +**Description**: Shared test helpers and assertions +- **Location**: `./common-test-utils/` +- **Dependencies**: core, Kotest assertions +- **Note**: not published; added as `testImplementation` to every JVM module by the `dfbuild.kotlinJvmCommon` + convention plugin. + +### build-logic, build-settings-logic +**Description**: The build itself — convention plugins (`dfbuild.*`, `dfsettings.*`) instead of inline build logic +- **Location**: `./build-logic/`, `./build-settings-logic/` +- **Note**: included with `includeBuild`, not as regular modules. + +## Examples and Documentation + +- `examples/projects/` — standalone Gradle/Maven example projects against the latest stable release, with + `examples/projects/dev/` holding the same set built against master. +- `examples/notebooks/` — Kotlin/Jupyter analytics notebooks. +- `docs/StardustDocs/` — the documentation website (WriterSide), with generated snippets and iframes. + +Each has its own `AGENTS.md` (`examples/AGENTS.md`, `docs/StardustDocs/AGENTS.md`) with the authoritative +layout and build instructions. + +## Module Dependencies Overview + +``` +dataframe (root aggregate) +├── core — base module, no module dependencies +├── dataframe-arrow — core +├── dataframe-csv — core, dataframe-json +├── dataframe-excel — core, dataframe-json +├── dataframe-jdbc — core +└── dataframe-json — core + +outside the aggregate: +dataframe-jupyter — dataframe (the whole aggregate) +dataframe-geo — core +dataframe-geo-jupyter — dataframe-geo, dataframe-jupyter +dataframe-openapi — core +dataframe-openapi-generator — core, dataframe-openapi +dataframe-compiler-plugin-core — core (shaded subset) +samples — dataframe (runtimeOnly) +common-test-utils — core (test-only, applied to every JVM module) +``` + +## Build Configuration + +- **Root project**: `dataframe`; group `org.jetbrains.kotlinx` +- **Build tool**: Gradle with the Kotlin DSL; the daemon JDK comes from `gradle/gradle-daemon-jvm.properties` +- **Target platform**: JVM (the Android example builds against the published artifacts) +- **Documentation**: Dokka for API docs, KoDEx for KDoc preprocessing, korro for the website samples +- **Public API**: guarded by binary-compatibility-validator in every publishable module (`api/*.api` dumps) diff --git a/core/README.md b/core/README.md index 744af1ce2f..503014cb85 100644 --- a/core/README.md +++ b/core/README.md @@ -5,6 +5,9 @@ It contains the DataFrame API and its implementation, as well as plenty of JUnit I/O operations are split off into other modules, like [:dataframe-excel](../dataframe-excel), [:dataframe-jdbc](../dataframe-jdbc), or [:dataframe-json](../dataframe-json). +See the [documentation website](https://kotlin.github.io/dataframe/) for how this API is meant to be used, and +[Operations](https://kotlin.github.io/dataframe/operations.html) for the reference of everything in it. + At the moment, these integrations are still part of the `:core` module: - (deprecated) csv/tsv diff --git a/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/api/indices.kt b/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/api/indices.kt index 0667eb35f4..59ecdc9b14 100644 --- a/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/api/indices.kt +++ b/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/api/indices.kt @@ -1,13 +1,68 @@ package org.jetbrains.kotlinx.dataframe.api import org.jetbrains.kotlinx.dataframe.DataFrame +import org.jetbrains.kotlinx.dataframe.DataRow import org.jetbrains.kotlinx.dataframe.RowFilter +import org.jetbrains.kotlinx.dataframe.documentation.AccessApis +import org.jetbrains.kotlinx.dataframe.documentation.DocumentationUrls +import org.jetbrains.kotlinx.dataframe.documentation.ExcludeFromSources import org.jetbrains.kotlinx.dataframe.indices // region DataFrame +/** + * {@comment + * "See also" list shared by both `indices` overloads. + * Include it into KDoc with `@include [CommonIndicesDocs]`. + * } + * + * See also: + * - [rowsCount][DataFrame.rowsCount] — returns the number of rows in this [DataFrame]. + * - [getRows][DataFrame.getRows] — returns the rows at the given indices, as a new [DataFrame]. + * - [filter][DataFrame.filter] — returns the rows that satisfy a condition, instead of their indices. + */ +@ExcludeFromSources +private typealias CommonIndicesDocs = Nothing + +/** + * Returns the indices of all rows in this [DataFrame], + * from `0` to the number of rows minus one, + * or an empty range if this [DataFrame] has no rows. + * + * For more information: {@include [DocumentationUrls.Indices]} + * + * @include [CommonIndicesDocs] + * + * @return An [IntRange] with the index of every row in this [DataFrame]. + */ public fun DataFrame<*>.indices(): IntRange = 0 until rowsCount() +/** + * Returns the indices of the rows in this [DataFrame] that satisfy the given [filter]. + * + * They are listed in the same order as the rows in this [DataFrame]. + * + * The [filter] is a [RowFilter] — a lambda that receives each [DataRow] as both `this` and `it` + * and is expected to return a [Boolean] value. + * + * It allows you to define conditions using the row's values directly, + * including through [extension properties][AccessApis.ExtensionPropertiesApi] + * for convenient and type-safe access. + * + * For more information: {@include [DocumentationUrls.Indices]} {@include [DocumentationUrls.DataRow.RowFilter]} + * + * @include [CommonIndicesDocs] + * + * ### Example + * ```kotlin + * // The indices of the rows where the "city" column is "Moscow" + * df.indices { city == "Moscow" } + * ``` + * + * @param [filter] A [RowFilter] that returns `true` for the rows whose indices should be returned. + * @return A [List] with the index of every row that satisfies the [filter], + * empty if no row satisfies it. + */ public inline fun DataFrame.indices(filter: RowFilter): List = indices().filter { val row = get(it) diff --git a/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/documentation/DocumentationUrls.kt b/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/documentation/DocumentationUrls.kt index b02292f137..05bd3a73e2 100644 --- a/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/documentation/DocumentationUrls.kt +++ b/core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/documentation/DocumentationUrls.kt @@ -502,6 +502,9 @@ public interface DocumentationUrls { /** [See `sortWith` on the documentation website.]({@include [Url]}/sortby.html#sortwith) */ public typealias SortWith = Nothing + /** [See `indices` on the documentation website.]({@include [Url]}/indexing.html#row-indices) */ + public typealias Indices = Nothing + /** [See `JsonPath` on the documentation website.]({@include [Url]}/read.html#specify-key-value-paths) */ public typealias JsonPath = Nothing diff --git a/core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/api/indices.kt b/core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/api/indices.kt new file mode 100644 index 0000000000..548e4aacb3 --- /dev/null +++ b/core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/api/indices.kt @@ -0,0 +1,125 @@ +package org.jetbrains.kotlinx.dataframe.api + +import io.kotest.matchers.shouldBe +import org.jetbrains.kotlinx.dataframe.AnyBaseCol +import org.jetbrains.kotlinx.dataframe.DataFrame +import org.jetbrains.kotlinx.dataframe.RowFilter +import org.jetbrains.kotlinx.dataframe.columns.ColumnGroup +import org.jetbrains.kotlinx.dataframe.columns.FrameColumn +import org.jetbrains.kotlinx.dataframe.indices +import org.jetbrains.kotlinx.dataframe.samples.api.TestBase +import org.jetbrains.kotlinx.dataframe.samples.api.age +import org.jetbrains.kotlinx.dataframe.samples.api.city +import org.jetbrains.kotlinx.dataframe.samples.api.firstName +import org.jetbrains.kotlinx.dataframe.samples.api.name +import org.junit.Test + +/** + * Tests the behavior of the `indices` API: + * + * - on a [DataFrame] without arguments: the index of every row, including a [DataFrame] without rows. + * + * - on a [DataFrame] with a [RowFilter]: the indices of the matching rows, their order, + * and the cases where nothing matches. + * + * - on a column, as the `indices()` function and the `indices` property: the index of every + * element, and what an element is for a [ColumnGroup] and for a [FrameColumn]. + */ +class IndicesTests : TestBase() { + + // `df` comes from `TestBase` and has seven rows; + // "Moscow" is the city in rows 2 and 6, "Charlie" the first name in rows 2, 3 and 6. + + // region on a DataFrame + + @Test + fun `indices returns the index of every row`() { + // the type annotation asserts the return type, which is part of the contract + val indices: IntRange = df.indices() + indices shouldBe 0..6 + } + + @Test + fun `indices of a dataframe without rows is empty`() { + // asserted as a range, like the non-empty case, so the return type stays part of the contract + df.take(0).indices() shouldBe IntRange.EMPTY + } + + @Test + fun `KDoc example -- indices with a filter returns the indices of the matching rows`() { + // the type annotation asserts the return type, which is part of the contract + val moscow: List = df.indices { city == "Moscow" } + moscow shouldBe listOf(2, 6) + } + + @Test + fun `indices with a filter written through the string API`() { + df.indices { "city"() == "Moscow" } shouldBe listOf(2, 6) + } + + @Test + fun `indices with a filter lists them in row order`() { + df.indices { name.firstName == "Charlie" } shouldBe listOf(2, 3, 6) + } + + @Test + fun `the filter receives the row as both this and it`() { + // `age` alone reads the value from `this`, `it.age` from the lambda argument + df.indices { age > 18 } shouldBe listOf(1, 2, 3, 4, 5, 6) + df.indices { it.age > 18 } shouldBe listOf(1, 2, 3, 4, 5, 6) + } + + @Test + fun `indices with a filter that no row satisfies is empty`() { + df.indices { city == "Paris" } shouldBe emptyList() + } + + @Test + fun `indices with a filter on a dataframe without rows is empty`() { + df.take(0).indices { city == "Moscow" } shouldBe emptyList() + } + + // endregion + + // region on a column + + // The column `indices` is an accessor on `DataColumn`/`BaseColumn`, not the `DataFrame` + // operation, and it carries no KDoc yet; these tests pin its behaviour down regardless. + + @Test + fun `indices of a column is the index of every element`() { + // the type annotation asserts the return type, which is part of the contract + val indices: IntRange = df.age.indices() + indices shouldBe 0..6 + } + + @Test + fun `the indices property gives the same range as the indices function`() { + // both are asserted against the expected value, not against each other + df.age.indices shouldBe 0..6 + df.age.indices() shouldBe 0..6 + } + + @Test + fun `indices of a column without elements is empty`() { + df.take(0).age.indices() shouldBe IntRange.EMPTY + } + + @Test + fun `indices of a column group are the row indices of the dataframe it forms`() { + // typed as a BaseColumn, so the receiver leaves only the column overload applicable + val group: AnyBaseCol = df.name + group.indices() shouldBe 0..6 + } + + @Test + fun `indices of a frame column has one index per cell`() { + // three first names, so three cells holding 2, 2 and 3 of the seven rows + val frames: FrameColumn<*> = df.groupBy { name.firstName }.into("frames")["frames"].asFrameColumn() + frames.indices() shouldBe 0..2 + // the seven rows are still there, spread over the three cells + frames.toList().sumOf { it.rowsCount() } shouldBe 7 + } + + // endregion +} diff --git a/core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api/Access.kt b/core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api/Access.kt index 51b14b642c..75ec7d49a6 100644 --- a/core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api/Access.kt +++ b/core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api/Access.kt @@ -160,24 +160,6 @@ class Access : TestBase() { // SampleEnd } - @Test - @TransformDataFrameExpressions - fun getCell_strings() { - // SampleStart - df["age"][1] - df[1]["age"] - // SampleEnd - } - - @Test - @TransformDataFrameExpressions - fun getCell_properties() { - // SampleStart - df.age[1] - df[1].age - // SampleEnd - } - @Test @TransformDataFrameExpressions fun getColumnsByName() { diff --git a/dataframe-compiler-plugin-core/README.md b/dataframe-compiler-plugin-core/README.md index 0450bfe02d..7908215b1f 100644 --- a/dataframe-compiler-plugin-core/README.md +++ b/dataframe-compiler-plugin-core/README.md @@ -2,3 +2,6 @@ Subset of :core used by [compiler plugin](https://github.com/JetBrains/kotlin/tree/master/plugins/kotlin-dataframe) to implement compile time interpreters of operations. Bundled together with compiler plugin in Kotlin and by extension in IntelliJ. Should aim to include only necessary code + +See [Compiler Plugin](https://kotlin.github.io/dataframe/compiler-plugin.html) for what the plugin this module is +bundled into does for a user. diff --git a/dataframe-csv/README.md b/dataframe-csv/README.md index 249bf559c3..6ef79c3124 100644 --- a/dataframe-csv/README.md +++ b/dataframe-csv/README.md @@ -3,5 +3,14 @@ This module, published as `dataframe-csv`, contains all logic and tests for DataFrame to be able to work with `csv` files. -At the moment, this module is in the experimental stage, so it's not included when -you add the `dataframe` dependency to your project. +This is the modern CSV/TSV integration, available since DataFrame v0.15 and based on +[Deephaven CSV](https://github.com/deephaven/deephaven-csv) for reading and +[Apache Commons CSV](https://commons.apache.org/proper/commons-csv/) for writing. The older csv/tsv code left in +[:core](../core) is deprecated in favor of it. + +The `dataframe` artifact re-exports this module, so `org.jetbrains.kotlinx:dataframe` already includes it. Depend +on `org.jetbrains.kotlinx:dataframe-csv` explicitly only if you use `dataframe-core` on its own. + +See [Read from CSV](https://kotlin.github.io/dataframe/read.html#read-from-csv) and +[Writing to CSV](https://kotlin.github.io/dataframe/write.html#writing-to-csv) +for more information about how to use it. diff --git a/docs/README.md b/docs/README.md index 3471d1bebf..c8557168a6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -59,6 +59,6 @@ Elements annotated with `@ExportAsHtml` will have their generated content be cop ### Korro code samples Code samples for the documentation website reside in [core/.../test/.../samples/api](../core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api) (for samples that depend solely on `dataframe-core`), -and [tests/.../samples/api](../tests/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api) (for samples can depend on other I/O modules) +and [samples/.../samples/api](../samples/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api) (for samples that may depend on other I/O modules) and they are copied over to Markdown files in [docs/StardustDocs/topics](./StardustDocs/topics) by [Korro](https://github.com/devcrocod/korro). diff --git a/docs/StardustDocs/AGENTS.md b/docs/StardustDocs/AGENTS.md index 4a4f9a164d..89dd9b56d1 100644 --- a/docs/StardustDocs/AGENTS.md +++ b/docs/StardustDocs/AGENTS.md @@ -36,3 +36,18 @@ Project marker: `project.ihp`; instance profile / table-of-contents: `d.tree` - `_shadow_resources.md` is regenerated by the `updateShadowResources` task in `:samples`. - The site itself is built/previewed by WriterSide (IDE plugin or CI builder) from `project.ihp` + `d.tree` + `cfg/build-script.xml`; Gradle only produces the injected snippets/iframes. + +## Gotchas + +- **One topic, one module.** A page importing samples from both `:core` and `:samples` builds, but don't: + it works against the #898 migration and blocks iframes, since `DataFrameSampleHelper` lives only in + `:samples`. Adding a sample to a `:core`-owned topic means moving that topic's samples over first. +- **A new iframe needs `:samples:updateShadowResources`** on top of the generated HTML — without the + `` entry in `_shadow_resources.md` the page silently shows no table. +- **An iframe `src` is the flat basename, never the on-disk path.** `updateShadowResources` registers + resources by basename only (422 `` entries, not one with a slash), so a nested + source file is still referenced as `./resources/.html`: `sliceRows.md:14` says + `./resources/sliceRowsDf.html` for a file at `resources/api/sliceRows/sliceRowsDf.html`. A nested + `src` builds fine and silently renders an empty frame. +- After a generator run `git status` over-reports on Windows (`core.autocrlf=true`, no `.gitattributes`) + — check the real set with `git diff --ignore-cr-at-eol`. diff --git a/docs/StardustDocs/resources/api/indexing/indexingDf.html b/docs/StardustDocs/resources/api/indexing/indexingDf.html new file mode 100644 index 0000000000..bf549bf28c --- /dev/null +++ b/docs/StardustDocs/resources/api/indexing/indexingDf.html @@ -0,0 +1,516 @@ + + + + + +
+ +

+ + + diff --git a/docs/StardustDocs/resources/api/indexing/indicesGetRows.html b/docs/StardustDocs/resources/api/indexing/indicesGetRows.html new file mode 100644 index 0000000000..cb4b0d2229 --- /dev/null +++ b/docs/StardustDocs/resources/api/indexing/indicesGetRows.html @@ -0,0 +1,516 @@ + + + + + +
+ +

+ + + diff --git a/docs/StardustDocs/topics/_shadow_resources.md b/docs/StardustDocs/topics/_shadow_resources.md index f4eedb68c4..568f46883f 100644 --- a/docs/StardustDocs/topics/_shadow_resources.md +++ b/docs/StardustDocs/topics/_shadow_resources.md @@ -154,6 +154,8 @@ + + diff --git a/docs/StardustDocs/topics/access.md b/docs/StardustDocs/topics/access.md index e8da516b6c..6c04aebb01 100644 --- a/docs/StardustDocs/topics/access.md +++ b/docs/StardustDocs/topics/access.md @@ -16,7 +16,8 @@ df.values() // Sequence **Learn how to:** -* [Access data by index](indexing.md) +* [Access a cell by index](indexing.md#access-a-cell-by-index) +* [Get row indices](indexing.md#row-indices) * [Iterate over data](iterate.md) * [Get a single row](getRow.md) * [Get single column](getColumns.md) diff --git a/docs/StardustDocs/topics/indexing.md b/docs/StardustDocs/topics/indexing.md index 4dd7dca1d5..4434d7d605 100644 --- a/docs/StardustDocs/topics/indexing.md +++ b/docs/StardustDocs/topics/indexing.md @@ -1,6 +1,19 @@ [//]: # (title: Indexing) - + + +The examples on this page use the following [`DataFrame`](DataFrame.md): + + + +```kotlin +df +``` + + + + +## Access a cell by index @@ -21,3 +34,51 @@ df[1]["age"] + +## Row indices + +`indices()` returns the indices of all rows of the [`DataFrame`](DataFrame.md) as an `IntRange` — +from `0` to the number of rows minus one: + + + +```kotlin +df.indices() // 0..9 +``` + + + +Pass a [row condition](DataRow.md#row-conditions) to get only the indices of the rows that satisfy it, +as a `List` in row order: + + + + + +```kotlin +val moscow = df.indices { city == "Moscow" } // [2, 6] +``` + + + + +```kotlin +val moscow = df.indices { "city"() == "Moscow" } // [2, 6] +``` + + + + +Use the result to [slice those rows](sliceRows.md) out of the dataframe: + + + +```kotlin +df.getRows(df.indices { city == "Moscow" }) +``` + + + + +See also [`rowsCount`](rowsCount.md), which returns how many rows there are rather than their indices, +and [`filter`](filter.md), which returns the matching rows themselves. diff --git a/samples/README.md b/samples/README.md index 126bb20ac2..826f187aa5 100644 --- a/samples/README.md +++ b/samples/README.md @@ -1,7 +1,8 @@ ## :samples Code samples, as well as DataFrame iframes and Kandy plot images, for the -[documentation website](https://github.com/Kotlin/dataframe). +[documentation website](https://kotlin.github.io/dataframe/) +(sources in [docs/StardustDocs](../docs/StardustDocs)). ### Korro diff --git a/samples/build.gradle.kts b/samples/build.gradle.kts index 6471b71dd0..f9cda28ee1 100644 --- a/samples/build.gradle.kts +++ b/samples/build.gradle.kts @@ -126,6 +126,7 @@ korro { include("count.md") include("valueCounts.md") include("sliceRows.md") + include("indexing.md") }, ) baseDir = rootProject.file("docs/StardustDocs/topics") diff --git a/samples/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api/IndexingSamples.kt b/samples/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api/IndexingSamples.kt new file mode 100644 index 0000000000..6563d87787 --- /dev/null +++ b/samples/src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api/IndexingSamples.kt @@ -0,0 +1,70 @@ +package org.jetbrains.kotlinx.dataframe.samples.api + +import org.jetbrains.kotlinx.dataframe.api.getRows +import org.jetbrains.kotlinx.dataframe.api.indices +import org.jetbrains.kotlinx.dataframe.samples.DataFrameSampleHelper +import org.junit.Test + +class IndexingSamples : DataFrameSampleHelper("indexing", "api") { + val df = peopleDf + + @Test + fun getCell_properties() { + // SampleStart + df.age[1] + df[1].age + // SampleEnd + df.age[1] willBe 45 + df[1].age willBe 45 + } + + @Test + fun getCell_strings() { + // SampleStart + df["age"][1] + df[1]["age"] + // SampleEnd + df["age"][1] willBe 45 + df[1]["age"] willBe 45 + } + + @Test + fun indexingDf() { + // SampleStart + df + // SampleEnd + .saveDfHtmlSample() + } + + @Test + fun indices() { + // SampleStart + df.indices() // 0..9 + // SampleEnd + df.indices() willBe 0..9 + } + + @Test + fun indicesCondition_properties() { + // SampleStart + val moscow = df.indices { city == "Moscow" } // [2, 6] + // SampleEnd + moscow willBe listOf(2, 6) + } + + @Test + fun indicesCondition_strings() { + // SampleStart + val moscow = df.indices { "city"() == "Moscow" } // [2, 6] + // SampleEnd + moscow willBe listOf(2, 6) + } + + @Test + fun indicesGetRows() { + // SampleStart + df.getRows(df.indices { city == "Moscow" }) + // SampleEnd + .saveDfHtmlSample() + } +}