Skip to content

Accept and preserve string or integer user and location IDs #429

Description

@paco-barreras

Intended identifier rules

user_id and location_id should default to pandas string when NOMAD must create a schema without dtype information. User-provided string and integer identifiers, including nullable Int64, should both be accepted and preserved.

Required changes

  • Update nomad.io.base._is_traj_df and nomad.io.base._is_stop_df to accept string or integer user_id values.
  • Apply the same validation rule to location_id when that column is present. _is_stop_df currently does not validate location_id.
  • Update nomad.io.base._cast_traj_cols so integer user_id values are not converted to strings. Apply the same preservation rule to location_id if it is cast during ingestion.
  • Keep SCHEMA_DTYPES["user_id"] and SCHEMA_DTYPES["location_id"] as string; these remain the defaults, not the only accepted input types.
  • Update nomad.stop_detection.utils._get_empty_stop_df: preserve a typed empty identifier column such as pd.Series(dtype="Int64"); use the canonical string dtype when the identifier column is absent or is an untyped empty object column.

Synthetic identifiers

nomad.city_gen.City.blocks_gdf uses (coord_x, coord_y) tuples as an internal MultiIndex. nomad.visit_attribution.visit_attribution.poi_map falls back to poi_table.index when no usable location_id column is supplied. Passing a synthetic block table through that fallback can therefore expose coordinate tuples as location_id values.

Keep block-coordinate tuples as internal spatial keys, but do not emit them as public identifiers. Synthetic trajectory locations should use the scalar IDs from City.buildings_gdf["id"]. If block rows need public location IDs, create a stable scalar string or integer column before calling poi_map rather than using the tuple index.

Tests

  • Cover string, NumPy integer, and nullable Int64 identifiers in from_df, _is_traj_df, and _is_stop_df.
  • Check that integer identifiers survive ingestion without conversion to strings.
  • Check empty and populated stop summaries for both string and integer identifier dtypes.
  • Check that imported stop tables still do not require a cluster column.
  • Check that synthetic attribution does not return tuple-valued location_id values.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions