Repository navigation

Feature/ndit 821 address ddb dc performance issues - #35

Closed

stevenhsd wants to merge 22 commits into

develop_v06from

feature/ndit-821_address_ddb_dc_performance_issues

Closed

Feature/ndit 821 address ddb dc performance issues#35
stevenhsd wants to merge 22 commits into
develop_v06from
feature/ndit-821_address_ddb_dc_performance_issues

Commits on Jan 26, 2026
- feat: Change how error messages are generated (by writing in batches). Duckdb no long relies on pandas- use of pyarrow, multiprocessing and background thread batch writing to avoid memory pressure
  stevenhsd
  committed
  efe0fa0
Commits on Jan 27, 2026
- style: run black, isort and resolve linting issues
  stevenhsd
  committed
  f1fd8aa
Commits on Jan 28, 2026
Commits on Jan 30, 2026
- build: add pytest dev settings with live log
  georgeRobertson
  committed
  4bbdf02
- refactor: improve the logging around dve processing errors and align reporting to module name rather than legacy name
  georgeRobertson
  committed
  cdd5fc8
Commits on Feb 2, 2026
Commits on Feb 3, 2026
- refactor: merge in main and resolve conflicts and linting issues
  stevenhsd
  committed
  141e1b3
Commits on Feb 6, 2026
- style: Additional logging around file transformation, business rules and error reports (#36)
  stevenhsd
  authored
  aaea473
- refactor: merging logging additions from release branch
  stevenhsd
  committed
  fc120b1