[Josiah Vinson (jovinson-ms)]

This PR combines three azure-health-deidentification fixes, plus a temporary shim for a regenerated-runtime regression. It supersedes #49625 and #49626.

1. Fix CI: stale generated code and regeneration

Problem

The python - healthdataaiservices - ci pipeline has failed on every run since July:

• pyright reportArgumentType: issubclass receives type | None in _serialization.py:793 and _utils/serialization.py:793.
• pylint C0123 unidiomatic-typecheck in _model_base.py and _utils/model_base.py, at lines 821, 824 and 830.

_serialization.py, _model_base.py, _vendor.py and aio/_vendor.py are leftovers from an older emitter. They only reference each other. The _utils/ copies were also out of date.

Fix

• Delete the four stale files.
• Regenerate from TypeSpec with the repo-pinned @azure-tools/typespec-python 0.63.10. The current emitter output passes pyright and pylint without any suppressions.

Regeneration side effects (please review)

• tsp-location.yaml: the spec moved to specification/healthdataaiservices/data-plane/HealthDataAIServices. It now points at 6de384008f55f42ea97ee0654f237c74fb05ee89, the merge commit of [HealthDataAIServices] Disable Python TypedDict generation for DeID data-plane SDK azure-rest-api-specs#47496 (see the TypedDict item below). Within this spec directory, the only change since the previously pinned c7d9e3379dd76fe55ce476f7aa312a4fb938615c is the generate-typeddict option in tspconfig.yaml. The old pinned commit no longer compiles with the current typespec-azure-core (@useDependency(Azure.Core.Versions.v1_0_Preview_2)). I diffed the TypeSpec, and the API surface is unchanged. The only differences are lint #suppress lines, removal of the Azure.Core.Versions dependency, main.tsp importing client.tsp, and tspconfig output paths.
• setup.py → pyproject.toml: the emitter now generates PEP 621 packaging.
• Minimum Python is now 3.10 (was 3.9), and the minimum azure-core is now 1.37.0 (was 1.30.0). README is updated.
• TypedDict generation is opted out (generate-typeddict: false, the emitter option added in [python] Add DPG TypedDict generation opt-out microsoft/typespec#11372). The emitter's request-body TypedDicts marked read-only fields as required (see [python]: typeddict generating non-input types microsoft/typespec#11562). With the opt-out, no types.py is generated and the JSON overloads match 1.1.0b1. The option is set in the spec's tspconfig.yaml by [HealthDataAIServices] Disable Python TypedDict generation for DeID data-plane SDK azure-rest-api-specs#47496, which is merged. tsp-location.yaml points at its merge commit, and regenerating from it produces no changes to the generated code.
• DELETE requests no longer send Accept: application/json: delete_job returns 204 with no body.
• Test matcher: tests/conftest.py adds set_custom_default_matcher(excluded_headers="Accept") so the existing recordings, which include Accept on DELETE, still match. The alternative is re-recording.
• polling=False in the sync test_create_cancel_delete: once Accept is excluded, the background LRO poller raced the test's own get_job calls for recorded GET responses. It failed about 3 of 5 runs. The test already polls the job status itself.
• CHANGELOG: new 1.1.0b2 (Unreleased) entry under Other Changes, and _version.py is bumped to 1.1.0b2.
• The api_version docstring ("Known values are 2025-07-15-preview and None") is emitter output. It is inaccurate, but runtime behavior is unchanged from 1.1.0b1. It needs an emitter fix (related: [http-client-python] Refine api_version argument doc microsoft/typespec#10347) and is not hand-edited here.

2. Surface job errors in live test failures; random live job names (from #49626)

Problem

Live runs of test_create_wait_finish, test_create_wait_finish_async, test_list_job_docs_pagination and deidentify_documents_async.py kept failing with HttpResponseError: Operation returned an invalid status 'OK'. That message is how azure-core LRO polling reports a job that reached Failed. It hid the actual job and document errors, so the underlying service issue went undiagnosed for weeks.

• Fixed live job names: conftest only regenerated the uniquifier when recording. Live CI sets AZURE_SKIP_LIVE_RECORDING=True, so every live run reused the checked-in uniquifier.conf value. Any live runs that share resources would then try to create jobs with the same names.
• Running after result(timeout=120): LROPoller.result(timeout) returns the current resource once the timeout expires. This is not an SDK bug. The tests just never checked whether the poller had finished.

Fix

• Test helper: tests/deid_base_test_case.py adds wait_for_job / wait_for_job_async.
  • They wait up to 600s and fail explicitly if the job hasn't finished.
  • On failure they fetch the job and report its status, job.error, summary counts, and up to 20 non-succeeded documents with their errors.
  • Fetching diagnostics never hides the original exception. If listing documents fails partway, the job details and the documents already collected are kept, and the listing error is appended.
• Batch tests: the three batch tests use the helper. Their per-document asserts now include the document location and error.
• tests/conftest.py: every live run gets a random uniquifier; AZURE_TEST_KEEP_UNIQUIFIER opts out. It is written to uniquifier.conf only when recording, so playback stays deterministic.
• Samples: deidentify_documents and list_job_documents, sync and async, print Job <name> ended with status <status>: <error> and re-raise on failure. The async samples also close their credential. The README snippet is regenerated from samples/deidentify_documents.py with azpysdk update_snippet.

3. Readiness probe in test-resources-post.ps1 (from #49625)

Problem

The post-deployment script opens the storage network ACL, then waits a fixed 30 seconds before uploading the test data. The role assignments and the ACL change can take longer to take effect, so live deployments sometimes fail with 403 AuthorizationFailure. The storage cmdlets also didn't stop the script on error, so a failed upload could be missed until tests ran.

Fix

• Added an Invoke-WithRetry helper with exponential backoff: it starts at 5s, doubles, caps at 60s, and gives up after about 5 minutes. When it gives up, it throws an error that names the operation and the last error.
• Get-AzStorageContainer (used as a readiness check) and each Set-AzStorageBlobContent upload run inside the helper with -ErrorAction Stop. Set-AzStorageAccount also uses -ErrorAction Stop.
• The fixed sleep is replaced by a 30-second minimum wait, measured from the network rule update. A successful upload proves the test principal's RBAC and the network rule are in effect, but not the DeID service managed identity's role assignment. The minimum wait gives that assignment time to propagate.
• Uploads use Get-ChildItem -File, so only files are uploaded, never directories.

4. Temporary shim: keep model list fields bound to one object

Problem

The regenerated _utils/model_base.py caches mutable fields when they are read as properties: _RestField.__get__ stores a newly deserialized list/dict/set back into the model. Any list previously returned by model["field"] is then detached. For example, filters = source["extensions"]; source.extensions; filters.append(".csv") dropped .csv on serialization, which worked in 1.1.0b1. The same emitter change fixed property-first mutations (source.extensions.append(...)), which were lost in 1.1.0b1. No released emitter fixes the dict-then-property case; the template on microsoft/typespec main is unchanged.

Fix

• patch_sdk() in azure/health/deidentification/_patch.py wraps the generated _RestField.__get__. If the generated getter replaced the container, the shim copies the new contents into the original object and keeps that one in the model. The cache flag and setter behavior are unchanged, and the wrap is applied once (the aio package shares _utils and imports the root package first).
• tests/test_model_container_identity.py (unit tests, no recordings) covers all five dict/property access orders for SourceStorageLocation.extensions (list of str) and TaggedPhiEntities.entities (list of models, built from kwargs and from a raw mapping), and checks the serialized output.
• Removal plan: the shim's TODO links [python] Repro: property access detaches container returned by __getitem__ microsoft/typespec#12359, the upstream emitter repro/fix PR, as the removal tracker. test_generated_get_still_needs_shim runs the unwrapped generated getter and fails once a regenerated emitter preserves the container itself, which signals that the shim should be deleted.
• CHANGELOG: Bugs Fixed entry under 1.1.0b2.

Validation

On the combined branch, run locally with the repo tooling (azpysdk, pyright 1.1.407, the same as CI):

• azpysdk black: clean
• azpysdk pyright: pass
• azpysdk pylint: 10.00/10
• azpysdk mypy: pass
• azpysdk update_snippet: README is in sync
• pytest tests in playback: 32/32 passed in each of 3 runs (13 recorded tests plus 19 new model unit tests). The new tests were confirmed to fail with the shim disabled.
• test-resources-post.ps1: no PowerShell parse errors

Earlier validation, done on the separate branches before they were combined:

• The retry helper and the whole post script were tested with the Az cmdlets and Start-Sleep mocked. This covered success after retries, the time-budget error, the 60s cap, and the final minimum wait.
• One live deployment per branch with New-TestResources.ps1, using the user's own (non-CI) credentials, torn down with Remove-TestResources.ps1 afterwards.
  • The test-resources branch: the post script succeeded on the first attempt. This run was before the 30-second minimum wait was added.
  • The diagnostics branch: 13/13 live tests and the 4 changed samples passed. Some tests needed a rerun after transient local Azure CLI credential failures.

Limitations

• The combined branch was not run live. The codegen and regeneration changes were validated in playback only.
• I did not re-record; the Accept matcher exclusion is used instead.
• The retry path was only exercised with mocks, because the live run hit no propagation delay. Nothing here has run in the CI live pipeline with the test service principal.
• The samples still call result(timeout=60/120), matching the README.
• Local setup workarounds that are not in this PR:
  • Generation ran through a short subst drive, because Windows MAX_PATH breaks the emitter's internal black step.
  • Packages came from an internal PyPI proxy.
  • Live auth used AZURE_TEST_USE_CLI_AUTH=true.

Created by Copilot on behalf of Josiah Vinson (Josiah Vinson (@jovinson-ms))

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

The new job-request TypedDict requires response-only fields, causing valid create requests to fail static type checking.

2 open issues

• Request type incorrectly requires response-only read-only fields · New
• API version docs contradict constructor defaults and None handling · New

What changed in this PR

Regenerates azure-health-deidentification to address persistent pyright/pylint failures and remove obsolete generated code.

Changes:

• Refreshes clients, models, serialization helpers, and request TypedDicts.
• Migrates packaging to pyproject.toml, raises dependency minimums, and prepares 1.1.0b2.
• Adjusts playback matching and polling to stabilize existing tests.

Paths below are relative to sdk/healthdataaiservices/azure-health-deidentification/.

File	Description
tsp-location.yaml	Updates the specification location and commit.
tests/​test_create_delete.py	Disables competing background polling.
tests/​conftest.py	Excludes Accept from playback matching.
setup.py	Removes legacy packaging configuration.
README.md	Documents Python 3.10 minimum.
pyproject.toml	Adds packaging metadata and dependency requirements.
CHANGELOG.md	Records regeneration and compatibility changes.
azure/​health/​deidentification/​types.py	Adds request TypedDicts.
azure/​health/​deidentification/​models/​_models.py	Refreshes generated models and annotations.
azure/​health/​deidentification/​aio/​_vendor.py	Removes obsolete async helper.
azure/​health/​deidentification/​aio/​_operations/​_operations.py	Updates async typing, streaming, and paging.
azure/​health/​deidentification/​aio/​_configuration.py	Refreshes generated configuration documentation.
azure/​health/​deidentification/​aio/​_client.py	Updates Self imports and documentation.
azure/​health/​deidentification/​_version.py	Bumps version to 1.1.0b2.
azure/​health/​deidentification/​_vendor.py	Removes obsolete helper.
azure/​health/​deidentification/​_utils/​serialization.py	Refreshes serialization and type handling.
azure/​health/​deidentification/​_utils/​model_base.py	Updates model caching and serialization behavior.
azure/​health/​deidentification/​_serialization.py	Removes stale serialization implementation.
azure/​health/​deidentification/​_operations/​_operations.py	Updates request typing, headers, streaming, and paging.
azure/​health/​deidentification/​_model_base.py	Removes stale model implementation.
azure/​health/​deidentification/​_configuration.py	Refreshes generated configuration documentation.
azure/​health/​deidentification/​_client.py	Updates Self imports and documentation.
apiview-properties.json	Adds cross-language version metadata.
_metadata.json	Adds per-service API-version metadata.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Extensive generated model-runtime changes and the packaging migration warrant human review beyond the reported static-analysis and playback checks.

No open issues 🎉

2 resolved since last review

• Request type incorrectly requires response-only read-only fields
• API version docs contradict constructor defaults and None handling

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Sample diagnostic lookups can replace the original failure and skip async credential cleanup.

1 open issue

• Guard diagnostic lookups to preserve original exceptions and cleanup · New

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Broad generated-runtime and packaging changes need human review, with an unresolved diagnostics issue and no combined-branch live validation.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Preserve job and partial document errors when enumeration fails

sdk/​healthdataaiservices/​azure-health-deidentification/​tests/​deid_base_test_case.py:102

In both describe_job and describe_job_async, if get_job succeeds but document enumeration fails—even on a later page—the outer catch discards the available job status/error and any collected document errors. The assertion then loses the service error this change is meant to expose. Catch document-listing failures separately and append that failure to the retained job and partial-document diagnostics.

1 resolved since last review

• Guard diagnostic lookups to preserve original exceptions and cleanup

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Broad regeneration and packaging changes warrant human review, particularly without combined live validation and with an unresolved diagnostic failure path.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Retain job diagnostics when document listing fails

sdk/​healthdataaiservices/​azure-health-deidentification/​tests/​deid_base_test_case.py:101

If get_job succeeds but document listing fails, including on a later page, the outer catch discards the fetched job and any collected documents. The assertion then omits the job's status, error, and summary—the diagnostics this change is intended to expose. In both describe_job and describe_job_async, catch document enumeration errors separately and retain the available diagnostics alongside the listing error.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The unresolved model-caching regression and broad runtime, packaging, and provisioning changes require correction and final human review.

1 open issue

• Preserve list identity when caching deserialized properties · New

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The broad model-runtime regeneration and descriptor shim warrant human review, and the combined branch has not been validated live.

No open issues 🎉

1 resolved since last review

• Preserve list identity when caching deserialized properties

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The broad model-runtime regeneration and descriptor shim warrant human review, especially with combined-branch live validation still outstanding.

No open issues 🎉

_{Review effort: Balanced}