Skip to content

[http-client-python] Escape generated string literals and wire names - #12342

Draft
Yuchao Yan (msyyc) wants to merge 12 commits into
mainfrom
users/msyyc/python-discriminator-string-literals
Draft

Yuchao Yan (msyyc) wants to merge 12 commits into
mainfrom
users/msyyc/python-discriminator-string-literals

Conversation

@msyyc

@msyyc Yuchao Yan (msyyc) commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fix Python string-literal generation described in this report and #12405.

  • Use Python literal serialization for string constants and discriminator class arguments instead of adding quotes around specification values.
  • Apply the same fix to discriminator initialization and JSON input templates.
  • Escape property and discriminator wire names in JSON-template keys, field metadata, and functional TypedDict declarations. Keep optional-property ordering consistent and escape wire names for their enclosing documentation strings.
  • Escape constant descriptions and literal type annotations in complete model docstrings. Encode discriminator wire names in both polymorphic input and response example headers.
  • Use the existing operation docstring escaping helper for parameter documentation so quoted default values remain valid in complete sync and async operations.
  • Apply the same helper to client and configuration parameter documentation without escaping closing docstring delimiters or double-escaping default values.
  • Preserve Python expressions and literal escapes through JSON template rendering. Keep code examples unwrapped and escape them for their enclosing docstrings.
  • Add regression coverage for quotes, backslashes, newlines, control characters, Unicode, and expression-like text. Check literal AST nodes, annotations, repeated model construction, discriminator selection, serialization, shared enum generation, required and optional wire names, and TypedDict annotation keys.
  • Update equivalent quotation-style expectations and add a fix changelog.

Validation

  • Python emitter build, package formatting checks, repository-wide pnpm format, and emitter/generator lint passed.
  • Emitter tests: 35 passed. A cold browser-import test initially exceeded its five-second timeout; rerunning the suite with a 20-second test timeout passed all tests without code changes.
  • Related Python generator tests: 281 passed, 1 deselected.
  • The string-literal regression file now covers 204 cases, including complete DPG/TypedDict serializer output, complete sync/async operations, clients and configurations with parameter defaults, and complete polymorphic input/response examples embedded in runtime operation docstrings. Each review finding was reproduced before applying its fix.
  • Generated and imported a Python SDK from a minimal TypeSpec service with quotes, backslashes, newlines, and tabs in encoded property and discriminator wire names. Parsed all 23 generated Python files and confirmed exact serialization keys, repeated construction, TypedDict keys, and discriminator dispatch.
  • Earlier discriminator-value validation also generated JSON-mode sync and async SDKs, parsed their operation docstrings, and verified four final code examples preserve the exact discriminator value.

The deselected test, test_models_mode_typeddict_does_not_write_empty_models_folder, fails identically against an isolated copy of the unchanged PR-head generator. It is unrelated to string serialization and is not changed here. Local Python runtime validation used Python 3.13.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ecfba725-b27d-416c-95a6-bc48a498ca8f
@pkg-pr-new

pkg-pr-new Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-python@12342

commit: bf6964b

@msyyc
Yuchao Yan (msyyc) marked this pull request as draft October 22, 2026 05:53
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

All changed packages have been documented.

  • ✅ @typespec/http-client-python
Show changes

@typespec/http-client-python - fix ✏️

Escape string literals and wire names in generated Python declarations, discriminator initializers, field metadata, and templates to preserve specification values as data.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Targeted verification confirms that downstream template rendering still corrupts escaped values and can produce invalid Python examples.

1 open issue
What changed in this PR

This PR updates the Python client generator to serialize specification strings as Python literals, preserving special characters and treating expression-like text as data.

Changes:

  • Replace manual quoting in string declarations, discriminator arguments, initializers, and template values.
  • Add regression tests for literal preservation and generated model behavior; adjust TypedDict quotation expectations.
  • Add a Python emitter fix changelog.
File Description
packages/​http-client-python/​tests/​unit/​test_typeddict.py Update two expectations for equivalent literal quoting.
packages/​http-client-python/​tests/​unit/​test_string_literals.py Add special-character and discriminator regression coverage.
packages/​http-client-python/​generator/​pygen/​codegen/​serializers/​model_serializer.py Escape discriminator initializers and class arguments.
packages/​http-client-python/​generator/​pygen/​codegen/​models/​primitive_types.py Serialize string declarations with repr.
packages/​http-client-python/​generator/​pygen/​codegen/​models/​model_type.py Escape discriminator values in template representations.
.chronus/​changes/​python-discriminator-string-literals-2026-10-22.md Document the Python string-literal fix.

Review effort: Balanced

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Python emitter diff

Baseline gh:d771b29578e9bcb5ad240e705cbabe4210e8e9d3 vs this PR.

No changes to generated output.

Rendered diff: inline on the run summary, or the emitter-diff-html artifact.

Informational check (eng/emitter-diff); does not block the PR.

@azure-sdk-automation

azure-sdk-automation Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

You can try these changes here

🛝 Playground 🌐 Website 🛝 VSCode Extension

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ecfba725-b27d-416c-95a6-bc48a498ca8f
@msyyc

Yuchao Yan (msyyc) commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

CI investigation: the optional Azure/typespec-azure integration check fails on both the original and follow-up heads, outside this Python change. Latest run: https://github.com/microsoft/typespec/actions/runs/39734024339/job/119675416698. The downstream playground imports registerMonacoDefaultWorkersForVite from @typespec/playground, but the current API is exported from @typespec/playground/vite/workers. The downstream import needs updating; this PR does not change playground exports or bypass the check. All other current-head checks have now passed. Tracked in Azure/typespec-azure#5806; monitoring continues.

{% for template_line in example_template %}
{% if template_line %}
{% set wrap_amount = (template_line | length) - (template_line.lstrip() | length) + 10 %}
{{ wrap_string(template_line, wrapstring='\n' + " " * wrap_amount, width=(95 - wrap_amount)) }}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iscai-msft I remember the code has special effect for specific SDK doc string, so not sure whether it is ok to remove the logic

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we added it to prevent overlong lines in generated docstring examples, while keeping wrapped continuation lines indented inside the sphinx code block .. code-block:: python, it might be one of the agent sdks, i can't remember exactly which sdk it was

can you have copilot have a look to see if we can still preserve this ability and write it in a better way?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Complete model rendering still produces invalid Python for triple-quote discriminator values, which the hand-built regression does not exercise.

1 open issue
1 resolved since last review

Review effort: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 27, 2026 02:13
@msyyc Yuchao Yan (msyyc) changed the title [http-client-python] Escape generated discriminator string literals [http-client-python] Escape generated string literals and wire names Oct 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unescaped docstring content and discriminator names can still produce invalid Python, and regression tests do not cover the complete generated output.

3 open issues

Review effort: Balanced

…tion rendering

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 27, 2026 02:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A string default containing triple quotes still produces an unparsable generated operation module because parameter descriptions are rendered without escaping quotes.

3 open issues
1 resolved since last review

Review effort: Balanced

…rings

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 27, 2026 02:53
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Client-level defaults containing triple quotes remain unescaped in generated client and configuration docstrings, making both modules unimportable.

4 open issues

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 27, 2026 03:01
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Client-scoped string defaults containing triple quotes remain unescaped in client and configuration docstrings, producing invalid Python and making generated SDKs unimportable.

5 open issues

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 27, 2026 03:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iscai-msft iscai-msft left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are a lot of diffs in the generated code, and i think most of them we don't want to change. i believe there's also a testserver case for the "too-long example code", you could see what changes when you remove the code in jinja

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aaeb82ef-b4f1-4a88-9442-edd2ea4061cf
Copilot AI balanced review requested due to automatic review settings October 29, 2026 09:03
@msyyc
Yuchao Yan (msyyc) marked this pull request as draft October 29, 2026 09:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Three test assertions expect single quotes even though the generated declarations retain double quotes, causing all three tests to fail.

1 open issue
5 resolved since last review

Review effort: Balanced

Yuchao Yan (msyyc) and others added 3 commits October 29, 2026 17:20
Merge main to align core dependency catalog references with Azure integration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aaeb82ef-b4f1-4a88-9442-edd2ea4061cf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aaeb82ef-b4f1-4a88-9442-edd2ea4061cf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aaeb82ef-b4f1-4a88-9442-edd2ea4061cf

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

emitter:client:python Issue for the Python client emitter: @typespec/http-client-python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants