cmd2 4.0 discussion

@kmvanbrunt

Now that cmd2 3.0.0 is out, it's time to start discussing what we would like to see in the 4.0.0 release for cmd2.

Features @kmvanbrunt and I have discussed for 4.x include:

Remove explicit dependency on Python's built-in cmd module, but retain backwards compatibility (PR Remove dependency on Python's built-in cmd module #1539)
Replace dependency on twine for publishing PyPI package with one on uv publish (Use uv for publishing to PyPI instead of twine #1548)
Replace use of invoke with use of make, primarily used for cleanup at this point (Add clean target to Makefile that runs what is in invoke clean #1549)
Migrate documentation to Zensical which is essentially MkDocs 2.0, but implemented in Rust and much faster (PR Migrate documentation to use Zensical instead of MkDocs #1557)
Remove dependency on Python's built-in readline module and replace it with a dependency on prompt-toolkit which is a pure Python replacement for GNU Readline (PR Migrate from using readline to prompt-toolkit #1553)
- Support adding a bottom toolbar to display real-time status via prompt-toolkit (PR Migrate from using readline to prompt-toolkit #1553)
- Add prompt-toolkit syntax highlighting of text entered by users, at least for command names (PR Migrate from using readline to prompt-toolkit #1553)
- Add support for prompt-toolkit Input Validation - I looked into this and honestly if we add support for more advanced input validation, I would prefer to add a dependency on pydantic and go that route. But that is more than I want to bite off for the 4.0 release of cmd2.
- Add auto-suggestions using prompt-toolkit similar to fish-shell where it will show a recommended completion grayed-out per user request in Add auto suggestion feature #1545 (PR Migrate from using readline to prompt-toolkit #1553)
Handle prompt-toolkit completion edge cases related to adding quotes (PR Refactor completion API #1575)
Consider implementing syntax that is essentially compatible with Typer for defining command arguments based on type hints. A user documented how to do this in Discussion A plugin for Typer command define within cmd2 #1530.
- For this I envision something like a @cmd2.typer decorator that will auto-generate an argparse.ArgumentParser based on argument annotations
- We won't be tackling this in the 4.0.0 release unless someone else wants to do the work
Remove the transcript testing feature set (PR Remove the Transcript Testing feature #1583)
Investigate migrating type-checking from mypy to ty now that ty is in beta
- Will revisit this in the future after releasing 4.0.0. This isn't a backwards incompatible change for anyone and I'd like to wait for this library to mature a little more before migrating to it
Replace restrictive type hints (e.g. list) with broader ones like Sequence or Iterable (PR Use more generic types where possible for input parameters #1584)
Ensure examples all work
Change async alert methods to rely on a queue (PR Refactored async alerts. #1586)
Remove the default_to_shell flag since it can result in unintended execution which can be dangerous (PR Removed default_to_shell. #1590)
Restore concatenating all lines of a multi-line command into one for the up-arrow history (PR Switch to using prompt-toolkit's multiline filtering. #1589)
Fix Cmd2Lexer coloring of command names used on continued lines in multiline commands (PR Fixed multiline command highlighting in Cmd2Lexer #1591)
Ensure documentation and CHANGELOG are fully up to date

Moving to prompt-toolkit and no longer depending on cmd will be a large paradigm shift for cmd2. But this will open doors for easier and more consistent cross-platform support as well as being able to provide some more advanced features that we cannot now.

PR #1539 handles removing the explicit dependency on Python's built-in cmd module

@kmvanbrunt

I plan to educate myself on Zensical over the holidays.

I think @kmvanbrunt has already started looking into prompt-toolkit.

@theagilehacker @joqual If you would like to help with anything here, we'd love some more developers to get involved. Also if you have ideas for what you would like to see in cmd2, please suggest them. Most of the planned work requires replacing readline with prompt-toolkit first. But the Typer stuff is fully independent of that. I suspect Kevin wouldn't mind help with prompt-tookit work, so familiarizing yourself with that also would be helpful.

I attempted to port the documentation build to use Zensical which is effectively the next generation of MkDocs. To do this I needed to eliminate use of the markdown-include plugin and replace it with usage of the pymdownx.snippets extension. That is all working fine.

Building with Zensical almost worked. The one one limitation I ran into is that currently the mkdocstrings extension for Zensical does not support cross-references yet:

The mkdocstrings integration is preliminary, which means some features are not yet supported, specifically cross-references and backlinks. We're working on bringing these features into Zensical.

mkdocstrings support was added to Zensical first on 2025-12-03, so this is under very rapid development and I expect within a month or so support for mkdocstrings cross-references will be available. Once that is there, everything should "just work". The documentation builds fast and looks even better than it did with MkDocs and the Material Theme.

I created an issue in the Zensical project about support for mkdocstrings cross-reference support: zensical/zensical#237

Once that is available, I will re-visit porting the documentation to Zensical.

I've started experimenting with migrating to prompt-toolkit from readline on the prompt-toolkit branch.

Things are getting close to working in a very basic rudimentary state. rl_utils.py has been deleted and all use of readline has been replaced with use of prompt-toollkit. All tests have been updated and are passing. A new file pt_utils.py was added as well as a new unit test file with tests to completely cover the code in the new file.

There are inevitably other things that need to be addressed related to asyncio functionality, signal handlers, and blocking popen calls. We need to do a lot of manual testing to make sure everything appears to work as intended

prompt-toolkit support is working and a PR is up to review for it: #1553

The documentation has been migrated to zensical in a separate PR which is also ready: #1557

Documentation has been ported to Zensical and is looking fine on ReadTheDocs (RTD) as well as locally. Differential build is much faster as is the live preview reloading in the browser.

Removed the Transcript Testing feature set in PR #1583

The 4.0.0b1 beta release of cmd2 is now available on PyPI and Release Notes are on GitHub.

The 4.0.0b1 beta release of cmd2 is now available on PyPI and Release Notes are on GitHub.

Thank you for the great work on the beta release. I migrated an application yesterday. It all went smoothly.

I have a few points you might want to consider:

When completing strings from suggestions that contain empty spaces, quotes are applied atomically. When tapping through those suggestions, the quotation is only closed when exactly one match remains. This is sometimes very useful, but not always. It would be quite useful to have some level of control over that behavior.
I found it surprising that the choice from prompt-toolkit is not used for cmd2.Cmd.select. I find its selection method (selecting with Tab, Up, Down etc. instead of or in addition to pressing numbers) more convenient.
I do not need it myself; however, I think a password input option from prompt would be a nice addition for cmd2.Cmd.read_input.

Thanks again for the new beta and especially for the detailed changelog. Each migration to a new beta went smoothly and did not take longer than 20 minutes.

I have some feedback:

There is a corner-case bug with the new completion: when completing an explicitly double-quoted string (e.g., "This is a completion item"), the final completion tap before the closing single quote causes the completer to duplicate the string instead of just inserting a single quote. The completion becomes '"This is a completion item""This is a completion item"'.

Example:

'"This is a completion item"(tap here) — completer → '"This is a completion item""This is a completion item"'

Instead of:

'"This is a completion item"(tap here) — completer → '"This is a completion item"'

This isn’t new and also applies to Cmd 3.x: when using cmd.poutput-like commands, only a few rich.console.ConsoleOptions are passed to an object's __rich_console__ dunder method via the intent interface (makeup, highlight, etc. are not passed through). I’m currently passing options these options by using the rich_print_kwargs parameter, which doesn’t seem intended for all ConsoleOptions fields since the TypedDict for that only lists a subset (see RichPrintKwargs), so the type checker complains about my workaround. Is there a reason for this?

@neoniobium

@neoniobium

Thanks, I'll look into that tab completion issue.
justify, emoji, markup, and highlight are direct arguments to the poutput-like commands.

RichPrintKwargs only contains infrequently used rich.console.print() keyword arguments. I put them in their own class to keep the already-long argument list of our print functions from getting even longer.

class RichPrintKwargs(TypedDict, total=False):
    overflow: OverflowMethod | None
    no_wrap: bool | None
    width: int | None
    height: int | None
    crop: bool
    new_line_start: bool

I might have phrased myself poorly. My concern isn’t how the rich arguments are organized, but that the keyword arguments (e.g., highlight, markup — the ones not in RichPrintKwargs) don’t work as I expect when controlling a Renderable that implements __rich_console__.

When I pass {"highlight": True} inside the dictionary given to rich_print_kwargs, the Renderable behaves as expected. When I pass highlight=True directly as a keyword argument, it behaves differently. Inside __rich_console__ I don’t see options.highlight being set accordingly, but I can access the highlight value when it’s passed via rich_print_kwargs.

I’m not sure if this is expected behavior or whether I’m intended to control Renderables with poutput etc. this way.

That's because of a workaround in cmd2 for a Rich bug. We are not passing emoji, markup, or highlight to console.print(). We are passing them directly to the Console constructor. This is because of two issues with Rich.

ConsoleOptions is missing an emoji field. Therefore, emoji overrides passed to console.print() are ignored.
Multiple classes (Table, Rule, Columns, and Measure) fail to pass some or all of emoji, markup, and highlight settings from their ConsoleOptions back to console.render_str().

I filed this bug:
Textualize/rich#4028

And created this PR to fix it:
Textualize/rich#4084

Since we aren't passing them console.print(), they aren't considered overrides to the underlying Console's behavior. Therefore console.print() isn't setting them in the ConsoleOptions class it passes around.

However, the Renderables should still print the same.

I can pass them to console.print() if you really need to see them in the ConsoleOptions object. But that's not really necessary since the underlying Console object is already configured properly and console.render_str() will use the correct value.

@neoniobium

@neoniobium The completion bug is fixed in #1641

@neoniobium

@neoniobium Beta 5 changed rich_print_kwargs from a TypedDict to a Mapping[str, Any]. So you can pass anything you want without receiving type errors.

@kmvanbrunt

@kmvanbrunt Thank you very much. I have checked out the latest beta and can confirm that both issues are fixed.

We now think that the 4.0.0 release is feature complete minus one experimental feature in development. The 4.0.0rc1 release candidate is now available on PyPI.

We would appreciate any and all help testing the Release Candidate and would like to hear about any bugs or suspected bugs. Suggestions for improving the documentation would be great, and PRs to do it would be even better ;-)

Please see the CHANGELOG for info on what has changed since 3.x and reference our documentation on cmd2 Major Versions Upgrades for tips on migrating.

The Release Notes in GitHub also give some great info on what has changed since the last Beta release.

@KelvinChung2000

@KelvinChung2000 Hey Kelvin, now that we are feature complete and put out a release candidate, the hectic pace of changes should slow down considerably. When you have time could you please merge or rebase the latest stuff in main to your feat/annotated-argparse branch and create a PR to merge that branch into main?

I'd like for @kmvanbrunt and myself to start taking a much closer look at that so we can get it into the final 4.0.0 release as an experimental feature.

Some feedback 4.0.0rc1 is building and testing okay in fedora rawhide with py3.14 and 3.15 on side branch.
https://src.fedoraproject.org/rpms/python-cmd2/pull-request/12

@traylenator

@traylenator Thanks for letting us know! We do really appreciate the positive feedback when downstream folks let us know there are no issues.

@traylenator

@traylenator Can you please elaborate on this comment in the link you gave?

In case anyone notices I know the "rc1" is wrong there in the version... No clue how you handle rc1 with %autorelease but I am not planning on releasing this anyway.

Did we someone now comply with idiomatic processes for release candidates?

@traylenator Can you please elaborate on this comment in the link you gave?

In case anyone notices I know the "rc1" is wrong there in the version... No clue how you handle rc1 with %autorelease but I am not planning on releasing this anyway.

Did we someone now comply with idiomatic processes for release candidates?

Sorry for the confusion, nothing to worry about. Everything is (very) good yourside.

The Version in that RPM spec file in that MR is wrong but your 4.0.0rc1 is perfect.

I am meant to do

Version: 4.0.0
Release: 0.1.rc1

and not what I did in the .spec so that it upgrades cleanly.

This is massively off topic for this thread so feel free to delete or hide these comments.

Thanks for making a RC, super useful.

The 4.0.0 release is now available on PyPI, the release notes are here on GitHub, and there is a Discussion open for chatting out it.

Its been a long-time coming and this is a big release.

Thanks to everyone that helped along the way to make it a reality!

Navigation Menu

Repository navigation

cmd2 4.0 discussion #1538

12 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

cmd2 4.0 discussion #1538

Description

Activity

tleonhardt commented on Dec 7, 2025

tleonhardt commented on Dec 12, 2025

tleonhardt commented on Dec 22, 2025

tleonhardt commented on Dec 22, 2025

tleonhardt commented on Dec 29, 2025

tleonhardt commented on Jan 20, 2026

tleonhardt commented on Jan 21, 2026

tleonhardt commented on Feb 21, 2026

tleonhardt commented on Mar 3, 2026

neoniobium commented on Mar 5, 2026

12 remaining items

neoniobium commented on Apr 19, 2026

kmvanbrunt commented on Apr 19, 2026

neoniobium commented on Apr 19, 2026

kmvanbrunt commented on Apr 19, 2026

kmvanbrunt commented on Apr 20, 2026

kmvanbrunt commented on Apr 23, 2026

neoniobium commented on Apr 24, 2026

tleonhardt commented on May 15, 2026

tleonhardt commented on May 15, 2026

traylenator commented on May 20, 2026

tleonhardt commented on May 20, 2026

tleonhardt commented on May 20, 2026

traylenator commented on May 20, 2026

tleonhardt commented on Jun 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions