Skip to content

[mypyc] Use vectorcall for calls to the type object of a native class - #22197

Merged
p-sawicki merged 2 commits into
python:masterfrom
rheard:fix-mypyc-1251
Oct 15, 2026
Merged

p-sawicki merged 2 commits into
python:masterfrom
rheard:fix-mypyc-1251

Conversation

@rheard

@rheard rheard commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Fixes mypyc/mypyc#1251.

A call to a native class is a call to the native constructor only when compiled code calls the class by name. Calls from interpreted code, and calls such as cls(...), type(self)(...) and self.__class__(...) in compiled code, call the type object. That goes through type.__call__, tp_new and the legacy wrapper of __init__: the arguments are packed into a tuple and a dict and parsed with CPyArg_ParseTupleAndKeywords.

With this PR a native class gets a tp_vectorcall, which CPython uses for calls to the type object instead of tp_new and tp_init. It's a generated function that parses the arguments like the vectorcall wrapper of a function does, and then calls the native constructor.

A class gets one when a call to the type object does the same as the native constructor:

  • It has a native __init__, its own or an inherited one. A decorated __init__ isn't called natively.
    If __init__ isn't native (a dataclass, a non-native base class) or takes *args / **kwargs, the
    constructor takes a tuple and a dict anyway. A class without __init__ has nothing to parse, and its
    tp_new accepts and ignores any arguments, which the wrapper wouldn't.
  • It has no __new__. If __new__ returns an instance of a subclass, type.__call__ calls __init__
    of the subclass and the native constructor doesn't.
  • It isn't abstract. The check of self in the wrapper of __init__ is the only thing that rejects an
    instance of an abstract class.

CPython never inherits tp_vectorcall, so interpreted subclasses of a class with allow_interpreted_subclasses=True still go through tp_new and tp_init. So do type.__call__(C, ...), C.__new__(C, ...), pickling, and on Python 3.10 and 3.11 calls of a class with ABCMeta as its metaclass (a heap type metaclass only uses vectorcall since 3.12).

Argument errors are the same as before: the parser uses the name __init__, and an argument of the wrong type adds the traceback entry that the wrapper of __init__ adds. __init__ can be inherited from a class in another module, so generate_class() now gets the source paths of all modules.

Other changes:

  • WrapperGenerator.set_target() and generate_wrapper_core() also take a FuncDecl, since the native
    constructor has no FuncIR.
  • generate_class() checked fields.get("tp_vectorcall") to decide whether a class with __call__
    needs the placeholder tp_call. Nothing set that field, so the placeholder was always there. Now that
    the field is used, the check is for tp_call.
  • The TODO in is_fastcall_supported() is replaced by a comment. The wrappers of __init__ and
    __new__ stay legacy wrappers, because tp_init and tp_new call them with a tuple and a dict.

Performance

The example from the issue, with python -m timeit:

statement master this PR
Point(1, 2) 103 ns 69 ns 1.49x
Point(1, y=2) 155 ns 80 ns 1.93x
Point(x=1, y=2) 172 ns 86 ns 2.00x
p + p (type(self)(...) in compiled code) 111 ns 76 ns 1.46x
Point.origin() (cls(...) in compiled code) 164 ns 129 ns 1.27x
make_point(1, 2) (a compiled function, for reference) 64 ns 64 ns

Comment on lines +263 to +270
if init is None or "__init__" not in init[1].method_decls:
# Without __init__ there are no arguments to parse, and tp_new accepts and
# ignores any that are passed. A decorated __init__ isn't called natively.
return False
if cl.has_method("__new__"):
# If __new__ returns an instance of a subclass, type.__call__ calls __init__ of
# the subclass, but the native constructor doesn't.
return False

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might need to also check if there is a decorated __new__ similar to the check for __init__.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. With #22192 merged in I was able to handle this easily with the has_non_native_new() method. Thanks!

@p-sawicki p-sawicki left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@p-sawicki
p-sawicki merged commit 318bc57 into python:master Oct 15, 2026
18 checks passed
@JukkaL

JukkaL commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator

This is a very nice performance win!

@rheard

rheard commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

@JukkaL @p-sawicki Might be worth checking in; this is the last of the performance/bug fix PRs that I opened because I personally ran into it. At this point I'm just fixing bugs I found while fixing bugs...

The following table shows performance comparisons for my project quadint between mypyc 2.4.0, current master, and current master with simplified code that the bug fixes have allowed:

  2.4.0 master master + simplified
a + b 268 ns 218 (−19%) 204 (−24%)
a * b 444 ns 391 (−12%) 384 (−13%)
a + 1 529 ns 405 (−23%) 336 (−37%)
a + 1j 551 ns 367 (−33%) 307 (−44%)
a == 1j / a == None 289 / 137 ns −45% / −56% same
a ** 3 1.18 µs −20% −23%
pow(a, 5, m) / a.gcd(b) 7.3 / 22.8 µs −19% / −16% −19% / −15%
complexint(3, 4) 214 ns 169 (−21%) −24%
x in ideal 343 ns −31% −59%
C ** 1000 21.9 µs −24% −23%
decompose_number, six split primes 288 µs −7% −8%
decompose_number, factored n ≈ 1e6 / d=5 9.9 / 46.2 µs −5% / −8% −7% / −7%
class groups (imaginary / real) 177 / 136 µs −5% / −9% −3% / −5%

As you can see this is quite a big improvement to a lot of operations, especially around builtin types. Thank you both for your solid reviews and prompt approvals (despite the hiccups around PRs clashing from this morning)! Given that quadint has an incredible userbase of 1, I cannot understate how impactful this will be.

Obviously if I was doing serious maths, I would use another language which makes this more an exercise in learning and optimization, but never-the-less, I'm enjoying it. I'm not sure how many more PRs I'll open, but thanks again to you both!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Calls to the type object of a native class use the slow tuple and dict calling convention

3 participants