Repository navigation
[mypyc] Use vectorcall for calls to the type object of a native class - #22197
Conversation
| if init is None or "__init__" not in init[1].method_decls: | ||
| # Without __init__ there are no arguments to parse, and tp_new accepts and | ||
| # ignores any that are passed. A decorated __init__ isn't called natively. | ||
| return False | ||
| if cl.has_method("__new__"): | ||
| # If __new__ returns an instance of a subclass, type.__call__ calls __init__ of | ||
| # the subclass, but the native constructor doesn't. | ||
| return False |
There was a problem hiding this comment.
might need to also check if there is a decorated __new__ similar to the check for __init__.
There was a problem hiding this comment.
Good catch. With #22192 merged in I was able to handle this easily with the has_non_native_new() method. Thanks!
148b022 to
191ffd7
Compare
|
This is a very nice performance win! |
|
@JukkaL @p-sawicki Might be worth checking in; this is the last of the performance/bug fix PRs that I opened because I personally ran into it. At this point I'm just fixing bugs I found while fixing bugs... The following table shows performance comparisons for my project
As you can see this is quite a big improvement to a lot of operations, especially around builtin types. Thank you both for your solid reviews and prompt approvals (despite the hiccups around PRs clashing from this morning)! Given that Obviously if I was doing serious maths, I would use another language which makes this more an exercise in learning and optimization, but never-the-less, I'm enjoying it. I'm not sure how many more PRs I'll open, but thanks again to you both! |
Fixes mypyc/mypyc#1251.
A call to a native class is a call to the native constructor only when compiled code calls the class by name. Calls from interpreted code, and calls such as
cls(...),type(self)(...)andself.__class__(...)in compiled code, call the type object. That goes throughtype.__call__,tp_newand the legacy wrapper of__init__: the arguments are packed into a tuple and a dict and parsed withCPyArg_ParseTupleAndKeywords.With this PR a native class gets a
tp_vectorcall, which CPython uses for calls to the type object instead oftp_newandtp_init. It's a generated function that parses the arguments like the vectorcall wrapper of a function does, and then calls the native constructor.A class gets one when a call to the type object does the same as the native constructor:
__init__, its own or an inherited one. A decorated__init__isn't called natively.If
__init__isn't native (a dataclass, a non-native base class) or takes*args/**kwargs, theconstructor takes a tuple and a dict anyway. A class without
__init__has nothing to parse, and itstp_newaccepts and ignores any arguments, which the wrapper wouldn't.__new__. If__new__returns an instance of a subclass,type.__call__calls__init__of the subclass and the native constructor doesn't.
selfin the wrapper of__init__is the only thing that rejects aninstance of an abstract class.
CPython never inherits
tp_vectorcall, so interpreted subclasses of a class withallow_interpreted_subclasses=Truestill go throughtp_newandtp_init. So dotype.__call__(C, ...),C.__new__(C, ...), pickling, and on Python 3.10 and 3.11 calls of a class withABCMetaas its metaclass (a heap type metaclass only uses vectorcall since 3.12).Argument errors are the same as before: the parser uses the name
__init__, and an argument of the wrong type adds the traceback entry that the wrapper of__init__adds.__init__can be inherited from a class in another module, sogenerate_class()now gets the source paths of all modules.Other changes:
WrapperGenerator.set_target()andgenerate_wrapper_core()also take aFuncDecl, since the nativeconstructor has no
FuncIR.generate_class()checkedfields.get("tp_vectorcall")to decide whether a class with__call__needs the placeholder
tp_call. Nothing set that field, so the placeholder was always there. Now thatthe field is used, the check is for
tp_call.is_fastcall_supported()is replaced by a comment. The wrappers of__init__and__new__stay legacy wrappers, becausetp_initandtp_newcall them with a tuple and a dict.Performance
The example from the issue, with
python -m timeit:Point(1, 2)Point(1, y=2)Point(x=1, y=2)p + p(type(self)(...)in compiled code)Point.origin()(cls(...)in compiled code)make_point(1, 2)(a compiled function, for reference)