Repository navigation
Use of ffi_type_complex_double is unguarded #128156
Description
Activity
I started taking a look at a patch for this, but it doesn't look trivial.
The existing checks use
defined(Py_HAVE_C_COMPLEX) && defined(Py_FFI_SUPPORT_C_COMPLEX)and I think it needs to checkdefined(USING_APPLE_OS_LIBFFI) && defined(HAVE_FFI_TYPE_COMPLEX_DOUBLE_RUNTIME)where the latter is defined with#ifdef HAVE_BUILTIN_AVAILABLE # define HAVE_FFI_TYPE_COMPLEX_DOUBLE_RUNTIME __builtin_available(macOS 10.15, *) #endifThe ergonomic way to do this is beyond my knowledge, but I'm very happy to submit a patch if someone wants to provide some guidance.
cc @skirpichev
Looks as a false positive to me. I don't understand why we need MacOS-specific workaround when nothing is broken.
@skirpichev won't CPython segfault when this function is called on macOS older than 10.15?
The typical pattern on macOS is to guard usage, e.g., in these similar situations:
- Availability of
ptsname_ris not checked at runtime on macOS #123797 - Availability of utimensat, futimens not checked correctly on macOS #75782
This is also blocking adding a guard against future regressions, see #128155 (comment).
cc @ned-deily
- Availability of
won't CPython segfault when this function is called on macOS older than 10.15?
Context: this is for cases where CPython is built to run on a range of macOS releases, like we do for the python.org macOS installer builds. This involves building on a newer (current) version of macOS (like
15) but with MACOSX_DEPLOYMENT_TARGET set to an earlier version, say,10.13.won't CPython segfault when this function is called on macOS older than 10.15?
How? If libffi has no proper support for complex types, then configure script will not define
Py_FFI_SUPPORT_C_COMPLEX.Edit:
This involves building on a newer (current) version of macOS (like 15) but with MACOSX_DEPLOYMENT_TARGET set to an earlier version, say, 10.13.
I see. So, for such builds you want to revert compile-time decision on
Py_FFI_SUPPORT_C_COMPLEXat runtime, right?That will require substantial rewrite of complex numbers support in ctypes.
I would suggest rather unset Py_FFI_SUPPORT_C_COMPLEX for such builds at compile time.
So, for such builds you want to revert compile-time decision on Py_FFI_SUPPORT_C_COMPLEX at runtime, right?
Basically, yes.
I would suggest rather unset Py_FFI_SUPPORT_C_COMPLEX for such builds at compile time.
That's a possibility, but weak symbols allow for portable builds which retain the functionality when available. This is common on macOS because there's robust support for it, but there's also interest in using weak symbols for Linux targets.
That will require substantial rewrite of complex numbers support in ctypes.
This is unfortunate. I'm not sure how to weigh the trade-offs here — but I'm curious if the idea that it's not macOS-specific changes your opinion?
Anyway, I don't think it's a great outcome to need to rewrite this module. I think an "easy" path forward here is to bump the minimum supported macOS version to 10.15 — it's not a big bump and @ned-deily had previously suggested this. There's some data on usage here.
We may need to revisit this module when adding weak symbols for Linux, but I have no clue how modern of library versions the functionality here depends on yet.
1 remaining item
We've set the minimum in CI to 10.15 but formalizing that in a policy will be a separate task (see #128155 (comment)).
I can't tell if it makes sense to leave this issue open since
ffi_type_complex_doubleusage is unguarded (and all of the other macOS 10.15 built-in usage is guarded). It sounds quite challenging to do anything about it though and we won't be enforcing guards for <=10.15 in the foreseeable future.On the
ctypesside, I think I can add run-time checks to all uses ofPy_FFI_SUPPORT_C_COMPLEX, and everything that depends on that. (There's a few changes in this area that I wanted to make anyway.)
But, I don't know much about macOS and I can only test on one macOS version.I'd appreciate a patch that defines, on any platform with
Py_FFI_SUPPORT_C_COMPLEX, a function/macro that returnstrueiffffi_type_complex_*can be used. With that I can write a PR; after that I'd ask you to test it.To be clear:
float complexorcsqrtf()are fine; justffi_type_complex_*, right?
Note that I changed the
cfield.ccode recently. If you have a draft in progress, you'll probably get conflicts with currentmain. If you need them resolved, push to a public branch and I can do the merge.To be clear: float complex or csqrtf() are fine; just ffi_type_complex_*, right?
Not sure what you meant. We have two defines:
Py_HAVE_C_COMPLEX- if Annex G complex types are present and correctly works, andPy_FFI_SUPPORT_C_COMPLEX- if libffi really support complex types (it might just defineFFI_TARGET_HAS_COMPLEX_TYPE, I'm not sure if this issue is valid also for MacOS libffi).For the struct module,
Py_HAVE_C_COMPLEX- is enough. But in the ctypes - adding complex types is useless withoutPy_FFI_SUPPORT_C_COMPLEX. In principle, we could add c_*_complex in pure-Python (if platform support Annex G complex types):class c_double_complex(ctypes.Structure): _fields_ = [("real", ctypes.c_double), ("imag", ctypes.c_double)] @property def value(self): return complex(self.real, self.imag)
And than call e.g. libm's functions like
csqrt(). That was working for me before. But I'm not sure if that will work for all platforms.Reacted by Petr ViktorinThanks for the clarification!
So,
ctypes.c_*_complexwill be missing if not available in libffi at runtime.(Later, we could add them, so that you can pass pointers to them as arguments or put them in Structures, but they wouldn't be usable as function arguments. But that's out of scope here.)
So, ctypes.c_*_complex will be missing if not available in libffi at runtime.
Yes, if not properly supported by libffi at runtime. We can add such types, if Annex G C types are available, but using them as function arguments will produce garbage (see #125206).
We can add such types, if Annex G C types are available, but using them as function arguments will produce garbage
Not that different from something like
c_int.__ctype_be__, then...Anyway, back to this issue: I'd appreciate if a macOS expert could write the guard code -- it feels like a few lines that would take a lot of research for me.
@encukou for what it's worth, the macOS-specific parts are pretty transferable from other cases where we check for runtime availability, e.g., #128156 (comment) and #128156 (comment).
Reacted by Petr Viktorin- added a commit that references this issue
on Jan 21, 2025 Fixed in 3.14.
How important is it to backport this? Due to recentctypeschanges a backport wouldn't be straightforward, but I can do it if needed.Reacted by Rene LeonhardtAFAIR, we don't have complex types in 3.13. (BTW, maybe it's worth a whatsnew entry.) So, I think this might be closed.
Reacted by Petr ViktorinOh! Well, that's one assumption I didn't check :)
Bug report
Bug description:
e.g.
See #100384 for context and similar issues.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
ffi_type_complex_doubleon macOS system libffi #128680