PEP 803: add maintainer comments; correct tag in table

[ngoldbaum]

@warsaw told me he'd like to see sections like this in C API PEPs. I checked with @encukou and he said he's happy for me to add this section and he thinks it improves the PEP. He already reviewed an earlier version of this text privately.

I've added myself as a PEP author, let me know if that's not the correct thing to do.

I might expand this with more quotes if I get more responses to queries I've sent out over the past couple of days.

---

📚 Documentation preview 📚: https://pep-previews--4831.org.readthedocs.build/

[MLopez-Ibanez]

Hi @ngoldbaum I am happy if you want to quote me in the way you did, but I think the first quote detracts from your point due to my mistake of calling the free-threaded version experimental, when it is not. What I meant is that there is no stable API that would avoid having to build wheels for each new version of Python, which is precisely what I wish to avoid. Feel free to paraphrase me instead of quoting me.

[MLopez-Ibanez]

This sentence: "It has a much smaller userbase than cryptography and has a smaller appetite for build complexity" makes it sound like the latter is an explanation for the former. The rationale for avoiding build complexity is a smaller developer base rather than userbase. Even if our userbase was larger, we would still aim to avoid build complexity (in particular, not having to chase new Python versions).

Also, one should maybe put "smaller userbase" in perspective to not detract from your argument (why should Python developers care about the needs of a Python package that nobody uses?) : moocore is now a hard-dependency of pymoo, which is among the top %1 Python packages according to https://clickpy.clickhouse.com/dashboard/pymoo Thus, moocore will reach the top %1 as old versions of pymoo are phased out.

[ngoldbaum]

Also, one should maybe put "smaller userbase" in perspective to not detract from your argument (why should Python developers care about the needs of a Python package that nobody uses?) : moocore is now a hard-dependency of pymoo, which is among the top %1 Python packages according to https://clickpy.clickhouse.com/dashboard/pymoo Thus, moocore will reach the top %1 as old versions of pymoo are phased out.

Good point, I'm just going to delete that whole sentence. It doesn't add anything. I also edited the last paragraph and your quote. Thanks for looking this over!

[ogrisel]

Another extra benefit: once major upstream projects ship stable ABI wheels (e.g. numpy and scipy), it will be easier for downstream projects (e.g. scikit-learn) to run their own tests against the main branch of CPython (or a specific PR branch) without having to rebuild those upstream dependencies from source (which can be computationally or operationally expensive).

[encukou]

Thank you, Nathan!

Do you have more requests in flight? If not, let's merge.

[ngoldbaum]

I think @davidhewitt is going to give me some text today.

[rgommers]

I was asked for a comment/quote, so here it is. Please feel free to use or not use as you see fit, or tweak a bit. And thanks again Petr, Steve, Nathan, and whoever else is helping out for all the energy poured into this topic.

"SciPy and a number of other projects in the scientific Python ecosystem are quite interested in starting to use the Stable ABI, in particular to reduce the maintenance load of providing more wheels. With recent CPython, Cython and NumPy releases, this now seems possible. The performance costs seem acceptable and small, although we'll only really build confidence in that assessment after having made the switch."

[ngoldbaum]

I'll update this tomorrow to give David just a little bit more time. He has useful opinions from the perspective of a maintainer of both pydantic and PyO3.

[encukou]

I pushed a correction found in the discussion to this PR.

[davidhewitt]

I can offer perspectives from both PyO3 as a binding framework and Pydantic as a wheel distributor:

"PyO3 greatly benefits from having a stable ABI - one of the biggest challenges for the framework is the need to abstract over a wide range of Python / OS / CPU / environment combinations. We also offer the possibility to build with the stable ABI for each of these environments (targeting a given minimum Python version's stable ABI). The goal is always that all functionality PyO3 offers works the same on all these combinations (sometimes with a Python version floor to access certain features). We currently support Python 3.7+. All functionality added to the stable ABI is a very welcome promise that PyO3 will not need to introduce further conditional code to support a given feature. In the long run this makes it possible for PyO3 to simplify code paths as support for older Python versions is dropped, helping to keep maintenance burden under control.

Pydantic distributes wheels for a native core built using Rust & PyO3. The latest release of pydantic-core distributed 112 wheels and this number is set to grow as more environments are to be added (e.g. Android, iOS, wasm). Pydantic has historically not distributed using the stable ABI because the feature set was too immature. Much Pydantic functionality interacts with Python objects via the C API in hot loops so performance is key concern. As the stable ABI matures it will be ideal for Pydantic to switch tier 2 platforms to the stable ABI (and perhaps eventually tier 1 platforms too), which will significantly reduce the number of wheels to build, test, and distribute.

I would like to highlight that if free-threading does not adopt a stable ABI, all the benefits above will be lost when free-threading becomes the default and only build (which seems the expected long-term plan)."

[ngoldbaum]

Thanks everyone, I think this is ready to merge now. This section got a little long - @encukou feel free to push commits to edit it down if you think it needs it.

[encukou]

Let's put it in; people can skip the section if it's too much.