Skip to content

Fix pixi pulling newer Python version#35

Merged
ipmb merged 2 commits intomainfrom
benchmark/pixi-pyproject
Mar 10, 2026
Merged

Fix pixi pulling newer Python version#35
ipmb merged 2 commits intomainfrom
benchmark/pixi-pyproject

Conversation

@ipmb
Copy link
Member

@ipmb ipmb commented Oct 9, 2025

Summary

Fixes two issues that were causing the pixi benchmark to be yanked:

1. Python version jumping to 3.14

Switched from pixi.toml to pyproject.toml (recommended by pixi docs). The old pixi.toml with python = "==3.11" wasn't reliably pinning the Python version. The new pyproject.toml uses the standard requires-python = "~= 3.11.0" constraint.

2. Cold install not being cold (Makefile)

pixi clean cache only clears the conda package cache (rattler, at ~/.cache/rattler/). Pixi uses uv as its PyPI backend, whose cache lives at ~/.cache/uv — a separate location that wasn't being cleared. This meant "cold" installs silently hit the warm uv cache, making cold and warm install times nearly identical.

Fix: add rm -rf ~/.cache/uv to pixi-clean-cache.

Verified in run 22916739188: cold install ~6.2s vs warm install ~1.1s — a clear difference now that the uv cache is actually cleared.

3. Re-enable pixi in benchmark

Uncommented the pixi job in benchmark.yml and added pixi to the gather job's needs list. All jobs pass including pixi.

Test plan

  • Trigger benchmark run and verify all jobs pass: run 22916739188
  • Confirm cold install takes longer than warm install (6.2s vs 1.1s)
  • Confirm Python 3.11 is used by pixi

@ipmb
Copy link
Member Author

ipmb commented Oct 9, 2025

The tests pass with this, but I don't think they are correct. Cold install doesn't seem to be doing anything https://github.com/lincolnloop/python-package-manager-shootout/actions/runs/18383090552/job/52374495457

@ipmb ipmb mentioned this pull request Oct 9, 2025
@edmorley
Copy link
Contributor

edmorley commented Oct 9, 2025

For the benchmarks to be comparable, it seems all package managers should run using the same Python version? (And ideally the newest Python version that supports all of the benchmark's dependencies)

Currently I see some inconsistent Python versions, eg:

@ipmb
Copy link
Member Author

ipmb commented Oct 9, 2025

Yes, the intent is that they're all on the same Python version. Not sure why Poetry is pulling 3.10. Would be happy for a PR to get that on 3.11 like the rest.

Also happy for a PR that bumps to a newer Python version. I'm not sure what the max supported by these dependencies is. That may be why we're on 3.11 :)

ipmb and others added 2 commits March 10, 2026 11:49
It's recommended in their docs and pixi.toml seemed to be jumping to
Python 3.14
- Add `rm -rf ~/.cache/uv` to `pixi-clean-cache`: pixi uses uv as its
  PyPI backend, and `pixi clean cache` only clears the conda package
  cache (~/.pixi/pkgs), leaving the uv cache (~/.cache/uv) intact.
  This meant "cold" installs were hitting the warm uv cache, making
  cold and warm install times identical.

- Uncomment pixi job in benchmark.yml and add pixi to gather's needs,
  now that the Python version and cold install issues are resolved.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ipmb ipmb force-pushed the benchmark/pixi-pyproject branch from 7713d4b to ebc984c Compare March 10, 2026 17:56
@ipmb
Copy link
Member Author

ipmb commented Mar 10, 2026

Changes in this PR

Three issues were addressed to get the pixi benchmark working correctly:

1. Python version jumping to 3.14

Switched from pixi.toml to pyproject.toml (the approach recommended by pixi docs). The old pixi.toml with python = "==3.11" wasn't reliably pinning the version. The new pyproject.toml uses the standard requires-python = "~= 3.11.0" constraint.

2. Cold install not actually being cold

pixi clean cache only clears the conda package cache (rattler, at ~/.cache/rattler/). Pixi uses uv as its PyPI backend, and uv's cache lives at ~/.cache/uv — a separate location that wasn't being cleared. This meant "cold" installs were silently hitting the warm uv cache, making cold and warm install times nearly identical (which was the "cold install doesn't seem to be doing anything" observation).

Fix: added rm -rf ~/.cache/uv to pixi-clean-cache in the Makefile.

3. Re-enabled pixi in the benchmark workflow

Uncommented the pixi job in benchmark.yml and added pixi to the gather job's needs list.


Proof

All six jobs pass in run 22916739188, including pixi. The cold vs warm install times now show a meaningful difference:

  • install-cold: ~6.2s (uv cache cleared, packages re-downloaded)
  • install-warm: ~1.1s (uv cache intact)

That ~6× difference confirms the cache is actually being cleared for cold runs.

@ipmb ipmb merged commit ce7759f into main Mar 10, 2026
7 checks passed
@ipmb ipmb deleted the benchmark/pixi-pyproject branch March 10, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants