I’m glad you asked.
What’s up?
Over the past few months, I’ve worked on a prototype for a resolver in pip, spent some time planning what rolling out a new dependency resolver in pip might look like, and continue to work on pip’s build logic to decouple it from the dependency resolution logic.
The prototype has helped me better understand the relationships and contracts between various portions of the codebase. This understanding has led me to conclude that it is a worthwhile investment to better decouple the build logic from the resolution logic, then introduce the abstractions defined in resolvelib/zazo, and then proceed to adopting better dependency resolution.
What’s next?
The work on refactoring pip’s build logic is in progress and we’re tracking it on GitHub. Significant technical debt has been accumulated in this part of pip’s codebase over the years, so this is a big project that will likely take multiple months. This would/will also make it easier to provide more user-friendly error messaging, though that is not something that I will pursue in this project.
This investment will enable us to untangle pip’s internals from the resolver, enabling pip to share code for dependency resolution with other packaging tooling (like pipenv and hatch) and reuse the work they have done (e.g., work done on pipenv’s resolver). This also makes a softer rollout possible, one that allows us to iterate based on feedback from users and minimize disruption.
Help us
How you can help us?
triage pip bugs! Around 20% of pip’s issues have not been triaged yet.
provide test cases where the latest released version of pip (19.1.1, at the time of writing) fails to resolve dependencies properly (on zazo’s issue tracker). They will help us test the new resolver.
look at New Resolver: Rollout, Feedback Loops and Development Flow and help us decide: Is the feature flag approach a good idea? Is it a good idea to get rollout feedback via some mechanism other than the pip GitHub issues? Any other thoughts to help us smooth out the rollout?
talk with your company about becoming a PSF sponsor. The Fundable Packaging Improvements page now summarizes what bugfixes/features/projects within and outside PyPA are blocked on the new resolver, some background context, and how funding would help us get it finished faster.
volunteer to help us build robust testing infrastructure
Thanks to Sumana Harihareswara for (a lot of) help in writing this, and Dan Ryan for reviewing the content!