This is an attempt to summarize the broader software architecture around dependency resolution in pip and how testing is being done around this area. The motivation behind writing this, is to make sure all the developers working on this project are on the same page, and to have a written record about the state of affairs. Architecture The “legacy” resolver in pip, is implemented as part of pip’s codebase and has been a part of it for many years....
I’m trying to post these roughly once a month. Here’s the January post. I am working on open source projects, as part of an internship at FOSSEE and as a part of grant-funded work on pip’s dependency resolver. Work I did (Jan 6 - Feb 5) Technical Co-worked with another developer, in person, for 1 week, on pip! Triaged pip’s issue tracker (a lot). Spend some time improving pip’s test suite infrastructure....
I’m trying to post these roughly once a month. Here’s the December post. I am working on open source projects, as part of an internship at FOSSEE and as a part of grant-funded work on pip’s dependency resolver. Work I did (Dec 11 - Jan 5) Technical Wrore and adopted vendoring for use in pip. Initiated conversation for pipenv to switch to it too. Made progress toward TOML 1.0, via useful clarifications in the specification Continued tackling pip’s build logic refactoring....
I’m trying to post these roughly once a month. Here’s the November post. This update has been a little late, since I have been traveling and recovering from an illness. I also have good news: I will be working on open source projects related to Python Packaging (full time) starting early Dec 2019 till May/Jun 2020. Work I did (Nov 6 - Dec 10) Technical Worked to separate pip’s dependency vendoring logic from rest of pip This would enable other projects that use similar mechanisms (eg....
I’m trying to post these roughly once a month. Here’s the October post. Work I did (Oct 6 - Nov 5) Technical Wrapped up refactoring metadata generation within pip. (yay!) Started refactoring wheel building logic within pip. A few minor fixes/cleanup within pip. Helped out with pip 19.3’s release. Significantly simplified pip’s release processes. Reviewed >50 PRs. Fixed a “major” regression in pip’s detection of virtual environments. Got some work done, progressing closer to TOML 1....
bugs.python.org is the issue tracker for CPython. “bpo”, as it is commonly referenced, is where implementation bugs, smaller feature requests, and documentation issues are tracked as part of Python’s development. PEP 581 proposes sunsetting bugs.python.org, in favor of GitHub issues. “Why not focus on improving Roundup / bpo?” section made me wonder: What is needed to improve Roundup / bpo? This blog post is my attempt at summarizing the state of bpo and how it could be improving in the future, based on a discussion with folks maintaining bpo....
I’m trying to post these roughly once a month. Here’s the September post. Work I did (Sept 6 - Oct 5) Technical Figured out a rough structure for approaching pip’s build logic refactor. Discussed and sketched a rough “goal” model for pip’s build logic, to adopt after initial cleanups and decoupling refactor work. Made significant progress on refactoring pip’s metadata handling and generation logic. (yay!) Identified a quirk in how pip’s current resolution logic, which makes it conceptually intertwined with pip’s index interaction logic (sigh)....
I’m trying to post these roughly once a month. Here’s the August post which includes the work I was planning to do this month. Work I did (Aug 6 - Sept 5) Technical Began working on splitting the code for handling of the two kinds of source distributions. Wrap up pip 19.2, with pip 19.2.3 release, unblocking Python 3.8’s final beta. Hooray! Fix documentation builds for pip, on ReadTheDocs – this took way longer to figure out than I’d expected....
I’m going to try to post these roughly once a month. Here’s the late June post. Work I was planning to do Technical Clearly determine what parts of the build system are intertwined with pip’s resolution logic. Communication Get confirmation from my college about required dates/duration of internships in final semester. Work I did (24 June - 6 Aug) Technical Identified portion of build system intertwined with pip’s resolver: “Get abstract distribution”: building installable artifacts is intertwined with pip’s resolver....
I’m glad you asked. What’s up? Over the past few months, I’ve worked on a prototype for a resolver in pip, spent some time planning what rolling out a new dependency resolver in pip might look like, and continue to work on pip’s build logic to decouple it from the dependency resolution logic. The prototype has helped me better understand the relationships and contracts between various portions of the codebase. This understanding has led me to conclude that it is a worthwhile investment to better decouple the build logic from the resolution logic, then introduce the abstractions defined in resolvelib/zazo, and then proceed to adopting better dependency resolution....