I’m trying to post these roughly once a month. Here’s the September post.
Work I did (Sept 6 - Oct 5)
Technical
- Figured out a rough structure for approaching pip’s build logic refactor.
- Discussed and sketched a rough “goal” model for pip’s build logic, to adopt after initial cleanups and decoupling refactor work.
- Made significant progress on refactoring pip’s metadata handling and generation logic. (yay!)
- Identified a quirk in how pip’s current resolution logic, which makes it conceptually intertwined with pip’s index interaction logic (sigh).
- out-of-scope for the ongoing “build logic refactor”, but we do need to resolve this at some point.
- Revamped pip’s linting pipeline to be… simpler (i.e. better).
- Multiple miscellaneous fixes/cleanup to pip.
- A lots of PRs reviewed (>80).
- Reviewed and merged ABNF-related PRs, for TOML.
- Sunk a good day to debugging 500s on GitHub (it was due to emojis). :(
Communication
- Reached out to quite a few folks regarding my semester-long internship.
- Discuss goals and approaches, for pip’s build logic refactor, with fellow pip maintainers across multiple issues on the issue tracker.
- Had a chat w/ a pipenv maintainer, discussing the resolver code reuse situation, and some design discussion about the ongoing build logic refactoring.
- Had multiple chats with fellow pip maintainers, discussing a rough plan and target, for the build logic refactor as well as our development workflow.
- Polished up my proposed “Status-Quo” PyPA governance model.
- Reviewed a proposal for better licensing metadata for Python Packages.
Additional notes on challenges
I had more exams! I’d hoped to get more work done but I hadn’t realize that I’d have exams over 2 weekends – Sunday, 29 Sept to Sunday, 6 Oct. (yay college)
In addition, I realized that my initial approach toward reworking pip’s “prepare distribution” logic was fatally flawed because I forgot about a usecase. My discussions with pipenv and pip maintainers were very fruitful and we pooled our combined knowledge of the tooling into text documents and diagrams. We’ve come up with a more feasible (and much better) plan than the one I had come up with on my own.
Recently, I read something on discuss.python.org, that just perfectly encapsulates how I feel about this stuff:
all of this stuff is really hard and there’s a lot of knowledge scattered around in people’s heads that could really do with being captured somewhere
Overall, I made time to work on pip this month, and made significant progress on cleaning up pip’s metadata generation and handling logic.
Goals for October
Technical
- Wrap up refactoring metadata generation and handling in pip.
- Work on a “blessed” TOML’s compliance suite, as we prepare for TOML 1.0.
- (stretch goal) Begin refactoring the next component – installation.
Communication
- Update the GitHub Project board for pip’s build logic refactor, based on the mental model built over the past few discussions.
- Attend PyCon India:
- Give a talk about the state of Python Packaging.
- Run a sprint on pip, packaging and pypa-bot (and more?).
- Figure out my internship situation and getting approval from my HoD for the internship I’d be doing.
Other commitments
I expect to have some time to work on pip in October, though not as much as September.
In mid-October, I’ll be speaking at PyCon India 2019 (noted above) and I expect from today until my talk, almost all of my free time would be going towards that talk preparation. After that, I am expecting that I’ll spend a few days going through the processes for approvals, at college, related to my final semester internship.
Help us
How can you help us?
- provide test cases where the latest released version of pip (19.2.3, at the time of writing) fails to resolve dependencies properly (on zazo’s issue tracker). They will help us design and test the new resolver.
- talk with your company about becoming a PSF sponsor. The Fundable Packaging Improvements page now summarizes what bugfixes/features/projects within and outside PyPA are blocked on the new resolver, some background context, and how funding would help us get it finished faster.
- triage pip bugs! Around 10% of pip’s issues have not been triaged yet and many others need of action/review.
- help improve our “how to triage pip bugs” guide.
- volunteer to help us build robust testing infrastructure