Repository navigation
New release on PyPI? #419
Description
Activity
This issue prevents to remove ABC aliases from the collections module in Python 3.9: https://bugs.python.org/issue37324
Reacted by Bernát Gábor, Hugo van Kemenade, Jon Dufresne, Dima Tisnek and Lumír 'Frenzy' BalharReacted by Bernát Gábor and Michael HowitzNow that Python 3.8 has been released, I imagine the urgency for releasing a new version to PyPI has become somewhat elevated.
As a side note, this tendency for package releases to lag behind master is one that I have personally encountered dozens of times... including when the fault is entirely my own, in projects that I maintain! It's a shame because it means there are features and bug fixes in master that nearly nobody is benefiting from, and all because publishing new releases to PyPI is just cumbersome enough to prevent maintainers from prioritizing it. (And once again, I'm including myself in that group.)
I recently solved this problem by creating AutoPub, an open-source project that helps automate package releases on PR merge, which I recently announced during my EuroPython talk a few months ago in Basel, Switzerland. Rather than explain its many benefits here, you can read more about them on the AutoPub project page.
Since the announcement last month, I've added AutoPub to several projects I maintain:
... and, of course, AutoPub releases itself via AutoPub.
As a maintainer, I have been thrilled with the results. I would be happy to submit a PR to this repository that automates package releases via AutoPub and Travis, assuming that would be of interest. What do you think, @gsnedders ?
https://bugs.python.org/issue37324 has been fixed: collections ABC aliases have been removed from Python 3.9. As a consequence, html5lib is no longer compatible with Python 3.9.
How can we help you to get a new release?
The issue broke pyperformance benchmark suite which has a benchmark on html5lib:
https://bugs.python.org/issue38739I've reviewed some PRs and left some comments on others, and made #430 which should help make releasing easier.
(Although I've just realised this is similar to Justin's suggestion so don't mean to step on any toes! #419 (comment))
Also, perhaps a transfer to JazzBand or the PSF may help unblock some of the more basic maintenance tasks such as releasing?
Reacted by James AddisonI contacted @jgraham and @gsnedders by email to see how I can help them to publish a new release.
Reacted by Hugo van Kemenade, Carlton Gibson, no, Zsolt Dollenstein, Steven Kessler, Nico Schlömer and Dima Tisnek@vstinner did you hear back from @jgraham or @gsnedders perhaps?
Otherwise, if all what's needed is to fork a repo and publish under a different
pypiname, then that should not be a big problem :)Reacted by Hugo van Kemenade@vstinner did you hear back from @jgraham or @gsnedders perhaps?
No.
Otherwise, if all what's needed is to fork a repo and publish under a different pypi name, then that should not be a big problem :)
There is a process to request to get the ownership of an existing PyPI name: see https://www.python.org/dev/peps/pep-0541/#abstract
html5lib is also bundled in bleach
@jgraham @gsnedders Thank you both very much for your work on this project!
It's been two years since a release and this project is widely used (2.5m downloads last month according to https://pypistats.org/packages/html5lib!), and it'd be great to keep things ticking over.
I understand if you've moved on and have other interests. Please would you consider giving merge and PyPI rights to one or more extra maintainers?
I would like to volunteer to help get a new release out, to unblock the
collections.abcissue, and for general maintenance tasks, at least as an interim maintainer like @willkg was for the 1.0 release, and would welcome other volunteers too.Otherwise, I suggest we make a community fork (I've made a start at html5lib-python/html5lib-python) and apply for a PEP 0541 transfer.
Again, thanks for all your work on this project!
Reacted by Carlton Gibson, James Addison and Paulo RibeiroReacted by Paulo RibeiroHi @hugovk.
I think at this point it's best to push forward with your fork. I see you've pulled in some the PRs here that are ready. It may be worth commenting on each that that's the case, and perhaps a ticket on the fork to track that progress?
I think beginning the transfer request would be wise. (I don't suppose those are instant.)
Thank you for your effort.
Reacted by Hugo van Kemenade and Seth LarsonPEP 0541 requires PyPI maintainers attempt contact at least three times over six weeks: https://www.python.org/dev/peps/pep-0541/#reachability, so I've opened a request at pypi/support#227. Although it would be preferable to continue things here.
I've commented the PRs here that have been merged into the fork.
Thanks!
Reacted by Carlton GibsonHi @hugovk. Super stuff. Thank you again.
Maybe it's worth opening a PR from your master branch to back here, so that at least there's the possibility that your effort can be picked up, if a sustainable solution here does turn up. 🤷♂️
Hi. I've basically been away from all tech stuff for the past month, and not seen any of the recent discussion.
To summarize, somehow I missed the offer of @jdufresne offering "to help out on a very limited time basis" in the email thread last October and was already on holiday by the time of @vstinner's email in December. Sorry!
Some of my disincentive since last summer has been undergoing a big move in my life, and spending very little time unpaid on tech stuff (as mentioned before, I'm more than happy to spend more time on html5lib if I'm paid to do so; I have relatively little interest in maintaining a major project used by enough for-profit companies without any compensation). That said, let's try and find a workable path forward.
What I said in October in email thread a bunch of you were on then still holds true:
I think my suggestion for a short-term path forward is we get some agreement on adding one or two maintainers (and preferably not just for the next release, because then we're back here in a year's time), we get master passing CI (inc on 3.8 and CPython master), and make a release. I think with other people involved I'm more likely to spend time on html5lib (me eventually landing changes after not getting anyone to review them for months, which has happened before, is a bit demotivating!), so it's probably worthwhile to assume that I'll be a part of whatever comes next.
So, let's:
- Figure out what PRs need to land prior to a next release (and I'll try and get more done on train later too).
- Document how to make a release (this has been another disincentive against me making a release, not knowing what I'm doing!).
- Make a release!
- Document a bunch of policies (there's enough users and institutional knowledge between myself and @jgraham that we should write a bunch of this down); Document Python version support policy #439 is an obvious one where we are at least constrained by being vendored by pip (and I think both us and those working on pip would rather avoid them needing to fork the last supported release!).
- At least add a few committers here (and hopefully with documentation as to how to I'll be much better at/willing to make releases!) and see how that goes, and probably later add some/all as maintainers on PyPI.
Though I will say I still feel like dropping the
collections.*aliases is a complete waste of time, it saves maintaining 15 lines in CPython and causes how many hours in way of work and changes throughout the ecosystem? Sure, it may be fractionally cleaner, but it's a breaking change that therefore necessitates work from plenty of Python developers rather than the Python developers supporting the ecosystem. I'd be much more understanding if it made a significant impact on maintenance work on CPython, but I can't imagine those 15 lines do. Is the benefit of breaking existing code really justified in this case?Let me know if this seems workable, either later this evening (and I realise many of you are in Europe and have previously been commenting during business hours) or over the next few days. In the mean time, I'll try and land a fair number of the open maintenance-task PRs.
Reacted by Hugo van Kemenade5 remaining items
- added 3 commits that reference this issue
on Feb 29, 2020 Though I will say I still feel like dropping the collections.* aliases is a complete waste of time (...)
FYI aliases to ABC in collections have been reverted in Python 3.9, but they will be removed again in Python 3.10: https://bugs.python.org/issue39674
But it doesn't solve the lack of new html5lib release ;-)
- added 2 commits that reference this issue
on Mar 6, 2020 @gsnedders @jgraham Hi, I created https://test.pypi.org/project/html5lib/ earlier for deploy testing, if you create accounts on TestPyPI, I'll give you the full rights.
OK, so status update from what I've done over the past few weeks:
- We now have support for pytest 4 and 5, which gets rid of one of the most common distro patches.
- We should land Don't use cElementTree on Python 3 #502 which avoids an import-time DeprecationWarning in 3.10.
- We should land Deprecate the sanitizer and recommend Bleach #501 which documents the sanitiser as being deprecated.
- And also excitingly, Add benchmarks and optimize html5lib a bit #493 gives pretty significant speed gains when parsing small documents and minor ones which larger documents.
Hopefully we'll get those three landed this week and make a release by the end of the week.
If I have the chance, I might try and fix #96 for wide Python builds only.
Reacted by Hugo van Kemenade, Carlton Gibson, Michael Howitz, Adam Johnson and Greg GutheReacted by Emil Stenström, Seth Larson, Takeshi KOMIYA, Greg Guthe, Storm Heg and Amadej Papež@gsnedders You are awesome! :)
Reacted by Carlton Gibson and Greg Guthe@gsnedders The end of the week has come; any word?
We've now landed all of the above, I think the changelog has all the potential breakage/notable changes, so… let me try to make a release tomorrow?
Reacted by FrankDMartinez, Hugo van Kemenade, Mathias Laurin, Emil Stenström and Greg GutheOK, that's 1.1 shipped. Shout at me if anything is broken (in a new issue, please!).
(I realise it's now Tuesday, so I failed at "tomorrow" as well as the end of the week!)
Reacted by Hugo van Kemenade, Takeshi KOMIYA and Adam JohnsonReacted by Stephen Moore and Seth LarsonReacted by Hugo van Kemenade- added a commit that references this issue
on Sep 26, 2025
Since this library is vendored by
pip, which is itself vendored in CPython'sensurepipmodule, the fact that there's no release that includes PR #403 is blocking the removal of the deprecated abstract base classes from thecollectionsmodule, long advertised as happening in Python 3.8, see python/cpython#10596Normally I am the last person to ask for releases (I assume maintainers have good reasons for not doing so, and I usually don't like being on the receiving end of such requests), but the deadline for the 3.8 beta release is coming up very soon, so if there's any way to expedite an
html5librelease it would really help that effort, and it would avoid stickier solutions like patchinghtml5libdirectly inpip.