Remove asyncore, asynchat and smtpd modules

@warsaw

BPO	28533
Nosy	@warsaw, @orsenthil, @vstinner, @asvetlov, @1st1, @grzgrzgrz3, @Mariatta, @nanjekyejoannah, @aeros, @jacksonriley, @OhSoDank, @iritkatriel
PRs	bpo-28533: Remove asyncore, asynchat, smtpd modules #29521 Revert "bpo-28533: Remove asyncore, asynchat, smtpd modules (GH-29521)" #29951

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-10-25.18:32:04.420>
labels = ['3.11', 'tests', 'expert-asyncio']
title = 'Remove asyncore, asynchat and smtpd modules'
updated_at = <Date 2022-02-24.13:39:30.092>
user = 'https://github.com/Mariatta'

bugs.python.org fields:

activity = <Date 2022-02-24.13:39:30.092>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests', 'asyncio']
creation = <Date 2016-10-25.18:32:04.420>
creator = 'Mariatta'
dependencies = []
files = []
hgrepos = []
issue_num = 28533
keywords = ['patch']
message_count = 37.0
messages = ['279439', '291907', '291908', '294710', '294712', '294771', '294773', '356355', '356364', '356369', '357088', '379342', '396295', '396297', '396319', '396323', '404575', '406159', '406160', '406161', '406162', '406170', '406173', '406208', '406210', '406213', '406214', '406215', '406372', '406377', '406434', '407559', '407560', '407897', '407900', '407915', '413909']
nosy_count = 12.0
nosy_names = ['barry', 'orsenthil', 'vstinner', 'asvetlov', 'yselivanov', 'grzgrzgrz3', 'Mariatta', 'nanjekyejoannah', 'aeros', 'jacksonriley', 'dankreso', 'iritkatriel']
pr_nums = ['29521', '29951']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue28533'
versions = ['Python 3.11']

PR: gh-72719: Remove deprecated asyncore and asynchat modules #96580

Deprecation warning was added to asyncore in https://bugs.python.org/issue25002

asyncore is still used in several tests and should be replaced.

asyncore is still used in several tests and should be replaced.

Here is the list of Python files using asyncore:

haypo@selma$ grep -l asyncore $(find -name "*.py")
./Lib/asyncore.py
./Lib/asynchat.py
./Lib/test/test_smtpd.py
./Lib/test/test_asyncore.py
./Lib/test/test_smtplib.py
./Lib/test/test_os.py
./Lib/test/test_ssl.py
./Lib/test/test_logging.py
./Lib/test/libregrtest/save_env.py
./Lib/test/test_asynchat.py
./Lib/test/test_pyclbr.py
./Lib/test/test_support.py
./Lib/test/test_poplib.py
./Lib/test/test_ftplib.py
./Lib/smtpd.py

smtpd has been rewritten with asyncio:
http://aiosmtpd.readthedocs.io/

The documentation of the smtpd module already points to aiosmtpd.

libregrtest/save_env.py only checks that asyncore.socket_map is not modified, we can keep it until asyncore is really removed.

test_support.py only tests that it's possible to have a "clean" import of asyncore, it can be easily be replaced later.

The list can reduced to:

./Lib/test/test_smtplib.py
./Lib/test/test_os.py
./Lib/test/test_ssl.py
./Lib/test/test_logging.py
./Lib/test/test_pyclbr.py
./Lib/test/test_poplib.py
./Lib/test/test_ftplib.py

asyncore was modified to emit a DeprecationWarning:
http://bugs.python.org/issue25002#msg279417

But then a lot of code starts with to fail -Werror.

Copy of Martin Panter's msg279469:

I normally run the tests with -Werror, and the failures I get are:

test_ssl, test_smtplib, test_poplib, test_logging, test_ftplib, test_support: tests use asyncore
test_os: test uses asynchat (When you import asynchat, the first error complains about importing asyncore, there are actually two warnings)
test_all: This seems to ignore DeprecationWarning; perhaps an exception for PendingDeprecationWarning should also be added?
test_asyncore and test_asynchat: Obviously these have to still test the modules, so they should anticipate the warnings, perhaps using Serhiy’s code
test_smtpd: smtpd module itself uses asyncore; see bpo-25008

I would like to work on this issue.

I think it's a good idea to split this task into few parts/PR.

Let me start from ./Lib/test/test_poplib.py.

What about rewriting pop3 server stub using asyncio, i think requests could be handled synchronously, there will be no benefits from asynchronous.

Please let me know what you think.

I think it's a good idea to split this task into few parts/PR. Let me start from ./Lib/test/test_poplib.py.

Once you have a patch, open a new issue and mention it here.

./Lib/test/test_poplib.py
sub-issue bpo-28533

./Lib/test/test_poplib.py
sub-issue bpo-30514

Fixed issue number from previous comment

Hi, is this still open?

I'm happy to work on replacing asyncore usage in one of the other test files.

@dankreso, the issue is still open and no one has claimed it yet. So feel free to open a pull request.

I'm happy to work on replacing asyncore usage in one of the other test files.

Sounds good, just let us know which one(s) you're working on. (:

Lib/test/test_pyclbr.py - subissue bpo-38866
A trivial one!

Since this issue is now a significant blocker for PEP-594 (remove stdlib dead batteries, which includes asyncore and asynchat), I'm going to prioritize working on this and assign it to myself.

Follow-up: PEP 594 has been accepted and asyncore, asynchat and smtpd are set for removal in 3.12.

Merged documentation PRs:

bpo-47022: Document asynchat, asyncore and smtpd removals in 3.12 #31891

bpo-47061: document module deprecations due to PEP 594 #31984

Pending removal PR:

bpo-42611: PEP 594 - Remove asyncore, asynchat, and smptd from stdlib #23728

Newer removal PR that was merged...

bpo-28533: Remove asyncore, asynchat, smtpd modules #29521

And reverted:

Revert "bpo-28533: Remove asyncore, asynchat, smtpd modules (GH-29521)" #29951

We should be able to un-revert this now.

I propose to remove the expert-asyncio label from this issue. Any objections? This doesn't require an asyncio expert at all, it's just documentation and removals (or deprecations?).

I think that was referring to tests that need to be rewritten with asyncio. The alternative is to move these modules (or parts of them) to test.support so the tests continue to work.

I think that was referring to tests that need to be rewritten with asyncio. The alternative is to move these modules (or parts of them) to test.support so the tests continue to work.

That still doesn't require pinging the asyncio experts, IMO (and my opinion counts since I'm the only active one :-). I'll remove the label.

The alternative is to move these modules (or parts of them) to test.support so the tests continue to work.

IMO that's a convenient way to unblock the issue. Moving away from asyncore/asynchat would be nice but nobody managed to update tests in 10 years, so maybe it's ok to not changed them until it becomes a major (blocking) issue :-)

pythongh-72719: Remove deprecated `asyncore` and `asynchat` modules

@iritkatriel

I've moved these modules to test/support as @iritkatriel and @vstinner proposed.
For now, they were coppied as-is (probably some parts are unused, but this is a subject to another piece of work).

Tests do continue to pass! 🎉

gh-72719: Remove asyncore and asynchat modules (#96580)

@aeros

asyncore, asynchat, smtpd are removed again in the main branch (future Python 3.12). Let's see how it goes this time.

Summary of this issue.

The asyncore/asynchat deprecation has a long history starting in 2015: https://bugs.python.org/issue25002

In 2020, Kyle Stanley @aeros worked on the removal in a branch: https://github.com/aeros/cpython/tree/remove-asycore-asynchat-smtpd

In 2021, I already removed the 3 modules (asyncore, asynchat, smtpd) in Python 3.11, but the Steering Council asked me to revert it since these modules only emitted a DeprecationWarning in 1 Python release (Python 3.10):

PR: bpo-28533: Remove asyncore, asynchat, smtpd modules #29521 (approved by 5 core devs)
commit: 9bf2cbc
I announced the removal on python-dev
In December 2021, 21 projects of PyPI top 5000 projects used these 3 removed modules
Brett Cannon wrote on python-dev that "the modules got yanked prematurely", the removal didn't follow PEP 387 process (updated to require a deprecation in at least 2 Python versions, rather than 1).
I formally requested a SC exception: python/steering-council#86
SC rejected my exception
I created PR Revert "bpo-28533: Remove asyncore, asynchat, smtpd modules (GH-29521)" #29951 to revert my removal

After my revert, I notified Mercurial and Ansible that these 3 projects are deprecated since 2015.

In 2022, these modules are removed again in Python 3.12:

smtpd: PR gh-93243: Make smtpd private before porting its users #93246 and commit 56d16e8
asyncore, asynchat: PR gh-72719: Remove deprecated asyncore and asynchat modules #96580 and commit e56e33d

The removal is now covered by PEP 594 which has been approved the SC.

You can now help the remaining projects still using the removed projects to be upgrade to asyncio and aiosmptd. People can also collaborate to maintain the 3 removed modules outside Python (on PyPI) if they want. Just copy code, tests and doc from Python 3.11.

Navigation Menu

Repository navigation

Remove asyncore, asynchat and smtpd modules #72719

43 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Remove asyncore, asynchat and smtpd modules #72719

Description

Activity

Mariatta commented on Oct 25, 2016

vstinner commented on Apr 19, 2017

vstinner commented on Apr 19, 2017

grzgrzgrz3 commented on May 29, 2017

vstinner commented on May 29, 2017

grzgrzgrz3 commented on May 30, 2017

grzgrzgrz3 commented on May 30, 2017

OhSoDank commented on Nov 11, 2019

nanjekyejoannah commented on Nov 11, 2019

aeros commented on Nov 11, 2019

jacksonriley commented on Nov 20, 2019

aeros commented on Oct 22, 2020

43 remaining items

iritkatriel commented on Aug 28, 2022

gvanrossum commented on Aug 28, 2022

iritkatriel commented on Aug 28, 2022

gvanrossum commented on Aug 28, 2022

vstinner commented on Aug 29, 2022

sobolevn commented on Sep 12, 2022

vstinner commented on Nov 8, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions