test_nntplib depends on unreliable external servers

@vstinner

BPO	40735
Nosy	@vstinner, @cjerdonek, @zware, @iritkatriel

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-13.19:06:04.946>
created_at = <Date 2020-05-22.21:07:45.966>
labels = ['type-feature', 'tests', '3.11']
title = 'test_nntplib depends on unreliable external servers'
updated_at = <Date 2022-03-14.15:50:45.562>
user = 'https://github.com/cjerdonek'

bugs.python.org fields:

activity = <Date 2022-03-14.15:50:45.562>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-13.19:06:04.946>
closer = 'iritkatriel'
components = ['Tests']
creation = <Date 2020-05-22.21:07:45.966>
creator = 'chris.jerdonek'
dependencies = []
files = []
hgrepos = []
issue_num = 40735
keywords = []
message_count = 5.0
messages = ['369648', '411529', '411649', '415081', '415150']
nosy_count = 4.0
nosy_names = ['vstinner', 'chris.jerdonek', 'zach.ware', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40735'
versions = ['Python 3.11']

A sporadic failure of test_nntplib.NetworkedNNTP_SSLTests.test_with_statement on the CI for macOS:
https://github.com/python/cpython/pull/20321/checks?check_run_id=700729471#step:6:612

See also:

https://bugs.python.org/issue19613 (test_article_head_body)
https://bugs.python.org/issue19756 (test_capabilities)

ERROR: test_with_statement (test.test_nntplib.NetworkedNNTP_SSLTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/test/test_nntplib.py", line 277, in test_with_statement
    server = self.NNTP_CLASS(self.NNTP_HOST,
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/nntplib.py", line 1025, in __init__
    super().__init__(host, port, user, password, readermode,
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/nntplib.py", line 334, in __init__
    self.sock = self._create_socket(timeout)
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/nntplib.py", line 1031, in _create_socket
    sock = _encrypt_on(sock, self.ssl_context, self.host)
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/nntplib.py", line 292, in _encrypt_on
    return context.wrap_socket(sock, server_hostname=hostname)
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1097)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/test/test_nntplib.py", line 250, in wrapped
    meth(self)
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/test/test_nntplib.py", line 293, in test_with_statement
    if re.search(r'(?i)KEY.TOO.SMALL', ssl_err.reason):
  File "/Users/runner/runners/2.262.1/work/cpython/cpython/Lib/re.py", line 201, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object

Following discussion with Zach and Victor: This is probably an issue with the external servers these tests used. The fix would be to rewrite the tests so that they don't rely on external infrastructure.

IMO for now it's better to *always* skip test_nnptlib tests which rely on external servers and close related issues.

https://bugs.python.org/issue19613 (test_article_head_body)
https://bugs.python.org/issue19756 (test_capabilities)

These issues are open for 9 years. NNTP protocol is getting less and less popular, I don't expect that anyone will come in 2022 and suddenly propose a fix.

nntplib is deprecated as per PEP-594, so there won't be further enhancements to it.

It tomorrow a test_nntplib test fails too often, I suggest to simply skip it, since PEP-594 is accepted. Especially tests using external real NNTP servers.

Navigation Menu

Repository navigation

test_nntplib depends on unreliable external servers #84912

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

test_nntplib depends on unreliable external servers #84912

Description

Activity

cjerdonek commented on May 22, 2020

iritkatriel commented on Jan 24, 2022

vstinner commented on Jan 25, 2022

iritkatriel commented on Mar 13, 2022

vstinner commented on Mar 14, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions