Skip to content

test_ssl is too verbose #94609

Description

@arhadthedev

Feature or enhancement

  • turn loud but ignored server tracebacks into silent exceptions passed straight to a main thread for assertRaises checks. An example of how it's currently (note "ok" in the last line):

    test_connect_ex (test.test_ssl.SimpleBackgroundTests.test_connect_ex) ...  server:  new connection from ('127.0.0.1', 51236)
     
    server:  bad connection attempt from ('127.0.0.1', 51236):
    Traceback (most recent call last):
       File "D:\a\cpython\cpython\Lib\test\test_ssl.py", line 2371, in wrap_conn
        self.sslconn = self.server.context.wrap_socket(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       File "D:\a\cpython\cpython\Lib\ssl.py", line 455, in wrap_socket
        return self.sslsocket_class._create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       File "D:\a\cpython\cpython\Lib\ssl.py", line 1013, in _create
        self.do_handshake()
        ^^^^^^^^^^^^^^^^^^^
       File "D:\a\cpython\cpython\Lib\ssl.py", line 1284, in do_handshake
        self._sslobj.do_handshake()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
     ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
    ok
    
  • stop printing a dialog between a client and a server; use the same assertion mechanism instead. An example:

    test_alpn_protocols (test.test_ssl.ThreadedTests.test_alpn_protocols) ...  server:  new connection from ('127.0.0.1', 51305)
     client:  sending b'FOO\n'...
     server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  closing connection.
     server:  new connection from ('127.0.0.1', 51307)
     client:  sending b'FOO\n'...
     server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  closing connection.
     server:  new connection from ('127.0.0.1', 51309)
     client:  sending b'FOO\n'...
     server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  closing connection.
     server:  new connection from ('127.0.0.1', 51311)
     client:  sending b'FOO\n'...
     server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  sending b'FOO\n'...
     client:  read b'foo\n'
     client:  closing connection.
    ok
    

Pitch

While (almost) all tests keep silence, test_ssl reports every nit into a console making the log bloated (>1200 lines) to the degree of being unusable for failure tracking.

In addition, all these reports must be looked through manually. Assertions should be used instead.

Linked PRs

Activity

  1. zware commented on Jul 6, 2022

    @zware
    Member

    I haven't looked at the PR, but in principle I agree with the goal here. I do think that the current verbosity should remain available with enough -v flags, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dirtopic-SSLtype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions