Skip to content

bpo-34271: Add ssl debugging helpers (GH-10031) #10031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tiran merged 2 commits into from
May 31, 2019
Merged

bpo-34271: Add ssl debugging helpers (GH-10031) #10031

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
16fa504
bpo-34271: Add ssl debugging helpers
tiran Oct 18, 2018
9bbf7a4
Make msg_callback a private debug function
tiran May 30, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
bpo-34271: Add ssl debugging helpers 
The ssl module now can dump key material to a keylog file and trace TLS
protocol messages with a tracing callback. The default and stdlib
contexts also support SSLKEYLOGFILE env var.
  • Loading branch information
@tiran
tiran committed May 31, 2019
commit 16fa504b113818d67f77d8572006b371972addc8
116 changes: 116 additions & 0 deletions Doc/library/ssl.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,10 @@ purposes.
*cadata* is given) or uses :meth:`SSLContext.load_default_certs` to load
default CA certificates.

When :attr:`~SSLContext.keylog_filename` is supported and the environment
variable :envvar:`SSLKEYLOGFILE` is set, :func:`create_default_context`
enables key logging.

.. note::
The protocol, options, cipher and other settings may change to more
restrictive values anytime without prior deprecation. The values
Expand Down Expand Up @@ -172,6 +176,10 @@ purposes.

3DES was dropped from the default cipher string.

.. versionchanged:: 3.8

Support for key logging to :envvar:`SSLKEYLOGFILE` was added.


Exceptions
^^^^^^^^^^
Expand Down Expand Up @@ -1056,6 +1064,28 @@ Constants

SSL 3.0 to TLS 1.3.

.. class:: TLSContentType

:class:`enum.IntEnum` collection of SSL and TLS content types from
:rfc:`8443`, section B.1.

.. versionadded:: 3.8

.. class:: TLSMessageType

:class:`enum.IntEnum` collection of SSL and TLS message types from
:rfc:`8443`, section B.3.

.. versionadded:: 3.8

.. class:: TLSAlertType

:class:`enum.IntEnum` collection of SSL and TLS alert types from
:rfc:`8443`, section B.2.

.. versionadded:: 3.8


SSL Sockets
-----------

Expand Down Expand Up @@ -1901,6 +1931,20 @@ to speed up repeated connections from the same clients.

This features requires OpenSSL 0.9.8f or newer.

.. attribute:: SSLContext.keylog_filename

Write TLS keys to a keylog file, whenever key material is generated or
received. The keylog file is designed for debugging purposes only. The
file format is specified by NSS and used by many traffic analyzers such
as Wireshark. The log file is opened in append-only mode. Writes a
Comment thread
tiran marked this conversation as resolved.
Outdated
synchronized between threads, but not between processes.

.. versionadded:: 3.8

.. note::

This features requires OpenSSL 1.1.1 or newer.

.. attribute:: SSLContext.maximum_version

A :class:`TLSVersion` enum member representing the highest supported
Expand Down Expand Up @@ -1936,6 +1980,78 @@ to speed up repeated connections from the same clients.

.. versionadded:: 3.7

.. attribute:: SSLContext.msg_callback

The message callback provides a debugging hook to analyze TLS connections.
The callback is called for any TLS protocol message (header, handshake,
alert, and more), but not for application data. Due to technical
limitations, the callback can't be used to filter traffic or to abort a
connection. Any exception raised in the callback is delayed until the
handshake, read, or write operation has been performed.

Example::

def msg_cb(conn, direction, version, content_type, msg_type, data):
print(f"{direction:5} {version._name_:7} {content_type._name_:18} "
f"{msg_type._name_:20} {len(data):4}")

ctx = ssl.create_default_context()
ctx.msg_callback = msg_cb
host = 'tls13.crypto.mozilla.org'
with ctx.wrap_socket(socket.socket(), server_hostname=host) as s:
s.connect((host, 443))
s.sendall(b'HEAD / HTTP/1.1\r\n')
s.sendall(b'Host: %s\r\n\r\n' % host)
s.recv(4096)
s.unwrap()

Output::

write TLSv1 HEADER HANDSHAKE 5
write TLSv1_3 HANDSHAKE CLIENT_HELLO 512
read TLSv1_2 HEADER HANDSHAKE 5
read TLSv1_3 HANDSHAKE SERVER_HELLO 122
read TLSv1_2 HEADER CHANGE_CIPHER_SPEC 5
read TLSv1_2 HEADER APPLICATION_DATA 5
read TLSv1_3 INNER_CONTENT_TYPE CERTIFICATE_STATUS 1
read TLSv1_3 HANDSHAKE ENCRYPTED_EXTENSIONS 6
read TLSv1_3 HANDSHAKE CERTIFICATE 3226
read TLSv1_3 HANDSHAKE CERTIFICATE_VERIFY 79
read TLSv1_3 HANDSHAKE FINISHED 52
write TLSv1_2 HEADER CHANGE_CIPHER_SPEC 5
write TLSv1_3 CHANGE_CIPHER_SPEC CHANGE_CIPHER_SPEC 1
write TLSv1_2 HEADER APPLICATION_DATA 5
write TLSv1_3 INNER_CONTENT_TYPE CERTIFICATE_STATUS 1
write TLSv1_3 HANDSHAKE FINISHED 52
write TLSv1_2 HEADER APPLICATION_DATA 5
write TLSv1_3 INNER_CONTENT_TYPE SUPPLEMENTAL_DATA 1
write TLSv1_2 HEADER APPLICATION_DATA 5
write TLSv1_3 INNER_CONTENT_TYPE SUPPLEMENTAL_DATA 1
read TLSv1_2 HEADER APPLICATION_DATA 5
read TLSv1_3 INNER_CONTENT_TYPE CERTIFICATE_STATUS 1
read TLSv1_3 HANDSHAKE NEWSESSION_TICKET 238
read TLSv1_3 HANDSHAKE NEWSESSION_TICKET 238


conn
:class:`SSLSocket` or :class:`SSLObject` instance
direction
``read`` or ``write``
version
:class:`TLSVersion` enum member or int for unknown version. For a
frame header, it's the header version.
content_type
:class:`TLSContentType` enum member or int for unsupported content type.
msg_type
Either a :class:`TLSContentType` enum number for a header message,
a :class:`TLSAlertType` enum member for an alert message,
a :class:`TLSMessageType` enum member for other messages, or int for
unsupported message types.
data
Raw, decrypted message content as bytes

.. versionadded:: 3.8

.. attribute:: SSLContext.options

An integer representing the set of SSL options enabled on this context.
Expand Down
137 changes: 136 additions & 1 deletion Lib/ssl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,90 @@ class TLSVersion(_IntEnum):
MAXIMUM_SUPPORTED = _ssl.PROTO_MAXIMUM_SUPPORTED


class TLSContentType(_IntEnum):
"""Content types (record layer)

See RFC 8446, section B.1
"""
CHANGE_CIPHER_SPEC = 20
ALERT = 21
HANDSHAKE = 22
APPLICATION_DATA = 23
# pseudo content types
HEADER = 0x100
INNER_CONTENT_TYPE = 0x101


class TLSAlertType(_IntEnum):
"""Alert types for TLSContentType.ALERT messages

See RFC 8466, section B.2
"""
CLOSE_NOTIFY = 0
UNEXPECTED_MESSAGE = 10
BAD_RECORD_MAC = 20
DECRYPTION_FAILED = 21
RECORD_OVERFLOW = 22
DECOMPRESSION_FAILURE = 30
HANDSHAKE_FAILURE = 40
NO_CERTIFICATE = 41
BAD_CERTIFICATE = 42
UNSUPPORTED_CERTIFICATE = 43
CERTIFICATE_REVOKED = 44
CERTIFICATE_EXPIRED = 45
CERTIFICATE_UNKNOWN = 46
ILLEGAL_PARAMETER = 47
UNKNOWN_CA = 48
ACCESS_DENIED = 49
DECODE_ERROR = 50
DECRYPT_ERROR = 51
EXPORT_RESTRICTION = 60
PROTOCOL_VERSION = 70
INSUFFICIENT_SECURITY = 71
INTERNAL_ERROR = 80
INAPPROPRIATE_FALLBACK = 86
USER_CANCELED = 90
NO_RENEGOTIATION = 100
MISSING_EXTENSION = 109
UNSUPPORTED_EXTENSION = 110
CERTIFICATE_UNOBTAINABLE = 111
UNRECOGNIZED_NAME = 112
BAD_CERTIFICATE_STATUS_RESPONSE = 113
BAD_CERTIFICATE_HASH_VALUE = 114
UNKNOWN_PSK_IDENTITY = 115
CERTIFICATE_REQUIRED = 116
NO_APPLICATION_PROTOCOL = 120


class TLSMessageType(_IntEnum):
"""Message types (handshake protocol)

See RFC 8446, section B.3
"""
HELLO_REQUEST = 0
CLIENT_HELLO = 1
SERVER_HELLO = 2
HELLO_VERIFY_REQUEST = 3
NEWSESSION_TICKET = 4
END_OF_EARLY_DATA = 5
HELLO_RETRY_REQUEST = 6
ENCRYPTED_EXTENSIONS = 8
CERTIFICATE = 11
SERVER_KEY_EXCHANGE = 12
CERTIFICATE_REQUEST = 13
SERVER_DONE = 14
CERTIFICATE_VERIFY = 15
CLIENT_KEY_EXCHANGE = 16
FINISHED = 20
CERTIFICATE_URL = 21
CERTIFICATE_STATUS = 22
SUPPLEMENTAL_DATA = 23
KEY_UPDATE = 24
NEXT_PROTO = 67
MESSAGE_HASH = 254
CHANGE_CIPHER_SPEC = 0x0101


if sys.platform == "win32":
from _ssl import enum_certificates, enum_crls

Expand Down Expand Up @@ -523,6 +607,48 @@ def hostname_checks_common_name(self, value):
def hostname_checks_common_name(self):
return True

@property
def msg_callback(self):
return super().msg_callback.user_function

@msg_callback.setter
def msg_callback(self, callback):
if callback is None:
super(SSLContext, SSLContext).msg_callback.__set__(self, None)
return

if not hasattr(callback, '__call__'):
raise TypeError(f"{callback} is not callable.")

def inner(conn, direction, version, content_type, msg_type, data):
try:
version = TLSVersion(version)
except TypeError:
pass

try:
content_type = TLSContentType(content_type)
except TypeError:
pass

if content_type == TLSContentType.HEADER:
msg_enum = TLSContentType
elif content_type == TLSContentType.ALERT:
msg_enum = TLSAlertType
else:
msg_enum = TLSMessageType
try:
msg_type = msg_enum(msg_type)
except TypeError:
pass

return callback(conn, direction, version,
content_type, msg_type, data)

inner.user_function = callback

super(SSLContext, SSLContext).msg_callback.__set__(self, inner)

@property
def protocol(self):
return _SSLMethod(super().protocol)
Expand Down Expand Up @@ -576,6 +702,11 @@ def create_default_context(purpose=Purpose.SERVER_AUTH, *, cafile=None,
# CERT_OPTIONAL or CERT_REQUIRED. Let's try to load default system
# root CA certificates for the given purpose. This may fail silently.
context.load_default_certs(purpose)
# OpenSSL 1.1.1 keylog file
if hasattr(context, 'keylog_filename'):
keylogfile = os.environ.get('SSLKEYLOGFILE')
if keylogfile and not sys.flags.ignore_environment:
context.keylog_filename = keylogfile
return context

def _create_unverified_context(protocol=PROTOCOL_TLS, *, cert_reqs=CERT_NONE,
Expand Down Expand Up @@ -617,7 +748,11 @@ def _create_unverified_context(protocol=PROTOCOL_TLS, *, cert_reqs=CERT_NONE,
# CERT_OPTIONAL or CERT_REQUIRED. Let's try to load default system
# root CA certificates for the given purpose. This may fail silently.
context.load_default_certs(purpose)

# OpenSSL 1.1.1 keylog file
if hasattr(context, 'keylog_filename'):
keylogfile = os.environ.get('SSLKEYLOGFILE')
if keylogfile and not sys.flags.ignore_environment:
context.keylog_filename = keylogfile
return context

# Used by http.client if no context is explicitly passed.
Expand Down
Loading