logging.config.dictConfig will convert namedtuple to ConvertingTuple #83323

Activity

1. hcoura commented on Dec 27, 2019

   

   hcouramannequin

   on Dec 27, 2019

   MannequinAuthor

   More actions

   While passing

   {
   "version": 1,
   "disable_existing_loggers": False,
   "formatters": {
   "verbose": {"format": "%(levelname)s %(asctime)s %(module)s %(message)s"}
   },
   "handlers": {
   "stackdriver": {
   "class": "google.cloud.logging.handlers.CloudLoggingHandler",
   "client": client,
   "resource": resource,
   },
   },
   "root": {"level": "INFO", "handlers": ["stackdriver"]},
   }

   to logging.config.dictConfig it will convert resource, which is a namedtuple to ConvertingTuple, this will make google.cloud.logging.handlers.CloudLoggingHandler break down the line.

   I am having to create a wrapper class like

   class Bla:
       resource = logging.resource.Resource(
           type="cloud_run_revision",
           labels={},
       )
   
       def _to_dict(self):
           return self.resource._to_dict()

   to go around this
2. hcoura

   added

   3.7 (EOL)end of lifeend of life

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   on Dec 27, 2019
3. tirkarthi commented on Dec 28, 2019

   

   tirkarthi

   on Dec 28, 2019

   Member

   More actions

   Thanks for the report. Can you please attach a sample script with the handler used to illustrate the issue?
4. hcoura commented on Dec 28, 2019

   

   hcouramannequin

   on Dec 28, 2019

   MannequinAuthor

   More actions

   Sorry for not sending a proper reproducible script when submitting the issue. End of the day and quite frustrated with the bug, anyway, I attached a full script that will show the error.

   This is the custom handler I used:

   class MyHandler(logging.StreamHandler):
       def __init__(self, resource, *args, **kwargs):
           super().__init__(*args, **kwargs)
           self.resource: namedtuple = resource
       
       def emit(self, record):
           record.msg += f" {self.resource.type}"
           return super().emit(record)

   self.resource.type will throw and AttributeError when logging, because resource gets converted from a namedtuple to a ConvertingTuple.
5. tirkarthi commented on Dec 28, 2019

   

   tirkarthi

   on Dec 28, 2019

   Member

   More actions

   Thanks for the details. Attached is a complete script. Looking at the source code, anything that looks like a tuple is converted into a tuple. namedtuple produces factory function that itself is a subclass of tuple. It led me to this interesting issue over detecting if it's a namedtuple bpo-7796 where detecting _fields in msg134186 is described as a workaround. So logging might detect namedtuple and skip conversion to ConvertingTuple using the workaround but it appears more of the class of special casing it here in logging that will depend on namedtuple internals. There are no test failures. I will leave it to vinay on this.

   >>> import collections
   >>> person = collections.namedtuple('Person', 'age')
   >>> isinstance(person(age=20), tuple)
   True
   >>> print(person._source) # works on 3.6 but verbose and _source were removed in 3.7 with 8b57d7363916869357848e666d03fa7614c47897
   from builtins import property as _property, tuple as _tuple
   from operator import itemgetter as _itemgetter
   from collections import OrderedDict

   class Person(tuple):
       'Person(age,)'
   // snip more source code for person

   # Detect namedtuple by checking for _fields attribute

   diff --git a/Lib/logging/config.py b/Lib/logging/config.py
   index 4a3b8966ed..ba6937e725 100644
   --- a/Lib/logging/config.py
   +++ b/Lib/logging/config.py
   @@ -448,7 +448,8 @@ class BaseConfigurator(object):
                value = ConvertingList(value)
                value.configurator = self
            elif not isinstance(value, ConvertingTuple) and\
   -                 isinstance(value, tuple):
   +                 isinstance(value, tuple) and\
   +                 not hasattr(value, '_fields'):
                value = ConvertingTuple(value)
                value.configurator = self
            elif isinstance(value, str): # str for py3k
6. vsajip

   added

   3.8 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   on Dec 31, 2019
7. vsajip commented on Jan 1, 2020

   

   vsajip

   on Jan 1, 2020

   Member

   More actions

   New changeset 46abfc1 by Vinay Sajip in branch 'master':
   bpo-39142: Avoid converting namedtuple instances to ConvertingTuple. (GH-17773)
   46abfc1
8. vsajip commented on Jan 1, 2020

   

   vsajip

   on Jan 1, 2020

   Member

   More actions

   New changeset 1d5a7e5 by Vinay Sajip (Miss Islington (bot)) in branch '3.8':
   bpo-39142: Avoid converting namedtuple instances to ConvertingTuple. (GH-17773) (GH-17785)
   1d5a7e5
9. vsajip commented on Jan 1, 2020

   

   vsajip

   on Jan 1, 2020

   Member

   More actions

   New changeset 0e0e4ac by Vinay Sajip (Miss Islington (bot)) in branch '3.7':
   bpo-39142: Avoid converting namedtuple instances to ConvertingTuple. (GH-17773) (GH-17786)
   0e0e4ac
10. vsajip

    closed this as completedon Jan 1, 2020
11. vsajip

    closed this as completedon Jan 1, 2020
12. ezio-melotti

    transferred this issue fromon Apr 10, 2022