Skip to content

platform.uname() not EINTR safe #65971

Closed
Closed
platform.uname() not EINTR safe#65971
Labels
type-bugAn unexpected behavior, bug, or error

Description

@TorColvin
TorColvin
mannequin
opened on Jun 15, 2014
BPO 21772
Nosy @malemburg, @vstinner, @stefanoborini
Superseder
  • bpo-18885: handle EINTR in the stdlib
    		•
    Files
  • platform.py.fix
  • platform_syscmd_uname.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2014-07-22.20:16:28.052>
created_at = <Date 2014-06-15.17:32:30.783>
labels = ['type-bug']
title = 'platform.uname() not EINTR safe'
updated_at = <Date 2014-07-22.20:16:28.052>
user = 'https://bugs.python.org/TorColvin'

    bugs.python.org fields:

    activity = <Date 2014-07-22.20:16:28.052>
actor = 'neologix'
assignee = 'none'
closed = True
closed_date = <Date 2014-07-22.20:16:28.052>
closer = 'neologix'
components = []
creation = <Date 2014-06-15.17:32:30.783>
creator = 'Tor.Colvin'
dependencies = []
files = ['35646', '35659']
hgrepos = []
issue_num = 21772
keywords = ['patch']
message_count = 12.0
messages = ['220654', '220738', '220769', '220774', '220811', '220812', '220813', '220814', '220923', '222551', '222552', '222558']
nosy_count = 5.0
nosy_names = ['lemburg', 'vstinner', 'neologix', 'Tor.Colvin', 'stefanoborini']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '18885'
type = 'behavior'
url = 'https://bugs.python.org/issue21772'
versions = ['Python 3.1', 'Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4', 'Python 3.5']

    Activity

    1. TorColvin commented on Jun 15, 2014

      @TorColvin
      TorColvinmannequin
      on Jun 15, 2014
      MannequinAuthor

      platform.uname() periodically spews EINTR errors on mac - so use subproces.communicate() which is EINTR safe after http://bugs.python.org/issue1068268

      Calling f.read() on naked os.open() output can produce "IOError: [Errno 4] Interrupted system call". Attached is a suggested fix.

    2. TorColvin
      added
      type-bugAn unexpected behavior, bug, or error
      on Jun 15, 2014

    3. vstinner commented on Jun 16, 2014

      @vstinner
      vstinner
      on Jun 16, 2014
      Member

      Here is a different but similar patch for _syscmd_uname():

      • use os.fsdecode() to use the locale encoding instead of latin-1
      • use subprocess.DEVNULL to redirect errors to /dev/null
      • use "with proc:" to ensure that the subprocesss is cleaned in case of error

    4. vstinner commented on Jun 16, 2014

      @vstinner
      vstinner
      on Jun 16, 2014
      Member

      Calling f.read() on naked os.open() output can produce "IOError: [Errno 4] Interrupted system call". Attached is a suggested fix.

      I cannot reproduce your issue. What is your exact Python version and OS?

      I tried to run 100 threads calling 100 times platform._syscmd_uname('-p'). I didn't reproduce the issue on Python 2.7 or 3.5.

      ---

      import platform
import threading

def test():
    for x in range(100):
        p = platform._syscmd_uname('-p')

threads = [threading.Thread(target=test) for n in range(100)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()

    5. TorColvin commented on Jun 16, 2014

      @TorColvin
      TorColvinmannequin
      on Jun 16, 2014
      MannequinAuthor

      2.7.6 or 3.4.1 with OS X 10.8. I can only reproduce it an automated test environment when the load is high on the machine. I've also seen it with 10.6/10.7. It's definitely inconsistently reproducible.

    6. malemburg commented on Jun 17, 2014

      @malemburg
      malemburg
      on Jun 17, 2014
      Member

      I'm not sure whether using os.fsencoding() is a good idea. The encoding used by uname is not defined anywhere and it's possible that Python using a wrong encoding may cause the call to fail (e.g. in case the host name includes non-ASCII chars).

      Then again: _syscmd_uname() is currently only used to determine the processor, which will most likely always be ASCII.

      BTW: It would be good to replace all other calls to os.popen() by subprocess as well. platform.py in Python 3 no longer has the requirement to stay compatible with earlier Python 2 releases.

    7. vstinner commented on Jun 17, 2014

      @vstinner
      vstinner
      on Jun 17, 2014
      Member

      "I'm not sure whether using os.fsencoding() is a good idea. The encoding used by uname is not defined anywhere and it's possible that Python using a wrong encoding may cause the call to fail (e.g. in case the host name includes non-ASCII chars)."

      I also expect ASCII, but I prefer os.fsdecode() because it's the encoding used almost everywhere in Python 3. It's for example the encoding (and error handler) currently used by _syscmd_uname() indirectly by TextIOWrapper.

    8. malemburg commented on Jun 17, 2014

      @malemburg
      malemburg
      on Jun 17, 2014
      Member

      On 17.06.2014 11:21, STINNER Victor wrote:

      STINNER Victor added the comment:

      "I'm not sure whether using os.fsencoding() is a good idea. The encoding used by uname is not defined anywhere and it's possible that Python using a wrong encoding may cause the call to fail (e.g. in case the host name includes non-ASCII chars)."

      I also expect ASCII, but I prefer os.fsdecode() because it's the encoding used almost everywhere in Python 3. It's for example the encoding (and error handler) currently used by _syscmd_uname() indirectly by TextIOWrapper.

      Ok.

    9. vstinner commented on Jun 17, 2014

      @vstinner
      vstinner
      on Jun 17, 2014
      Member

      "2.7.6 or 3.4.1 with OS X 10.8. I can only reproduce it an automated test environment when the load is high on the machine. I've also seen it with 10.6/10.7. It's definitely inconsistently reproducible."

      I'm surprised that the Python read() method doesn't handle EINTR internally.

      I'm in favor of handling EINTR internally almost everywhere, I mean in the Python modules implemented in the C, not in each call using these C methods. "handling EINTR" means calling PyErr_CheckSignals() which may raises a Python exception (ex: KeyboardInterrupt).

    10. neologix commented on Jun 18, 2014

      @neologix
      neologixmannequin
      on Jun 18, 2014
      Mannequin

      I'm surprised that the Python read() method doesn't handle EINTR internally.

      I'm in favor of handling EINTR internally almost everywhere, I mean in the Python modules implemented in the C, not in each call using these C methods. "handling EINTR" means calling PyErr_CheckSignals() which may raises a Python exception (ex: KeyboardInterrupt).

      It's good, we all agree on that.
      I think the different EINTR-related reports should be closed as bpo-18885
      duplicates.
      Then we need a patch for bpo-18885: my idea was to write a pair of macros
      similar to Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS that would
      handle the retry logic, instead of having to check for EINTR and
      calling Py_CheckPendingSignals() everywhere.

    11. stefanoborini commented on Jul 8, 2014

      @stefanoborini
      stefanoborinimannequin
      on Jul 8, 2014
      Mannequin

      You can't use subprocess. platform is used during build. subprocess needs select, but select is a compiled module and at that specific time in the build process is not compiled yet.

    12. malemburg commented on Jul 8, 2014

      @malemburg
      malemburg
      on Jul 8, 2014
      Member

      On 08.07.2014 11:40, Stefano Borini wrote:

      You can't use subprocess. platform is used during build. subprocess needs select, but select is a compiled module and at that specific time in the build process is not compiled yet.

      Good point :-)

    13. stefanoborini commented on Jul 8, 2014

      @stefanoborini
      stefanoborinimannequin
      on Jul 8, 2014
      Mannequin

      Wouldn't it make sense to use the same strategy used in the subprocess fix (that is, retry?). See http://hg.python.org/cpython/rev/6e664bcc958d/

    14. neologix
      closed this as completedon Jul 22, 2014
    15. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      type-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions