Skip to content

test_context and test_asyncio crash on Windows 7 #76822

Closed
Closed
test_context and test_asyncio crash on Windows 7#76822
Labels
3.7 (EOL)end of lifetopic-asyncio

Description

@vstinner
vstinner
opened on Jan 23, 2018
BPO 32641
Nosy @db3l, @vstinner, @asvetlov, @1st1
PRs
  • bpo-32436: Don't use native popcount() (also fixes bpo-32641) #5292
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-01-24.03:17:07.414>
created_at = <Date 2018-01-23.22:07:58.159>
labels = ['3.7', 'expert-asyncio']
title = 'test_context and test_asyncio crash on Windows 7'
updated_at = <Date 2018-01-24.09:50:40.281>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2018-01-24.09:50:40.281>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-01-24.03:17:07.414>
closer = 'yselivanov'
components = ['asyncio']
creation = <Date 2018-01-23.22:07:58.159>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 32641
keywords = ['patch']
message_count = 11.0
messages = ['310528', '310529', '310530', '310531', '310532', '310538', '310541', '310552', '310554', '310555', '310578']
nosy_count = 4.0
nosy_names = ['db3l', 'vstinner', 'asvetlov', 'yselivanov']
pr_nums = ['5292']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue32641'
versions = ['Python 3.7']

    Activity

    1. vstinner commented on Jan 23, 2018

      @vstinner
      vstinner
      on Jan 23, 2018
      MemberAuthor

      http://buildbot.python.org/all/#/builders/58/builds/498

      0:36:25 [252/414/1] test_context crashed (Exit code 3) -- running: test_largefile (239 sec)
      Windows fatal exception: code 0xc000001d

      Current thread 0x00000944 (most recent call first):
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_context.py", line 321 in ctx1_fun
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_context.py", line 339 in test_context_copy_1
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\case.py", line 615 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\case.py", line 663 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\runner.py", line 176 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\support\init.py", line 1861 in _run_suite
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\support\init.py", line 1951 in run_unittest
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 175 in test_runner
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 176 in runtest_inner
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 130 in runtest
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest_mp.py", line 67 in run_tests_slave
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 517 in _main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 510 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 585 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\regrtest.py", line 46 in _main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\regrtest.py", line 50 in <module>
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\runpy.py", line 85 in _run_code
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\runpy.py", line 193 in _run_module_as_main
      Fatal Python error: Illegal instruction

      Current thread 0x00000944 (most recent call first):
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_context.py", line 321 in ctx1_fun
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_context.py", line 339 in test_context_copy_1
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\case.py", line 615 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\case.py", line 663 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\runner.py", line 176 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\support\init.py", line 1861 in _run_suite
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\support\init.py", line 1951 in run_unittest
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 175 in test_runner
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 176 in runtest_inner
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 130 in runtest
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest_mp.py", line 67 in run_tests_slave
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 517 in _main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 510 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 585 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\regrtest.py", line 46 in _main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\regrtest.py", line 50 in <module>
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\runpy.py", line 85 in _run_code
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\runpy.py", line 193 in _run_module_as_main

      0:46:50 [306/414/2] test_asyncio crashed (Exit code 3) -- running: test_io (487 sec)
      Windows fatal exception: code 0xc000001d

      Current thread 0x00000d24 (most recent call first):
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\test_tasks.py", line 2191 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\test_tasks.py", line 2289 in _step
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\events.py", line 88 in _run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\base_events.py", line 1529 in _run_once
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\base_events.py", line 391 in run_forever
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\base_events.py", line 423 in run_until_complete
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\test_tasks.py", line 2199 in test_context_1
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\case.py", line 615 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\case.py", line 663 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\runner.py", line 176 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\support\init.py", line 1861 in _run_suite
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\support\init.py", line 1951 in run_unittest
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 175 in test_runner
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 176 in runtest_inner
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 130 in runtest
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest_mp.py", line 67 in run_tests_slave
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 517 in _main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 510 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 585 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\regrtest.py", line 46 in _main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\regrtest.py", line 50 in <module>
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\runpy.py", line 85 in _run_code
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\runpy.py", line 193 in _run_module_as_main
      Fatal Python error: Illegal instruction

      Current thread 0x00000d24 (most recent call first):
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\test_tasks.py", line 2191 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\test_tasks.py", line 2289 in _step
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\events.py", line 88 in _run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\base_events.py", line 1529 in _run_once
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\base_events.py", line 391 in run_forever
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\asyncio\base_events.py", line 423 in run_until_complete
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_asyncio\test_tasks.py", line 2199 in test_context_1
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\case.py", line 615 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\case.py", line 663 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 122 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\suite.py", line 84 in __call__
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\unittest\runner.py", line 176 in run
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\support\init.py", line 1861 in _run_suite
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\support\init.py", line 1951 in run_unittest
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 175 in test_runner
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 176 in runtest_inner
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest.py", line 130 in runtest
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\runtest_mp.py", line 67 in run_tests_slave
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 517 in _main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 510 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\libregrtest\main.py", line 585 in main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\regrtest.py", line 46 in _main
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\regrtest.py", line 50 in <module>
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\runpy.py", line 85 in _run_code
      File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\runpy.py", line 193 in _run_module_as_main

      Note: the commit 0bad4d6 doesn't fix these crashes. (The running build 399 hash this fix, and test_context does still crash.)

    2. vstinner
      added
      3.7 (EOL)end of life
      topic-asyncio
      on Jan 23, 2018

    3. vstinner commented on Jan 23, 2018

      @vstinner
      vstinner
      on Jan 23, 2018
      MemberAuthor

      test_asyncio started to crash at build 491:

      http://buildbot.python.org/all/#/builders/58/builds/491

      This build contains the commit f23746a: "bpo-32436: Implement PEP-567". I'm not 100% sure that it's the cause of the regression. Since test_context added by this commit also crash, it really smells this the cause of the regression.

    4. 1st1 commented on Jan 23, 2018

      @1st1
      1st1
      on Jan 23, 2018
      Member

      It's not a regression, the newly added code doesn't work on some Windows buildbot... looking at it.

    5. vstinner commented on Jan 23, 2018

      @vstinner
      vstinner
      on Jan 23, 2018
      MemberAuthor

      Some hints. Compiler warnings:

      ..\Python\hamt.c(625): warning C4018: '<': signed/unsigned mismatch [D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\PCbuild\pythoncore.vcxproj]
      ..\Python\hamt.c(733): warning C4018: '<': signed/unsigned mismatch [D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\PCbuild\pythoncore.vcxproj]
      ..\Python\hamt.c(939): warning C4018: '<': signed/unsigned mismatch [D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\PCbuild\pythoncore.vcxproj]
      ..\Python\hamt.c(1122): warning C4018: '<': signed/unsigned mismatch [D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\PCbuild\pythoncore.vcxproj]

    6. 1st1 commented on Jan 23, 2018

      @1st1
      1st1
      on Jan 23, 2018
      Member

      No, these are trivial uint32_t/Py_SIZE: for (uint32_t i = val_idx + 1; i < Py_SIZE(o); i++); it's something else. I suspect popcount instruction...

    7. 1st1 commented on Jan 24, 2018

      @1st1
      1st1
      on Jan 24, 2018
      Member

      I suspect this is a compiler bug. I can't reproduce it on my windows 7 virtual machine and on AppVeyor.

      It's not related to HAMT, btw, as there is a multitude of HAMT-specific tests that all pass. The crash is specifically in context.c, and the code there is pretty trivial.

    8. vstinner commented on Jan 24, 2018

      @vstinner
      vstinner
      on Jan 24, 2018
      MemberAuthor

      I suspect this is a compiler bug.

      It may be interesting to add the compiler version to test.pythoninfo. It would be simpler to compare Windows buildbots to identify a bug.

      Currently, the only info is:

      "sys.version: 3.7.0a4+ (heads/master:9d411c1, Jan 23 2018, 15:13:42) [MSC v.1900 32 bit (Intel)]"

      Some environment variables are also dumped:

      os.environ[VS100COMNTOOLS]: C:\Program Files\Microsoft Visual Studio 10.0\Common7\Tools\
      os.environ[VS140COMNTOOLS]: D:\Program Files\Microsoft Visual Studio 14.0\Common7\Tools\
      os.environ[VS90COMNTOOLS]: D:\Program Files\Microsoft Visual Studio 9.0\Common7\Tools\

      In the compile step, I see lines like:

      "Microsoft (R) Build Engine version 14.0.24730.2"

      I recall also issues with outdated ucrtbase DLLs. Maybe we can also dump the version of such DLL?

    9. 1st1 commented on Jan 24, 2018

      @1st1
      1st1
      on Jan 24, 2018
      Member

      In an off-list conversation with David, it was confirmed that the buildbot in question runs on an old hardware without SSE4.2 support.

      I decided to simply stop using native popcount instructions as there's no detectable performance difference when using them vs using the fallback code.

      See #5292 for details.

    10. 1st1 commented on Jan 24, 2018

      @1st1
      1st1
      on Jan 24, 2018
      Member

      New changeset b7a80d5 by Yury Selivanov in branch 'master':
      bpo-32436: Don't use native popcount() (also fixes bpo-32641) (bpo-5292)
      b7a80d5

    11. 1st1
      closed this as completedon Jan 24, 2018

    12. 1st1 commented on Jan 24, 2018

      @1st1
      1st1
      on Jan 24, 2018
      Member

      I've pushed a commit that should fix the buildbot. Please reopen if it doesn't.

    13. vstinner commented on Jan 24, 2018

      @vstinner
      vstinner
      on Jan 24, 2018
      MemberAuthor

      I've pushed a commit that should fix the buildbot. Please reopen if it doesn't.

      test_context and test_asyncio do'nt crash anymore on this buildbot, thanks Yury.

      test_asyncio still fail on this buildbot, I created a new issue: bpo-32645, test_asyncio: TLS tests fail on "x86 Windows7" buildbot.

    14. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.7 (EOL)end of lifetopic-asyncio

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions