Skip to content

Full unicode import system #47330

Closed
Closed

Description

@amauryfa
BPO 3080
Nosy @brettcannon, @birkenfeld, @terryjreedy, @amauryfa, @ncoghlan, @abalkin, @pitrou, @vstinner, @benjaminp, @merwok, @bitdancer, @asvetlov
Files
  • issue3080-5.patch
  • issue3080.py
  • typo.diff
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/vstinner'
    closed_at = <Date 2011-03-21.00:01:35.123>
    created_at = <Date 2008-06-11.18:24:56.218>
    labels = ['interpreter-core', 'type-bug']
    title = 'Full unicode import system'
    updated_at = <Date 2011-03-23.15:58:18.116>
    user = 'https://github.com/amauryfa'

    bugs.python.org fields:

    activity = <Date 2011-03-23.15:58:18.116>
    actor = 'vstinner'
    assignee = 'vstinner'
    closed = True
    closed_date = <Date 2011-03-21.00:01:35.123>
    closer = 'vstinner'
    components = ['Interpreter Core']
    creation = <Date 2008-06-11.18:24:56.218>
    creator = 'amaury.forgeotdarc'
    dependencies = []
    files = ['20477', '21296', '21305']
    hgrepos = []
    issue_num = 3080
    keywords = ['patch']
    message_count = 60.0
    messages = ['68005', '68015', '109844', '112028', '119107', '123963', '123993', '124756', '125752', '126514', '126515', '126516', '126606', '126608', '126612', '126613', '126672', '126673', '126676', '126678', '126680', '126681', '126695', '126705', '126706', '126708', '126752', '126755', '126756', '126760', '127591', '127674', '129141', '129143', '129185', '129196', '130050', '130473', '130492', '130507', '130645', '130935', '131457', '131464', '131472', '131473', '131474', '131483', '131484', '131516', '131547', '131571', '131572', '131575', '131606', '131612', '131625', '131664', '131711', '131890']
    nosy_count = 14.0
    nosy_names = ['brett.cannon', 'georg.brandl', 'terry.reedy', 'amaury.forgeotdarc', 'ncoghlan', 'belopolsky', 'pitrou', 'vstinner', 'benjamin.peterson', 'eric.araujo', 'Arfrever', 'r.david.murray', 'asvetlov', 'python-dev']
    pr_nums = []
    priority = 'high'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue3080'
    versions = ['Python 3.3']

    Activity

    1. ncoghlan commented on Jan 20, 2011

      @ncoghlan
      Contributor

      Victor, could you please create a Reitveld review for this? The auto-review creator can't cope with the Git diffs.

    2. vstinner commented on Jan 20, 2011

      @vstinner
      Member

      Victor, could you please create a Reitveld review for this?

      Yes, but not yet. I have first to cleanup the patch.

    3. ncoghlan commented on Jan 20, 2011

      @ncoghlan
      Contributor

      OK - I'll wait until that is ready before digging into this.

    4. vstinner commented on Jan 20, 2011

      @vstinner
      Member

      Use "U" format to parse a module name, and "%R" to format a module name
      (to escape surrogates characters and add quotes, instead of
      "... '%.200s' ...").

      See also bpo-8754: repr() is better than str() for other reasons, eg. to see a space at the end of a module name (import('space ')) thanks to the quotes.

    5. vstinner commented on Jan 21, 2011

      @vstinner
      Member

      Version 4 of the patch.

    6. vstinner commented on Jan 21, 2011

      @vstinner
      Member

      Same patch (version 4) generated by svn.

    7. vstinner commented on Jan 21, 2011

      @vstinner
      Member

      You can review the patch with Rietveld:
      http://codereview.appspot.com/3972045

    8. vstinner commented on Jan 21, 2011

      @vstinner
      Member

      Oops, there is a dummy typo in imp_init_builtin() that makes test_importlib to crash (which proves that importlib has a good coverage :-)): replace "s:" by "U:" in if (!PyArg_ParseTuple(args, "s:init_builtin", &name)).

    9. vstinner commented on Jan 21, 2011

      @vstinner
      Member

      test_reprlib fails on Windows, because '\' in quoted '\\' in the filename on repr(module). Workaround:

      *******
      index b0dc4d7..e476941 100644
      --- a/Lib/test/test_reprlib.py
      +++ b/Lib/test/test_reprlib.py
      @@ -234,7 +234,7 @@ class LongReprTest(unittest.TestCase):
               touch(os.path.join(self.subpkgname, self.pkgname + '.py'))
               from areallylongpackageandmodulenametotestreprtruncation.areallylongpackageandmodulenametotestreprtruncation import areallylongpackageandmodulenametotestreprtruncation
               eq(repr(areallylongpackageandmodulenametotestreprtruncation),
      -           "<module '%s' from '%s'>" % (areallylongpackageandmodulenametotestreprtruncation.__name__, areallylongpackageandmodulenametotestreprtruncation.__file__))
      +           "<module %r from %r>" % (areallylongpackageandmodulenametotestreprtruncation.__name__, areallylongpackageandmodulenametotestreprtruncation.__file__))
               eq(repr(sys), "<module 'sys' (built-in)>")
       
           def test_type(self):
      *******

      It is maybe not a good idea to use %R to format the filename in module.__repr__().

    10. vstinner commented on Jan 21, 2011

      @vstinner
      Member

      test_runpy fails on Windows on make_legacy_pyc() (of test.support), I don't know why.

    11. ncoghlan commented on Jan 21, 2011

      @ncoghlan
      Contributor

      After applying the patch, doing a make clean and rebuild, I found that test_importlib fails with a segmentation fault, but the default test suite otherwise runs without error (that's on Linux with a UTF-8 filesystem, though).

      I'll see how a -uall run fares.

    12. ncoghlan commented on Jan 21, 2011

      @ncoghlan
      Contributor

      As for the more limited run, I get a clean run with -uall except for the segfault in test_importlib.

      I'll switch to a pydebug build and see how a verbose run of that test fares.

    13. ncoghlan commented on Jan 21, 2011

      @ncoghlan
      Contributor

      I haven't investigated in detail yet, but this is the final line showing the failing test:

      test_module (importlib.test.builtin.test_loader.LoaderTests) ... Segmentation fault

    14. vstinner commented on Jan 21, 2011

      @vstinner
      Member

      except for the segfault in test_importlib.

      Yes, as reported in my previous comment :-) Let's update the patch for practical reasons. But I don't want to touch http://codereview.appspot.com/1874048 (based on patch version 4).

    15. ncoghlan commented on Jan 21, 2011

      @ncoghlan
      Contributor

      Oops, missed that post - that was indeed the problem. With that fixed, tests are all good on this system. I'll give the patch a look anyway, but I'm going to have trouble diagnosing things that don't fail on my development machine.

      As far as the test_reprlib failure goes, I seem to recall addressing a similar problem elsewhere in the standard lib by replace a "%r" code with "'%s'" to get the single quotes without the backslash escaping. A similar change should probably do the trick here.

    16. vstinner commented on Jan 21, 2011

      @vstinner
      Member

      but I'm going to have trouble diagnosing things that don't fail
      on my development machine.

      On Windows, try any character not encodable into your ANSI code page (eg. Ł with cp1252) in the module path and non-ASCII characters in the module name.

      On Mac OS X, sorry, it already works.

      On other OSes, set the locale to something else than UTF-8 (and than ASCII because ASCII is not very interesting) and try non-ASCII module names. The patch includes bpo-3080.py: set the locale to fr_FR.iso88591 to have ISO-8859-1 as locale encoding and try to load a module called 'bpo-3080\xE4'. U+00E4 is encoded to b'\xE4' in ISO-8859-1 and b'\xC3\xA4' to UTF-8.

    17. vstinner commented on Jan 21, 2011

      @vstinner
      Member

      I tried bpo-3080-5.patch. The whole test suite pass on Windows. It pass also on Linux with "-Wd -Werror -R 3:3:" (except bpo-10971 which is unrelated to this issue).

      I should maybe add some unit tests for non-ASCII module paths and non-ASCII module names :-)

    18. ncoghlan commented on Jan 21, 2011

      @ncoghlan
      Contributor

      On Sat, Jan 22, 2011 at 3:08 AM, STINNER Victor <report@bugs.python.org> wrote:

      I should maybe add some unit tests for non-ASCII module paths and non-ASCII module names :-)

      Indeed. There are a few tests in test_runpy that could be adapted to
      that task fairly easily (it creates the test packages at run time in a
      temporary directory, so copying that to make a non-ASCII path and
      module name test should be too difficult).

    19. vstinner commented on Jan 31, 2011

      @vstinner
      Member

      As explained in issue bpo-10828: Python 3.2 doesn't support non-ASCII module names on Windows because module names are encoded to UTF-8 instead of the filesystem encoding (the ANSI code page).

    20. vstinner commented on Feb 1, 2011

      @vstinner
      Member

      See also bpo-6011.

    21. vstinner commented on Feb 22, 2011

      @vstinner
      Member

      I started to commit some parts of the huge patch:

      r88515: Mark PyWin_FindRegisteredModule() as private
      r88516: Remove unused argument of _PyImport_GetDynLoadFunc()
      r88517 (3.3), r88518 (3.2): document encoding used by import functions

    22. vstinner commented on Feb 23, 2011

      @vstinner
      Member

      r88519: Mark _PyImport_FindBuiltin() argument as constant
      r88520: Add PyModule_GetNameObject()

    23. pitrou commented on Feb 23, 2011

      @pitrou
      Member

      This new failure is perhaps related:

      http://www.python.org/dev/buildbot/all/builders/AMD64%20Windows%20Server%202008%203.x/builds/572/steps/test/logs/stdio

      ======================================================================
      FAIL: test_module (test.test_reprlib.LongReprTest)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
        File "c:\buildslave-py3k\3.x.curtin-win2008-amd64\build\lib\test\test_reprlib.py", line 237, in test_module
          "<module '%s' from '%s'>" % (areallylongpackageandmodulenametotestreprtruncation.__name__, areallylongpackageandmodulenametotestreprtruncation.__file__))
      AssertionError: "<module 'areallylongpackageandmodulenametotestreprtruncation.areallylongpackage [truncated]... != "<module 'areallylongpackageandmodulenametotestreprtruncation.areallylongpackage [truncated]...
      Diff is 825 characters long. Set self.maxDiff to None to see it.
    24. vstinner commented on Feb 23, 2011

      @vstinner
      Member

      This new failure is perhaps related: (...) test_reprlib

      Ah yes, yesterday, I tried to remember which test was impacted by the module change, but all tests passed on Linux. Anyway, it's now fixed by r88533.

    25. vstinner commented on Mar 4, 2011

      @vstinner
      Member

      r88746: Add PyModule_NewObject() function
      r88747: Add PyImport_AddModuleObject() and PyImport_ExecCodeModuleObject()

    26. vstinner commented on Mar 9, 2011

      @vstinner
      Member

      I created the features/unicode_import repository with a "unicode_import" branch:
      http://hg.python.org/features/unicode_import/

      It's my huge patch splitted into small and atomic commits.

    27. amauryfa commented on Mar 10, 2011

      @amauryfa
      ContributorAuthor

      Nice work! Is there a specific place for comments? Here are some of them already:

        pathsize = PyUnicode_GET_SIZE(prefix) + PyUnicode_GET_SIZE(name);
        result = PyUnicode_FromUnicode(NULL, pathsize);
        path = PyUnicode_AS_UNICODE(ret);
        ...
        return result;
        lastdot = Py_UNICODE_strrchr(nameuni, '.');
        if (lastdot == NULL)
            shortname = namenuni;
        else:
            shortname = lastdot + 1;
      • _PyImport_GetDynLoadFunc still takes char* arguments. Can this fail on win32
        for example, in case the pathname cannot be encoded to mbcs?
    28. vstinner commented on Mar 10, 2011

      @vstinner
      Member

      Is there a specific place for comments?

      Yes, but my work is not done. I still have parts to commit.

      _PyImport_GetDynLoadFunc still takes char* arguments.

      Oh. This one is not easy because this function has many implementations and all implementations have the same prototype. I will maybe fix it later.

    29. vstinner commented on Mar 11, 2011

      @vstinner
      Member

      See also bpo-9319: when this issue will be fixed, it will be easier to fix bpo-9319.

    30. vstinner commented on Mar 15, 2011

      @vstinner
      Member

      I finished to split the huge patch into smaller commits. You can now test the unicode_import Mercurial branch. Especially, it should be tested on Windows.

      I don't know if I should merge the branch as an unique commit or as multiple commits. Some of them can be simply be merged.

      You can try bpo-3080.py (file attached to this issue, extracted from the patch): a short script testing this issue.

      --

      The parser and _PyImport_GetDynLoadFunc() (on Windows) do still store the filename as byte strings, and so I don't think that Python is ready to use full Unicode range for filenames on Windows. But at least, it should now support non-ASCII module names and paths which are encodable to the ANSI code page.

      Issue bpo-10785 should improve the situation at least for the parser.

      But for _PyImport_GetDynLoadFunc(), I don't know if there is a Unicode version of LoadLibraryEx().

      --

      Modules/zipimport.c::make_filename: remove the limit buffer

      Implemented in f286d3b514e0.

      Python/importdl.c::_PyImport_LoadDynamicModule: shortnameobj is not necessary

      Done in 76907d413b99

    31. vstinner commented on Mar 19, 2011

      @vstinner
      Member

      Replace open_exclusive() by fopen(name, "wb") on Windows: is it correct?

      I reverted this change in my Mercurial branch (unicode_import).

      rename xxxobj => xxx to keep original names and have a short patch

      done

      catch encoding errors in case_ok()

      done

      don't encode in case_ok() if case_ok() does nothing (eg. on Linux)

      done

      find a better name for find_module2()

      done: find_module_path_list() and find_module_path()

    32. vstinner commented on Mar 19, 2011

      @vstinner
      Member

      test_runpy fails on Windows on make_legacy_pyc() (of test.support),
      I don't know why.

      Gotcha: I replaced mkdir() by CreateDirectoryW(), but the "directory already exists" error was not ignored. Fixed by 2debe178697b.

    33. python-dev commented on Mar 20, 2011

      python-devmannequin
      Mannequin

      New changeset 6c80ac44ae9c by Victor Stinner in branch 'default':
      Issue bpo-3080: zipimport has a full unicode suppport
      http://hg.python.org/cpython/rev/6c80ac44ae9c

      New changeset b50a0d44545a by Victor Stinner in branch 'default':
      Issue bpo-3080: PyImport_Cleanup() uses Unicode
      http://hg.python.org/cpython/rev/b50a0d44545a

      New changeset e7c1019b27b9 by Victor Stinner in branch 'default':
      Issue bpo-3080: Add PyImport_ImportFrozenModuleObject()
      http://hg.python.org/cpython/rev/e7c1019b27b9

      New changeset 2425717c6430 by Victor Stinner in branch 'default':
      Issue bpo-3080: Import builtins using Unicode strings
      http://hg.python.org/cpython/rev/2425717c6430

      New changeset ced52fcd95f6 by Victor Stinner in branch 'default':
      Issue bpo-3080: Use PyUnicode_InternFromString() for builtins
      http://hg.python.org/cpython/rev/ced52fcd95f6

      New changeset e63a583ec689 by Victor Stinner in branch 'default':
      Issue bpo-3080: Document the name attribute of the _inittab structure
      http://hg.python.org/cpython/rev/e63a583ec689

      New changeset bab42673674a by Victor Stinner in branch 'default':
      Issue bpo-3080: _PyWin_FindRegisteredModule() returns the path as Unicode
      http://hg.python.org/cpython/rev/bab42673674a

      New changeset ef2b6305d395 by Victor Stinner in branch 'default':
      Issue bpo-3080: _PyImport_LoadDynamicModule() uses Unicode for name and path
      http://hg.python.org/cpython/rev/ef2b6305d395

      New changeset d52f471fbbeb by Victor Stinner in branch 'default':
      Issue bpo-3080: find_module() initialize buf and *p_fp
      http://hg.python.org/cpython/rev/d52f471fbbeb

      New changeset bdf5820f5a39 by Victor Stinner in branch 'default':
      Issue bpo-3080: Remove useless name buffer from find_module()
      http://hg.python.org/cpython/rev/bdf5820f5a39

      New changeset a4d797b9ff63 by Victor Stinner in branch 'default':
      Issue bpo-3080: Create find_module_path_list() subfunction
      http://hg.python.org/cpython/rev/a4d797b9ff63

      New changeset 09aaac73d9cf by Victor Stinner in branch 'default':
      Issue bpo-3080: Create find_module_path() subfunction
      http://hg.python.org/cpython/rev/09aaac73d9cf

      New changeset f6507eb8e689 by Victor Stinner in branch 'default':
      Issue bpo-3080: get_sourcefile(), make_source_pathname(), load_package()
      http://hg.python.org/cpython/rev/f6507eb8e689

      New changeset d24decc8c97e by Victor Stinner in branch 'default':
      Issue bpo-3080: Use Unicode to import source and compiled modules
      http://hg.python.org/cpython/rev/d24decc8c97e

      New changeset 64c21f364519 by Victor Stinner in branch 'default':
      Issue bpo-3080: load_module() expects name and path as Unicode
      http://hg.python.org/cpython/rev/64c21f364519

      New changeset e55e7f197649 by Victor Stinner in branch 'default':
      Issue bpo-3080: PyImport_ImportModuleNoBlock() uses Unicode
      http://hg.python.org/cpython/rev/e55e7f197649

      New changeset 7c67aa3ab531 by Victor Stinner in branch 'default':
      Issue bpo-3080: Use Unicode for the "The Magnum Opus of dotted-name import"
      http://hg.python.org/cpython/rev/7c67aa3ab531

      New changeset 23fe237afa81 by Victor Stinner in branch 'default':
      Issue bpo-3080: Use %R to format module name in error messages
      http://hg.python.org/cpython/rev/23fe237afa81

      New changeset 2ee0ab9d2e8a by Victor Stinner in branch 'default':
      Issue bpo-3080: Reindent and simplify import_submodule()
      http://hg.python.org/cpython/rev/2ee0ab9d2e8a

      New changeset 340f76a6a792 by Victor Stinner in branch 'default':
      Issue bpo-3080: Drop OS/2 support for the import machinery
      http://hg.python.org/cpython/rev/340f76a6a792

      New changeset 156818529636 by Victor Stinner in branch 'default':
      Issue bpo-3080: find_module() expects module fullname and subname as Unicode
      http://hg.python.org/cpython/rev/156818529636

      New changeset fe1d421ca3fa by Victor Stinner in branch 'default':
      Issue bpo-3080: Rename some path variables to path_list
      http://hg.python.org/cpython/rev/fe1d421ca3fa

      New changeset c1a5a7dca1ec by Victor Stinner in branch 'default':
      Issue bpo-3080: find_module() sets an empty path for builtin and frozen modules
      http://hg.python.org/cpython/rev/c1a5a7dca1ec

      New changeset c4ccf02456d6 by Victor Stinner in branch 'default':
      Issue bpo-3080: Refactor find_module_path(), use return instead of break
      http://hg.python.org/cpython/rev/c4ccf02456d6

      New changeset 298a70b27497 by Victor Stinner in branch 'default':
      Issue bpo-3080: find_init_module() expects Unicode
      http://hg.python.org/cpython/rev/298a70b27497

      New changeset 066b399a8477 by Victor Stinner in branch 'default':
      Issue bpo-3080: case_ok() expects Unicode strings
      http://hg.python.org/cpython/rev/066b399a8477

      New changeset 9aec6f0e4076 by Victor Stinner in branch 'default':
      Issue bpo-3080: find_module() returns the path as Unicode
      http://hg.python.org/cpython/rev/9aec6f0e4076

      New changeset c17bc2026145 by Victor Stinner in branch 'default':
      Issue bpo-3080: imp.new_module() uses Unicode
      http://hg.python.org/cpython/rev/c17bc2026145

      New changeset c4361bab6914 by Victor Stinner in branch 'default':
      Issue bpo-3080: Use repr() to format the module name on error
      http://hg.python.org/cpython/rev/c4361bab6914

      New changeset 80f4bd647695 by Victor Stinner in branch 'default':
      Issue bpo-3080: Add PyImport_ImportModuleLevelObject() function
      http://hg.python.org/cpython/rev/80f4bd647695

      New changeset cc7c0f6f60bf by Victor Stinner in branch 'default':
      Issue bpo-3080: skip test_bdist_rpm if sys.executable is not encodable to UTF-8
      http://hg.python.org/cpython/rev/cc7c0f6f60bf

    34. python-dev commented on Mar 20, 2011

      python-devmannequin
      Mannequin

      New changeset f8d6f6797909 by Victor Stinner in branch 'default':
      Issue bpo-3080: Fix case_ok() using case_bytes()
      http://hg.python.org/cpython/rev/f8d6f6797909

    35. python-dev commented on Mar 20, 2011

      python-devmannequin
      Mannequin

      New changeset dc38c4d65cd9 by Victor Stinner in branch 'default':
      Issue bpo-3080: Fix call to case_ok() in find_init_module()
      http://hg.python.org/cpython/rev/dc38c4d65cd9

    36. vstinner commented on Mar 20, 2011

      @vstinner
      Member

      http://www.python.org/dev/buildbot/all/builders/PPC%20Tiger%203.x/builds/1599/steps/test/logs/stdio

      ======================================================================
      ERROR: testImpWrapper (test.test_importhooks.ImportHooksTestCase)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
        File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/test_importhooks.py", line 239, in testImpWrapper
          m = __import__(mname, globals(), locals(), ["__dummy__"])
        File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/test_importhooks.py", line 132, in load_module
          mod = imp.load_module(fullname, self.file, self.filename, self.stuff)
        File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/distutils/core.py", line 19, in <module>
          from distutils.cmd import Command
        File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/test_importhooks.py", line 132, in load_module
          mod = imp.load_module(fullname, self.file, self.filename, self.stuff)
        File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/distutils/cmd.py", line 11, in <module>
          from distutils import util, dir_util, file_util, archive_util, dep_util
        File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/test_importhooks.py", line 132, in load_module
          mod = imp.load_module(fullname, self.file, self.filename, self.stuff)
        File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/distutils/dir_util.py", line 8, in <module>
          import errno
        File "/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/test_importhooks.py", line 132, in load_module
          mod = imp.load_module(fullname, self.file, self.filename, self.stuff)
      TypeError: 'NoneType' object is not iterable

    37. vstinner commented on Mar 20, 2011

      @vstinner
      Member
      mod = imp.load_module(fullname, self.file, self.filename, self.stuff)
      

      TypeError: 'NoneType' object is not iterable

      The problem is that imp.find_module() now returns None as the filename, but imp.load_module() doesn't support None.

    38. merwok commented on Mar 20, 2011

      @merwok
      Member

      Attached patch fixes a typo in Doc/c-api/import.rst. You can merge it in your next commit.

    39. python-dev commented on Mar 20, 2011

      python-devmannequin
      Mannequin

      New changeset 7f4a4e393058 by Victor Stinner in branch 'default':
      Issue bpo-3080: imp.load_module() accepts None for the module path
      http://hg.python.org/cpython/rev/7f4a4e393058

    40. vstinner commented on Mar 20, 2011

      @vstinner
      Member

      Ok. Python 3.3 does now support non-ASCII characters in module paths and names on Windows, but only characters encodable to the ANSI code page. To support the full Unicode range, we should remove all calls to PyUnicode_EncodeFSDefault() on Windows:

      a) parse_source_module() has to encode the filename because the parser has no function expecting a filename as a Python object. It uses currently PyParser_ASTFromFile().

      b) write_compiled_module() encodes the filename to call open_exclusive(). I don't know how to implement open_exclusive() for Windows using Unicode filename: open() expects the filename as a byte string. Can we use _Py_fopen() (_wfopen)? Or do you need the O_EXCL flag?

      c) _PyImport_LoadDynamicModule() encodes the filename for _PyImport_GetDynLoadFunc(). The prototype should be changed, but only on Windows, to accept a filename as a Unicode string.

      Issue bpo-10785 is the right fix to (a). When bpo-10785 will be fixed, it will be easier to fix bpo-9319 crash.

    41. vstinner commented on Mar 20, 2011

      @vstinner
      Member

      c) _PyImport_LoadDynamicModule() encodes the filename for
      _PyImport_GetDynLoadFunc(). The prototype should be changed,
      but only on Windows, to accept a filename as a Unicode string.

      Hum, the difficult part is to use Unicode in _PyImport_GetDynLoadFunc() for:

      hDLL = LoadLibraryEx(pathname, NULL, LOAD_WITH_ALTERED_SEARCH_PATH);

      There is a LoadLibraryW() function, but it doesn't have a flag argument. And I suppose that the LOAD_WITH_ALTERED_SEARCH_PATH option is important.

    42. vstinner commented on Mar 21, 2011

      @vstinner
      Member

      Ok, I think that the most important part is now implemented in Python 3.3: use Unicode for module names and paths in the import machinery. Remaing parts are specific to Windows, and so I opened a new issue: bpo-11619. Let's close this 3 years old issue.

    43. python-dev commented on Mar 21, 2011

      python-devmannequin
      Mannequin

      New changeset ee4e780a6b7a by Éric Araujo in branch 'default':
      Fix a typo (see bpo-3080)
      http://hg.python.org/cpython/rev/ee4e780a6b7a

    44. asvetlov commented on Mar 21, 2011

      @asvetlov
      Contributor

      As I see Victor has dropped OS/2 support from Python/import.c
      Perhaps file Python/dynload_os2.c should be removed also.
      Not sure about other dynload_* files.

    45. vstinner commented on Mar 21, 2011

      @vstinner
      Member

      As I see Victor has dropped OS/2 support from Python/import.c
      Perhaps file Python/dynload_os2.c should be removed also.
      Not sure about other dynload_* files.

      340f76a6a792 just removes few lines in import.c: they can easily be rewritten. And this commit doesn't drop completly the support of OS/2 from the import machinery, as you wrote: dynload_os2.c still exists.

      If we drop completly the support of OS/2, it should be done completly using a PEP (I don't remember its number), and it should be discussed. At least with Andrew I MacIntyre :-)

    46. asvetlov commented on Mar 21, 2011

      @asvetlov
      Contributor

      Understood. Sorry.
      I thought Python support only Windows and posix (Linux, BSD, MacOSX etc) systems now, all other OSes are not maintained.

      Anyway please don't care about that.

    47. python-dev commented on Mar 22, 2011

      python-devmannequin
      Mannequin

      New changeset 15f9eca5e956 by Victor Stinner in branch 'default':
      Issue bpo-3080: On DJGPP, case_bytes() returns -1 to signal an error if the file
      http://hg.python.org/cpython/rev/15f9eca5e956

    48. vstinner commented on Mar 23, 2011

      @vstinner
      Member

      test the fixed nosy list

    49. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions