Skip to content

time.strftime() and Unicode characters on Windows #52551

Closed
#125193
Closed
time.strftime() and Unicode characters on Windows#52551
#125193
Assignees
serhiy-storchaka
Labels
3.12only security fixes3.13only security fixes3.14bugs and security fixesOS-windowsextension-modulesC modules in the Modules dirstdlibStandard Library Python modules in the Lib/ directorytopic-unicodetype-bugAn unexpected behavior, bug, or error

Description

@AndiDogold
AndiDogold
mannequin
opened on Apr 3, 2010
BPO 8304
Nosy @terryjreedy, @pfmoore, @abalkin, @vstinner, @ericvsmith, @tjguk, @ezio-melotti, @shimizukawa, @zware, @eryksun, @zooba

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2010-04-03.15:08:42.673>
labels = ['3.8', '3.9', 'extension-modules', 'expert-unicode', 'type-bug', '3.10', 'library', 'OS-windows']
title = 'time.strftime() and Unicode characters on Windows'
updated_at = <Date 2021-03-08.19:17:29.084>
user = 'https://bugs.python.org/AndiDogold'

bugs.python.org fields:

activity = <Date 2021-03-08.19:17:29.084>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules', 'Library (Lib)', 'Unicode', 'Windows']
creation = <Date 2010-04-03.15:08:42.673>
creator = 'AndiDog_old'
dependencies = []
files = []
hgrepos = []
issue_num = 8304
keywords = []
message_count = 16.0
messages = ['102269', '102298', '102310', '102332', '102335', '159341', '222667', '226114', '251554', '251558', '251560', '255043', '255133', '388241', '388277', '388286']
nosy_count = 12.0
nosy_names = ['terry.reedy', 'paul.moore', 'belopolsky', 'vstinner', 'eric.smith', 'tim.golden', 'ezio.melotti', 'AndiDog_old', 'shimizukawa', 'zach.ware', 'eryksun', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8304'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

Linked PRs

Activity

  1. AndiDogold commented on Apr 3, 2010

    @AndiDogold
    AndiDogoldmannequin
    on Apr 3, 2010
    MannequinAuthor

    There is inconsistent behavior in time.strftime, comparing Python 2.6 and 3.1. In 3.1, non-ASCII Unicode characters seem to get dropped whereas in 2.6 you can keep them using the necessary Unicode-to-UTF8 workaround.

    This should be fixed if it isn't intended behavior.

    Python 2.6

    >>> time.strftime(u"%d\u200F%A".encode("utf-8"), time.gmtime()).decode("utf-8")
u'03\u200fSaturday'
>>> time.strftime(u"%d\u0041%A".encode("utf-8"), time.gmtime()).decode("utf-8")
u'03ASaturday'

    Python 3.1

    >>> time.strftime("%d\u200F%A", time.gmtime())
''
>>> len(time.strftime("%d\u200F%A", time.gmtime()))
0
>>> time.strftime("%d\u0041%A", time.gmtime())
'03ASaturday'
  2. AndiDogold
    added
    stdlibStandard Library Python modules in the Lib/ directory
    topic-unicode
    type-bugAn unexpected behavior, bug, or error
    on Apr 3, 2010

  3. ezio-melotti commented on Apr 3, 2010

    @ezio-melotti
    ezio-melotti
    on Apr 3, 2010
    Member

    This seems to be fixed now, on both 3.1 and 3.2.
    Can you try with 3.1.2 and see if it works?
    What operating system are you using?

  4. ezio-melotti commented on Apr 4, 2010

    @ezio-melotti
    ezio-melotti
    on Apr 4, 2010
    Member

    Actually the bug seems related to Windows.

  5. ezio-melotti
    added
    OS-windows
    on Apr 4, 2010

  6. AndiDogold commented on Apr 4, 2010

    @AndiDogold
    AndiDogoldmannequin
    on Apr 4, 2010
    MannequinAuthor

    Just installed Python 3.1.2, same problem. I'm using Windows XP SP2 with two Python installations (2.6.4 and now 3.1.2).

  7. AndiDogold commented on Apr 4, 2010

    @AndiDogold
    AndiDogoldmannequin
    on Apr 4, 2010
    MannequinAuthor

    Definitely a Windows problem. I did this on Visual Studio 2008:

    wchar_t out[1000];
time_t currentTime;
time(&currentTime);
tm *timeStruct = gmtime(&currentTime);

        size_t ret = wcsftime(out, 1000, L"%d%A", timeStruct);
    wprintf(L"ret = %d, out = (%s)\n", ret, out);
        ret = wcsftime(out, 1000, L"%d\u200f%A", timeStruct);
    wprintf(L"ret = %d, out = (%s)\n", ret, out);

    and the output was

        ret = 8, out = (04Sunday)
    ret = 0, out = ()

    Python really shouldn't use any so-called standard functions on Windows. They never work as expected ^^...

  8. vstinner commented on Apr 25, 2012

    @vstinner
    vstinner
    on Apr 25, 2012
    Member

    Actually the bug seems related to Windows.

    See also the issue bpo-10653: wcsftime() doesn't format correctly time zones, so Python 3 uses strftime() instead.

  9. BreamoreBoy commented on Jul 10, 2014

    @BreamoreBoy
    BreamoreBoymannequin
    on Jul 10, 2014
    Mannequin

    Using 3.4.1 and 3.5.0 I get:-

    time.strftime("%d\u200F%A", time.gmtime())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'locale' codec can't encode character '\u200f' in position 2: Illegal byte sequence

  10. terryjreedy commented on Aug 30, 2014

    @terryjreedy
    terryjreedy
    on Aug 30, 2014
    Member

    I verified Marks 3.4.1 result with Idle.

    It strikes me as a bug that a function that maps a unicode format string to a unicode string with interpolations added should ever encode the format to bytes, lets alone using using an encoding that fails or loses information. It is especially weird given that % formatting does not even work (at present) for bytes.

    It seems to me that strftime should never encode the non-special parts of the format text. Instead, it could split the format (re.split) into a list of alternatine '%x' pairs and running text segments, replace the '%x' entries with the proper entries, and return the list joined back into a string. Some replacements would be locale dependent, other not.

    (Just wondering, are the locate names of days and months bytes restricted to ascii or unrestricted unicode using native characters?)

  11. vstinner
    changed the title [-]strftime and Unicode characters[/-] [+]time.strftime() and Unicode characters on Windows[/+] on Oct 1, 2014

  12. BreamoreBoy commented on Sep 24, 2015

    @BreamoreBoy
    BreamoreBoymannequin
    on Sep 24, 2015
    Mannequin

    @alexander what is you take on this please? I can confirm that it is still a problem on Windows in 3.5.0.

  13. abalkin commented on Sep 25, 2015

    @abalkin
    abalkin
    on Sep 25, 2015
    Member

    Mark, I am no expert on Windows. I believe Victor is most knowledgable in this area.

  14. 22 remaining items

  15. serhiy-storchaka
    added a commit that references this issue on Oct 17, 2024

    gh-52551: Fix encoding issues in strftime() (GH-125193)

    ad3eac1
  16. serhiy-storchaka
    closed this as completedin #125193on Oct 17, 2024
  17. github-project-automation
    moved this to Done in Date and time issues 🕰️on Oct 17, 2024
  18. serhiy-storchaka
    added a commit that references this issue on Oct 17, 2024

    [3.13] pythongh-52551: Fix encoding issues in strftime() (pythonGH-12…

    13ef787
  19. serhiy-storchaka
    reopened this on Oct 17, 2024
  20. serhiy-storchaka
    added a commit that references this issue on Oct 17, 2024

    pythongh-52551: Fix encoding issues in strftime() (pythonGH-125193)

    4019173

  21. serhiy-storchaka commented on Oct 17, 2024

    @serhiy-storchaka
    serhiy-storchaka
    on Oct 17, 2024
    Member

    Python's time.strftime() now passes only ASCII strings to C's strftime(), so there are no encoding errors. But the result of strftime() still can contain non-ASCII data and it need to be decoded. I think that switching to wcsftime() would make decoding less prone to locale misconfiguration errors.

  22. serhiy-storchaka
    added 2 commits that reference this issue on Oct 17, 2024

    pythongh-52551: Use wcsftime() to implement time.strftime() on Windows

    629ff02

    [3.13] gh-52551: Fix encoding issues in strftime() (GH-125193) (GH-12…

    08ccbb9
  23. miss-islington
    added a commit that references this issue on Oct 17, 2024

    [3.13] pythongh-52551: Fix encoding issues in strftime() (pythonGH-12…

    a34caba
  24. serhiy-storchaka
    added 2 commits that reference this issue on Oct 17, 2024

    [3.12] gh-52551: Fix encoding issues in strftime() (GH-125193) (GH-12…

    232b303

    gh-52551: Use wcsftime() to implement time.strftime() on Windows (GH-…

    a7443a1
  25. serhiy-storchaka
    closed this as completedon Oct 19, 2024
  26. ebonnal
    added 2 commits that reference this issue on Jan 12, 2025

    pythongh-52551: Fix encoding issues in strftime() (pythonGH-125193)

    3a7c158

    pythongh-52551: Use wcsftime() to implement time.strftime() on Windows (

    2cf7336
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.12only security fixes3.13only security fixes3.14bugs and security fixesOS-windowsextension-modulesC modules in the Modules dirstdlibStandard Library Python modules in the Lib/ directorytopic-unicodetype-bugAn unexpected behavior, bug, or error

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions