Skip to content

Fixed an issue decoding Dataclass and TypedDict instances - #1126

Open
mishmish66 wants to merge 1 commit into
msgspec:mainfrom
mishmish66:main
Open

Fixed an issue decoding Dataclass and TypedDict instances#1126
mishmish66 wants to merge 1 commit into
msgspec:mainfrom
mishmish66:main

Conversation

@mishmish66

@mishmish66 mishmish66 commented Jul 19, 2026

Copy link
Copy Markdown

To reproduce try this snippet (in a fresh process):

from dataclasses import dataclass
import msgspec

@dataclass
class Foo:
    π: float
    μ: int

foo = msgspec.json.decode(b'{"\xcf\x80":3.14,"\xce\xbc":0}', type=Foo)
print(foo)

Can't decode π because it is unicode and hasn't been loaded into the unicode cache yet

But this:

from dataclasses import dataclass
import msgspec

@dataclass
class Foo:
    π: float
    μ: int

print(msgspec.json.encode(Foo(3.1416, 1)))
foo = msgspec.json.decode(b'{"\xcf\x80":3.14,"\xce\xbc":0}', type=Foo)
print(foo)

works fine since the cache is populated during encoding.
This PR should have a fix for this and also a test that fails the old unfixed version.

@mishmish66
Added a test for issue with Dataclass and TypedDict classes where…
47a4a56 
… unicode fields wouldn't decode properly since the nocheck path was run without cached unicode values.
provinzkraut
provinzkraut reviewed Jul 19, 2026
View reviewed changes

@provinzkraut provinzkraut left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhm. I'm not sure if this is the best way to fix the issue. Invoking PyUnicode_AsUTF8 purely for its side effect doesn't seem to great. We're not doing this elsewhere for UTF8 either, so I would suggest not introducing this pattern for this particular path only.

Have you checked if other types (Struct, NamedTuple, attrs classes) are also affected? If not, dataclasses / TypedDicts should handle this the same way these do.

@Siyet

Siyet commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Pls run each of these tests in a fresh subprocess. If an earlier test encodes a dataclass with π and μ fields, both new tests pass even without this fix. The UTF-8 representation of those field names is already cached, and gc.collect() does not clear it. Running each test in a fresh process would prevent earlier tests from affecting the result.

This branch had an error being deployed

1 failed deployment
docs-preview — 47a4a565 Deployed Jul 19, 2026 by mishmish66 via Deploy preview #347
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@provinzkraut provinzkraut provinzkraut left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mishmish66 @Siyet @provinzkraut