Repository navigation
Some data (groups and scalars) stored as pickles in H5 #1184
Description
Activity
The fix is too involved for a bug fix release, so moving to 0.36. The difficulty will be to allow (and test!) loading the old format for backward compatibility. Forward compatibility (i.e. the new format could be read using old code) would be nice but I do not think it is entirely possible (e.g. for scalar groups). Forward compatibility is not worth too much effort at this point IMO.
FWIW, this is "very high priority" not because of the loss of efficiency, the loss of disk space, or the annoying warnings. Those problems are ugly, embarrassing and annoying but frankly low priority BUT the ticking time bomb nature of pickled data (see #1178) DOES make this very high priority (because this will make old files unreadable at some point in the future)
- changed the title
[-]Some groups still stored as pickles in H5[/-][+]Some data (groups and scalars) are stored as pickles in H5[/+]on Apr 3, 2026 - changed the title
[-]Some data (groups and scalars) are stored as pickles in H5[/-][+]Some data (groups and scalars) stored as pickles in H5[/+]on Apr 3, 2026
e.g. demo_python/.../baseline_parameters.h5
Pickles are a mess (see #1178) and make some H5 files much larger than necessary (see #788)
Btw, the test for groups introduced in #789 does not test what it should test (the slice object itself is pickled)