Skip to content

Fix problems in to_mdtraj and notebook tests - #776

Merged
dwhswenson merged 10 commits into
openpathsampling:masterfrom
dwhswenson:fix_to_mdtraj
May 30, 2018
Merged

dwhswenson merged 10 commits into
openpathsampling:masterfrom
dwhswenson:fix_to_mdtraj

Conversation

@dwhswenson

@dwhswenson dwhswenson commented May 28, 2018 •

Copy link
Copy Markdown
Member

A recent update to netcdf seems to be causing errors when we use to_mdtraj. This PR aims to fix that.

UPDATE: this PR will fix several problems. They are:

  • netcdf returns np.maskedarray by default, which doesn't seem to play nicely with simtk.unit (and was causing problems in our to_mdtraj); using set_auto_mask(False) fixes
  • something (probably pandas) changed such that there were problems in Py3k runs of the toy MSTIS analysis notebook; updating the notebook fixes
  • something (probably numpy) changed the default output, making strict comparisons in several notebooks incorrect in Py 2.7; updating the notebooks fixes (with forced formatting for some cells)

@dwhswenson dwhswenson changed the title Fix problem in to_mdtraj [WIP] Fix problem in to_mdtraj May 28, 2018
@dwhswenson dwhswenson added the bugfix PRs fixing bugs label May 28, 2018
@sroet

sroet commented May 28, 2018 •

Copy link
Copy Markdown
Member

This is actually happening due to an incompatibility between previous nc files and this netcdf4 version (might be a genuine upstream issue):
The following code is runs fine when fully done in a single netcdf version:

import openpathsampling as paths
store = paths.Storage('test.nc','w')
frame = paths.engines.openmm.snapshot_from_pdb('resources/AD_initial_frame.pdb') #In the examples dir
traj = paths.Trajectory([frame,frame])
store.save(traj)
store.close()
#update netcdf from 1.3 to 1.4 here to break
import openpathsampling as paths
store = paths.Storage('test.nc','r')
traj = store.trajectories[0]
print(traj.box_vectors)
print(type(traj.box_vectors))
traj.to_mdtraj()

and the print statements return

[[[2.558 0.    0.   ]
  [0.    2.558 0.   ]
  [0.    0.    2.558]]

 [[2.558 0.    0.   ]
  [0.    2.558 0.   ]
  [0.    0.    2.558]]] nm
<class 'simtk.unit.quantity.Quantity'>

however if the storage is saved in version 1.3 and loaded in version 1.4 the code breaks and the print statements are

[Quantity(value=masked_array(
   data=[[2.558000087738037, 0.0, 0.0],
         [0.0, 2.558000087738037, 0.0],
         [0.0, 0.0, 2.558000087738037]],
   mask=[[False, False, False],
         [False, False, False],
         [False, False, False]],
   fill_value=9.96921e+36,
   dtype=float32), unit=nanometer), Quantity(value=masked_array(
   data=[[2.558000087738037, 0.0, 0.0],
         [0.0, 2.558000087738037, 0.0],
         [0.0, 0.0, 2.558000087738037]],
   mask=[[False, False, False],
         [False, False, False],
         [False, False, False]],
   fill_value=9.96921e+36,
   dtype=float32), unit=nanometer)]
<class 'list'>

@dwhswenson

Copy link
Copy Markdown
Member Author

Seems like they're aware of the possibility that this could break existing code: Unidata/netcdf4-python#787 (comment). At least the fix for that was easy, although it's yet another time that a minor release of netCDF broke our code.

There are still some other errors in these runs. I'll try to track them down before merging this.

Looks like recent pandas changes were causing problems in Py3
@dwhswenson dwhswenson changed the title [WIP] Fix problem in to_mdtraj [WIP] Fix problems in to_mdtraj and notebook tests May 29, 2018
@dwhswenson dwhswenson changed the title [WIP] Fix problems in to_mdtraj and notebook tests Fix problems in to_mdtraj and notebook tests May 30, 2018
@dwhswenson
dwhswenson requested a review from jhprinz May 30, 2018 10:48
@dwhswenson

Copy link
Copy Markdown
Member Author

This is ready for review. Because the changes are small (no new features; only updates so tests still pass after upstream changes) and because they are important (tests are failing until this is merged, especially affecting users who upgrade netCDF), I will leave this up for review for 24 hours. After that, I'll merge it myself.

@jhprinz

jhprinz commented May 30, 2018

Copy link
Copy Markdown
Contributor

Looks fine from my limited quick view. Please go ahead and merge!

@dwhswenson
dwhswenson merged commit 7a948c5 into openpathsampling:master May 30, 2018
@dwhswenson
dwhswenson deleted the fix_to_mdtraj branch May 30, 2018 10:55
@dwhswenson dwhswenson mentioned this pull request Jun 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants