Skip to content

run_until_decorrelated();TypeError: iteration over a 0-d array #1115

Description

@WenHuiShen-Bio

1_traj_prepair_mstis.zip
When I run ops with the trajectory file I generated, an error occurs at equilibration.run_until_decorrelated(). The attachment is the corresponding code and error message.
thanks

Activity

  1. dwhswenson commented on Jul 24, 2022

    @dwhswenson
    Member

    Thanks for the report. Without the full setup, I can't reproduce it, and I don't recall seeing this specific error before. So I'm doing a little guesswork here. What I can determine from the error message: The error happens when you are trying to save a result. In particular, it seems to be a problem with saving the results of your CV.

    My recommendations:

    1. Use the new "experimental" storage and CVs. The new versions are listed as experimental, but have been largely in place and reasonably stable for a couple years — the new storage is much faster than the old storage, and will replace it when we release OPS 2.0.
    2. Make sure you've installed OPS from the current master. There are changes that haven't made it into a release yet. That specifically may include bugfixes in storage.

    If you do both of these and there's still an error (it would be a different error, but perhaps still an error) then we can try to debug in more detail.

    Switching to new storage: This basically involves running a special "monkey patch" command, and getting certain objects (Storage and CVs) from the openpathsampling.experimental.storage subpackage. A few resources to help you switch over to new storage/new CVs:

    Hope this helps, and let me know if this fixes or if you run into further problems!

  2. WenHuiShen-Bio commented on Jul 30, 2022

    @WenHuiShen-Bio
    Author

    Thank you very much for your reply!
    Now, i have installed OPS from the current master. I tried two ways to load my gromacs' trr trajectories, but both failed.The first way of loading trajectories took me ten hours, which seems too slow.
    All initial files can be obtained from here : https://drive.google.com/file/d/1ZscRc11vVOxezYb7y5_Q1EC1n-B6Xjxh/view?usp=sharing , where the gromacs trr trajectory file is converted from the lammps trajectory file.

    • The first way I load the trajectories is:

    `from tqdm import trange

    trr_file = "traj.trr"
    gro_file = "p.gro"
    n_frames = len(md.load(trr_file, top=gro_file))
    external_traj = paths.Trajectory([equil_engine.read_frame_from_file(trr_file, num) for num in trange(n_frames)])
    #define states
    rmsd_cv = MDTrajFunctionCV(func=md.rmsd, topology=template.topology, reference=equil_engine.gro, frame=0).named("rmsd_cv")
    rg_cv = MDTrajFunctionCV(func=md.compute_rg, topology=template.topology).named("rg_cv")
    def circle(snapshot, center, rmsd_cv, rg_cv):
    import math
    return math.sqrt((rmsd_cv(snapshot)-center[0])**2
    + (rg_cv(snapshot)-center[1])**2)
    opA = paths.CoordinateFunctionCV(name="opA", f=circle, rmsd_cv=rmsd_cv, rg_cv=rg_cv, center=[0.25, 0.8])
    opB = paths.CoordinateFunctionCV(name="opB", f=circle, rmsd_cv=rmsd_cv, rg_cv=rg_cv, center=[0.5, 0.8])
    opC = paths.CoordinateFunctionCV(name="opC", f=circle, rmsd_cv=rmsd_cv, rg_cv=rg_cv, center=[0.65, 0.73])

    stateA = paths.CVDefinedVolume(opA, 0.0, 0.01)
    stateB = paths.CVDefinedVolume(opB, 0.0, 0.01)
    stateC = paths.CVDefinedVolume(opC, 0.0, 0.01)

    interfacesA = paths.VolumeInterfaceSet(opA, 0.0, [0.01, 0.02, 0.03])
    interfacesB = paths.VolumeInterfaceSet(opB, 0.0, [0.01, 0.02, 0.03])
    interfacesC = paths.VolumeInterfaceSet(opC, 0.0, [0.01, 0.02, 0.03])
    ms_outers = paths.MSOuterTISInterface.from_lambdas(
    {ifaces: 0.05
    for ifaces in [interfacesA, interfacesB, interfacesC]}
    )
    mstis = paths.MSTISNetwork(
    [(stateA, interfacesA),
    (stateB, interfacesB),
    (stateC, interfacesC)],
    ms_outers=ms_outers
    ).named('mstis')
    scheme = paths.DefaultScheme(mstis, engine=equil_engine).named("scheme")
    tmp_network = paths.TPSNetwork.from_states_all_to_all([stateA, stateB, stateC])`

    When i run below:

    subtrajectories = [] for ens in tmp_network.analysis_ensembles: subtrajectories += ens.split(external_traj) print(subtrajectories)

    I got error :
    `---------------------------------------------------------------------------
    AttributeError Traceback (most recent call last)
    in
    1 subtrajectories = []
    2 for ens in tmp_network.analysis_ensembles:
    ----> 3 subtrajectories += ens.split(external_traj)
    4 print(subtrajectories)

    ~/whshen/anaconda3/lib/python3.8/site-packages/openpathsampling/ensemble.py in split(self, trajectory, max_length, min_length, overlap, reverse, n_results)
    790 for part in itertools.islice(indices, n_results)]
    791 else:
    --> 792 return [trajectory[part] for part in indices]
    793
    794 @Property

    ~/whshen/anaconda3/lib/python3.8/site-packages/openpathsampling/ensemble.py in (.0)
    790 for part in itertools.islice(indices, n_results)]
    791 else:
    --> 792 return [trajectory[part] for part in indices]
    793
    794 @Property

    ~/whshen/anaconda3/lib/python3.8/site-packages/openpathsampling/ensemble.py in iter_valid_slices(self, trajectory, max_length, min_length, overlap, reverse)
    462 can_append_tt = self.strict_can_append(tt)
    463 else:
    ...
    405 for val in values]

    mdtraj/rmsd/_rmsd.pyx in mdtraj._rmsd.rmsd()

    AttributeError: 'str' object has no attribute 'xyz'`

    • The sencond way I load the trajectories is:

    mdt = md.load("traj.trr", top="p.gro") trr = md.formats.TRRTrajectoryFile("traj.trr") vel = trr._read(n_frames=len(mdt), atom_indices=None, get_velocities=True)[5] external_traj = trajectory_from_mdtraj(mdt, velocities=vel)

    This gives me this error:
    --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-27-d5a6b13c2609> in <module> 5 vel = trr._read(n_frames=len(mdt), atom_indices=None, get_velocities=True)[5] 6 # # combine mdtandvel` in an OPS trajectory
    ----> 7 external_traj = trajectory_from_mdtraj(mdt, velocities=vel)
    8 # external_traj = ops_load_trajectory("traj.trr", top="p.gro")
    9 # vel1.shape

    ~/whshen/anaconda3/lib/python3.8/site-packages/openpathsampling/engines/openmm/tools.py in trajectory_from_mdtraj(mdtrajectory, simple_topology, velocities)
    234 topology = Topology(*mdtrajectory.xyz[0].shape)
    235 else:
    --> 236 topology = MDTrajTopology(mdtrajectory.topology)
    237
    238 if velocities is None:

    NameError: name 'MDTrajTopology' is not defined`

    Maybe because I am not very familiar with some ops modules, and some of my codes may be wrong, so I bother you again, and your reply has also benefited me a lot, much appreciated!

  3. dwhswenson commented on Aug 1, 2022

    @dwhswenson
    Member

    OPS snapshots depend on the engine the engine they're to be used for (although MD snapshots are mostly interoperable). The first approach you give is for going from TRR file to a trajectory for OpenMM: you get the error about MDTrajTopology because you don't have OpenMM installed. The second approach is correct for Gromacs, although I hadn't realized how long it would take for such a long input trajectory. That's a performance issue for me to solve.

    Here's a workaround that quickly loads a trr, after defining n_frames and trr_file as you did in the second version -- however, it won't really solve your problem, only delay it:

    external_traj = paths.Trajectory([
        equil_engine.SnapshotClass(trr_file, num, equil_engine)
        for num in range(n_frames)
    ])

    The real issue here combines two facts:

    • Snapshots for the Gromacs engine are stored in external TRR files.
    • OPS needs to be able to treat each snapshot as an individual (because OPS trajectories will include snapshots from multiple TRR files as your path sampling progresses.)

    Loading coordinates/velocities one snapshot at a time is very slow, and that's what you're seeing. In fact, I suspect that the seek function we use from MDTraj may not actually be instantaneous, but might have a cost that scales linearly with the length of the trajectory. That would mean that loading all snapshots that way costs N^2 with the length of the trajectory -- that would explain the extremely bad performance. But doing the approach above to get your external_traj will only delay that slow loading (I think it hits when you try to calculate your CVs now). Once loaded, the snapshots keep their data, so you only need to load once -- but let's see if we can get it to take less than 10 hours.

    Here's an ugly workaround that should work for now, although I'll implement something to make this work in a more general way. Temporary workaround, assuming you've defined the MDTraj trajectory mdt and numpy array vel as you did in OpenMM-like "first way" to generate the trajectory, and that external_traj is generated from the workaround above:

    for snap, xyz, vel, box in zip(external_traj, mdt.xyz, vel, mdt.unitcell_vectors):
        snap._xyz = xyz
        snap._velocities = vel
        snap._box_vectors = box

    I notice that even here that calculating the MDTraj CVs don't go as quickly as I would expect (it takes a few minutes per CV to calculate for the whole trajectory; this is better than 10 hours, but it should be almost as fast as plain MDTraj). I think this means that something I thought I'd implemented as a performance improvement either isn't implemented, or is implemented in a way that doesn't actually help performance.

    So, my to-dos coming out of this:

    • Loading all information into a trajectory of external snapshots should be a simple function on the trajectory object. Things that need all the data from a trajectory (like CVs) should be able to call trajectory.load_all_data(), which should be a fast implementation of this (engine-specific).
    • Within MDTraj CVs, a single trajectory should be made for all snapshots that need to be calculated. The function from MDTraj should only be called once when an OPS trajectory is passed to it.

    When I get PRs for those posted, I'll @-mention you, @ShenWenHuibit -- if you can test it out and see if it helps you, that would be useful, but I'm hoping the workarounds above can at least let you proceed with your research.

    As for the AttributeError: your RMSD CV uses reference=equil_engine.gro, which is a string; it should be an MDTraj Trajectory. Easily fixed by wrapping as reference=md.load(equil_engine.gro). This is the source of the 'str' object has no attribute 'xyz' that you're getting -- MDTraj expects the reference to be its Trajectory object (which has an xyz attribute), but you're giving it a string instead of a trajectory.

  4. WenHuiShen-Bio commented on Aug 1, 2022

    @WenHuiShen-Bio
    Author

    Glad to hear back again, The process of importing trajectories as you suggested became very fast, taking only a dozen seconds, but I still have problems setting up the equilibration : TypeError: Object of type Trajectory is not JSON serializable , I don't know what's wrong here, and I don't understand what the variable snap does, where should it be called ? Excuse me again .
    The implementation is in the following pdf file :

    1_traj_prepair_mstis.pdf

  5. dwhswenson commented on Aug 1, 2022

    @dwhswenson
    Member

    Ah, that appears to be a very silly bug on my part. Basically, I have a way to directly turn MDTraj trajectories into something that can be saved by SimStore, and I tested it in isolation. But I forgot to connect that functionality to the rest of the storage system. I guess no one else has used RMSD as a CV (where you need to save the reference) within SimStore! PR to fix that coming very soon.

    As for the snap in the previous workaround -- that workaround just loops over all the snapshots in your trajectory. The first part of the workaround loaded them as empty references -- just the filename and frame number. The second part takes each snapshot snap and loads the data into the private variables it uses. Direct access to private variables is not a usually recommended practice, but this was just intended to be a quick workaround until I write code that does this more cleanly.

  6. WenHuiShen-Bio commented on Aug 3, 2022

    @WenHuiShen-Bio
    Author

    After I saved the setup.db file, I got the following error when using the openpathsampling command line openpathsampling equilibrate setup.db -o equil.db :

    Traceback (most recent call last):
    File "/home/xhshi/whshen/anaconda3/envs/opsmaster/bin/openpathsampling", line 33, in
    sys.exit(load_entry_point('openpathsampling-cli', 'console_scripts', 'openpathsampling')())
    File "/home/xhshi/whshen/anaconda3/envs/opsmaster/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 1130, in call
    return self.main(*args, **kwargs)
    File "/home/xhshi/whshen/anaconda3/envs/opsmaster/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
    File "/home/xhshi/whshen/anaconda3/envs/opsmaster/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/xhshi/whshen/anaconda3/envs/opsmaster/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
    File "/home/xhshi/whshen/anaconda3/envs/opsmaster/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-cli-main/paths_cli/commands/equilibrate.py", line 34, in equilibrate
    equilibrate_main(
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-cli-main/paths_cli/commands/equilibrate.py", line 53, in equilibrate_main
    simulation.run_until_decorrelated()
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathsimulators/path_sampling.py", line 245, in run_until_decorrelated
    self.run(1)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathsimulators/path_sampling.py", line 261, in run
    hook_state, mcstep = self.run_one_step(step_info, hook_state)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathsimulators/path_sampling.py", line 291, in run_one_step
    movepath = self._mover.move(self.sample_set, step=self.step)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathmover.py", line 2635, in move
    self.mover.move(sample_set),
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathmover.py", line 1605, in move
    subchange = mover.move(sample_set)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathmover.py", line 1605, in move
    subchange = mover.move(sample_set)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathmover.py", line 1605, in move
    subchange = mover.move(sample_set)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathmover.py", line 619, in move
    change = self.move_core(samples)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathmover.py", line 646, in move_core
    trials, call_details = self(*samples)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathmover.py", line 793, in call
    trial_trajectory, run_details = self._run(initial_trajectory,
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathmover.py", line 931, in _run
    trial_trajectory = self._make_backward_trajectory(
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathmover.py", line 900, in _make_backward_trajectory
    partial_trajectory = self.engine.generate(initial_snapshot,
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/engines/dynamics_engine.py", line 437, in generate
    for trajectory in it:
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/engines/dynamics_engine.py", line 627, in iter_generate
    stop = self.stop_conditions(trajectory=trajectory,
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/engines/dynamics_engine.py", line 385, in stop_conditions
    stop = (not condition(trajectory, trusted)) or stop
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 2392, in can_prepend
    return self._new_ensemble.can_prepend(self._alter(trajectory),
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 2392, in can_prepend
    return self._new_ensemble.can_prepend(self._alter(trajectory),
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 1386, in can_prepend
    return self._generalized_short_circuit(
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 1342, in _generalized_short_circuit
    a = f1(trajectory, trusted)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 2073, in can_prepend
    return self._generic_can_prepend(trajectory, trusted, strict=False)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 1971, in _generic_can_prepend
    subtraj_first = self._find_subtraj_first(
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 1704, in _find_subtraj_first
    while ((ens.can_prepend(subtraj, trusted=True) or
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 2392, in can_prepend
    return self._new_ensemble.can_prepend(self._alter(trajectory),
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 1386, in can_prepend
    return self._generalized_short_circuit(
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 1360, in _generalized_short_circuit
    b = f2(trajectory, trusted)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 2245, in can_prepend
    return self._trusted_call(trajectory, self._cache_can_prepend)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/ensemble.py", line 2227, in _trusted_call
    cache.contents['previous'] = self._volume(frame)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/volume.py", line 206, in call
    return not self.volume(snapshot)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/volume.py", line 137, in call
    a = self.volume1(snapshot)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/volume.py", line 424, in call
    l = self._get_cv_float(snapshot)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/volume.py", line 418, in _get_cv_float
    val = self.collectivevariable(snapshot)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/netcdfplus/chaindict.py", line 229, in getitem
    return self._post[items]
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/netcdfplus/chaindict.py", line 260, in getitem
    return self._post[[items]][0]
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/netcdfplus/chaindict.py", line 74, in getitem
    rep = list(self._post[nones])
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/netcdfplus/chaindict.py", line 67, in getitem
    results = self._get_list(items)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/netcdfplus/chaindict.py", line 365, in _get_list
    results = [self._eval(obj) for obj in items]
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/netcdfplus/chaindict.py", line 365, in
    results = [self._eval(obj) for obj in items]
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/collectivevariable.py", line 342, in _eval
    return self.cv_callable(items, **self.kwargs)
    File "/tmp/ipykernel_122005/1369615926.py", line 4, in circle
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/experimental/simstore/storable_functions.py", line 486, in call
    stage_results, missing = stage(missing)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/experimental/simstore/storable_functions.py", line 405, in _eval
    values = [self.func(item, **self.kwargs) for item in preprocessed]
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/experimental/simstore/storable_functions.py", line 405, in
    values = [self.func(item, **self.kwargs) for item in preprocessed]
    File "mdtraj/rmsd/_rmsd.pyx", line 175, in mdtraj._rmsd.rmsd
    File "stringsource", line 658, in View.MemoryView.memoryview_cwrapper
    File "stringsource", line 349, in View.MemoryView.memoryview.cinit
    ValueError: buffer source array is read-only

    Then when I try to run the command with notebook: equilibration.run_until_decorrelated(), it give me error : RuntimeError: External engine died unexpectedly
    The implementation is in the following pdf file :
    1_traj_prepair_mstis.pdf

  7. dwhswenson commented on Aug 4, 2022

    @dwhswenson
    Member

    I'm able to reproduce this last error in MDTraj. My guess is that there's something incompatible in the interaction between OPS and MDTraj, specifically related to using RMSD as a CV. Mentioning this here to let you know it's under investigation -- as it happens, I may not be available for the next two days due to travel.

  8. dwhswenson commented on Aug 4, 2022

    @dwhswenson
    Member

    It looks like for some reason NumPy is expecting that your reference trajectory be writable. I don't think MDTraj actually writes to that (it really shouldn't) but this is probably NumPy being overly safe.

    I think the following workaround will fix (create a custom function that wraps MDTraj's RMSD by making a copy of the coordinates):

    def rmsd(traj, reference):
        """Both traj and reference are MDTraj trajectories"""
        import mdtraj as md
        ref = md.Trajectory(reference.xyz.copy(), reference.topology)
        return md.rmsd(traj, reference=ref, frame=0)
    
    rmsd_cv = MDTrajFunctionCV(func=rmsd, topology=template.topology, reference=md.load(equil_engine.gro)).named("rmsd_cv")

    It's a bit annoying since it requires copying the reference trajectory coordinates each time. I would recommended ensuring that reference is always a single-frame trajectory -- I assume it is, since it comes from a .gro file.

  9. WenHuiShen-Bio commented on Aug 5, 2022

    @WenHuiShen-Bio
    Author

    I tried it according to your idea, still can't pass the equilibrition, and I got other error:

    Traceback (most recent call last):
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/bin/openpathsampling", line 8, in
    sys.exit(main())
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/site-packages/click/core.py", line 1130, in call
    return self.main(*args, **kwargs)
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/site-packages/paths_cli/commands/equilibrate.py", line 33, in equilibrate
    equilibrate_main(
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/site-packages/paths_cli/commands/equilibrate.py", line 47, in equilibrate_main
    simulation = paths.PathSampling(
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/pathsimulators/path_sampling.py", line 84, in init
    self.storage.save([self.move_scheme, self.root_mover,
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/experimental/simstore/storage.py", line 313, in save
    storables_list = [serialize(o) for o in by_table[table].values()]
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/experimental/simstore/storage.py", line 313, in
    storables_list = [serialize(o) for o in by_table[table].values()]
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/experimental/simstore/custom_json.py", line 92, in simobj_serializer
    return self._sim_serialization.serializer(obj)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/experimental/simstore/custom_json.py", line 18, in serializer
    'json': self.json_encoder(obj)}
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/json/init.py", line 234, in dumps
    return cls(
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
    File "/home/xhshi/whshen/anaconda3/envs/ops-master/lib/python3.9/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/experimental/simstore/custom_json.py", line 105, in default
    result = coding_method.default(obj)
    File "/home/xhshi/whshen/OPS_code_test/openpathsampling-master/openpathsampling/experimental/simstore/callable_codec.py", line 97, in default
    root_mod = obj.module.split('.')[0]
    AttributeError: 'NoneType' object has no attribute 'split'

    And the result of running on notebook is very confusing, sometimes gromacs will run two log files and sometimes one, but both end up getting the error :RuntimeError: External engine died unexpectedly

  10. dwhswenson commented on Aug 8, 2022

    @dwhswenson
    Member

    I'm hoping to explore this in more detail today or tomorrow. But as a first check, can you try using the branch from #1118? I noticed that I was getting the External engine died unexpectedly error because of problems in the mdp file (basically, Gromacs is now warning on things it didn't used to warn on, so my maxwarn was no longer correct -- also, I have no idea why our example files use such bad thermostats/barostats.)

    If the problem is the grompp command, #1118 will give a clearer error.

  11. WenHuiShen-Bio commented on Aug 9, 2022

    @WenHuiShen-Bio
    Author

    I used this branch from #1118 and still got the same error : External engine died unexpectedly, and then I modified the mdp file so that the grompp program no longer generates the warning, but still got the same error(The mdp file I use here is from openpathsampling-master/examples/gromacs/md.mdp, because I only have protein in my box, so I modified Pcoupl = no and coulombtype = Cut-off to avoid warnings.) I don't see the error about grompp command, and I'm not sure if the current problem is related to mdp file.

    error message is below :

    RuntimeError                              Traceback (most recent call last)
    /home/xhshi/whshen/vscode/lammps_FDM/test_fdm/1LE3-16resi/data_analysis/ops/test/1_traj_prepair_mstis.ipynb Cell 44 in <cell line: 5>()
          <a href='vscode-notebook-cell://ssh-remote%2B10.2.1.240/home/xhshi/whshen/vscode/lammps_FDM/test_fdm/1LE3-16resi/data_analysis/ops/test/1_traj_prepair_mstis.ipynb#X60sdnNjb2RlLXJlbW90ZQ%3D%3D?line=0'>1</a> # !rm -rf ./initial_frame.trr
          <a href='vscode-notebook-cell://ssh-remote%2B10.2.1.240/home/xhshi/whshen/vscode/lammps_FDM/test_fdm/1LE3-16resi/data_analysis/ops/test/1_traj_prepair_mstis.ipynb#X60sdnNjb2RlLXJlbW90ZQ%3D%3D?line=1'>2</a> 
          <a href='vscode-notebook-cell://ssh-remote%2B10.2.1.240/home/xhshi/whshen/vscode/lammps_FDM/test_fdm/1LE3-16resi/data_analysis/ops/test/1_traj_prepair_mstis.ipynb#X60sdnNjb2RlLXJlbW90ZQ%3D%3D?line=2'>3</a> # NBVAL_SKIP
          <a href='vscode-notebook-cell://ssh-remote%2B10.2.1.240/home/xhshi/whshen/vscode/lammps_FDM/test_fdm/1LE3-16resi/data_analysis/ops/test/1_traj_prepair_mstis.ipynb#X60sdnNjb2RlLXJlbW90ZQ%3D%3D?line=3'>4</a> # don't run this during testing
    ----> <a href='vscode-notebook-cell://ssh-remote%2B10.2.1.240/home/xhshi/whshen/vscode/lammps_FDM/test_fdm/1LE3-16resi/data_analysis/ops/test/1_traj_prepair_mstis.ipynb#X60sdnNjb2RlLXJlbW90ZQ%3D%3D?line=4'>5</a> equilibration.run_until_decorrelated()
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathsimulators/path_sampling.py:245, in PathSampling.run_until_decorrelated(self, time_reversal)
        239     out_str = "Step {}: {} of {} trajectories still correlated\n"
        240     paths.tools.refresh_output(
        241         out_str.format(self.step + 1, to_decorrelate, len(originals)),
        242         refresh=False,
        243         output_stream=original_output_stream
        244     )
    --> 245     self.run(1)
        246     to_decorrelate = n_correlated(self.sample_set, originals)
        248 paths.tools.refresh_output(
        249     "Step {}: All trajectories decorrelated!\n".format(self.step+1),
        250     refresh=False,
        251     output_stream=original_output_stream
        252 )
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathsimulators/path_sampling.py:261, in PathSampling.run(self, n_steps)
        259 for nn in range(n_steps):
        260     step_info = nn, n_steps
    --> 261     hook_state, mcstep = self.run_one_step(step_info, hook_state)
        263 # after simulation hooks
        264 self.run_hooks('after_simulation', sim=self, hook_state=hook_state)
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathsimulators/path_sampling.py:291, in PathSampling.run_one_step(self, step_info, hook_state)
        289 # MCStep, i.e. actual sample move
        290 time_start = time.time()  # we time **only** the MCStep (no hooks!)
    --> 291 movepath = self._mover.move(self.sample_set, step=self.step)
        292 samples = movepath.results
        293 new_sampleset = self.sample_set.apply_samples(samples)
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathmover.py:2635, in PathSimulatorMover.move(self, sample_set, step)
       2629 def move(self, sample_set, step=-1):
       2630     details = Details(
       2631         step=step
       2632     )
       2634     return paths.PathSimulatorMoveChange(
    -> 2635         self.mover.move(sample_set),
       2636         mover=self,
       2637         details=details
       2638     )
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathmover.py:1605, in SelectionMover.move(self, sample_set)
       1603 weights = self._selector(sample_set)
       1604 mover, details = self.select_mover(weights)
    -> 1605 subchange = mover.move(sample_set)
       1607 path = paths.RandomChoiceMoveChange(
       1608     subchange=subchange,
       1609     mover=self,
       1610     details=details
       1611 )
       1613 return path
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathmover.py:1605, in SelectionMover.move(self, sample_set)
       1603 weights = self._selector(sample_set)
       1604 mover, details = self.select_mover(weights)
    -> 1605 subchange = mover.move(sample_set)
       1607 path = paths.RandomChoiceMoveChange(
       1608     subchange=subchange,
       1609     mover=self,
       1610     details=details
       1611 )
       1613 return path
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathmover.py:1605, in SelectionMover.move(self, sample_set)
       1603 weights = self._selector(sample_set)
       1604 mover, details = self.select_mover(weights)
    -> 1605 subchange = mover.move(sample_set)
       1607 path = paths.RandomChoiceMoveChange(
       1608     subchange=subchange,
       1609     mover=self,
       1610     details=details
       1611 )
       1613 return path
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathmover.py:619, in SampleMover.move(self, sample_set)
        617 def move(self, sample_set):
        618     samples = self.get_samples_from_sample_set(sample_set)
    --> 619     change = self.move_core(samples)
        620     return change
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathmover.py:646, in SampleMover.move_core(self, samples)
        639 # this is separated out for reuse and remove dependence core MC move
        640 # dependence on the entire sample set (for parallelization)
        641 try:
        642     # pass these samples to the trial move which might throw
        643     # engine-specific exceptions if something goes wrong.
        644     # Most common should be `EngineNaNError` if nan is detected and
        645     # `EngineMaxLengthError`
    --> 646     trials, call_details = self(*samples)
        648 except SampleNaNError as e:
        649     e.details.update({'rejection_reason': 'nan'})
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathmover.py:793, in EngineMover.__call__(self, input_sample)
        790 shooting_index = self.selector.pick(initial_trajectory)
        792 try:
    --> 793     trial_trajectory, run_details = self._run(initial_trajectory,
        794                                               shooting_index)
        796 except paths.engines.EngineNaNError as e:
        797     trial, details = self._build_sample(
        798         input_sample, shooting_index, e.last_trajectory, 'nan')
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathmover.py:927, in EngineMover._run(self, trajectory, shooting_index)
        920 logger.info(shoot_str.format(
        921     fnum=shooting_index,
        922     maxt=len(trajectory) - 1,
        923     sh_dir=self.direction
        924 ))
        926 if self.direction == "forward":
    --> 927     trial_trajectory = self._make_forward_trajectory(
        928         trajectory, shooting_index
        929     )
        930 elif self.direction == "backward":
        931     trial_trajectory = self._make_backward_trajectory(
        932         trajectory, shooting_index
        933     )
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/pathmover.py:887, in EngineMover._make_forward_trajectory(self, trajectory, shooting_index)
        883 initial_snapshot = trajectory[shooting_index]  # .copy()
        884 run_f = paths.PrefixTrajectoryEnsemble(self.target_ensemble,
        885                                        trajectory[0:shooting_index]
        886                                        ).can_append
    --> 887 partial_trajectory = self.engine.generate(initial_snapshot,
        888                                           running=[run_f])
        889 trial_trajectory = (trajectory[0:shooting_index] +
        890                     partial_trajectory)
        891 # TODO: this should check for overshoot; only works now if ensemble
        892 # doesn't overshoot
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/engines/dynamics_engine.py:437, in DynamicsEngine.generate(self, snapshot, running, direction)
        429 trajectory = None
        430 it = self.iter_generate(
        431     snapshot,
        432     running,
        433     direction,
        434     intervals=0,
        435     max_length=self.options['n_frames_max'])
    --> 437 for trajectory in it:
        438     pass
        440 return trajectory
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/engines/dynamics_engine.py:671, in DynamicsEngine.iter_generate(self, initial, running, direction, intervals, max_length)
        669     logger.info("Through frame: %d", len(trajectory))
        670     self._clear_snapshot_cache(snapshot)
    --> 671     raise final_error
        673 logger.info("Finished trajectory, length: %d", len(trajectory))
        674 yield trajectory
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/engines/dynamics_engine.py:562, in DynamicsEngine.iter_generate(self, initial, running, direction, intervals, max_length)
        560 try:
        561     with self.interrupter():
    --> 562         snapshot = self.generate_next_frame()
        564         # if self.on_nan != 'ignore' and \
        565         if not self.is_valid_snapshot(snapshot):
    
    File ~/whshen/OPS_code_test/openpathsampling-gmx-fast-fail-grompp/openpathsampling-gmx-fast-fail-grompp/openpathsampling/engines/external_engine.py:227, in ExternalEngine.generate_next_frame(self)
        225 elif next_frame is None:
        226     if self.proc.poll() is not None:
    --> 227         raise RuntimeError("External engine died unexpectedly")
        228     logger.debug("Sleeping for {:.2f}ms".format(self.sleep_ms))
        229     time.sleep(self.sleep_ms/1000.0)
    
    RuntimeError: External engine died unexpectedly
    
  12. dwhswenson commented on Aug 9, 2022

    @dwhswenson
    Member

    So, for the engine dying unexpectedly: Your mdp file's nsteps is not compatible with the n_frames_max parameter in the engine options.

    OPS "frames" are number of points saved by Gromacs, so total number if the gmx process was allowed to run until it completes would be nsteps / nstxout (and nstxout should equal nstvout). In your case, thats 10000 / 10 = 1000. The value of n_frames_max in the engine must be smaller than that, but you give 10000.

    The idea is that OPS should always be in charge of stopping the engine. That way it doesn't need to worry about why the external process died; if it dies for any reason other than "OPS told it to stop", then it is unexpected to OPS. In your case, it was "dying" because it stopped normally -- it got to the end of the trajectory.

    Note that you need to make sure the time is long enough to get trajectories that hit the states -- I tested that it worked better be reducing n_frames_max, but that wasn't getting accepted trajectories because the maximum length was too short. OPS stopped the trajectory at max length, which wasn't enough to hit a state.

    A couple of side comments:

    • You see a warning about filename_setter in the engine options. You might want to explicitly use 'filename_setter': RandomStringFilenames() (see #933 for discussion).
    • You currently set nstxout/nstvout to 10, meaning that you're writing every 20ps. This is useful transitions that are very fast (like the AD bond rotation. I haven't dug into the science of what you're doing, but this might mean you're writing more data than you need (more disk space, also slows Gromacs down). Typically, you want your output trajectories in TIS to be 50-1000 frames each (less than 10 is definitely bad, more than 5000 is probably more data than you need).
      If your input trajectory is a plain MD run at the same conditions, then the subtrajectories you're getting from that look like they're fine. But if it comes from high-temperature or other modified dynamics (e.g., metadynamics), those subtrajectories may be much shorter than what you'll find on equilibration.

    I'll try to take a look at the CLI issue later -- that looks like another problem with serialization.

  13. WenHuiShen-Bio commented on Aug 11, 2022

    @WenHuiShen-Bio
    Author

    That's right, it was a careless mistake I forgot to modify the n_frames_max, which caused gromacs to end unexpectedly. Now I set n_frames_max to 1000 and the program works fine. As per your suggestion I modified the simulation time in mdp (nsteps / nstxout = 100000/100) as there is no accepted trajectories. But this takes more time, and my remote server connection is not stable, so I haven't got the final sampled product after two disconnections. I'm going to submit a python script to the server to run in the background, and I'll see how it turns out later.

    Then I tried running the equilibration again with the CLI (openpathsampling equilibrate setup.db -o equil.db) and still got the error : AttributeError: 'NoneType' object has no attribute 'split'

    In fact, this part of the work simulates the folding process of the protein from the random state to the native state (pdb id: 1LE3, 16 amino acids). The initial trajectory I used here is obtained by modifying the potential function of the force field, so the sub-trajectories should be the same as Consistent with what you said, they are shorter compared to the equilibrium trajectories. It's not clear to me if unbiased trajectories can be obtained with the openpathsampling program, maybe a longer simulation time is needed here to test.

  14. dwhswenson commented on Aug 11, 2022

    @dwhswenson
    Member

    It looks like your CVs opA, opB, and opC are still using old-style CVs (paths.CoordinateFunctionCV). Use the CoordinateFunctionCV in openpathsampling.experimental.storage.collective_variables instead. This should fix the CLI issue. (You did catch the paths.InterfaceSet.simstore = True, which took me a few minutes to find when trying to debug this issue.)

    The equilibration stage in OPS should get you unbiased trajectories (assuming the biased initial trajectories are reasonably close to the unbiased ones), although you might want multiplier to be greater than 1 (which is the bare minimum). This the same as any "burn-in" process in Monte Carlo (or MD) -- you'll eventually get to something physical, but the closer your initial conditions are to something that is physical, the faster that happens.

    The current run_until_decorrelated isn't the most efficient decorrelation process; the more ensembles you have, the less efficient it is. However, its inefficiency just means that some ensembles may get more decorrelation than necessary, which isn't always bad. Adding a faster decorrelation process in on the to-do list, but not a super high priority right now.

  15. WenHuiShen-Bio commented on Aug 12, 2022

    @WenHuiShen-Bio
    Author

    After I use the new-style CVs (openpathsampling.experimental.storage.collective_variables.CoordinateFunctionCV), the CLI still gives me the error : AttributeError: 'NoneType' object has no attribute 'split'. The issue reported by the CLI does not appear to be resolved.

  16. dwhswenson commented on Aug 12, 2022

    @dwhswenson
    Member

    @ShenWenHuibit : Are you getting the error immediately after it starts the dynamics? (i.e., after the stage where it ensures that there are trajectories for each ensemble) Or have you already seen some Gromacs output when you see it?

    I get to the point that it runs a few Gromacs trajectories with the attached notebook (based on your earlier notebooks; obviously I had to comment out the module load stuff and use a gmx_executable that works locally -- I also added some timing checks that are useful for me to know where to improve performance later). I run that notebook up to cell where I save to file, before the "Equilibration heading." Then I run openpathsampling equilibrate setup.db -o equil.db. I'm on the OPS master and OPS CLI main branches.

    I get a couple Gromacs trajectories and then kill the process (I don't actually need to fully equilibrate this) -- so if the error happens later, I wouldn't see it, but I was previously getting the error before any dynamics started.

    1_traj_mstis.ipynb.zip

  17. WenHuiShen-Bio commented on Aug 15, 2022

    @WenHuiShen-Bio
    Author

    Thank you for your patience in answering!

    First of all the CLI issue before might be caused by the way I installed (at that time I used the openpathsampling-cli installed online by conda), anyway, the CLI can be used normally.

    But it seems that the running speed is not particularly efficient. I found that when a Monte Carlo step needs to consume the full number of steps I set in the mdp file, the CLI run becomes very slow(it will take a few minutes before the next Monte Carlo step begins), but when the number of steps set in the mdp file does not run to completion (that is, ops terminates gromacs running ), then next Monte Carlo step will start immediately.

    As for the simulation speed, there seem to be two modes of operation here. One is that OPS actively stops gromas running, and the other is that gromacs runs the nstep steps set in mdp file, and then starts the next step of Monte Carlo. I don't know if both run modes are going to generate new config files through the gmx grompp program, but I'm guessing that generating all run config files might lead to slower runs, which is what I'm seeing. Is there a way to speed up this process?

    Secondly, I have some questions about analyzing data that I would like to ask you. I used the fully equilibrated trajectories as the initial condition, then ran TIS sampling and finally all the outputs were saved to the mstis.db file.

    Here are some questions I'm confused about :

    • In TIS analysis, I want to know how to set the parameter bin_range in max_lambda_calcs properly, because different bin_range values make me get different Rate matrix.
    • I noticed that in my example the Total crossing probability between A-B and A-C exactly overlap, but in the Rate matrix only B-C has value, others are 0, is this normal? Does this mean that my sample results only contain paths from B to C states, but not A to C states?

    I noticed that in my example the Total crossing probability between A-B and A-C exactly overlap, but in the Rate matrix only B-C has value, others are 0, is this normal?

    I noticed that I didn't name the volums when I defined it earlier, this may caused the program not to recognize the volumes, is this why the two lines are completely overlaped?

    • Here I got only the values from B to C in the rate matrix. I am not sure whether it is because the number of steps in TIS sampling is not enough (Monte Carlo steps) or because the number of steps set in the mdp file is not enough?
    • Then I visualized part of the path tree and the result is not displayed correctly (see path tree in 2_analysis.ipynb). How to solve display problems? I also tried the method described from examples/gromacs/AD_tps_3a_analysis_flex.ipynb to analyze the path tree, and also got the wrong display result.
    • What I want in the end is to get the free energy of a certain path in the result of TIS sampling. So I want to know how to extract a certain path in the result to analyze.

    The initial trajectory conditions and final product results are stored in the setup.db and mstis.db files, respectively, which can be obtained from here : https://drive.google.com/file/d/1FLNbTL0GIStviQBaU36zu-rtETKkqiD1/view?usp=sharing

    The ipynb files can be find here:
    Desktop.zip

  18. WenHuiShen-Bio commented on Aug 18, 2022

    @WenHuiShen-Bio
    Author

    Attached is the analysis method I tried recently. I refer to the official website tutorial, but the move "decision tree" visualizer cannot be used correctly.

    Next, the analysis of the path density made me realize that there might be a problem with the states I defined, which caused the transition rate between states A and C to be 0. I would probably define more states as being very close between them. After I plotted the trajectories of all the replicas on the reaction coordinates, I found that the three states seem to be connected, but it still doesn't seem to be enough to get a path through the three states.

    Next I will try to recalculate with more states and a longer simulation time to see if the problem can be solved.

    When I call some built-in functions of OPS, there may be some usage errors, so the path tree cannot be displayed correctly or the analysis results are wrong. If you don't mind, I hope you can take a few minutes to look at my example and give me some suggestions to try. Greatful!
    2_analysis.pdf

  19. WenHuiShen-Bio commented on Aug 19, 2022

    @WenHuiShen-Bio
    Author

    As for the simulation speed, there seem to be two modes of operation here. One is that OPS actively stops gromas running, and the other is that gromacs runs the nstep steps set in mdp file, and then starts the next step of Monte Carlo. I don't know if both run modes are going to generate new config files through the gmx grompp program, but I'm guessing that generating all run config files might lead to slower runs, which is what I'm seeing. Is there a way to speed up this process?

    Well, I found that after I submit the OPS program to the remote server, when OPS runs gromacs, it will randomly assign the gromacs program to a core. So this will cause gromacs to run slower when there are other jobs on this node. So I need to make specific core for OPS and gromacs separately(eg: set 'mdrun_args' : ' -ntomp 1 && taskset -pc 1 $! && wait' in 'options') and see if the problem is solved.

    And I noticed that there is a way to parallelize the program(https://github.com/dwhswenson/ops_tutorial/blob/main/7_parallel_tis_setup.ipynb), I'll try it out.

  20. dwhswenson commented on Aug 19, 2022

    @dwhswenson
    Member

    @ShenWenHuibit : Just a few quick comments here -- I'm pretty swamped with stuff for the American Chemical Society National Meeting next week, and so probably won't be responsive in the near future, but I wanted to make a couple quick points:

    • Path trees: I thought that path trees kind of worked, but gave results that didn't have the correct colors when using the new storage. In any case, PR #1048 gives with a new way of generating path trees that should work. That gives matplotlib output instead of an SVG; it should work for basic path trees, but doesn't have some of the much fancier stuff that the older path trees can do (coloring based on CVs, nice hover highlight in browser, etc.) It does mean that it is easier to save a figure to file, since you can save using standard matplotlib savefig.
    • Parallel TIS: Keep in mind that this does not allow for replica exchange between path ensembles (RETIS), which is often really helpful in practical simulations, especially when dealing with more than 2 stable states. Parallel RETIS is still in progress, but it technically much more difficult.
    • Gromacs with other processes on the same node: Assuming your system is large enough to benefit from Gromacs's parallelization within a node (which most are), it may be best to write your job submission script to request a whole node (or at least an allocation of more than 1 core on a node). Overloading the core with the OPS process in it (i.e., having Gromacs also run there) usually won't be a problem, since (in most cases) the OPS process is usually sleeping while waiting for Gromacs to write out the next frame. Then there's a brief burst while OPS calculates CVs on the new frame, but that's usually much less expensive than actually running the dynamics. If using taskset as you are works for you, please let me know -- that would be a good fix to be aware of for use cases where you'll really run on only 1 core.
    • Move decision tree: This is a known issue that broke somehow in Python 3. It seems to be used rarely enough that it hasn't been a priority to fix it. (The code there is high complexity and last time I went down that rabbit hole, it seemed like I just made things worse.)
  21. WenHuiShen-Bio commented on Aug 20, 2022

    @WenHuiShen-Bio
    Author

    @dwhswenson :Thanks for your suggestions and discussions. In the future, I may mainly rely on the OPS framework for my research work, so I will continue to pay attention to the update of OPS and the testing of calculation examples.

    • Gromacs with other processes on the same node:
      I wrote the taskset command in a shell script and ran it, and it can indeed specify OPS to run on a specific core(the way i wrote in shell script:openpathsampling equilibrate setup.db -o equil.db & taskset -pc 1 $! wait ), the output:
      image
      However, the gromacs program executed by OPS cannot run on a specific core. When I submit multiple OPS programs to the same node, although I specify that each OPS program runs on a different core, all gromacs programs are always running on core 0, which results in extremely slow running efficiency.
      In OPS python script, we can bind the gromacs program to a specific core to run through the parameter -pin on -pinoffset $core_id of the mdrun program. When I try to parallelize MSTIS, I need to create multiple engines to a list, and then assign them to replica_schemes, so that multiple gromacs programs can run in parallel on different cores without affecting each other.

    • Path tree:
      master branch does not contain new path tree:
      image
      Then I tried dwhswenson:new_path_tree branch and got the following error:
      image

    • Thermodynamics and dynamics:
      Standard Monte Carlo sampling of microstates follows from the principles of equilibrium statistical mechanics, and quantities computed from it are thermodynamic properties. Similarly, transition path sampling follows from a statistical mechanics of trajectory space, and quantities computed from it are dynamical properties, like rate constants. Can OPS analyze thermodynamic properties? such as free energy.

  22. WenHuiShen-Bio commented on Sep 3, 2022

    @WenHuiShen-Bio
    Author

    @dwhswenson :When I use OPS-cli to run pathsampling calculations, I always get an os error, which does not appear when I run it with jupyter notebook.

    The log files are attached:
    logfiles.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions