Skip to content

Faster AppendedTrajectoryEnsembles - #231

Merged
jhprinz merged 20 commits into
openpathsampling:masterfrom
dwhswenson:faster_sequential_ensemble
May 7, 2015
Merged

jhprinz merged 20 commits into
openpathsampling:masterfrom
dwhswenson:faster_sequential_ensemble

Conversation

@dwhswenson

Copy link
Copy Markdown
Member

Once this is done, trajectory generation should be blazing fast.

  • Caching for ForwardAppendedTrajectoryEnsemble
  • Linear scaling for ForwardAppendedTrajectoryEnsemble
  • Caching for BackwardPrependedTrajectoryEnsemble
  • Linear scaling for BackwardPrependedTrajectoryEnsemble
  • Remove/refactor debugging statements to reduce prefactor

I'm also overriding the unused of can_append/can_prepend for these to raise an error if called. ForwardAppendedTrajectoryEnsemble.can_prepend is nonsense: I'm not even sure what the expected behavior would be, so I'm calling it an error.

Fixes #222.

@dwhswenson dwhswenson self-assigned this Apr 29, 2015
@dwhswenson dwhswenson changed the title Faster AppendedTrajectoryEnsembles [WIP] Faster AppendedTrajectoryEnsembles Apr 29, 2015
@dwhswenson

Copy link
Copy Markdown
Member Author

As mentioned in #222, the prepend cost is actually only "near-linear." The nonlinearity is visible (see shooting_speed_test.ipynb in my gist for details), but the effect is small enough that I don't think we need to worry about it.

I've done what I can to speed things up here. Hypothetically, a different implementation of the sequential ensemble could speed it up a bit (by replacing the current approach with an approach which assigns frames to the trajectories, and then uses that assignment as a cache).

I've attached a recent call graph (from a 200-cycle RETIS calculation). You'll have to load it to read anything, of course, but a few main points:

  • storing SampleSets: 27% of total time (done after each MC cycle).
  • sanity check: 14% of total time (done every 10 MC cycles, although that is easily adjusted)
  • chaindict-related stuff: 46% of the total time

Part of the problem with chaindict is, of course, that it gets called a lot. But it still makes a lot more calls within itself. So anything that could reduce that would probably speed things up considerably.

Anyway, this is ready to merge now. I'm getting reasonably faster (and much more consistent) calculation speeds.

retis_200

@dwhswenson dwhswenson assigned jhprinz and unassigned dwhswenson May 1, 2015
@dwhswenson dwhswenson changed the title [WIP] Faster AppendedTrajectoryEnsembles Faster AppendedTrajectoryEnsembles May 1, 2015
dwhswenson added 2 commits May 4, 2015 01:02
Now all test trajs in testensemble are passed through make_1d_traj. Before,
we added this on the trajs for BackwardPrepended tests. Advanced in openpathsampling#242
make that redundant.
@dwhswenson

Copy link
Copy Markdown
Member Author

Application of other PRs made this unmergeable, and #242 made a step in the tests here redundant (and thus they would throw errors). All of that should now be fixed, and (assuming it passes tests, which it should) this should be ready for review and merge.

@dwhswenson

Copy link
Copy Markdown
Member Author

Fixed so that this is ready to be merged again (at least, if we merge it before merging anything else this time!)

@jhprinz

jhprinz commented May 7, 2015

Copy link
Copy Markdown
Contributor

lets go...

jhprinz added a commit that referenced this pull request May 7, 2015
@jhprinz
jhprinz merged commit 7afed59 into openpathsampling:master May 7, 2015
@dwhswenson
dwhswenson deleted the faster_sequential_ensemble branch October 26, 2015 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AppendedTrajectoryEnsembles are slow

2 participants