Codecov Report

❗ No coverage uploaded for pull request base (master@3a681c1). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #959   +/-   ##
=========================================
  Coverage          ?   74.62%           
=========================================
  Files             ?      128           
  Lines             ?    13998           
  Branches          ?        0           
=========================================
  Hits              ?    10446           
  Misses            ?     3552           
  Partials          ?        0           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a681c1...a2760a0. Read the comment docs.

Oh, and also I'm moving from Coveralls to CodeCov for coverage reports. I never liked the constructivism-inspired design of Coveralls, but it used to be that Coveralls was the best for handling parallel builds. Now I think that CodeCov does almost as well, and looks better. And I've finally given in and accepted getting a comment from CodeCov on each PR.

Different behaviors if a PR is marked as a draft. For example, don't run notebook tests in draft PRs (encouraging faster development cycle then).

I have no strong opinion on this (run the test suite on my own machine anyway), the one drawback I could think if the CI is not retriggered after someone just removes the Draft tag on a PR (normally editing the title does not retrigger the CI), therefor leading to a possible broken code after merging

Separate testing aspects into different workflows: Perhaps move the notebook tests to run in parallel with unit tests. Perhaps have the docs build/deploy as a separate workflow that only happens after merging a PR.

I'm always in favor of running as much parallel as allowed/socially accepted. I do think the notebook test, unit-test, doc building should be parallel jobs (especially if we don't want to run the docs or the notebooks for the whole test matrix)

Also, I would build docs on the PR (but maybe only if doc files changed?), but deploy should only be done from protected branches.

On this PR: the code looks decent (did not look too deeply at the autorelease-* vendored scripts). I would enable 3.8 testing soon as well, but you might want to do that in a different PR.

I have no strong opinion on this (run the test suite on my own machine anyway), the one drawback I could think if the CI is not retriggered after someone just removes the Draft tag on a PR (normally editing the title does not retrigger the CI), therefor leading to a possible broken code after merging

This is where GitHub actions may provide a little more flexibility (or, at least, better integration with GitHub) than other CI platforms can. Note activity types in the details of how pull requests trigger workflows. Also, you have access to the whole JSON object for the PR (from the REST API), which includes the boolean "draft" attribute. I'll have to experiment a bit, but I think it can be done such that notebook tests trigger when (1) you open/push to/reopen a PR that is not a draft; (2) you convert a PR from draft to ready-to-review.

I'm always in favor of running as much parallel as allowed/socially accepted. I do think the notebook test, unit-test, doc building should be parallel jobs (especially if we don't want to run the docs or the notebooks for the whole test matrix)

Also, I would build docs on the PR (but maybe only if doc files changed?), but deploy should only be done from protected branches.

Thanks; these are both the direction I've been leaning, too.

For this PR, I may keep things more analogous to what we do on Travis, but it's good to hear feedback on what others would find useful in the future.

I would enable 3.8 testing soon as well, but you might want to do that in a different PR.

Waiting on OpenMMTools. Just set up a conda env to see if I can fix their failing tests and encourage a conda-forge release, now that OpenMM is there.

Since Travis seems to be dead (nothing is happening in #961), I'm going to go ahead and merge this one so we have CI. I'll add docs support later.