Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
1850972
Add core & engines for yaml compiling
dwhswenson Mar 21, 2021
4392e41
Remove named_obj param in engines, topologies
dwhswenson Mar 21, 2021
2a81273
CVs and tests (without named_objs)
dwhswenson Mar 21, 2021
3be9450
volumes and partial tests for volumes
dwhswenson Mar 21, 2021
dfbf36c
Moved tests over to testing dir
dwhswenson Mar 21, 2021
e1a778f
remove named objs; still seems to work
dwhswenson Mar 24, 2021
b3edf04
Seems to work up to networks now!
dwhswenson Mar 24, 2021
2ed6990
Seems to work through TPS schemes!
dwhswenson Mar 25, 2021
3c45b87
steps toward parsing simulations
dwhswenson Mar 26, 2021
248cef0
Add compile command
dwhswenson Apr 8, 2021
10ce9d2
Merge branch 'main' into sim-setup
dwhswenson Apr 10, 2021
0657882
Merge branch 'wizard' into sim-setup
dwhswenson Jun 18, 2021
45e77d6
long forgotten stash with lots of useful stuff
dwhswenson Jul 24, 2021
728083c
add tests/parsing/__init__.py
dwhswenson Jul 24, 2021
87029dd
tests for volumes combinations
dwhswenson Jul 24, 2021
ef1f086
fix for location of MDTrajTopology
dwhswenson Jul 24, 2021
55c2dbd
improvements to testing
dwhswenson Jul 24, 2021
e3ecd1e
Add tests for shooting
dwhswenson Jul 25, 2021
b6254ec
fix shooting tests
dwhswenson Jul 25, 2021
3597dd6
first steps toward autodocumenting yaml structures
dwhswenson Jul 26, 2021
1b2bab0
add json schema support to MDTraj CV
dwhswenson Jul 26, 2021
28d37f7
start using new parameters in volumes
dwhswenson Jul 27, 2021
1181edc
Merge branch 'main' of github.com:openpathsampling/openpathsampling-c…
dwhswenson Jul 27, 2021
76da8bc
mock => unittest.mock
dwhswenson Jul 27, 2021
345b77c
Some more refactoring of sim-setup
dwhswenson Jul 27, 2021
f703605
switched everything over to parameter/builder
dwhswenson Jul 27, 2021
548b389
remove now-unused classes
dwhswenson Jul 27, 2021
8f9fe2e
Start switch to plugins
dwhswenson Jul 29, 2021
379e5f1
add plugins files
dwhswenson Jul 29, 2021
e513680
Start toward registering parsing via plugins
dwhswenson Aug 10, 2021
bdb1ab9
proof of principle for plugin registration
dwhswenson Aug 10, 2021
f31b1c9
more updates to use the new plugin model
dwhswenson Aug 10, 2021
d4b3f29
remove old parsing from engines, cvs, volumes
dwhswenson Aug 11, 2021
c8bf0c3
add tests/parsing/utils
dwhswenson Aug 11, 2021
8163cfe
Remove old parsers: networks, schemes, strategies
dwhswenson Aug 11, 2021
769d3f7
skeleton of root_parser tests
dwhswenson Aug 13, 2021
1cd4885
some root parser tests
dwhswenson Aug 13, 2021
eb78804
more tests for root_parser
dwhswenson Aug 13, 2021
59b890c
Some docstrings and tests for core
dwhswenson Sep 5, 2021
656a22e
Merge branch 'main' of github.com:openpathsampling/openpathsampling-c…
dwhswenson Sep 5, 2021
cb69db6
(most) tests for InstanceBuilder
dwhswenson Sep 5, 2021
eab9b4a
more tests in parsing core
dwhswenson Sep 5, 2021
2bd0eea
tests for helpers in compile command
dwhswenson Sep 5, 2021
f7cc88c
steps toward testing compile command
dwhswenson Sep 6, 2021
b64ea30
major refactor of root_paser
dwhswenson Sep 7, 2021
d77b8aa
Major rename: parsing => compiling
dwhswenson Sep 7, 2021
030679c
more tests for root_compiler
dwhswenson Sep 7, 2021
4b7dd7c
more cleanup to get tests running
dwhswenson Sep 7, 2021
ddc6a17
add test/commands/utils
dwhswenson Sep 7, 2021
ffbf6c5
cleanup; start to tests move scheme compiling
dwhswenson Sep 7, 2021
1b6f505
Fix the error_on_duplicate issue
dwhswenson Sep 8, 2021
ad714d9
fix other problem with duplication
dwhswenson Sep 8, 2021
1e531ba
more tests for compiling.schemes
dwhswenson Sep 8, 2021
71719d5
more testing/coverage
dwhswenson Sep 8, 2021
2f9dc3d
remove unneeded options for InputError
dwhswenson Sep 8, 2021
fdbd7a4
fix typo
dwhswenson Sep 8, 2021
43fec22
Renamings
dwhswenson Sep 8, 2021
4e74899
Rename: compiler_name => category
dwhswenson Sep 8, 2021
0318227
refactor for reusable get_installed_plugins
dwhswenson Sep 8, 2021
3daa542
add docs/compiling.rst
dwhswenson Sep 8, 2021
7fce8a5
compiling/__init__; and always error on duplicate
dwhswenson Sep 9, 2021
9d61206
Apply suggestions from code review
dwhswenson Sep 23, 2021
823ef0c
Update paths_cli/tests/compiling/test_volumes.py
dwhswenson Sep 23, 2021
454ff28
some fixes from review
dwhswenson Sep 23, 2021
571d283
redo reload modules after test
dwhswenson Sep 24, 2021
8904fd3
complete skipped tests in compiling/test_core
dwhswenson Sep 25, 2021
fb518ba
finish skipped tests in test_root_compiler
dwhswenson Sep 25, 2021
e463087
add compat.openmm to handle OpenMM imports
dwhswenson Sep 26, 2021
2dad4d8
fix name of unit when None
dwhswenson Sep 26, 2021
0b106e6
fix skiptest
dwhswenson Sep 26, 2021
ff4bc78
use custom_eval_int_strict_pos
dwhswenson Sep 26, 2021
63a0f33
bring coverage back to 100%
dwhswenson Sep 26, 2021
3fd2509
don't use endswith on possible None
dwhswenson Sep 26, 2021
854137c
cleanup per review comments
dwhswenson Sep 27, 2021
a43b2c0
Update paths_cli/compiling/core.py
dwhswenson Sep 27, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Seems to work through TPS schemes!
  • Loading branch information
dwhswenson committed Mar 25, 2021
commit 2ed6990de1b29529bc526b4671abfe597550d32e
4 changes: 4 additions & 0 deletions paths_cli/commands/contents.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@
'Snapshots': 'snapshots'
}

import logging
logger = logging.getLogger(__name__)

@click.command(
'contents',
short_help="list named objects from an OPS .nc file",
Expand Down Expand Up @@ -48,6 +51,7 @@ def contents(input_file, table):

def get_section_string(label, store):
attr = NAME_TO_ATTR.get(label, label.lower())
logger.debug(f"Working on {attr}")
if attr in UNNAMED_SECTIONS:
string = get_unnamed_section_string(label, store)
elif attr in ['tag', 'tags']:
Expand Down
100 changes: 32 additions & 68 deletions paths_cli/parsing/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,79 +22,25 @@ def unlistify(obj, listified):
obj = obj[0]
return obj

# class NamedObjects(abc.MutableMapping):
# """Class to track named objects and their descriptions"""
# def __init__(self, global_dct):
# names, dcts = self.all_name_descriptions(global_dct)
# self.objects = {name: None for name in names}
# self.descriptions = {name: dct for name, dct in zip(names, dcts)}

# def __getitem__(self, key):
# return self.objects[key]

# def __setitem__(self, key, value):
# self.objects[key] = value

# def __delitem__(self, key):
# del self.objects[key]

# def __iter__(self):
# return iter(self.objects)

# def __len__(self):
# return len(self.objects)

# @staticmethod
# def all_name_descriptions(dct):
# """Find all the named objets and their dict descriptions

# Parameters
# ----------
# dct : dict
# output from loading YAML
# """
# names = []
# dcts = []
# name = None
# if isinstance(dct, list):
# for item in dct:
# name_list, dct_list = \
# NamedObjects.all_name_descriptions(item)
# names.extend(name_list)
# dcts.extend(dct_list)
# elif isinstance(dct, dict):
# for k, v in dct.items():
# if isinstance(v, (dict, list)):
# name_list, dct_list = \
# NamedObjects.all_name_descriptions(v)
# names.extend(name_list)
# dcts.extend(dct_list)
# try:
# name = dct['name']
# except KeyError:
# pass
# else:
# names.append(name)
# dcts.append(dct)

# return names, dcts


class InstanceBuilder:
# TODO: add schema as an input so we can autogenerate our JSON schema!
def __init__(self, builder, attribute_table, defaults=None, module=None,
remapper=None):
def __init__(self, builder, attribute_table, optional_attributes=None,
defaults=None, module=None, remapper=None):
self.module = module
self.builder = builder
self.builder_name = str(self.builder)
self.attribute_table = attribute_table
if optional_attributes is None:
optional_attributes = {}
self.optional_attributes = optional_attributes
# TODO use none_to_default
if remapper is None:
remapper = lambda x: x
self.remapper = remapper
if defaults is None:
defaults = {}
self.defaults = defaults
self.logger = logging.getLogger(f"parser.InstanceBuilder[{builder}]")
self.logger = logging.getLogger(f"parser.InstanceBuilder.{builder}")

def select_builder(self, dct):
if self.module is not None:
Expand All @@ -103,16 +49,30 @@ def select_builder(self, dct):
builder = self.builder
return builder


def __call__(self, dct):
def _parse_attrs(self, dct):
# TODO: support aliases in dct[attr]
input_dct = self.defaults.copy().update(dct)
# new_dct = {attr: func(dct[attr], named_objs)
# for attr, func in self.attribute_table.items()}
input_dct = self.defaults.copy()
self.logger.debug(f"defaults: {input_dct}")
input_dct.update(dct)
self.logger.debug(f"effective input: {input_dct}")

new_dct = {}
for attr, func in self.attribute_table.items():
self.logger.debug(f"{attr}: {dct[attr]}")
new_dct[attr] = func(dct[attr])
try:
value = input_dct[attr]
except KeyError:
raise InputError(f"'{self.builder_name}' missing required "
f"parameter '{attr}'")
self.logger.debug(f"{attr}: {input_dct[attr]}")
new_dct[attr] = func(input_dct[attr])

optionals = set(self.optional_attributes) & set(dct)
for attr in optionals:
new_dct[attr] = self.attribute_table[attr](dct[attr])

return new_dct

def _build(self, new_dct):
builder = self.select_builder(new_dct)
ops_dct = self.remapper(new_dct)
self.logger.debug("Building...")
Expand All @@ -121,6 +81,10 @@ def __call__(self, dct):
self.logger.debug(obj)
return obj

def __call__(self, dct):
new_dct = self._parse_attrs(dct)
return self._build(new_dct)


class Parser:
"""Generic parse class; instances for each category"""
Expand Down
2 changes: 2 additions & 0 deletions paths_cli/parsing/root_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from paths_cli.parsing.cvs import cv_parser
from paths_cli.parsing.volumes import volume_parser
from paths_cli.parsing.networks import network_parser
from paths_cli.parsing.schemes import scheme_parser


TYPE_MAPPING = {
Expand All @@ -11,6 +12,7 @@
'volumes': volume_parser,
'states': volume_parser,
'networks': network_parser,
'moveschemes': scheme_parser,
}

def parse(dct):
Expand Down
76 changes: 76 additions & 0 deletions paths_cli/parsing/schemes.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
from paths_cli.parsing.core import InstanceBuilder, Parser
from paths_cli.parsing.tools import custom_eval
from paths_cli.parsing.shooting import shooting_selector_parser
from paths_cli.parsing.engines import engine_parser
from paths_cli.parsing.networks import network_parser
from paths_cli.parsing.strategies import strategy_parser

build_spring_shooting_scheme = InstanceBuilder(
module='openpathsampling',
builder='SpringShootingMoveScheme',
attribute_table={
'network': network_parser,
'k_spring': custom_eval,
'delta_max': custom_eval,
'engine': engine_parser,
}
)

class StrategySchemeInstanceBuilder(InstanceBuilder):
"""
Variant of the InstanceBuilder that appends strategies to a MoveScheme
"""
def __init__(self, builder, attribute_table, defaults=None, module=None,
remapper=None, default_global_strategy=False):
from openpathsampling import strategies
super().__init__(builder, attribute_table, defaults=defaults,
module=module, remapper=remapper)
if default_global_strategy is True:
self.default_global = [strategies.OrganizeByMoveGroupStrategy()]
elif default_global_strategy is False:
self.default_global = []
else:
self.default_global= [default_global_strategy]

def __call__(self, dct):
new_dct = self._parse_attrs(dct)
strategies = new_dct.pop('strategies')
scheme = self._build(new_dct)
for strat in strategies + self.default_global:
scheme.append(strat)

self.logger.debug(f"strategies: {scheme.strategies}")
return scheme


build_one_way_shooting_scheme = StrategySchemeInstanceBuilder(
module='openpathsampling',
builder='OneWayShootingMoveScheme',
attribute_table={
'network': network_parser,
'selector': shooting_selector_parser,
'engine': engine_parser,
'strategies': strategy_parser,
}
)

build_scheme = StrategySchemeInstanceBuilder(
module='openpathsampling',
builder='MoveScheme',
attribute_table={
'network': network_parser,
'strategies': strategy_parser,
},
default_global_strategy=True,
)

scheme_parser = Parser(
type_dispatch={
'one-way-shooting': build_one_way_shooting_scheme,
'spring-shooting': build_spring_shooting_scheme,
'scheme': build_scheme,
'default-tis': ...,
},
label='movescheme'
)

31 changes: 31 additions & 0 deletions paths_cli/parsing/shooting.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
from paths_cli.parsing.core import InstanceBuilder, Parser
from paths_cli.parsing.cvs import cv_parser
from paths_cli.parsing.tools import custom_eval
import numpy as np

build_uniform_selector = InstanceBuilder(
module='openpathsampling',
builder='UniformSelector',
attribute_table={}
)

def remapping_gaussian_stddev(dct):
dct['alpha'] = 0.5 / dct.pop('stddev')**2
return dct

build_gaussian_selector = InstanceBuilder(
module='openpathsampling',
builder='GaussianSelector',
attribute_table={'cv': cv_parser,
'mean': custom_eval,
'stddev': custom_eval},
remapper=remapping_gaussian_stddev
)

shooting_selector_parser = Parser(
type_dispatch={
'uniform': build_uniform_selector,
'gaussian': build_gaussian_selector,
},
label='shooting_selector'
)
97 changes: 97 additions & 0 deletions paths_cli/parsing/strategies.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
from paths_cli.parsing.core import InstanceBuilder, Parser
from paths_cli.parsing.shooting import shooting_selector_parser
from paths_cli.parsing.engines import engine_parser

build_one_way_shooting_strategy = InstanceBuilder(
module='openpathsampling.strategies',
builder='OneWayShootingStrategy',
attribute_table={
'selector': shooting_selector_parser,
'engine': engine_parser,
},
optional_attributes={
'group': str,
'replace': bool,
}
)

build_two_way_shooting_strategy = InstanceBuilder(
module='openpathsampling.strategies',
builder='TwoWayShootingStrategy',
attribute_table={
'modifier': ...,
'selector': shooting_selector_parser,
'engine': engine_parser,
},
optional_attributes={
'group': str,
'replace': bool,
}
)

build_nearest_neighbor_repex_strategy = InstanceBuilder(
module='openpathsampling.strategies',
builder='NearestNeighborRepExStrategy',
attribute_table={},
optional_attributes={
'group': str,
'replace': bool,
},
)

build_all_set_repex_strategy = InstanceBuilder(
module='openpathsampling.strategies',
builder='AllSetRepExStrategy',
attribute_table={},
optional_attributes={
'group': str,
'replace': bool,
}
)

build_path_reversal_strategy = InstanceBuilder(
module='openpathsampling.strategies',
builder='PathReversalStrategy',
attribute_table={},
optional_attributes={
'group': str,
'replace': bool,
}
)

build_minus_move_strategy = InstanceBuilder(
module='openpathsampling.strategies',
builder='MinusMoveStrategy',
attribute_table={
'engine': engine_parser,
},
optional_attributes={
'group': str,
'replace': bool,
}
)

build_single_replica_minus_move_strategy = InstanceBuilder(
module='openpathsampling.strategies',
builder='SingleReplicaMinusMoveStrategy',
attribute_table={
'engine': engine_parser,
},
optional_attributes={
'group': str,
'replace': bool,
}
)

strategy_parser = Parser(
type_dispatch={
'one-way-shooting': build_one_way_shooting_strategy,
'two-way-shooting': build_two_way_shooting_strategy,
'nearest-neighbor-repex': build_nearest_neighbor_repex_strategy,
'all-set-repex': build_all_set_repex_strategy,
'path-reversal': build_path_reversal_strategy,
'minus': build_minus_move_strategy,
'single-rep-minux': build_single_replica_minus_move_strategy,
},
label="strategy"
)