Skip to content

Commit e384cb5

Browse files
authored
Instantiating objects with Hydra Example (ebezzam#8)
* Add hydra_instantiate example * Instantiate Example update * Instantiate Example Fix Docs * Instantiate Example Fix Docs * Instantiate Example Fix Docs and Config * Instantiate Example Fix Docs
1 parent cbde197 commit e384cb5

7 files changed

Lines changed: 105 additions & 1 deletion

File tree

‎docs/source/reproducible.rst‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,3 +104,47 @@ configuration file and use it the script like so:
104104
# Set the seed for numpy
105105
np.random.seed(config.seed)
106106
# application-specific seed setting
107+
108+
Instantiating objects
109+
---------------------
110+
111+
Another cool feature of Hydra is `object instantiating <https://hydra.cc/docs/advanced/instantiate_objects/overview/>`_.
112+
Imagine you want to try different optimizers for your Deep Neural Network (DNN) or you want to try different DNNs in the same pipeline.
113+
Instead of doing ``if-else`` statements, you write one line of code and let Hydra choose the
114+
appropriate object class based on your configuration. See the script
115+
`examples/real_convolve.py <https://github.com/ebezzam/python-dev-tips/blob/main/examples/real_convolve.py>`_
116+
for the example.
117+
118+
.. code-block:: python
119+
120+
@hydra.main(version_base=None, config_path="configs", config_name="defaults")
121+
def main(config):
122+
# instantiate object from config
123+
signal = instantiate(config.signal)
124+
# application specific choice of object class
125+
126+
``instantiate`` function from ``hydra.utils`` allows you to define an object in a YAML file
127+
without being tied to a particular class. To do this, you need to define ``_target_`` in
128+
your config (see configs in ``configs/signal``) and object initialization arguments. Object class
129+
can be either defined in your project (``configs/signal/ExampleZeros``, ``configs/signal/ExampleCustom``)
130+
or taken from a package (``configs/signal/ExampleNumpy``).
131+
132+
Note that here we use another Hydra feature: config grouping and splitting. Instead of writing
133+
configurations for all objects in the main config and copying configuration files, we create a sub-directory ``signal``,
134+
where all ``signal`` configs are defined. Now we can run the main config with the ``signal`` of
135+
our choice simply by specifying it in the command line. For example, ``python examples/real_convolve.py signal=ExampleNumpy``
136+
or ``python examples/real_convolve.py signal=ExampleZeros``.
137+
138+
If we need to define some of the arguments inside the code before creating an object, we can pass them directly to the ``instantiate`` function.
139+
For example, we did not define ``signal_len`` in the ``signal`` configuration file and passed it by hand:
140+
``signal = instantiate(config.signal, config.signal_len)``. This is especially useful when you have positional-only arguments
141+
like ``numpy.random.randn`` in our example. Note that we can both define arguments in the configuration file and pass new ones to ``instantiate`` like we did for
142+
``ExampleCustom``.
143+
144+
Object instantiating is recursive, i.e. some of the arguments of the class can also be
145+
defined using ``_target_`` and they will be created automatically. For example,
146+
``python examples/real_convolve.py signal=ExampleCustom +signal/transform=power`` defines the ``transform`` argument of
147+
the ``ExampleCustom`` class as the ``PowerTransform`` class. The ``+signal/transform=power`` in the command line
148+
means adding the ``transform`` argument to the current ``signal`` configuration from the ``power.yaml`` config defined
149+
in ``configs/signal/transform``. That is, you can have sub-sub-directories. The default values from sub-sub-directories
150+
can also be changed in the command-line: ``python examples/real_convolve.py signal=ExampleCustom +signal/transform=power signal.transform.pow=3``

‎examples/configs/defaults.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
defaults:
2+
- signal: ExampleNumpy
3+
- _self_
4+
15
hydra:
26
job:
37
chdir: True # change to output folder
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
_target_: real_convolve.ExampleCustom
2+
numpy_method: arange
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
_target_: numpy.random.randn
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
_target_: real_convolve.ExampleZeros
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
_target_: real_convolve.PowerTransform
2+
pow: 2

‎examples/real_convolve.py‎

Lines changed: 51 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,67 @@
22
import matplotlib.pyplot as plt
33
from pydevtips.fftconvolve import RFFTConvolve
44
import hydra
5+
from hydra.utils import instantiate
56
import os
67

78

9+
class PowerTransform:
10+
def __init__(self, pow):
11+
self.pow = pow
12+
13+
def __call__(self, x):
14+
return np.power(x, self.pow)
15+
16+
17+
class ExampleZeros:
18+
"""
19+
Wrapper over np.zeros
20+
"""
21+
22+
def __init__(self, signal_len) -> None:
23+
self.data = np.zeros(signal_len)
24+
25+
def max(self):
26+
return self.data.max()
27+
28+
def __array__(self):
29+
return self.data
30+
31+
def __len__(self):
32+
return len(self.data)
33+
34+
35+
class ExampleCustom:
36+
"""
37+
Wrapper over custom np.ndarray creation method with an optional transform
38+
"""
39+
40+
def __init__(self, signal_len, numpy_method, transform=None) -> None:
41+
self.data = getattr(np, numpy_method)(signal_len)
42+
if transform is not None:
43+
self.data = transform(self.data)
44+
45+
def max(self):
46+
return self.data.max()
47+
48+
def __array__(self):
49+
return self.data
50+
51+
def __len__(self):
52+
return len(self.data)
53+
54+
855
@hydra.main(version_base=None, config_path="configs", config_name="defaults")
956
def main(config):
1057

1158
output_dir = os.getcwd()
1259
np.random.seed(config.seed)
1360

1461
# Create a signal
15-
signal = np.random.randn(config.signal_len)
62+
signal = instantiate(config.signal, config.signal_len)
63+
# Note that we added extra argument signal_len which was not defined in signal config
64+
print(f"Signal class, len and max: {type(signal), len(signal), signal.max()}")
65+
signal = np.array(signal) # for the following computations np.ndarray is required
1666

1767
# Create a moving average filter (low pass)
1868
n = config.filter_len

0 commit comments

Comments
 (0)