adapt.parameter_based.FineTuning

class adapt.parameter_based.FineTuning(encoder=None, task=None, Xt=None, yt=None, training=True, pretrain=False, verbose=1, copy=True, random_state=None, **params)

FineTuning : finetunes pretrained networks on target data.

A pretrained source encoder should be given. A task network, pretrained or not, should be given too.

Finetuning train both networks. The task network can be fitted first using the pretrain parameter. The layers to train in the encoder can be set via the parameter training.

Transferring parameters of a CNN pretrained on Imagenet (source: [1])

Parameters

encodertensorflow Model (default=None)

Encoder netwok. If None, a shallow network with 10 neurons and ReLU activation is used as encoder network.

tasktensorflow Model (default=None)

Task netwok. If None, a two layers network with 10 neurons per layer and ReLU activation is used as task network.

Xtnumpy array (default=None)

Target input data.

ytnumpy array (default=None)

Target output data.

trainingbool or list of bool, optional (default=True)

Weither to train the encoder or not. If a list is given, values from training are assigned successively to the trainable attribute of the encoder layers going from the last layer to the first one. If the length of training is smaller than the length of the encoder layers list, the last value of training will be asigned to the remaining layers.

pretrainbool (default=False)

If True, the task network is first trained alone on the outputs of the encoder.

copyboolean (default=True)

Whether to make a copy of estimator or not.

verboseint (default=1)

Verbosity level.

random_stateint (default=None)

Seed of random generator.

paramskey, value arguments

Arguments given at the different level of the adapt object. It can be, for instance, compile or fit parameters of the estimator or kernel parameters etc… Accepted parameters can be found by calling the method _get_legal_params(params).

Yields

optimizerstr or instance of tf.keras.optimizers (default=”rmsprop”)

Optimizer for the task. It should be an instance of tf.keras.optimizers as: tf.keras.optimizers.SGD(0.001) or tf.keras.optimizers.Adam(lr=0.001, beta_1=0.5). A string can also be given as "adam". Default optimizer is rmsprop.

lossstr or instance of tf.keras.losses (default=”mse”)

Loss for the task. It should be an instance of tf.keras.losses as: tf.keras.losses.MeanSquaredError() or tf.keras.losses.CategoricalCrossentropy(). A string can also be given as "mse" or categorical_crossentropy. Default loss is mse.

metricslist of str or list of tf.keras.metrics.Metric instance

List of metrics to be evaluated by the model during training and testing. Typically you will use metrics=['accuracy'].

optimizer_encstr or instance of tf.keras.optimizers

If the Adapt Model has an encoder attribute, a specific optimizer for the encoder network can be given. Typically, this parameter can be used to give a smaller learning rate to the encoder. If not specified, optimizer_enc=optimizer.

optimizer_discstr or instance of tf.keras.optimizers

If the Adapt Model has a discriminator attribute, a specific optimizer for the discriminator network can be given. If not specified, optimizer_disc=optimizer.

kwargskey, value arguments

Any arguments of the fit method from the Tensorflow Model can be given, as epochs and batch_size. Specific arguments from optimizer can also be given as learning_rate or beta_1 for Adam. This allows to perform GridSearchCV from scikit-learn on these arguments.

References

1

[1] Oquab M., Bottou L., Laptev I., Sivic J. “Learning and transferring mid-level image representations using convolutional neural networks”. In CVPR, 2014.

Examples

>>> from adapt.utils import make_regression_da
>>> from adapt.parameter_based import FineTuning
>>> Xs, ys, Xt, yt = make_regression_da()
>>> src_model = FineTuning(loss="mse", pretrain=False, random_state=0)
>>> src_model.fit(Xs, ys, epochs=100, verbose=0)
>>> print(src_model.score(Xt, yt))
1/1 [==============================] - 0s 82ms/step - loss: 0.2645
0.26447802782058716
>>> tgt_model = FineTuning(encoder=src_model.encoder_, loss="mse", pretrain=True,
...                        pretrain__epochs=30, random_state=0)
>>> tgt_model.fit(Xt[:3], yt[:3], epochs=100, verbose=0)
>>> tgt_model.score(Xt, yt)
1/1 [==============================] - 0s 78ms/step - loss: 0.1114
0.11144775152206421

Attributes

encoder_tensorflow Model

encoder network.

task_tensorflow Model

Network.

history_dict

history of the losses and metrics across the epochs of the network training.

Methods

__init__([encoder, task, Xt, yt, training, ...])
compile([optimizer, loss, metrics, ...])	Configures the model for training.
fit([Xt, yt])	Fit FineTuning.
get_params([deep])	Get parameters for this estimator.
predict(x[, batch_size, verbose, steps, ...])	Generates output predictions for the input samples.
predict_disc(X)	Not used.
predict_task(X)	Return predictions of the task on the encoded features.
pretrain_step(data)
score(X, y[, sample_weight])	Return the evaluation of the model on X, y.
set_params(**params)	Set the parameters of this estimator.
transform(X)	Return the encoded features of X.
unsupervised_score(Xs, Xt)	Return unsupervised score.

__init__(encoder=None, task=None, Xt=None, yt=None, training=True, pretrain=False, verbose=1, copy=True, random_state=None, **params)

compile(optimizer=None, loss=None, metrics=None, loss_weights=None, weighted_metrics=None, run_eagerly=None, steps_per_execution=None, **kwargs)

Configures the model for training.

Parameters

optimizer: str or `tf.keras.optimizer` instance

Optimizer

loss: str or `tf.keras.losses.Loss` instance

Loss function. A loss function is any callable with the signature loss = fn(y_true, y_pred), where y_true are the ground truth values, and y_pred are the model’s predictions. y_true should have shape (batch_size, d0, .. dN) (except in the case of sparse loss functions such as sparse categorical crossentropy which expects integer arrays of shape (batch_size, d0, .. dN-1)). y_pred should have shape (batch_size, d0, .. dN). The loss function should return a float tensor. If a custom Loss instance is used and reduction is set to None, return value has shape (batch_size, d0, .. dN-1) i.e. per-sample or per-timestep loss values; otherwise, it is a scalar. If the model has multiple outputs, you can use a different loss on each output by passing a dictionary or a list of losses. The loss value that will be minimized by the model will then be the sum of all individual losses, unless loss_weights is specified.

metrics: list of str or list of `tf.keras.metrics.Metric` instance

List of metrics to be evaluated by the model during training and testing. Typically you will use metrics=[‘accuracy’]. A function is any callable with the signature result = fn(y_true, y_pred). To specify different metrics for different outputs of a multi-output model, you could also pass a dictionary, such as metrics={‘output_a’: ‘accuracy’, ‘output_b’: [‘accuracy’, ‘mse’]}. You can also pass a list to specify a metric or a list of metrics for each output, such as metrics=[[‘accuracy’], [‘accuracy’, ‘mse’]] or metrics=[‘accuracy’, [‘accuracy’, ‘mse’]]. When you pass the strings ‘accuracy’ or ‘acc’, we convert this to one of tf.keras.metrics.BinaryAccuracy, tf.keras.metrics.CategoricalAccuracy, tf.keras.metrics.SparseCategoricalAccuracy based on the loss function used and the model output shape. We do a similar conversion for the strings ‘crossentropy’ and ‘ce’ as well.

loss_weights: List or dict of floats

Scalars to weight the loss contributions of different model outputs. The loss value that will be minimized by the model will then be the weighted sum of all individual losses, weighted by the loss_weights coefficients. If a list, it is expected to have a 1:1 mapping to the model’s outputs. If a dict, it is expected to map output names (strings) to scalar coefficients.

weighted_metrics: list of metrics

List of metrics to be evaluated and weighted by sample_weight or class_weight during training and testing.

run_eagerly: bool (default=False)

If True, this Model’s logic will not be wrapped in a tf.function. Recommended to leave this as None unless your Model cannot be run inside a tf.function. run_eagerly=True is not supported when using tf.distribute.experimental.ParameterServerStrategy.

steps_per_execution: int (default=1)

The number of batches to run during each tf.function call. Running multiple batches inside a single tf.function call can greatly improve performance on TPUs or small models with a large Python overhead. At most, one full epoch will be run each execution. If a number larger than the size of the epoch is passed, the execution will be truncated to the size of the epoch. Note that if steps_per_execution is set to N, Callback.on_batch_begin and Callback.on_batch_end methods will only be called every N batches (i.e. before/after each tf.function execution).

**kwargs: key, value arguments

Arguments supported for backwards compatibility only.

Returns

None: None

fit(Xt=None, yt=None, **fit_params)

Fit FineTuning.

Parameters

Xtnumpy array (default=None)

Target input data.

ytnumpy array (default=None)

Target output data.

fit_paramskey, value arguments

Arguments given to the fit method of the model (epochs, batch_size, callbacks…).

Returns

selfreturns an instance of self

get_params(deep=True)

Get parameters for this estimator.

Parameters

deepbool, default=True

Not used, here for scikit-learn compatibility.

Returns

paramsdict

Parameter names mapped to their values.

predict(x, batch_size=None, verbose=0, steps=None, callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False)

Generates output predictions for the input samples.

Computation is done in batches. This method is designed for performance in large scale inputs. For small amount of inputs that fit in one batch, directly using __call__() is recommended for faster execution, e.g., model(x), or model(x, training=False) if you have layers such as tf.keras.layers.BatchNormalization that behaves differently during inference. Also, note the fact that test loss is not affected by regularization layers like noise and dropout.

Parameters

x: array

Input samples.

batch_size: int (default=`None`)

Number of samples per batch. If unspecified, batch_size will default to 32. Do not specify the batch_size if your data is in the form of dataset, generators, or keras.utils.Sequence instances (since they generate batches).

verbose: int (default=0)

Verbosity mode, 0 or 1.

steps: int (default=None)

Total number of steps (batches of samples) before declaring the prediction round finished. Ignored with the default value of None. If x is a tf.data dataset and steps is None, predict() will run until the input dataset is exhausted.

callbacks: List of `keras.callbacks.Callback` instances.

List of callbacks to apply during prediction. See [callbacks](/api_docs/python/tf/keras/callbacks).

max_queue_size: int (default=10)

Used for generator or keras.utils.Sequence input only. Maximum size for the generator queue. If unspecified, max_queue_size will default to 10.

workers: int (default=1)

Used for generator or keras.utils.Sequence input only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1.

use_multiprocessing: bool (default=False)

Used for generator or keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can’t be passed easily to children processes.

Returns

y_predarray

Numpy array(s) of predictions.

predict_disc(X)

Not used.

predict_task(X)

Return predictions of the task on the encoded features.

Parameters

Xarray

input data

Returns

y_taskarray

predictions of task network

pretrain_step(data)

score(X, y, sample_weight=None)

Return the evaluation of the model on X, y.

Call evaluate on tensorflow Model.

Parameters

Xarray

input data

yarray

output data

sample_weightarray (default=None)

Sample weights

Returns

scorefloat

Score.

set_params(**params)

Set the parameters of this estimator.

Parameters

**paramsdict

Estimator parameters.

Returns

selfestimator instance

Estimator instance.

transform(X)

Return the encoded features of X.

Parameters

Xarray

input data

Returns

X_encarray

predictions of encoder network

unsupervised_score(Xs, Xt)

Return unsupervised score.

The normalized discrepancy distance is computed between the reweighted/transformed source input data and the target input data.

Parameters

Xsarray

Source input data.

Xtarray

Source input data.

Returns

scorefloat

Unsupervised score.

Examples

Model-Based Transfer Learning

Deep Domain Adaptation: Transfer one domain to another