adapt.instance_based.TwoStageTrAdaBoostR2
- class adapt.instance_based.TwoStageTrAdaBoostR2(estimator=None, Xt=None, yt=None, n_estimators=10, n_estimators_fs=10, cv=5, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]
Two Stage Transfer AdaBoost for Regression
TwoStageTrAdaBoostR2 algorithm is a supervised instances-based domain adaptation method suited for regression tasks.
The method is based on a “reverse boosting” principle where the weights of source instances poorly predicted decrease at each boosting iteration whereas the ones of target instances increase.
This “two stages” version of TrAdaBoostR2 algorithm update separately the weights of source and target instances.
In a first stage, the weights of source instances are frozen whereas the ones of target instances are updated according to the classical AdaBoostR2 algorithm. In a second stage, the weights of target instances are now frozen whereas the ones of source instances are updated according to the TrAdaBoost algorithm.
At each first stage, a cross-validation score is computed with the labeled target data available. The CV scores obtained are used at the end to select the best estimator whithin all boosting iterations.
The algorithm performs the following steps:
1. Normalize weights: \(\sum w_S + \sum w_T = 1\).
2. Fit an AdaBoostR2 estimator \(f\) on source and target labeled data \((X_S, y_S), (X_T, y_T)\) with the respective importances initial weights: \(w_S, w_T\). During training of the AdaBoost estimator, the source weights \(w_S\) are frozen.
3. Compute a cross-validation score on \((X_T, y_T)\)
4. Compute error vectors of training instances:
\(\epsilon_S = L(f(X_S), y_S)\).
\(\epsilon_T = L(f(X_T), y_T)\).
5 Normalize error vectors:
\(\epsilon_S = \epsilon_S \setminus max_{\epsilon \in \epsilon_S \cup \epsilon_T} \epsilon\).
\(\epsilon_T = \epsilon_T \setminus max_{\epsilon \in \epsilon_S \cup \epsilon_T} \epsilon\).
6. Update source and target weights:
\(w_S = w_S \beta_S^{\epsilon_S} \setminus Z\).
\(w_T = w_T \setminus Z\).
Where:
\(Z\) is a normalizing constant.
\(\beta_S\) is chosen such that the sum of target weights \(w_T\) is equal to \(\frac{n_T}{n_T + n_S} + \frac{t}{N - 1}(1 - \frac{n_T}{n_T + n_S})\) with \(t\) the current boosting iteration number. \(\beta_S\) is found using binary search.
7. Return to step 1 and loop until the number \(N\) of boosting iteration is reached.
The prediction are then given by the best estimator according to cross-validation scores.
- Parameters
- estimatorsklearn estimator or tensorflow Model (default=None)
Estimator used to learn the task. If estimator is
None, aLinearRegressioninstance is used as estimator.- Xtnumpy array (default=None)
Target input data.
- ytnumpy array (default=None)
Target output data.
- n_estimatorsint (default=10)
Number of boosting iteration.
- n_estimators_fsint (default=10)
Number of boosting iteration in first stage (given to AdaboostR2 estimators)
- cv: int, optional (default=5)
Split cross-validation parameter.
- lrfloat (default=1.)
Learning rate. For higher
lr, the sample weights are updating faster.- copyboolean (default=True)
Whether to make a copy of
estimatoror not.- verboseint (default=1)
Verbosity level.
- random_stateint (default=None)
Seed of random generator.
- paramskey, value arguments
Arguments given at the different level of the adapt object. It can be, for instance, compile or fit parameters of the estimator or kernel parameters etc… Accepted parameters can be found by calling the method
_get_legal_params(params).
See also
References
- 1
[1] D. Pardoe and P. Stone. “Boosting for regression transfer”. In ICML, 2010.
Examples
>>> from sklearn.linear_model import Ridge >>> from adapt.utils import make_regression_da >>> from adapt.instance_based import TrAdaBoostR2 >>> Xs, ys, Xt, yt = make_regression_da() >>> model = TrAdaBoostR2(Ridge(), n_estimators=10, Xt=Xt[:10], yt=yt[:10], random_state=0) >>> model.fit(Xs, ys) Iteration 0 - Cross-validation score: 0.2956 (0.0905) Iteration 1 - Cross-validation score: 0.2956 (0.0905) Iteration 2 - Cross-validation score: 0.2614 (0.1472) Iteration 3 - Cross-validation score: 0.2701 (0.1362) Iteration 4 - Cross-validation score: 0.2745 (0.1280) Iteration 5 - Cross-validation score: 0.2768 (0.1228) Iteration 6 - Cross-validation score: 0.2782 (0.1195) Iteration 7 - Cross-validation score: 0.2783 (0.1170) Iteration 8 - Cross-validation score: 0.2767 (0.1156) Iteration 9 - Cross-validation score: 0.2702 (0.1155) >>> model.score(Xt, yt) 0.5997110100905185
- Attributes
- estimators_list of object
List of fitted AdaboostR2 estimators for each first stage.
- estimator_errors_1D array of float
Array of cross-validation MAE computed on labeled target data.
- sample_weights_src_list of numpy arrays
List of source sample weight for each iteration.
- sample_weights_tgt_list of numpy arrays
List of target sample weight for each iteration.
Methods
__init__([estimator, Xt, yt, n_estimators, ...])fit(X, y[, Xt, yt, sample_weight_src, ...])Fit TrAdaBoost
fit_estimator(X, y[, sample_weight, ...])Fit estimator on X, y.
get_params([deep])Get parameters for this estimator.
predict(X)Return predictions of the best estimator according to cross-validation scores.
predict_estimator(X, **predict_params)Return estimator predictions for X.
predict_weights([domain])Return sample weights.
score(X, y)Return the TrAdaboost score on X, y.
set_params(**params)Set the parameters of this estimator.
unsupervised_score(Xs, Xt)Return unsupervised score.
- __init__(estimator=None, Xt=None, yt=None, n_estimators=10, n_estimators_fs=10, cv=5, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]
- fit(X, y, Xt=None, yt=None, sample_weight_src=None, sample_weight_tgt=None, **fit_params)[source]
Fit TrAdaBoost
- Parameters
- Xnumpy array
Source input data.
- ynumpy array
Source output data.
- Xtarray (default=None)
Target input data. If None, the Xt argument given in init is used.
- ytarray (default=None)
Target input data. If None, the Xt argument given in init is used.
- sample_weight_srcnumpy array, (default=None)
Initial sample weight of source data
- sample_weight_tgtnumpy array, (default=None)
Initial sample weight of target data
- fit_paramskey, value arguments
Arguments given to the fit method of the estimator.
- Returns
- selfreturns an instance of self
- Other Parameters
- Xtarray (default=self.Xt)
Target input data.
- ytarray (default=self.yt)
Target output data.
- fit_estimator(X, y, sample_weight=None, random_state=None, warm_start=True, **fit_params)[source]
Fit estimator on X, y.
- Parameters
- Xarray
Input data.
- yarray
Output data.
- sample_weightarray
Importance weighting.
- random_stateint (default=None)
Seed of the random generator
- warm_startbool (default=True)
If True, continue to fit
estimator_, else, a new estimator is fitted based on a copy ofestimator. (Be sure to setcopy=Trueto usewarm_start=False)- fit_paramskey, value arguments
Arguments given to the fit method of the estimator and to the compile method for tensorflow estimator.
- Returns
- estimator_fitted estimator
- get_params(deep=True)[source]
Get parameters for this estimator.
- Parameters
- deepbool, default=True
Not used, here for scikit-learn compatibility.
- Returns
- paramsdict
Parameter names mapped to their values.
- predict(X)[source]
Return predictions of the best estimator according to cross-validation scores.
- Parameters
- Xarray
Input data.
- Returns
- y_predarray
Best estimator predictions.
- predict_estimator(X, **predict_params)[source]
Return estimator predictions for X.
- Parameters
- Xarray
input data
- Returns
- y_predarray
prediction of estimator.
- predict_weights(domain='src')[source]
Return sample weights.
Return the source importance weighting of the best estimator.
You can secify between “source” and “target” weights with the domain parameter.
- Parameters
- domainstr (default=”tgt”)
Choose between
"source", "src"and"target", "tgt".
- Returns
- weightssource sample weights
- score(X, y)[source]
Return the TrAdaboost score on X, y.
- Parameters
- Xarray
input data
- yarray
output data
- Returns
- scorefloat
estimator score.