adapt.instance_based.TrAdaBoostR2
- class adapt.instance_based.TrAdaBoostR2(estimator=None, Xt=None, yt=None, n_estimators=10, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]
Transfer AdaBoost for Regression
TrAdaBoostR2 algorithm is a supervised instances-based domain adaptation method suited for regression tasks.
The method is based on a “reverse boosting” principle where the weights of source instances poorly predicted decrease at each boosting iteration whereas the ones of target instances increase.
The algorithm performs the following steps:
1. Normalize weights: \(\sum w_S + \sum w_T = 1\).
2. Fit an estimator \(f\) on source and target labeled data \((X_S, y_S), (X_T, y_T)\) with the respective importances weights: \(w_S, w_T\).
3. Compute error vectors of training instances:
\(\epsilon_S = L(f(X_S), y_S)\).
\(\epsilon_T = L(f(X_T), y_T)\).
4 Normalize error vectors:
\(\epsilon_S = \epsilon_S \setminus max_{\epsilon \in \epsilon_S \cup \epsilon_T} \epsilon\).
\(\epsilon_T = \epsilon_T \setminus max_{\epsilon \in \epsilon_S \cup \epsilon_T} \epsilon\).
5. Compute total weighted error of target instances: \(E_T = \frac{1}{n_T} w_T^T \epsilon_T\).
6. Update source and target weights:
\(w_S = w_S \beta^{\epsilon_S}\).
\(w_T = w_T \beta_T^{-\epsilon_T}\).
Where:
\(\beta = 1 \setminus (1 + \sqrt{2 \text{ln} n_S \setminus N})\).
\(\beta_T = E_T \setminus (1 - E_T)\).
7. Return to step 1 and loop until the number \(N\) of boosting iteration is reached.
The prediction are then given by the weighted median of the \(N \setminus 2\) last estimators.
- Parameters
- estimatorsklearn estimator or tensorflow Model (default=None)
Estimator used to learn the task. If estimator is
None, aLinearRegressioninstance is used as estimator.- Xtnumpy array (default=None)
Target input data.
- ytnumpy array (default=None)
Target output data.
- n_estimatorsint (default=10)
Number of boosting iteration.
- lrfloat (default=1.)
Learning rate. For higher
lr, the sample weights are updating faster.- copyboolean (default=True)
Whether to make a copy of
estimatoror not.- verboseint (default=1)
Verbosity level.
- random_stateint (default=None)
Seed of random generator.
- paramskey, value arguments
Arguments given at the different level of the adapt object. It can be, for instance, compile or fit parameters of the estimator or kernel parameters etc… Accepted parameters can be found by calling the method
_get_legal_params(params).
See also
References
- 1
[1] D. Pardoe and P. Stone. “Boosting for regression transfer”. In ICML, 2010.
Examples
>>> from sklearn.linear_model import Ridge >>> from adapt.utils import make_regression_da >>> from adapt.instance_based import TrAdaBoostR2 >>> Xs, ys, Xt, yt = make_regression_da() >>> model = TrAdaBoostR2(Ridge(), n_estimators=10, Xt=Xt[:10], yt=yt[:10], random_state=0) >>> model.fit(Xs, ys) Iteration 0 - Error: 0.4862 Iteration 1 - Error: 0.5711 Iteration 2 - Error: 0.6709 Iteration 3 - Error: 0.7095 Iteration 4 - Error: 0.7154 Iteration 5 - Error: 0.6987 Iteration 6 - Error: 0.6589 Iteration 7 - Error: 0.5907 Iteration 8 - Error: 0.4930 Iteration 9 - Error: 0.3666 >>> model.score(Xt, yt) 0.6998064452649377
- Attributes
- estimators_list of object
List of fitted estimators
- estimator_errors_1D array of float
Array of weighted estimator errors computed on labeled target data.
- estimator_weights_1D array of float
Array of estimator importance weights.
- sample_weights_src_list of numpy arrays
List of source sample weight for each iteration.
- sample_weights_tgt_list of numpy arrays
List of target sample weight for each iteration.
Methods
__init__([estimator, Xt, yt, n_estimators, ...])fit(X, y[, Xt, yt, sample_weight_src, ...])Fit TrAdaBoost
fit_estimator(X, y[, sample_weight, ...])Fit estimator on X, y.
get_params([deep])Get parameters for this estimator.
predict(X)Return weighted median of estimators.
predict_estimator(X, **predict_params)Return estimator predictions for X.
predict_weights([domain])Return sample weights.
score(X, y)Return the TrAdaboost score on X, y.
set_params(**params)Set the parameters of this estimator.
unsupervised_score(Xs, Xt)Return unsupervised score.
- __init__(estimator=None, Xt=None, yt=None, n_estimators=10, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]
- fit(X, y, Xt=None, yt=None, sample_weight_src=None, sample_weight_tgt=None, **fit_params)[source]
Fit TrAdaBoost
- Parameters
- Xnumpy array
Source input data.
- ynumpy array
Source output data.
- Xtarray (default=None)
Target input data. If None, the Xt argument given in init is used.
- ytarray (default=None)
Target input data. If None, the Xt argument given in init is used.
- sample_weight_srcnumpy array, (default=None)
Initial sample weight of source data
- sample_weight_tgtnumpy array, (default=None)
Initial sample weight of target data
- fit_paramskey, value arguments
Arguments given to the fit method of the estimator.
- Returns
- selfreturns an instance of self
- Other Parameters
- Xtarray (default=self.Xt)
Target input data.
- ytarray (default=self.yt)
Target output data.
- fit_estimator(X, y, sample_weight=None, random_state=None, warm_start=True, **fit_params)[source]
Fit estimator on X, y.
- Parameters
- Xarray
Input data.
- yarray
Output data.
- sample_weightarray
Importance weighting.
- random_stateint (default=None)
Seed of the random generator
- warm_startbool (default=True)
If True, continue to fit
estimator_, else, a new estimator is fitted based on a copy ofestimator. (Be sure to setcopy=Trueto usewarm_start=False)- fit_paramskey, value arguments
Arguments given to the fit method of the estimator and to the compile method for tensorflow estimator.
- Returns
- estimator_fitted estimator
- get_params(deep=True)[source]
Get parameters for this estimator.
- Parameters
- deepbool, default=True
Not used, here for scikit-learn compatibility.
- Returns
- paramsdict
Parameter names mapped to their values.
- predict(X)[source]
Return weighted median of estimators.
- Parameters
- Xarray
Input data.
- Returns
- y_predarray
Median results.
- predict_estimator(X, **predict_params)[source]
Return estimator predictions for X.
- Parameters
- Xarray
input data
- Returns
- y_predarray
prediction of estimator.
- predict_weights(domain='src')[source]
Return sample weights.
Return the final importance weighting.
You can secify between “source” and “target” weights with the domain parameter.
- Parameters
- domainstr (default=”tgt”)
Choose between
"source", "src"and"target", "tgt".
- Returns
- weightssource sample weights
- score(X, y)[source]
Return the TrAdaboost score on X, y.
- Parameters
- Xarray
input data
- yarray
output data
- Returns
- scorefloat
estimator score.
