adapt.instance_based.TrAdaBoost
- class adapt.instance_based.TrAdaBoost(estimator=None, Xt=None, yt=None, n_estimators=10, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]
Transfer AdaBoost for Classification
TrAdaBoost algorithm is a supervised instances-based domain adaptation method suited for classification tasks.
The method is based on a “reverse boosting” principle where the weights of source instances poorly predicted decrease at each boosting iteration whereas the ones of target instances increase.
The algorithm performs the following steps:
1. Normalize weights: \(\sum w_S + \sum w_T = 1\).
2. Fit an estimator \(f\) on source and target labeled data \((X_S, y_S), (X_T, y_T)\) with the respective importances weights: \(w_S, w_T\).
3. Compute error vectors of training instances:
\(\epsilon_S = L_{01}(f(X_S), y_S)\).
\(\epsilon_T = L_{01}(f(X_T), y_T)\).
4. Compute total weighted error of target instances: \(E_T = \frac{1}{n_T} w_T^T \epsilon_T\).
5. Update source and target weights:
\(w_S = w_S \beta^{\epsilon_S}\).
\(w_T = w_T \beta_T^{-\epsilon_T}\).
Where:
\(\beta = 1 \setminus (1 + \sqrt{2 \text{ln} n_S \setminus N})\).
\(\beta_T = E_T \setminus (1 - E_T)\).
6. Return to step 1 and loop until the number \(N\) of boosting iteration is reached.
The prediction are then given by the vote of the \(N \setminus 2\) last computed estimators weighted by their respective parameter \(\beta_T\).
- Parameters
- estimatorsklearn estimator or tensorflow Model (default=None)
Estimator used to learn the task. If estimator is
None, aLinearRegressioninstance is used as estimator.- Xtnumpy array (default=None)
Target input data.
- ytnumpy array (default=None)
Target output data.
- n_estimatorsint (default=10)
Number of boosting iteration.
- lrfloat (default=1.)
Learning rate. For higher
lr, the sample weights are updating faster.- copyboolean (default=True)
Whether to make a copy of
estimatoror not.- verboseint (default=1)
Verbosity level.
- random_stateint (default=None)
Seed of random generator.
- paramskey, value arguments
Arguments given at the different level of the adapt object. It can be, for instance, compile or fit parameters of the estimator or kernel parameters etc… Accepted parameters can be found by calling the method
_get_legal_params(params).
See also
References
- 1
[1] Dai W., Yang Q., Xue G., and Yu Y. “Boosting for transfer learning”. In ICML, 2007.
Examples
>>> from sklearn.linear_model import RidgeClassifier >>> from adapt.utils import make_classification_da >>> from adapt.instance_based import TrAdaBoost >>> Xs, ys, Xt, yt = make_classification_da() >>> model = TrAdaBoost(RidgeClassifier(), n_estimators=10, Xt=Xt[:10], yt=yt[:10], random_state=0) >>> model.fit(Xs, ys) Iteration 0 - Error: 0.2550 Iteration 1 - Error: 0.2820 Iteration 2 - Error: 0.3011 Iteration 3 - Error: 0.3087 Iteration 4 - Error: 0.3046 Iteration 5 - Error: 0.2933 Iteration 6 - Error: 0.2819 Iteration 7 - Error: 0.2747 Iteration 8 - Error: 0.2712 Iteration 9 - Error: 0.2698 >>> model.score(Xt, yt) 0.66
- Attributes
- estimators_list of object
List of fitted estimators
- estimator_errors_1D array of float
Array of weighted estimator errors computed on labeled target data.
- estimator_weights_1D array of float
Array of estimator importance weights.
- sample_weights_src_list of numpy arrays
List of source sample weight for each iteration.
- sample_weights_tgt_list of numpy arrays
List of target sample weight for each iteration.
Methods
__init__([estimator, Xt, yt, n_estimators, ...])fit(X, y[, Xt, yt, sample_weight_src, ...])Fit TrAdaBoost
fit_estimator(X, y[, sample_weight, ...])Fit estimator on X, y.
get_params([deep])Get parameters for this estimator.
predict(X)Return weighted vote of estimators.
predict_estimator(X, **predict_params)Return estimator predictions for X.
predict_weights([domain])Return sample weights.
score(X, y)Return the TrAdaboost score on X, y.
set_params(**params)Set the parameters of this estimator.
unsupervised_score(Xs, Xt)Return unsupervised score.
- __init__(estimator=None, Xt=None, yt=None, n_estimators=10, lr=1.0, copy=True, verbose=1, random_state=None, **params)[source]
- fit(X, y, Xt=None, yt=None, sample_weight_src=None, sample_weight_tgt=None, **fit_params)[source]
Fit TrAdaBoost
- Parameters
- Xnumpy array
Source input data.
- ynumpy array
Source output data.
- Xtarray (default=None)
Target input data. If None, the Xt argument given in init is used.
- ytarray (default=None)
Target input data. If None, the Xt argument given in init is used.
- sample_weight_srcnumpy array, (default=None)
Initial sample weight of source data
- sample_weight_tgtnumpy array, (default=None)
Initial sample weight of target data
- fit_paramskey, value arguments
Arguments given to the fit method of the estimator.
- Returns
- selfreturns an instance of self
- Other Parameters
- Xtarray (default=self.Xt)
Target input data.
- ytarray (default=self.yt)
Target output data.
- fit_estimator(X, y, sample_weight=None, random_state=None, warm_start=True, **fit_params)[source]
Fit estimator on X, y.
- Parameters
- Xarray
Input data.
- yarray
Output data.
- sample_weightarray
Importance weighting.
- random_stateint (default=None)
Seed of the random generator
- warm_startbool (default=True)
If True, continue to fit
estimator_, else, a new estimator is fitted based on a copy ofestimator. (Be sure to setcopy=Trueto usewarm_start=False)- fit_paramskey, value arguments
Arguments given to the fit method of the estimator and to the compile method for tensorflow estimator.
- Returns
- estimator_fitted estimator
- get_params(deep=True)[source]
Get parameters for this estimator.
- Parameters
- deepbool, default=True
Not used, here for scikit-learn compatibility.
- Returns
- paramsdict
Parameter names mapped to their values.
- predict(X)[source]
Return weighted vote of estimators.
- Parameters
- Xarray
Input data.
- Returns
- y_predarray
Vote results.
- predict_estimator(X, **predict_params)[source]
Return estimator predictions for X.
- Parameters
- Xarray
input data
- Returns
- y_predarray
prediction of estimator.
- predict_weights(domain='src')[source]
Return sample weights.
Return the final importance weighting.
You can secify between “source” and “target” weights with the domain parameter.
- Parameters
- domainstr (default=”tgt”)
Choose between
"source", "src"and"target", "tgt".
- Returns
- weightssource sample weights
- score(X, y)[source]
Return the TrAdaboost score on X, y.
- Parameters
- Xarray
input data
- yarray
output data
- Returns
- scorefloat
estimator score.
