An AdaBoost classifier.
An AdaBoost 1 classifier is a meta-estimator that begins by fitting a classifier on the original dataset and then fits additional copies of the classifier on the same dataset but where the weights of incorrectly classified instances are adjusted such that subsequent classifiers focus more on difficult cases.
This class implements the algorithm known as AdaBoost-SAMME 2.
Read more in the :ref:`User Guide <adaboost>`.
.. versionadded:: 0.14
Parameters ---------- base_estimator : object, default=None The base estimator from which the boosted ensemble is built. Support for sample weighting is required, as well as proper ``classes_`` and ``n_classes_`` attributes. If ``None``, then the base estimator is ``DecisionTreeClassifier(max_depth=1)``.
n_estimators : int, default=50 The maximum number of estimators at which boosting is terminated. In case of perfect fit, the learning procedure is stopped early.
learning_rate : float, default=1. Learning rate shrinks the contribution of each classifier by ``learning_rate``. There is a trade-off between ``learning_rate`` and ``n_estimators``.
algorithm : 'SAMME', 'SAMME.R', default='SAMME.R' If 'SAMME.R' then use the SAMME.R real boosting algorithm. ``base_estimator`` must support calculation of class probabilities. If 'SAMME' then use the SAMME discrete boosting algorithm. The SAMME.R algorithm typically converges faster than SAMME, achieving a lower test error with fewer boosting iterations.
random_state : int or RandomState, default=None Controls the random seed given at each `base_estimator` at each boosting iteration. Thus, it is only used when `base_estimator` exposes a `random_state`. Pass an int for reproducible output across multiple function calls. See :term:`Glossary <random_state>`.
Attributes ---------- base_estimator_ : estimator The base estimator from which the ensemble is grown.
estimators_ : list of classifiers The collection of fitted sub-estimators.
classes_ : ndarray of shape (n_classes,) The classes labels.
n_classes_ : int The number of classes.
estimator_weights_ : ndarray of floats Weights for each estimator in the boosted ensemble.
estimator_errors_ : ndarray of floats Classification error for each estimator in the boosted ensemble.
feature_importances_ : ndarray of shape (n_features,) The impurity-based feature importances if supported by the ``base_estimator`` (when based on decision trees).
Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values). See :func:`sklearn.inspection.permutation_importance` as an alternative.
See Also -------- AdaBoostRegressor An AdaBoost regressor that begins by fitting a regressor on the original dataset and then fits additional copies of the regressor on the same dataset but where the weights of instances are adjusted according to the error of the current prediction.
GradientBoostingClassifier GB builds an additive model in a forward stage-wise fashion. Regression trees are fit on the negative gradient of the binomial or multinomial deviance loss function. Binary classification is a special case where only a single regression tree is induced.
sklearn.tree.DecisionTreeClassifier A non-parametric supervised learning method used for classification. Creates a model that predicts the value of a target variable by learning simple decision rules inferred from the data features.
References ---------- .. 1 Y. Freund, R. Schapire, 'A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting', 1995.
.. 2 J. Zhu, H. Zou, S. Rosset, T. Hastie, 'Multi-class AdaBoost', 2009.
Examples -------- >>> from sklearn.ensemble import AdaBoostClassifier >>> from sklearn.datasets import make_classification >>> X, y = make_classification(n_samples=1000, n_features=4, ... n_informative=2, n_redundant=0, ... random_state=0, shuffle=False) >>> clf = AdaBoostClassifier(n_estimators=100, random_state=0) >>> clf.fit(X, y) AdaBoostClassifier(n_estimators=100, random_state=0) >>> clf.predict([0, 0, 0, 0]) array(1) >>> clf.score(X, y) 0.983...