.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/basic/plot_training_log.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_basic_plot_training_log.py: ========================================================= Customize ensemble training log ========================================================= This example illustrates how to enable and customize the training log when training an :mod:`imbens.ensemble` classifier. This example uses: - :class:`imbens.ensemble.SelfPacedEnsembleClassifier` .. GENERATED FROM PYTHON SOURCE LINES 13-18 .. code-block:: Python # Authors: Zhining Liu # License: MIT .. GENERATED FROM PYTHON SOURCE LINES 19-33 .. code-block:: Python print(__doc__) # Import imbalanced-ensemble import imbens # Import utilities import sklearn from sklearn.datasets import make_classification from sklearn.model_selection import train_test_split RANDOM_STATE = 42 # sphinx_gallery_thumbnail_path = '../../docs/source/_static/training_log_thumbnail.png' .. GENERATED FROM PYTHON SOURCE LINES 34-37 Prepare data ---------------------------- Make a toy 3-class imbalanced classification task. .. GENERATED FROM PYTHON SOURCE LINES 37-57 .. code-block:: Python # make dataset X, y = make_classification( n_classes=3, class_sep=2, weights=[0.1, 0.3, 0.6], n_informative=3, n_redundant=1, flip_y=0, n_features=20, n_clusters_per_class=2, n_samples=2000, random_state=0, ) # train valid split X_train, X_valid, y_train, y_valid = train_test_split( X, y, test_size=0.5, stratify=y, random_state=RANDOM_STATE ) .. GENERATED FROM PYTHON SOURCE LINES 58-65 Customize training log --------------------------------------------------------------------------- Take ``SelfPacedEnsembleClassifier`` as example, training log is controlled by 3 parameters of the ``fit()`` method: - ``eval_datasets``: Dataset(s) used for evaluation during the ensemble training. - ``eval_metrics``: Metric(s) used for evaluation during the ensemble training. - ``train_verbose``: Controls the granularity and content of the training log. .. GENERATED FROM PYTHON SOURCE LINES 65-68 .. code-block:: Python clf = imbens.ensemble.SelfPacedEnsembleClassifier(random_state=RANDOM_STATE) .. GENERATED FROM PYTHON SOURCE LINES 69-72 Set training log format ----------------------- (``fit()`` parameter: ``train_verbose``: bool, int or dict) .. GENERATED FROM PYTHON SOURCE LINES 74-75 **Enable auto training log** .. GENERATED FROM PYTHON SOURCE LINES 75-83 .. code-block:: Python clf.fit( X_train, y_train, train_verbose=True, ) .. rst-class:: sphx-glr-script-out .. code-block:: none ┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ Data: train ┃ ┃ #Estimators ┃ Class Distribution ┃ Metric ┃ ┃ ┃ ┃ acc balanced_acc weighted_f1 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ 1 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 0.944 0.955 0.945 ┃ ┃ 5 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 0.991 0.991 0.991 ┃ ┃ 10 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 0.999 0.997 0.999 ┃ ┃ 15 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┃ 20 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 0.999 0.997 0.999 ┃ ┃ 25 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┃ 30 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┃ 35 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┃ 40 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┃ 45 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┃ 50 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ final ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┗━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ .. raw:: html 
SelfPacedEnsembleClassifier(random_state=RandomState(MT19937) at 0x7C5E51B77040)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 84-85 **Customize training log granularity** .. GENERATED FROM PYTHON SOURCE LINES 85-95 .. code-block:: Python clf.fit( X_train, y_train, train_verbose={ 'granularity': 10, }, ) .. rst-class:: sphx-glr-script-out .. code-block:: none ┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ Data: train ┃ ┃ #Estimators ┃ Class Distribution ┃ Metric ┃ ┃ ┃ ┃ acc balanced_acc weighted_f1 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ 1 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 0.958 0.965 0.958 ┃ ┃ 10 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 0.999 0.999 0.999 ┃ ┃ 20 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┃ 30 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┃ 40 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┃ 50 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ final ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ ┗━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ .. raw:: html 
SelfPacedEnsembleClassifier(random_state=RandomState(MT19937) at 0x7C5E51B77040)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 96-97 **Customize training log content column** .. GENERATED FROM PYTHON SOURCE LINES 97-109 .. code-block:: Python clf.fit( X_train, y_train, train_verbose={ 'granularity': 10, 'print_distribution': False, 'print_metrics': True, }, ) .. rst-class:: sphx-glr-script-out .. code-block:: none ┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ ┃ Data: train ┃ ┃ #Estimators ┃ Metric ┃ ┃ ┃ acc balanced_acc weighted_f1 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ 1 ┃ 0.936 0.947 0.937 ┃ ┃ 10 ┃ 0.999 0.997 0.999 ┃ ┃ 20 ┃ 0.999 0.997 0.999 ┃ ┃ 30 ┃ 1.000 1.000 1.000 ┃ ┃ 40 ┃ 1.000 1.000 1.000 ┃ ┃ 50 ┃ 1.000 1.000 1.000 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ final ┃ 1.000 1.000 1.000 ┃ ┗━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ .. raw:: html 
SelfPacedEnsembleClassifier(random_state=RandomState(MT19937) at 0x7C5E51B77040)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 110-113 Add additional evaluation dataset(s) ------------------------------------ (``fit()`` parameter: ``eval_datasets``: dict) .. GENERATED FROM PYTHON SOURCE LINES 113-126 .. code-block:: Python clf.fit( X_train, y_train, eval_datasets={ 'valid': (X_valid, y_valid), # add validation data }, train_verbose={ 'granularity': 10, }, ) .. rst-class:: sphx-glr-script-out .. code-block:: none ┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ Data: train ┃ Data: valid ┃ ┃ #Estimators ┃ Class Distribution ┃ Metric ┃ Metric ┃ ┃ ┃ ┃ acc balanced_acc weighted_f1 ┃ acc balanced_acc weighted_f1 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ 1 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 0.937 0.945 0.938 ┃ 0.909 0.899 0.911 ┃ ┃ 10 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ 0.969 0.962 0.969 ┃ ┃ 20 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ 0.966 0.954 0.966 ┃ ┃ 30 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ 0.964 0.947 0.964 ┃ ┃ 40 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ 0.966 0.946 0.966 ┃ ┃ 50 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ 0.966 0.949 0.966 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ ┃ final ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 1.000 1.000 ┃ 0.966 0.949 0.966 ┃ ┗━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ .. raw:: html 
SelfPacedEnsembleClassifier(random_state=RandomState(MT19937) at 0x7C5E51B77040)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. GENERATED FROM PYTHON SOURCE LINES 127-130 Specify evaluation metric(s) ---------------------------- (``fit()`` parameter: ``eval_metrics``: dict) .. GENERATED FROM PYTHON SOURCE LINES 130-148 .. code-block:: Python clf.fit( X_train, y_train, eval_datasets={ 'valid': (X_valid, y_valid), }, eval_metrics={ 'weighted_f1': ( sklearn.metrics.f1_score, {'average': 'weighted'}, ), # use weighted_f1 }, train_verbose={ 'granularity': 10, }, ) .. rst-class:: sphx-glr-script-out .. code-block:: none ┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓ ┃ ┃ ┃ Data: train ┃ Data: valid ┃ ┃ #Estimators ┃ Class Distribution ┃ Metric ┃ Metric ┃ ┃ ┃ ┃ weighted_f1 ┃ weighted_f1 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━╋━━━━━━━━━━━━━┫ ┃ 1 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 0.945 ┃ 0.906 ┃ ┃ 10 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 ┃ 0.969 ┃ ┃ 20 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 ┃ 0.975 ┃ ┃ 30 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 ┃ 0.974 ┃ ┃ 40 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 ┃ 0.974 ┃ ┃ 50 ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 ┃ 0.973 ┃ ┣━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━╋━━━━━━━━━━━━━┫ ┃ final ┃ {np.int64(0): 100, np.int64(1): 100, np.int64(2): 100} ┃ 1.000 ┃ 0.973 ┃ ┗━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━┻━━━━━━━━━━━━━┛ .. raw:: html 
SelfPacedEnsembleClassifier(random_state=RandomState(MT19937) at 0x7C5E51B77040)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.


.. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 1.651 seconds) .. _sphx_glr_download_auto_examples_basic_plot_training_log.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_training_log.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_training_log.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_training_log.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_