Loading...

vastorbit.machine_learning.vast.cluster.NearestCentroid.classification_report

NearestCentroid.classification_report(metrics: None | str | list[Literal['aic', 'bic', 'accuracy', 'acc', 'balanced_accuracy', 'ba', 'auc', 'roc_auc', 'prc_auc', 'best_cutoff', 'best_threshold', 'false_discovery_rate', 'fdr', 'false_omission_rate', 'for', 'false_negative_rate', 'fnr', 'false_positive_rate', 'fpr', 'recall', 'tpr', 'precision', 'ppv', 'specificity', 'tnr', 'negative_predictive_value', 'npv', 'negative_likelihood_ratio', 'lr-', 'positive_likelihood_ratio', 'lr+', 'diagnostic_odds_ratio', 'dor', 'log_loss', 'logloss', 'f1', 'f1_score', 'mcc', 'bm', 'informedness', 'mk', 'markedness', 'ts', 'csi', 'critical_success_index', 'fowlkes_mallows_index', 'fm', 'prevalence_threshold', 'pm', 'confusion_matrix', 'classification_report']] = None, cutoff: Annotated[int | float | Decimal, 'Python Numbers'] = None, labels: None | str | list[str] = None, nbins: int = 9999) → float | TableSample

Computes a classification report using multiple model evaluation metrics (auc, accuracy, f1…). For multiclass classification, it considers each category as positive and switches to the next one during the computation.

Parameters:

  • metrics (list, optional) –

    List of the metrics used to compute the final report.

    • accuracy:

      Accuracy.

      \[Accuracy = \frac{TP + TN}{TP + TN + FP + FN}\]
    • aic:

      Akaike’s Information Criterion

      \[AIC = 2k - 2\ln(\hat{L})\]
    • auc:

      Area Under the Curve (ROC).

      \[AUC = \int_{0}^{1} TPR(FPR) \, dFPR\]
    • ba:

      Balanced Accuracy.

      \[BA = \frac{TPR + TNR}{2}\]
    • best_cutoff:

      Cutoff which optimised the ROC Curve prediction.

    • bic:

      Bayesian Information Criterion

      \[BIC = -2\ln(\hat{L}) + k \ln(n)\]
    • bm:

      Informedness

      \[BM = TPR + TNR - 1\]
    • csi:

      Critical Success Index

      \[index = \frac{TP}{TP + FN + FP}\]
    • f1:

      F1 Score

      \[F_1 Score = 2 \times \frac{Precision \times Recall}{Precision + Recall}\]
    • fdr:

      False Discovery Rate

      \[FDR = 1 - PPV\]
    • fm:

      Fowlkes-Mallows index

      \[FM = \sqrt{PPV * TPR}\]
    • fnr:

      False Negative Rate

      \[FNR = \frac{FN}{FN + TP}\]
    • for:

      False Omission Rate

      \[FOR = 1 - NPV\]
    • fpr:

      False Positive Rate

      \[FPR = \frac{FP}{FP + TN}\]
    • logloss:

      Log Loss.

      \[Loss = -\frac{1}{N} \sum_{i=1}^{N} \left( y_i \log(p_i) + (1 - y_i) \log(1 - p_i) \right)\]
    • lr+:

      Positive Likelihood Ratio.

      \[LR+ = \frac{TPR}{FPR}\]
    • lr-:

      Negative Likelihood Ratio.

      \[LR- = \frac{FNR}{TNR}\]
    • dor:

      Diagnostic Odds Ratio.

      \[DOR = \frac{TP \times TN}{FP \times FN}\]
    • mc:

      Matthews Correlation Coefficient .. math:

      MCC = \frac{TP \times TN - FP \times FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}
    • mk:

      Markedness

      \[MK = PPV + NPV - 1\]
    • npv:

      Negative Predictive Value

      \[NPV = \frac{TN}{TN + FN}\]
    • prc_auc:

      Area Under the Curve (PRC)

      \[AUC = \int_{0}^{1} Precision(Recall) \, dRecall\]
    • precision:

      Precision

      \[Precision = TP / (TP + FP)\]
    • pt:

      Prevalence Threshold.

      \[threshold = \frac{\sqrt{FPR}}{\sqrt{TPR} + \sqrt{FPR}}\]
    • recall:

      Recall.

      \[Recall = \frac{TP}{TP + FN}\]
    • specificity:

      Specificity.

      \[Specificity = \frac{TN}{TN + FP}\]

  • cutoff (PythonNumber, optional) – Cutoff for which the tested category is accepted as a prediction. For multiclass classification, each tested category becomes the positives and the others are merged into the negatives. The cutoff represents the classes threshold. If it is empty, the regular cutoff (1 / number of classes) is used.

  • labels (str | list, optional) – List of the different labels to be used during the computation.

  • nbins (int, optional) – [Used to compute ROC AUC, PRC AUC and the best cutoff] An integer value that determines the number of decision boundaries. Decision boundaries are set at equally spaced intervals between 0 and 1, inclusive. Greater values for nbins give more precise estimations of the metrics, but can potentially decrease performance. The maximum value is 999,999. If negative, the maximum value is used.

Returns:

report.

Return type:

TableSample

Examples

For this example, we will use the Iris dataset.

import vastorbit.datasets as vod
data = vod.load_iris()
train, test = data.train_test_split(test_size = 0.2)
123
sepallengthcm
Decimal(5, 2)
123
sepalwidthcm
Decimal(5, 2)
123
petallengthcm
Decimal(5, 2)
123
petalwidthcm
Decimal(5, 2)
Abc
species
Varchar(30)
15.13.51.40.2Iris-setosa
24.93.01.40.2Iris-setosa
34.73.21.30.2Iris-setosa
44.63.11.50.2Iris-setosa
55.03.61.40.2Iris-setosa
65.43.91.70.4Iris-setosa
74.63.41.40.3Iris-setosa
85.03.41.50.2Iris-setosa
94.42.91.40.2Iris-setosa
104.93.11.50.1Iris-setosa
115.43.71.50.2Iris-setosa
124.83.41.60.2Iris-setosa
134.83.01.40.1Iris-setosa
144.33.01.10.1Iris-setosa
155.84.01.20.2Iris-setosa
165.74.41.50.4Iris-setosa
175.43.91.30.4Iris-setosa
185.13.51.40.3Iris-setosa
195.73.81.70.3Iris-setosa
205.13.81.50.3Iris-setosa
215.43.41.70.2Iris-setosa
225.13.71.50.4Iris-setosa
234.63.61.00.2Iris-setosa
245.13.31.70.5Iris-setosa
254.83.41.90.2Iris-setosa
265.03.01.60.2Iris-setosa
275.03.41.60.4Iris-setosa
285.23.51.50.2Iris-setosa
295.23.41.40.2Iris-setosa
304.73.21.60.2Iris-setosa
314.83.11.60.2Iris-setosa
325.43.41.50.4Iris-setosa
335.24.11.50.1Iris-setosa
345.54.21.40.2Iris-setosa
354.93.11.50.1Iris-setosa
365.03.21.20.2Iris-setosa
375.53.51.30.2Iris-setosa
384.93.11.50.1Iris-setosa
394.43.01.30.2Iris-setosa
405.13.41.50.2Iris-setosa
415.03.51.30.3Iris-setosa
424.52.31.30.3Iris-setosa
434.43.21.30.2Iris-setosa
445.03.51.60.6Iris-setosa
455.13.81.90.4Iris-setosa
464.83.01.40.3Iris-setosa
475.13.81.60.2Iris-setosa
484.63.21.40.2Iris-setosa
495.33.71.50.2Iris-setosa
505.03.31.40.2Iris-setosa
517.03.24.71.4Iris-versicolor
526.43.24.51.5Iris-versicolor
536.93.14.91.5Iris-versicolor
545.52.34.01.3Iris-versicolor
556.52.84.61.5Iris-versicolor
565.72.84.51.3Iris-versicolor
576.33.34.71.6Iris-versicolor
584.92.43.31.0Iris-versicolor
596.62.94.61.3Iris-versicolor
605.22.73.91.4Iris-versicolor
615.02.03.51.0Iris-versicolor
625.93.04.21.5Iris-versicolor
636.02.24.01.0Iris-versicolor
646.12.94.71.4Iris-versicolor
655.62.93.61.3Iris-versicolor
666.73.14.41.4Iris-versicolor
675.63.04.51.5Iris-versicolor
685.82.74.11.0Iris-versicolor
696.22.24.51.5Iris-versicolor
705.62.53.91.1Iris-versicolor
715.93.24.81.8Iris-versicolor
726.12.84.01.3Iris-versicolor
736.32.54.91.5Iris-versicolor
746.12.84.71.2Iris-versicolor
756.42.94.31.3Iris-versicolor
766.63.04.41.4Iris-versicolor
776.82.84.81.4Iris-versicolor
786.73.05.01.7Iris-versicolor
796.02.94.51.5Iris-versicolor
805.72.63.51.0Iris-versicolor
815.52.43.81.1Iris-versicolor
825.52.43.71.0Iris-versicolor
835.82.73.91.2Iris-versicolor
846.02.75.11.6Iris-versicolor
855.43.04.51.5Iris-versicolor
866.03.44.51.6Iris-versicolor
876.73.14.71.5Iris-versicolor
886.32.34.41.3Iris-versicolor
895.63.04.11.3Iris-versicolor
905.52.54.01.3Iris-versicolor
915.52.64.41.2Iris-versicolor
926.13.04.61.4Iris-versicolor
935.82.64.01.2Iris-versicolor
945.02.33.31.0Iris-versicolor
955.62.74.21.3Iris-versicolor
965.73.04.21.2Iris-versicolor
975.72.94.21.3Iris-versicolor
986.22.94.31.3Iris-versicolor
995.12.53.01.1Iris-versicolor
1005.72.84.11.3Iris-versicolor
Rows: 1-100 | Columns: 5

Let’s import the model:

from vastorbit.machine_learning.vast import NearestCentroid

Then we can create the model:

model = NearestCentroid(p = 2)

We can now fit the model:

model.fit(
    train,
    [
        "SepalLengthCm",
        "SepalWidthCm",
        "PetalLengthCm",
        "PetalWidthCm",
    ],
    "Species",
    test,
)

We can get all the classification metrics using the classification_report:

model.classification_report()
Iris-setosaIris-versicolorIris-virginicaavg_macroavg_weightedavg_micro
auc0.94117647058823531.00.98295454545454540.97471033868092680.9713569518716577[null]
prc_auc0.278273809523809530.210532044583768620.1897167487684730.22617420095868370.23138416181519628[null]
accuracy1.00.96666666666666670.96666666666666670.97777777777777790.980.9777777777777777
log_loss0.229483600274010570.301431754281224930.279066222533111530.2699938590294490.26594283315390516[null]
precision1.00.91.00.96666666666666670.97000000000000010.9666666666666667
recall1.01.00.88888888888888880.96296296296296290.96666666666666670.9666666666666667
f1_score1.00.94736842105263160.94117647058823530.96284829721362230.96656346749226010.9666666666666667
mcc1.00.92582009977255140.92113237294367670.9489841575720760.95408574181486840.95
informedness1.00.95238095238095230.88888888888888880.9470899470899470.95238095238095230.95
markedness1.00.89999999999999990.95454545454545460.95151515151515150.95636363636363650.95
csi1.00.90.88888888888888880.92962962962962960.93666666666666680.9354838709677419
Rows: 1-11 | Columns: 7

Important

For this example, a specific model is utilized, and it may not correspond exactly to the model you are working with. To see a comprehensive example specific to your class of interest, please refer to that particular class.