Loading...

vastorbit.VastFrame.score

VastFrame.score(y_true: str, y_score: str, metric: Literal['aic', 'bic', 'accuracy', 'acc', 'balanced_accuracy', 'ba', 'auc', 'roc_auc', 'prc_auc', 'best_cutoff', 'best_threshold', 'false_discovery_rate', 'fdr', 'false_omission_rate', 'for', 'false_negative_rate', 'fnr', 'false_positive_rate', 'fpr', 'recall', 'tpr', 'precision', 'ppv', 'specificity', 'tnr', 'negative_predictive_value', 'npv', 'negative_likelihood_ratio', 'lr-', 'positive_likelihood_ratio', 'lr+', 'diagnostic_odds_ratio', 'dor', 'log_loss', 'logloss', 'f1', 'f1_score', 'mcc', 'bm', 'informedness', 'mk', 'markedness', 'ts', 'csi', 'critical_success_index', 'fowlkes_mallows_index', 'fm', 'prevalence_threshold', 'pm', 'confusion_matrix', 'classification_report', 'r2', 'rsquared', 'mae', 'mean_absolute_error', 'mse', 'mean_squared_error', 'msle', 'mean_squared_log_error', 'max', 'max_error', 'median', 'median_absolute_error', 'var', 'explained_variance']) float

Computes the score using the input columns and the input metric.

Parameters:
  • y_true (str) – Response column.

  • y_score (str) – Prediction.

  • metric (str) –

    The metric used to compute the score.

    For Classification

    • accuracy:

      Accuracy.

      \[Accuracy = \frac{TP + TN}{TP + TN + FP + FN}\]
    • auc:

      Area Under the Curve (ROC).

      \[AUC = \int_{0}^{1} TPR(FPR) \, dFPR\]
    • ba:

      Balanced Accuracy.

      \[BA = \frac{TPR + TNR}{2}\]
    • best_cutoff:

      Cutoff which optimised the ROC Curve prediction.

    • bm:

      Informedness.

      \[BA = TPR + TNR - 1\]
    • csi:

      Critical Success Index.

      \[index = \frac{TP}{TP + FN + FP}\]
    • f1:

      F1 Score

      \[F_1 Score = 2 \times \frac{Precision \times Recall}{Precision + Recall}\]
    • fdr:

      False Discovery Rate.

      \[FDR = 1 - PPV\]
    • fm:

      Fowlkes-Mallows index.

      \[FM = \sqrt{PPV * TPR}\]
    • fnr:

      False Negative Rate.

      \[FNR = \frac{FN}{FN + TP}\]
    • for:

      False Omission Rate.

      \[FOR = 1 - NPV\]
    • fpr:

      False Positive Rate.

      \[\frac{FP}{FP + TN}\]
    • logloss:

      Log Loss.

      \[Loss = -\frac{1}{N} \sum_{i=1}^{N} \left( y_i \log(p_i) + (1 - y_i) \log(1 - p_i) \right)\]
    • lr+:

      Positive Likelihood Ratio.

      \[LR+ = \frac{TPR}{FPR}\]
    • lr-:

      Negative Likelihood Ratio.

      \[LR- = \frac{FNR}{TNR}\]
    • dor:

      Diagnostic Odds Ratio.

      \[DOR = \frac{TP \times TN}{FP \times FN}\]
    • mcc:

      Matthews Correlation Coefficient.

      \[MCC = \frac{TP \times TN - FP \times FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}\]
    • mk:

      Markedness.

      \[MK = PPV + NPV - 1\]
    • npv:

      Negative Predictive Value.

      \[NPV = \frac{TN}{TN + FN}\]
    • prc_auc:

      Area Under the Curve (PRC).

      \[AUC = \int_{0}^{1} Precision(Recall) \, dRecall\]
    • precision:

      Precision.

      \[Precision = TP / (TP + FP)\]
    • pt:

      Prevalence Threshold.

      \[threshold = \frac{\sqrt{FPR}}{\sqrt{TPR} + \sqrt{FPR}}\]
    • recall:

      Recall.

      \[Recall = \frac{TP}{TP + FN}\]
    • specificity:

      Specificity.

      \[Specificity = \frac{TN}{TN + FP}\]

    For Regression

    • max:

      Max Error.

      \[ME = \max_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]
    • mae:

      Mean Absolute Error.

      \[MAE = \frac{1}{n} \sum_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]
    • median:

      Median Absolute Error.

      \[MedAE = \text{median}_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]
    • mse:

      Mean Squared Error.

      \[MSE = \frac{1}{n} \sum_{i=1}^{n} \left( y_i - \hat{y}_i \right)^2\]
    • msle:

      Mean Squared Log Error.

      \[MSLE = \frac{1}{n} \sum_{i=1}^{n} (\log(1 + y_i) - \log(1 + \hat{y}_i))^2\]
    • r2:

      R squared coefficient.

      \[R^2 = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2}\]
    • var:

      Explained Variance.

      \[\text{Explained Variance} = 1 - \frac{Var(y - \hat{y})}{Var(y)}\]

Returns:

score.

Return type:

float

Examples

Let us build a quick ML model and calculate the score of its predictions.

Load data for machine learning

For this example, we will use the winequality dataset.

import vastorbit.datasets as vod

data = vod.load_winequality()
123
fixed_acidity
Decimal(6, 3)
123
volatile_acidity
Decimal(7, 4)
123
citric_acid
Decimal(6, 3)
123
residual_sugar
Decimal(7, 3)
123
chlorides
Double
123
free_sulfur_dioxide
Decimal(7, 2)
123
total_sulfur_dioxide
Decimal(7, 2)
123
density
Double
123
ph
Decimal(6, 3)
123
sulphates
Decimal(6, 3)
123
alcohol
Double
123
quality
Integer
123
good
Integer
Abc
color
Varchar(20)
16.30.670.4812.60.05257.0222.00.99793.170.529.360white
27.40.40.295.40.04431.0122.00.9943.30.511.181white
37.10.260.312.20.04429.0128.00.99373.340.6410.981white
49.00.310.486.60.04311.073.00.99382.90.3811.650white
56.30.390.246.90.0699.0117.00.99423.150.3510.240white
68.20.220.366.80.03412.090.00.99443.010.3810.581white
77.10.190.283.60.03316.078.00.9932.910.7811.460white
87.30.250.3613.10.0535.0200.00.99863.040.468.971white
97.90.20.341.20.0429.0118.00.99323.140.4110.660white
107.10.260.325.90.03739.097.00.99343.310.411.660white
117.00.20.345.70.03532.083.00.99283.190.4611.560white
126.90.30.334.10.03526.0155.00.99253.250.7912.381white
138.10.290.497.10.04222.0124.00.99443.140.4110.860white
145.80.170.31.40.03755.0130.00.99093.290.3811.360white
155.90.4150.020.80.03822.063.00.99323.360.369.350white
166.60.230.261.30.04516.0128.00.99343.360.610.060white
178.60.550.3515.550.05735.5366.51.00013.040.6311.030white
186.90.350.741.00.04418.0132.00.9923.130.5510.250white
197.60.140.741.60.0427.0103.00.99163.070.410.871white
209.20.280.4911.80.04229.0137.00.9983.10.3410.140white
216.20.180.494.50.04717.090.00.99193.270.3711.660white
225.30.1650.241.10.05125.0105.00.99253.320.479.150white
239.80.250.7410.00.05636.0225.00.99773.060.4310.040white
248.10.290.497.10.04222.0124.00.99443.140.4110.860white
256.80.220.490.90.05226.0128.00.9913.250.3511.460white
267.20.220.491.00.04534.0140.00.993.050.3412.760white
277.40.250.491.10.04235.0156.00.99173.130.5511.350white
288.20.180.491.10.03328.081.00.99233.00.6810.471white
296.10.220.491.50.05118.087.00.99283.30.469.650white
307.00.390.241.00.0488.0119.00.99233.00.3110.140white
316.10.220.491.50.05118.087.00.99283.30.469.650white
326.50.360.492.90.0316.094.00.99023.10.4912.171white
337.10.290.491.20.03132.099.00.98933.070.3312.260white
347.40.250.491.10.04235.0156.00.99173.130.5511.350white
356.90.230.2414.20.05319.094.00.99823.170.59.650white
368.50.560.7417.850.05151.0243.01.00052.990.79.250white
378.20.180.491.10.03328.081.00.99233.00.6810.471white
386.30.230.497.10.0567.0210.00.99513.230.349.550white
396.10.250.497.60.05267.0226.00.99563.160.478.950white
407.20.260.7413.60.0556.0162.00.9983.030.448.850white
417.20.310.241.40.05717.0117.00.99283.160.3510.550white
428.00.250.491.20.06127.0117.00.99383.080.349.450white
437.00.180.495.30.0434.0125.00.99143.240.412.260white
447.80.430.4913.00.03337.0158.00.99553.140.3511.360white
458.30.20.744.450.04433.0130.00.99243.250.4212.260white
466.30.270.491.20.06335.092.00.99113.380.4212.260white
477.40.160.491.20.05518.0150.00.99173.230.4711.260white
487.40.160.491.20.05518.0150.00.99173.230.4711.260white
496.90.190.496.60.03649.0172.00.99323.20.2711.560white
507.80.430.4913.00.03337.0158.00.99553.140.3511.360white
517.20.40.491.10.04811.0138.00.99293.010.429.350white
527.80.430.4913.00.03337.0158.00.99553.140.3511.360white
537.60.520.4914.00.03437.0156.00.99583.140.3811.871white
548.30.210.4919.80.05450.0231.01.00122.990.549.250white
556.90.340.7411.20.06944.0150.00.99683.00.819.250white
566.30.270.491.20.06335.092.00.99113.380.4212.260white
578.30.20.744.450.04433.0130.00.99243.250.4212.260white
587.10.220.742.70.04442.0144.00.9913.310.4112.260white
597.90.110.494.50.04827.0133.00.99463.240.4210.660white
608.50.170.743.60.0529.0128.00.99283.280.412.460white
616.40.1450.495.40.04854.0164.00.99463.560.4410.860white
627.40.160.491.20.05518.0150.00.99173.230.4711.260white
638.30.190.491.20.05111.0137.00.99183.060.4611.060white
648.00.440.499.10.03146.0151.00.99263.160.2712.781white
657.00.20.740.80.04419.0163.00.99313.460.5310.250white
666.90.190.496.60.03649.0172.00.99323.20.2711.560white
677.10.250.493.00.0330.096.00.99033.130.3912.371white
686.50.240.241.60.04615.060.00.99283.190.399.850white
697.20.40.491.10.04811.0138.00.99293.010.429.350white
707.60.520.4914.00.03437.0156.00.99583.140.3811.871white
717.80.430.4913.00.03337.0158.00.99553.140.3511.360white
727.80.210.491.350.0526.048.00.99113.150.2811.450white
737.00.20.495.90.03839.0128.00.99383.210.4810.860white
746.90.250.243.60.05713.085.00.99422.990.489.540white
757.20.080.491.30.0518.0148.00.99453.460.4410.260white
767.10.850.498.70.02840.0184.00.99623.220.3610.750white
777.60.510.241.20.0410.0104.00.9923.050.2910.860white
787.90.220.244.60.04439.0159.00.99272.990.2811.560white
797.70.160.492.00.05620.0124.00.99483.320.4910.760white
807.20.080.491.30.0518.0148.00.99453.460.4410.260white
816.60.250.241.70.04826.0124.00.99423.370.610.160white
826.70.160.492.40.04657.0187.00.99523.620.8110.460white
836.90.250.243.60.05713.085.00.99422.990.489.540white
847.50.320.244.60.0538.0134.00.99583.140.59.130white
857.40.280.491.50.03420.0126.00.99182.980.3910.660white
866.20.150.490.90.03317.051.00.99323.30.79.460white
876.70.250.7419.40.05444.0169.01.00043.510.459.860white
886.50.260.7413.30.04468.0224.00.99723.180.549.560white
897.90.160.7417.850.03752.0187.00.99982.990.419.350white
905.60.1850.491.10.0328.0117.00.99183.550.4510.360white
917.50.20.491.30.0318.097.00.99183.060.6211.150white
928.00.30.499.40.04647.0188.00.99643.140.4810.050white
938.00.340.499.00.03339.0180.00.99363.130.3812.381white
947.70.350.498.650.03342.0186.00.99313.140.3812.481white
957.60.290.499.60.0345.0197.00.99383.130.3812.371white
966.70.620.241.10.0396.062.00.99343.410.3210.450white
976.80.270.491.20.04435.0126.00.993.130.4812.171white
987.70.270.491.80.04123.086.00.99143.160.4212.560white
996.70.510.242.10.04314.0155.00.99043.220.613.060white
1007.40.190.499.30.0326.0132.00.9942.990.3211.071white
Rows: 1-100 | Columns: 14

Note

vastorbit offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the vastorbit environment.

You can easily divide your dataset into training and testing subsets using the VastFrame.train_test_split() method.

data = vod.load_winequality()
train, test = data.train_test_split(test_size = 0.2)

Model Initialization

First we import the LinearRegression model:

from vastorbit.machine_learning.vast import LinearRegression

Then we can create the model:

model = LinearRegression(
    tol = 1e-6,
    fit_intercept = True,
)

Model Training

We can now fit the model:

model.fit(
    train,
    [
        "fixed_acidity",
        "volatile_acidity",
        "citric_acid",
        "residual_sugar",
        "chlorides",
        "density"
    ],
    "quality",
    test,
)

Prediction

Prediction is straight-forward:

result = model.predict(
    test,
    [
        "fixed_acidity",
        "volatile_acidity",
        "citric_acid",
        "residual_sugar",
        "chlorides",
        "density"
    ],
    "prediction",
)
123
fixed_acidity
Decimal(6, 3)
123
volatile_acidity
Decimal(7, 4)
123
citric_acid
Decimal(6, 3)
123
residual_sugar
Decimal(7, 3)
123
chlorides
Double
123
free_sulfur_dioxide
Decimal(7, 2)
123
total_sulfur_dioxide
Decimal(7, 2)
123
density
Double
123
ph
Decimal(6, 3)
123
sulphates
Decimal(6, 3)
123
alcohol
Double
123
quality
Integer
123
good
Integer
Abc
color
Varchar(20)
123
seedrand
Decimal(26, 6)
123
prediction
Double
17.40.70.01.90.07611.034.00.99783.510.569.450red0.125.0160030375046745
27.80.760.042.30.09215.054.00.9973.260.659.850red0.115.154165628980678
37.40.660.01.80.07513.040.00.99783.510.569.450red0.115.042160469606813
47.50.50.366.10.07117.0102.00.99783.350.810.550red0.185.321140644907786
58.50.490.112.30.0849.067.00.99683.170.539.450red0.15.482509891283485
67.80.6450.02.00.08199999999999998.016.00.99643.380.599.860red0.135.3193499127730774
76.90.6050.1210.70.07340.083.00.99933.450.529.460red0.115.169084603365491
88.70.290.521.60.11312.037.00.99693.250.589.550red0.115.567550140224029
96.40.40.231.60.0665.012.00.99583.340.569.250red0.065.341099158553561
105.60.310.371.40.07412.096.00.99543.320.589.250red0.075.322832083240257
117.50.630.125.10.11150.0110.00.99833.260.779.450red0.145.135049695149519
127.30.390.312.40.0749.046.00.99623.410.549.460red0.075.448270388433826
137.50.520.161.90.08512.035.00.99683.380.629.571red0.095.289546490791139
147.20.7250.054.650.0864.011.00.99623.410.3910.950red0.045.307507777184554
159.30.320.572.00.07427.065.00.99693.280.7910.750red0.15.646053056178914
168.00.7050.051.90.0748.019.00.99623.340.9510.560red0.085.321387905926201
177.70.690.221.90.08418.094.00.99613.310.489.550red0.035.281616475657728
189.70.320.542.50.09428.083.00.99843.280.829.650red0.175.515896944285089
198.30.6250.21.50.0827.0119.00.99723.161.129.140red0.05.2481234461306485
206.20.450.21.60.0693.015.00.99583.410.569.250red0.125.277245528785613
217.30.670.261.80.40116.051.00.99693.161.149.450red0.045.097760906235635
228.60.490.281.90.1120.0136.00.99722.931.959.960red0.085.4007071371740665
237.00.620.081.80.0768.024.00.99783.480.539.050red0.095.0040916109269915
248.60.490.281.90.1120.0136.00.99722.931.959.960red0.095.4007071371740665
254.70.60.172.30.057999999999999917.0106.00.99323.850.612.960red0.175.347151597133632
268.00.330.532.50.09118.080.00.99763.370.89.660red0.145.37409888945902
2710.10.310.442.30.0822.046.00.99883.320.679.760red0.145.529700086968404
287.80.560.122.00.08199999999999997.028.00.9973.370.59.460red0.145.284089412034518
296.10.380.151.80.0726.019.00.99553.420.579.450red0.145.373109138619384
307.20.4150.362.00.08113.045.00.99723.480.649.250red0.05.248678421520566
317.60.490.261.60.23610.088.00.99683.110.89.350red0.095.2933875751626545
327.30.330.472.10.0775.011.00.99583.330.5310.360red0.125.517934714577137
337.10.430.425.50.0729.0129.00.99733.420.7210.550red0.185.3534592107420735
346.80.60.181.90.07918.086.00.99683.590.579.360red0.135.123930333905605
357.60.950.032.00.097.020.00.99593.20.569.650red0.165.1248240023767835
367.30.590.267.20.0735.0121.00.99813.370.499.450red0.045.240758743954018
377.20.730.022.50.07616.042.00.99723.440.529.350red0.115.072646889188235
387.90.50.332.00.08415.0143.00.99683.20.559.550red0.195.346939180082984
397.80.590.332.00.07424.0120.00.99683.250.549.450red0.065.264345038505667
407.00.430.361.60.08914.037.00.996153.340.569.260red0.115.339289262376383
417.80.440.282.70.118.095.00.99663.220.679.450red0.185.441956284457149
4211.00.30.582.10.0547.019.00.9983.310.8810.571red0.015.758462074287053
4311.60.440.642.10.0595.015.00.9983.210.6710.260red0.075.732241847565831
447.00.490.495.60.0626.0121.00.99743.340.7610.550red0.065.275240156780228
456.80.610.041.50.0575.010.00.995253.420.69.550red0.15.337521758380177
467.70.430.252.60.07329.063.00.996153.370.5810.560red0.155.499662096103606
478.21.00.092.30.0657.037.00.996853.320.559.060red0.085.046833210997164
487.20.630.01.90.096999999999999914.038.00.996753.370.589.060red0.15.187927239087713
4915.00.210.442.20.07510.024.01.000053.070.849.271red0.166.142504562720916
509.00.310.486.60.04311.073.00.99382.90.3811.650white0.076.262277673240931
516.30.390.246.90.0699.0117.00.99423.150.3510.240white0.055.789798179270804
528.20.220.366.80.03412.090.00.99443.010.3810.581white0.116.152166326034205
535.80.170.31.40.03755.0130.00.99093.290.3811.360white0.166.108488144047726
545.30.1650.241.10.05125.0105.00.99253.320.479.150white0.115.805312374789622
556.10.220.491.50.05118.087.00.99283.30.469.650white0.195.824799592249121
566.10.250.497.60.05267.0226.00.99563.160.478.950white0.075.668798745054744
577.20.310.241.40.05717.0117.00.99283.160.3510.550white0.015.943567170561323
588.30.20.744.450.04433.0130.00.99243.250.4212.260white0.076.31701016384082
597.60.520.4914.00.03437.0156.00.99583.140.3811.871white0.035.933410044071763
606.90.340.7411.20.06944.0150.00.99683.00.819.250white0.045.671675605966783
618.30.20.744.450.04433.0130.00.99243.250.4212.260white0.196.31701016384082
626.40.1450.495.40.04854.0164.00.99463.560.4410.860white0.015.839514114120476
637.80.430.4913.00.03337.0158.00.99553.140.3511.360white0.076.030509331287192
647.20.080.491.30.0518.0148.00.99453.460.4410.260white0.075.842511014648437
656.70.160.492.40.04657.0187.00.99523.620.8110.460white0.145.6568435794747245
667.40.280.491.50.03420.0126.00.99182.980.3910.660white0.096.11269027188132
677.70.270.491.80.04123.086.00.99143.160.4212.560white0.136.2337171194849645
688.10.20.498.10.05151.0205.00.99543.10.5211.060white0.086.050185950292473
697.80.260.747.50.04459.0160.00.9963.220.6410.060white0.145.819017472263283
708.30.20.491.70.03838.0167.00.99393.050.3710.160white0.076.015682142899465
716.60.30.241.20.03417.0121.00.99333.130.369.250white0.025.784802230256162
728.00.250.499.00.04431.0185.00.9983.340.4910.060white0.015.667609235351335
736.60.210.4918.150.04241.0158.00.99973.280.398.760white0.025.648343348656454
747.90.280.497.70.04548.0195.00.99543.040.5511.060white0.145.942970864378907
757.40.270.491.10.03733.0156.00.9923.150.5411.160white0.156.074376999092294
767.10.180.7415.60.04444.0176.00.99963.380.679.060white0.025.6172850726999
776.00.170.491.00.03426.0106.00.9923.210.429.860white0.075.941072781921491
787.20.160.491.30.03727.0104.00.99243.230.5710.660white0.046.0806273399980455
797.30.140.491.10.03828.099.00.99283.20.7210.660white0.166.044961902634725
807.90.120.495.20.04933.0152.00.99523.180.4710.660white0.015.984478138136694
818.50.150.491.50.03117.0122.00.99323.030.410.360white0.196.174188240804256
827.30.210.491.80.03844.0152.00.99123.320.4412.671white0.056.249312066144881
837.90.180.495.20.05136.0157.00.99533.180.4810.660white0.025.9245045492462225
847.80.30.741.80.03333.0156.00.9913.290.5212.860white0.136.252131640149116
857.80.30.741.80.03333.0156.00.9913.290.5212.860white0.036.252131640149116
866.70.30.745.00.03835.0157.00.99453.210.469.950white0.025.732274426181135
876.60.30.744.60.04136.0159.00.99463.210.459.950white0.15.685895451259711
886.80.170.742.40.05361.0182.00.99533.630.7610.560white0.145.618628878766344
897.00.270.741.50.03627.0122.00.99263.350.4811.260white0.135.917250325736319
906.90.270.4923.50.05759.0235.01.00242.980.478.650white0.035.494388150204742
917.50.320.491.70.03144.0109.00.99063.070.4612.560white0.026.2761941594745565
927.80.260.493.20.02728.087.00.99193.030.3211.371white0.066.246468465176122
938.90.320.491.60.0517.0131.00.99563.130.349.450white0.075.765857140803007
947.00.140.495.90.05322.0118.00.99543.360.369.460white0.115.839057104010095
957.80.150.247.70.04721.098.00.99512.940.319.660white0.126.100412451506088
966.90.250.241.80.0536.0121.00.9933.230.711.450white0.095.934412427243586
978.10.120.491.20.04243.0160.00.99343.130.489.760white0.16.096351271957161
987.30.190.4915.550.05850.0134.00.99983.420.369.171white0.015.638315272647333
997.50.240.499.40.04850.0149.00.99623.170.5910.571white0.115.874714492249041
1006.90.260.491.60.05839.0166.00.99653.650.529.440white0.15.390052860592874
Rows: 1-100 | Columns: 16

Score

Finally we can calculate the scores:

# R2
result.score("quality", "prediction", metric = "r2")
#  MSE
result.score("quality", "prediction", metric = "mse")
#  Max Error
result.score("quality", "prediction", metric = "max")

Note

If the prediction is already part of the dataset, there is no need to use a model to compute a prediction column. Use your column directly.

See also

LinearRegression : Linear Regression model.