Classification¶

Build and evaluate models to predict categorical outcomes.

Overview¶

Classification algorithms predict categorical response variables. They are used for:

Binary classification – Two classes (e.g., fraud/not fraud, churn/no churn)
Multiclass classification – Multiple classes (e.g., flower species, product categories)

Common use cases:

Customer churn prediction
Fraud detection
Image classification
Sentiment analysis
Disease diagnosis

Build a Classification Model¶

We’ll predict flower species using the Iris dataset with a Random Forest Classifier.

Import the model:

from vastorbit.machine_learning.vast import RandomForestClassifier

Load the dataset:

from vastorbit.datasets import load_iris

iris = load_iris()
iris.head(5)

Initialize the model:

model = RandomForestClassifier(
    n_estimators = 5,
    max_depth = 3,
)

Train the model:

model.fit(
    iris,
    X=["PetalLengthCm", "SepalLengthCm"],
    y="Species",
)

Tip

All computation happens in-database. No data is moved to Python memory.

Evaluate Model Performance¶

Generate classification report:

model.report()

	Iris-setosa	Iris-versicolor	Iris-virginica	avg_macro	avg_weighted	avg_micro
auc	1.0	0.9843000000000001	0.9873	0.9905333333333334	0.9905333333333333	[null]
prc_auc	0.6166666666666667	0.34597069597069596	0.5387475734142402	0.5004616453505343	0.5004616453505343	[null]
accuracy	1.0	0.9533333333333334	0.9533333333333334	0.9688888888888889	0.9688888888888889	0.9688888888888889
log_loss	0.022117501878744335	0.16603928383851718	0.15073279027665273	0.1129631919979714	0.11296319199797142	[null]
precision	1.0	0.9777777777777777	0.8909090909090909	0.9562289562289562	0.9562289562289561	0.9533333333333334
recall	1.0	0.88	0.98	0.9533333333333333	0.9533333333333334	0.9533333333333334
f1_score	1.0	0.9263157894736842	0.9333333333333333	0.9532163742690059	0.9532163742690059	0.9533333333333334
mcc	1.0	0.8949594297801331	0.899973418001914	0.9316442825940157	0.9316442825940158	0.93
informedness	1.0	0.8700000000000001	0.9199999999999999	0.93	0.93	0.9300000000000002
markedness	1.0	0.9206349206349205	0.8803827751196172	0.9336725652515124	0.9336725652515125	0.9300000000000002
csi	1.0	0.8627450980392157	0.875	0.9125816993464052	0.9125816993464052	0.910828025477707

Key metrics:

Accuracy – Overall correctness (correct predictions / total predictions)
Precision – Of predicted positives, how many are actually positive
Recall – Of actual positives, how many were correctly identified
F1-Score – Harmonic mean of precision and recall
AUC – Area under ROC curve (discrimination ability)

Make Predictions¶

Predict class labels:

model.predict(iris, name="prediction")

	123 sepallengthcm Decimal(5,2) 100%	123 sepalwidthcm Decimal(5,2) 100%	123 petallengthcm Decimal(5,2) 100%	123 petalwidthcm Decimal(5,2) 100%	Abc species Varchar(30) 100%	Abc prediction Varchar(15) 100%
1	5.1	3.5	1.4	0.2	Iris-setosa	Iris-setosa
2	4.9	3.0	1.4	0.2	Iris-setosa	Iris-setosa
3	4.7	3.2	1.3	0.2	Iris-setosa	Iris-setosa
4	4.6	3.1	1.5	0.2	Iris-setosa	Iris-setosa
5	5.0	3.6	1.4	0.2	Iris-setosa	Iris-setosa
6	5.4	3.9	1.7	0.4	Iris-setosa	Iris-setosa
7	4.6	3.4	1.4	0.3	Iris-setosa	Iris-setosa
8	5.0	3.4	1.5	0.2	Iris-setosa	Iris-setosa
9	4.4	2.9	1.4	0.2	Iris-setosa	Iris-setosa
10	4.9	3.1	1.5	0.1	Iris-setosa	Iris-setosa
11	5.4	3.7	1.5	0.2	Iris-setosa	Iris-setosa
12	4.8	3.4	1.6	0.2	Iris-setosa	Iris-setosa
13	4.8	3.0	1.4	0.1	Iris-setosa	Iris-setosa
14	4.3	3.0	1.1	0.1	Iris-setosa	Iris-setosa
15	5.8	4.0	1.2	0.2	Iris-setosa	Iris-setosa
16	5.7	4.4	1.5	0.4	Iris-setosa	Iris-setosa
17	5.4	3.9	1.3	0.4	Iris-setosa	Iris-setosa
18	5.1	3.5	1.4	0.3	Iris-setosa	Iris-setosa
19	5.7	3.8	1.7	0.3	Iris-setosa	Iris-setosa
20	5.1	3.8	1.5	0.3	Iris-setosa	Iris-setosa

Predict class probabilities:

model.predict_proba(iris, name="prob")

	123 sepallengthcm Decimal(5,2) 100%	123 sepalwidthcm Decimal(5,2) 100%	123 petallengthcm Decimal(5,2) 100%	123 petalwidthcm Decimal(5,2) 100%	Abc species Varchar(30) 100%	Abc prediction Varchar(15) 100%	123 prob_irissetosa Decimal(17,12) 100%	123 prob_irisversicolor Decimal(17,12) 100%	123 prob_irisvirginica Decimal(17,12) 100%
1	5.1	3.5	1.4	0.2	Iris-setosa	Iris-setosa	1.0	1.0	1.0
2	4.9	3.0	1.4	0.2	Iris-setosa	Iris-setosa	1.0	1.0	1.0
3	4.7	3.2	1.3	0.2	Iris-setosa	Iris-setosa	1.0	1.0	1.0
4	4.6	3.1	1.5	0.2	Iris-setosa	Iris-setosa	1.0	1.0	1.0
5	5.0	3.6	1.4	0.2	Iris-setosa	Iris-setosa	1.0	1.0	1.0
6	5.4	3.9	1.7	0.4	Iris-setosa	Iris-setosa	1.0	1.0	1.0
7	4.6	3.4	1.4	0.3	Iris-setosa	Iris-setosa	1.0	1.0	1.0
8	5.0	3.4	1.5	0.2	Iris-setosa	Iris-setosa	1.0	1.0	1.0
9	4.4	2.9	1.4	0.2	Iris-setosa	Iris-setosa	1.0	1.0	1.0
10	4.9	3.1	1.5	0.1	Iris-setosa	Iris-setosa	1.0	1.0	1.0
11	5.4	3.7	1.5	0.2	Iris-setosa	Iris-setosa	1.0	1.0	1.0
12	4.8	3.4	1.6	0.2	Iris-setosa	Iris-setosa	1.0	1.0	1.0
13	4.8	3.0	1.4	0.1	Iris-setosa	Iris-setosa	1.0	1.0	1.0
14	4.3	3.0	1.1	0.1	Iris-setosa	Iris-setosa	1.0	1.0	1.0
15	5.8	4.0	1.2	0.2	Iris-setosa	Iris-setosa	0.6	0.6	0.6
16	5.7	4.4	1.5	0.4	Iris-setosa	Iris-setosa	0.8	0.8	0.8
17	5.4	3.9	1.3	0.4	Iris-setosa	Iris-setosa	1.0	1.0	1.0
18	5.1	3.5	1.4	0.3	Iris-setosa	Iris-setosa	1.0	1.0	1.0
19	5.7	3.8	1.7	0.3	Iris-setosa	Iris-setosa	0.8	0.8	0.8
20	5.1	3.8	1.5	0.3	Iris-setosa	Iris-setosa	1.0	1.0	1.0

Visualize Results¶

ROC Curve:

model.roc_curve()

Confusion Matrix:

model.confusion_matrix()

Feature Importance:

model.features_importance()

Understanding Metrics¶

The Accuracy Trap

Accuracy alone can be misleading, especially with imbalanced datasets.

Example: Fraud Detection

Suppose fraudulent transactions represent only 1% of data:

# Naive model: predict "no fraud" for everything
# Accuracy: 99% ✓
# Usefulness: 0% ✗ (misses all fraud!)

Better metrics for imbalanced data:

ROC AUC – Measures discrimination ability across all thresholds
PRC AUC – Precision-Recall curve (better for rare events)
F1-Score – Balances precision and recall
Class-specific metrics – Precision/recall per class

When to use which metric:

Metric	Best For
Accuracy	Balanced datasets with equal class importance
Precision	When false positives are costly (e.g., spam detection)
Recall	When false negatives are costly (e.g., disease screening)
F1-Score	Balance between precision and recall
ROC AUC	Overall model discrimination ability
PRC AUC	Imbalanced datasets with rare positive class

Train/Test Split¶

The example above used the entire dataset for training. For real-world applications, always split data:

from vastorbit.machine_learning.model_selection import train_test_split

# Split data: 80% train, 20% test
train, test = iris.train_test_split(test_size=0.2)

# Train on training set
model.fit(
    train,
    X=["PetalLengthCm", "SepalLengthCm"],
    y="Species",
)

# Evaluate on test set
predictions = model.predict(test, name="prediction")
model.report()

Warning

Training and testing on the same data leads to overfitting and unrealistic performance metrics.

Advanced Techniques¶

Cross-validation:

from vastorbit.machine_learning.model_selection import cross_validate

# 5-fold cross-validation
scores = cross_validate(
    model,
    iris,
    X=["PetalLengthCm", "SepalLengthCm"],
    y="Species",
    cv=5,
)

Hyperparameter tuning:

# Grid search for best parameters
best_model = RandomForestClassifier(
    n_estimators = 5,
    max_depth = 3,
)

Feature engineering:

# Create interaction features
iris["petal_sepal_ratio"] = iris["PetalLengthCm"] / iris["SepalLengthCm"]

# Train with new feature
model.fit(
    iris,
    X=["PetalLengthCm", "SepalLengthCm", "petal_sepal_ratio"],
    y="Species",
)

Available Classifiers¶

VAST Orbit supports multiple classification algorithms:

RandomForestClassifier – Ensemble of decision trees
LogisticRegression – Linear classification
NaiveBayes – Probabilistic classifier
LinearSVC – Support Vector Classifier
KNeighborsClassifier – K-nearest neighbors

Each algorithm has strengths for different data types and problem characteristics.

Next Steps¶

Now that you understand classification, explore:

Regression – Predict continuous values
Time Series – Analyze temporal patterns
Clustering – Discover data groups