Train Test Split

Before you test a supervised model, you’ll need separate, non-overlapping sets for training and testing.

In vastorbit, the train_test_split() method uses a random number generator to assign each row to either the training or the testing set, ensuring that the two sets never overlap.

from vastorbit.datasets import load_titanic

titanic = load_titanic()
# LinearRegression is ``scikit-learn`` backed and rejects NaN, so we drop rows
# with missing values in the columns we model on before splitting.
titanic = titanic.dropna(columns = ["age", "fare", "survived"])
train, test = titanic.train_test_split()

titanic.shape()

train.shape()

test.shape()

Because the split is driven by a random assignment, it depends on the order in which the rows are processed. If your data isn’t sorted by a unique (or near-unique) feature, the same row could end up in a different set from one run to the next. To make the split consistent and reproducible, pass the order_by parameter so the rows are processed in a deterministic order.

train, test = titanic.train_test_split(order_by = {"fare": "asc"})

Even if fare has duplicates, ordering the data this way drastically decreases the likelihood of a collision.

Let’s create a model and evaluate it.

from vastorbit.machine_learning.vast import LinearRegression

model = LinearRegression()

When fitting the model with the fit() method, you can use the parameter test_relation to score your data on a specific relation.

model.fit(
    train,
    ["age", "fare"],
    "survived",
    test,
)

model.report()

	value
explained_variance	0.06834558226202614
max_error	0.8148086579154292
median_absolute_error	0.41007540688564303
mean_absolute_error	0.4532428335681724
mean_squared_error	0.28173659962403746
root_mean_squared_error	0.4724297906777503
r2	0.06775762580934841
r2_adj	0.06230591601875979
aic	-430.9168603498986
bic	-419.5093942542298

All model evaluation abstractions will now use the test relation for the scoring. After that, you can evaluate the efficiency of your model.