Learning Curve

General

vastorbit’s Learning Curve tool is an essential asset for evaluating machine learning models. It enables users to visualize a model’s performance by plotting key metrics against varying training dataset sizes. By analyzing these curves, data analysts can identify issues such as overfitting or underfitting, make informed decisions about dataset size, and optimize model performance. This feature plays a crucial role in enhancing model robustness and facilitating data-driven decision-making.

Let’s begin by importing vastorbit.

import vastorbit as vo

Let’s generate a dataset using the following data.

import random

N = 200 # Number of Records
k = 10 # step

# Normal Distributions
x = np.random.normal(5, 1, round(N / 2))
y = np.random.normal(3, 1, round(N / 2))
z = np.random.normal(3, 1, round(N / 2))

# Creating a VastFrame with two clusters
data = vo.VastFrame({
    "x": np.concatenate([x, x + k]),
    "y": np.concatenate([y, y + k]),
    "z": np.concatenate([z, z + k]),
    "c": [random.randint(0, 1) for _ in range(N)]
})

Let’s proceed by creating a Random Forest Classifier model using the complete dataset.

# Importing the VAST ML module
import vastorbit.machine_learning.vast as vml

# Importing the model selection module
import vastorbit.machine_learning.model_selection as vms

# Defining the Model
model = vml.RandomForestClassifier(n_estimators = 4, max_depth = 2)

In the context of data visualization, we have the flexibility to harness multiple plotting libraries to craft a wide range of graphical representations. vastorbit, as a versatile tool, provides support for several graphic libraries, such as Matplotlib and Plotly. Each of these libraries offers unique features and capabilities, allowing us to choose the most suitable one for our specific data visualization needs.

Note

To select the desired plotting library, we simply need to use the set_option() function. vastorbit offers the flexibility to smoothly transition between different plotting libraries. In instances where a particular graphic is not supported by the chosen library or is not supported within the vastorbit framework, the tool will automatically generate a warning and then switch to an alternative library where the graphic can be created.

Please click on the tabs to view the various graphics generated by the different plotting libraries.

Plotly

We can switch to using the plotly module.

vo.set_option("plotting_lib", "plotly")

Efficiency

fig = vms.learning_curve(
  model,
  data,
  X = ["x", "y", "z"],
  y = "c",
  method = "efficiency",
  cv = 3,
  metric = "auc",
  return_chart = True,
)
fig

Scalability

fig = vms.learning_curve(
  model,
  data,
  X = ["x", "y", "z"],
  y = "c",
  method = "scalability",
  cv = 3,
  metric = "auc",
  return_chart = True,
)
fig

Performance

fig = vms.learning_curve(
  model,
  data,
  X = ["x", "y", "z"],
  y = "c",
  method = "performance",
  cv = 3,
  metric = "auc",
  return_chart = True,
)
fig

Matplotlib

We can switch to using the matplotlib module.

vo.set_option("plotting_lib", "matplotlib")

Efficiency

 vms.learning_curve(
   model,
   data,
   X = ["x", "y", "z"],
   y = "c",
   method = "efficiency",
   cv = 3,
   metric = "auc",
 )

Scalability

 vms.learning_curve(
   model,
   data,
   X = ["x", "y", "z"],
   y = "c",
   method = "scalability",
   cv = 3,
   metric = "auc",
 )

Performance

 vms.learning_curve(
   model,
   data,
   X = ["x", "y", "z"],
   y = "c",
   method = "performance",
   cv = 3,
   metric = "auc",
 )

---

Chart Customization

vastorbit empowers users with a high degree of flexibility when it comes to tailoring the visual aspects of their plots. This customization extends to essential elements such as color schemes, text labels, and plot sizes, as well as a wide range of other attributes that can be fine-tuned to align with specific design preferences and analytical requirements. Whether you want to make your visualizations more visually appealing or need to convey specific insights with precision, vastorbit’s customization options enable you to craft graphics that suit your exact needs.

Note

As learning curves are essentially range plots, customization options are identical to those available for Range Plots.