Machine Learning - LOF Plot¶
General¶
vastorbit’s Local Outlier Factor (LOF) functionality is a powerful tool for detecting local anomalies or outliers within your data. LOF operates by evaluating the density of data points in the vicinity of each observation, highlighting instances that exhibit significantly different local behavior compared to their neighbors. This feature helps data analysts uncover irregularities in their datasets that may not be apparent through traditional global outlier detection methods. By providing insights into local data structures, vastorbit’s LOF empowers users to make informed decisions and identify critical data points that may require further investigation or action.
Let’s begin by importing vastorbit.
import vastorbit as vo
Let’s also import numpy to create a random dataset.
import numpy as np
Let’s generate a dataset using the following data.
N = 100 # Number of Records
k = 10 # step
# Normal Distributions
x = np.random.normal(5, 1, round(N / 2))
y = np.random.normal(3, 1, round(N / 2))
z = np.random.normal(3, 1, round(N / 2))
# Creating a VastFrame with two clusters
data = vo.VastFrame({
"x": np.concatenate([x, x + k]),
"y": np.concatenate([y, y + k]),
"z": np.concatenate([z, z + k]),
})
Let’s proceed by creating Local Outlier Factors (LOFs) for various dimensions, including 1D, 2D, and 3D.
# Importing the VAST ML module
import vastorbit.machine_learning.vast as vml
# Defining the Models
model_lof_1d = vml.LocalOutlierFactor()
model_lof_2d = vml.LocalOutlierFactor()
model_lof_3d = vml.LocalOutlierFactor()
# Fitting the models
model_lof_1d.fit(data, "x")
model_lof_2d.fit(data, ["x", "y"])
model_lof_3d.fit(data, ["x", "y", "z"])
# Displaying the VastFrame
display(data)
In the context of data visualization, we have the flexibility to harness multiple plotting libraries to craft a wide range of graphical representations. vastorbit, as a versatile tool, provides support for several graphic libraries, such as Matplotlib and Plotly. Each of these libraries offers unique features and capabilities, allowing us to choose the most suitable one for our specific data visualization needs.
Note
To select the desired plotting library, we simply need to use the set_option() function. vastorbit offers the flexibility to smoothly transition between different plotting libraries. In instances where a particular graphic is not supported by the chosen library or is not supported within the vastorbit framework, the tool will automatically generate a warning and then switch to an alternative library where the graphic can be created.
Please click on the tabs to view the various graphics generated by the different plotting libraries.
We can switch to using the plotly module.
vo.set_option("plotting_lib", "plotly")
model_lof_1d.plot()
model_lof_2d.plot()
model_lof_3d.plot()