vastorbit.VastFrame.outliers¶

VastFrame.outliers(columns: Annotated[str | list[str], 'STRING representing one column or a list of columns'] | None = None, name: str = 'distribution_outliers', threshold: float = 3.0, robust: bool = False) → VastFrame¶

Adds a new VastColumn labeled with 0 or 1, where 1 indicates that the record is a global outlier.

Parameters:

columns (SQLColumns, optional) – List of the VastColumn names. If empty, all numerical VastColumn are used.
name (str, optional) – Name of the new VastColumn.
threshold (float, optional) – Threshold equal to the critical score.
robust (bool) – If set to True, uses the Robust Z-Score instead of the Z-Score.

Returns:

self

Return type:

VastFrame

Examples

Let’s begin by importing vastorbit.

import vastorbit as vo

Hint

By assigning an alias to vastorbit, we mitigate the risk of code collisions with other libraries. This precaution is necessary because vastorbit uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions from vastorbit are used as intended without interfering with functions from other libraries.

Let us create a VastFrame that has some outliers:

import numpy as np

data = np.random.normal(
    loc = 0,
    scale = 1,
    size = 10,
)
data = np.append(data, [100])
vdf = vo.VastFrame({"vals": data})

	123 vals Decimal(19, 16)
1	1.0238377252508037
2	0.5287504688558247
3	-0.1785129022533889
4	-0.7494816555248112
5	100.0
6	-0.8238795757236385
7	0.847378126053947
8	2.2753217834369743
9	0.6783591155305716
10	0.7945278832355748
11	-2.055084187529675

Rows: 1-11 | Column: vals | Type: decimal(19, 16)

Now we can see which values are outliers by using the VastFrame.outliers() method:

vdf.outliers()

	123 vals Decimal(19, 16)	123 distribution_outliers Integer
1	0.7945278832355748	0
2	-0.8238795757236385	0
3	-0.7494816555248112	0
4	1.0238377252508037	0
5	0.5287504688558247	0
6	100.0	1
7	-0.1785129022533889	0
8	0.847378126053947	0
9	0.6783591155305716	0
10	2.2753217834369743	0
11	-2.055084187529675	0

Rows: 1-11 | Columns: 2

Note

This function can only identify global outliers in the distribution. For other types of outliers, it is recommended to create machine learning models.