vastorbit.VastColumn.describe¶

VastColumn.describe(method: Literal['auto', 'numerical', 'categorical', 'cat_stats'] = 'auto', max_cardinality: int = 6, numcol: str | None = None) → TableSample¶

This function aggregates the VastColumn using multiple statistical aggregations such as minimum (min), maximum (max), median, cardinality (unique), and other relevant statistics. The specific aggregations applied depend on the data types of the VastColumn. For example, numeric columns are aggregated with numerical aggregations (min, median, max…), while categorical columns are aggregated using categorical ones (cardinality, mode…). This versatile function provides valuable insights into the dataset’s statistical properties and can be customized to meet specific analytical requirements.

Parameters:

method (str, optional) –
The describe method.
- auto:
  Sets the method to numerical if the VastColumn is numerical, categorical otherwise.
- categorical:
  Uses only categorical aggregations during the computation.
- cat_stats:
  Computes statistics of a numerical column for each VastColumn category. In this case, the parameter numcol must be defined.
- numerical:
  Uses popular numerical aggregations during the computation.
max_cardinality (int, optional) – Cardinality threshold to use to determine if the VastColumn is considered as categorical.
numcol (str, optional) – Numerical VastColumn to use when the parameter method is set to cat_stats.

Returns:

result.

Return type:

TableSample

Examples

For this example, we will use the following dataset:

import vastorbit as vo

data = vo.VastFrame(
    {
        "x": [1, 2, 4, 9, 10, 15, 20, 22],
        "y": [1, 2, 1, 2, 1, 1, 2, 1],
        "z": [10, 12, 2, 1, 9, 8, 1, 3],
        "c": ['A', 'A', 'A', 'A', 'B', 'B', 'C', 'D'],
    }
)

The describe method provides you with a variety of statistical methods.

The numerical parameter allows for the computation of numerical aggregations.

data["x"].describe(method = "numerical")

	value
name	"x"
dtype	integer
unique	8.0
count	8.0
mean	10.375
std	8.016723591521485
min	1.0
approx_25%	4.0
approx_50%	10.0
approx_75%	20.0
max	22.0

Rows: 1-11 | Columns: 2

The categorical parameter allows for the computation of categorical aggregations.

data["x"].describe(method = "categorical")

	value
name	"x"
dtype	integer
unique	8.0
count	8.0
15	1
Others	1
22	1
4	1
20	1
1	1
9	1

Rows: 1-11 | Columns: 2

The cat_stats parameter enables grouping by a categorical column and computing various aggregations on a numerical one.

data["c"].describe(
    method = "cat_stats",
    numcol = "x"
)

	count	percent	mean	std	min	approx_10%	approx_25%	approx_50%	approx_75%	approx_90%	max
C	1	12.5	20.0	[null]	20	20	20	20	20	20	20
D	1	12.5	22.0	[null]	22	22	22	22	22	22	22
B	2	25.0	12.5	3.5355339059327378	10	10	10	15	15	15	15
A	4	50.0	4.0	3.559026084010437	1	1	2	4	9	9	9

Rows: 1-4 | Columns: 12

Note

All the calculations are pushed to the database.