vastorbit.VastFrame.cdt¶

VastFrame.cdt(columns: Annotated[str | list[str], 'STRING representing one column or a list of columns'] | None = None, max_cardinality: int = 20, nbins: int = 10, tcdt: bool = True, drop_transf_cols: bool = True) → VastFrame¶

Returns the complete disjunctive table of the VastFrame. Numerical features are transformed to categorical using the VastFrame.discretize() method. Applying PCA on TCDT leads to MCA (Multiple correspondence analysis).

Warning

This method can become computationally expensive when used with categorical variables with many categories.

Parameters:

columns (SQLColumns, optional) – List of the VastColumns names.
max_cardinality (int, optional) – For any categorical variable, keeps the most frequent categories and merges the less frequent categories into a new unique category.
nbins (int, optional) – Number of bins used for the discretization (must be > 1).
tcdt (bool, optional) – If set to True, returns the transformed complete disjunctive table (TCDT).
drop_transf_cols (bool, optional) – If set to True, drops the columns used during the transformation.

Returns:

the CDT relation.

Return type:

VastFrame

Examples

Let’s begin by importing vastorbit.

import vastorbit as vo

Hint

By assigning an alias to vastorbit, we mitigate the risk of code collisions with other libraries. This precaution is necessary because vastorbit uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions from vastorbit are used as intended without interfering with functions from other libraries.

Let us create a VastFrame with multiple columns:

vdf = vo.VastFrame(
    {
        "id": [0, 1, 2, 3, 4, 5],
        "cats": ["A", "B", "C", "A", "B", "C"],
        "vals": [2, 4, 8, 1, 4, 2],
    },
)

	123 id Integer	Abc cats Varchar(1)	123 vals Integer
1	1	B	4
2	3	A	1
3	5	C	2
4	4	B	4
5	2	C	8
6	0	A	2

Rows: 1-6 | Columns: 3

We can create the complete disjunctive table of the VastFrame:

vdf.cdt(columns=["cats", "vals"], tcdt = False)

	123 id Integer	123 cats_A Integer	123 cats_B Integer	123 cats_C Integer	123 vals_1 Integer	123 vals_2 Integer	123 vals_4 Integer	123 vals_8 Integer
1	1	0	1	0	0	0	1	0
2	0	1	0	0	0	1	0	0
3	3	1	0	0	1	0	0	0
4	5	0	0	1	0	1	0	0
5	2	0	0	1	0	0	0	1
6	4	0	1	0	0	0	1	0

Rows: 6 | Columns: 8

Same can be done to create the transformed complete disjunctive table of the VastFrame:

vdf.cdt(columns=["cats", "vals"], tcdt = True)

	123 id Integer	123 cats_A Decimal(18, 6)	123 cats_B Decimal(18, 6)	123 cats_C Decimal(18, 6)	123 vals_1 Decimal(18, 6)	123 vals_2 Decimal(18, 6)	123 vals_4 Decimal(18, 6)	123 vals_8 Decimal(18, 6)
1	2	-1.0	-1.0	-0.5	-1.0	-1.0	-1.0	0.0
2	4	-1.0	-0.5	-1.0	-1.0	-1.0	-0.5	-1.0
3	0	-0.5	-1.0	-1.0	-1.0	-0.5	-1.0	-1.0
4	3	-0.5	-1.0	-1.0	0.0	-1.0	-1.0	-1.0
5	5	-1.0	-1.0	-0.5	-1.0	-0.5	-1.0	-1.0
6	1	-1.0	-0.5	-1.0	-1.0	-1.0	-0.5	-1.0

Rows: 1-6 | Columns: 8

Note

This method can be useful to build an MCA (Multiple Correspondence Analysis) model based on a PCA (Principal Component Analysis) one. The transformed complete disjunctive table refers to a table used in MCA, where the original categorical data is transformed into binary indicators to represent the absence or presence of categories.