Loading...

vastorbit.VastFrame.one_hot_encode

VastFrame.one_hot_encode(columns: Annotated[str | list[str], 'STRING representing one column or a list of columns'] | None = None, max_cardinality: int = 12, prefix_sep: str = '_', drop_first: bool = True, use_numbers_as_suffix: bool = False) VastFrame

Encodes the VastColumns using the One Hot Encoding algorithm.

Parameters:
  • columns (SQLColumns, optional) – List of the VastColumns used to train the One Hot Encoding model. If empty, only the VastColumns with a cardinality less than ‘max_cardinality’ are used.

  • max_cardinality (int, optional) – Cardinality threshold used to determine whether the VastColumn is taken into account during the encoding This parameter is used only if the parameter ‘columns’ is empty.

  • prefix_sep (str, optional) – Prefix delimitor of the dummies names.

  • drop_first (bool, optional) – Drops the first dummy to avoid the creation of correlated features.

  • use_numbers_as_suffix (bool, optional) – Uses numbers as suffix instead of the VastColumns categories.

Returns:

self

Return type:

VastFrame

Examples

We import vastorbit:

import vastorbit as vo

Hint

By assigning an alias to vastorbit, we mitigate the risk of code collisions with other libraries. This precaution is necessary because vastorbit uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions from vastorbit are used as intended without interfering with functions from other libraries.

For this example, we will use the Titanic dataset.

import vastorbit.datasets as vod

data = vod.load_titanic()
123
pclass
Integer
123
survived
Integer
Abc
name
Varchar(164)
Abc
sex
Varchar(20)
123
age
Double
123
sibsp
Integer
123
parch
Integer
Abc
ticket
Varchar(36)
123
fare
Double
Abc
cabin
Varchar(30)
Abc
embarked
Varchar(20)
Abc
boat
Varchar(100)
123
body
Integer
Abc
home.dest
Varchar(100)
131McCormack, Mr. Thomas Josephmale[null]003672287.75[null]Q[null][null][null]
231McCoy, Miss. Agnesfemale[null]2036722623.25[null]Q16[null][null]
331McCoy, Miss. Aliciafemale[null]2036722623.25[null]Q16[null][null]
431McCoy, Mr. Bernardmale[null]2036722623.25[null]Q16[null][null]
531McDermott, Miss. Brigdet Deliafemale[null]003309327.7875[null]Q13[null][null]
630McEvoy, Mr. Michaelmale[null]003656815.5[null]Q[null][null][null]
731McGovern, Miss. Maryfemale[null]003309317.8792[null]Q13[null][null]
831McGowan, Miss. Anna "Annie"female15.0003309238.0292[null]Q[null][null][null]
930McGowan, Miss. Katherinefemale35.00092327.75[null]Q[null][null][null]
1030McMahon, Mr. Martinmale[null]003703727.75[null]Q[null][null][null]
1130McNamee, Mr. Nealmale24.01037656616.1[null]S[null][null][null]
1230McNamee, Mrs. Neal (Eileen O'Leary)female19.01037656616.1[null]S[null]53[null]
1330McNeill, Miss. Bridgetfemale[null]003703687.75[null]Q[null][null][null]
1430Meanwell, Miss. (Marion Ogden)female[null]00SOTON/O.Q. 3920878.05[null]S[null][null][null]
1530Meek, Mrs. Thomas (Annie Louise Rowley)female[null]003430958.05[null]S[null][null][null]
1630Meo, Mr. Alfonzomale55.500A.5. 112068.05[null]S[null]201[null]
1730Mernagh, Mr. Robertmale[null]003687037.75[null]Q[null][null][null]
1831Midtsjo, Mr. Karl Albertmale21.0003455017.775[null]S15[null][null]
1930Miles, Mr. Frankmale[null]003593068.05[null]S[null][null][null]
2030Mineff, Mr. Ivanmale24.0003492337.8958[null]S[null][null][null]
2130Minkoff, Mr. Lazarmale21.0003492117.8958[null]S[null][null][null]
2230Mionoff, Mr. Stoytchomale28.0003492077.8958[null]S[null][null][null]
2330Mitkoff, Mr. Mitomale[null]003492217.8958[null]S[null][null][null]
2431Mockler, Miss. Helen Mary "Ellie"female[null]003309807.8792[null]Q16[null][null]
2530Moen, Mr. Sigurd Hansenmale25.0003481237.65F G73S[null]309[null]
2631Moor, Master. Meiermale6.00139209612.475E121S14[null][null]
2731Moor, Mrs. (Beila)female27.00139209612.475E121S14[null][null]
2830Moore, Mr. Leonard Charlesmale[null]00A4. 545108.05[null]S[null][null][null]
2931Moran, Miss. Berthafemale[null]1037111024.15[null]Q16[null][null]
3030Moran, Mr. Daniel Jmale[null]1037111024.15[null]Q[null][null][null]
3130Moran, Mr. Jamesmale[null]003308778.4583[null]Q[null][null][null]
3230Morley, Mr. Williammale34.0003645068.05[null]S[null][null][null]
3330Morrow, Mr. Thomas Rowanmale[null]003726227.75[null]Q[null][null][null]
3431Moss, Mr. Albert Johanmale[null]003129917.775[null]SB[null][null]
3531Moubarek, Master. Geriosmale[null]11266115.2458[null]CC[null][null]
3631Moubarek, Master. Halim Gonios ("William George")male[null]11266115.2458[null]CC[null][null]
3731Moubarek, Mrs. George (Omine "Amenia" Alexander)female[null]02266115.2458[null]CC[null][null]
3831Moussa, Mrs. (Mantoura Boulos)female[null]0026267.2292[null]C[null][null][null]
3930Moutal, Mr. Rahamin Haimmale[null]003747468.05[null]S[null][null][null]
4031Mullens, Miss. Katherine "Katie"female[null]00358527.7333[null]Q16[null][null]
4131Mulvihill, Miss. Bertha Efemale24.0003826537.75[null]Q15[null][null]
4230Murdlin, Mr. Josephmale[null]00A./5. 32358.05[null]S[null][null][null]
4331Murphy, Miss. Katherine "Kate"female[null]1036723015.5[null]Q16[null][null]
4431Murphy, Miss. Margaret Janefemale[null]1036723015.5[null]Q16[null][null]
4531Murphy, Miss. Norafemale[null]003656815.5[null]Q16[null][null]
4630Myhrman, Mr. Pehr Fabian Oliver Malkolmmale18.0003470787.75[null]S[null][null][null]
4730Naidenoff, Mr. Penkomale22.0003492067.8958[null]S[null][null][null]
4831Najib, Miss. Adele Kiamie "Jane"female15.00026677.225[null]CC[null][null]
4931Nakid, Miss. Maria ("Mary")female1.002265315.7417[null]CC[null][null]
5031Nakid, Mr. Sahidmale20.011265315.7417[null]CC[null][null]
5131Nakid, Mrs. Said (Waika "Mary" Mowad)female19.011265315.7417[null]CC[null][null]
5230Nancarrow, Mr. William Henrymale33.000A./5. 33388.05[null]S[null][null][null]
5330Nankoff, Mr. Minkomale[null]003492187.8958[null]S[null][null][null]
5430Nasr, Mr. Mustafamale[null]0026527.2292[null]C[null][null][null]
5530Naughton, Miss. Hannahfemale[null]003652377.75[null]Q[null][null][null]
5630Nenkoff, Mr. Christomale[null]003492347.8958[null]S[null][null][null]
5731Nicola-Yarred, Master. Eliasmale12.010265111.2417[null]CC[null][null]
5831Nicola-Yarred, Miss. Jamilafemale14.010265111.2417[null]CC[null][null]
5930Nieminen, Miss. Manta Josefinafemale29.00031012977.925[null]S[null][null][null]
6030Niklasson, Mr. Samuelmale28.0003636118.05[null]S[null][null][null]
6131Nilsson, Miss. Berta Oliviafemale18.0003470667.775[null]SD[null][null]
6231Nilsson, Miss. Helmina Josefinafemale26.0003474707.8542[null]S13[null][null]
6330Nilsson, Mr. August Ferdinandmale21.0003504107.8542[null]S[null][null][null]
6430Nirva, Mr. Iisakki Antino Aijomale41.000SOTON/O2 31012727.125[null]S[null][null]Finland Sudbury, ON
6531Niskanen, Mr. Juhamale39.000STON/O 2. 31012897.925[null]S9[null][null]
6630Nosworthy, Mr. Richard Catermale21.000A/4. 398867.8[null]S[null][null][null]
6730Novel, Mr. Mansouermale28.50026977.2292[null]C[null]181[null]
6831Nysten, Miss. Anna Sofiafemale22.0003470817.75[null]S13[null][null]
6930Nysveen, Mr. Johan Hansenmale61.0003453646.2375[null]S[null][null][null]
7030O'Brien, Mr. Thomasmale[null]1037036515.5[null]Q[null][null][null]
7130O'Brien, Mr. Timothymale[null]003309797.8292[null]Q[null][null][null]
7231O'Brien, Mrs. Thomas (Johanna "Hannah" Godfrey)female[null]1037036515.5[null]Q[null][null][null]
7330O'Connell, Mr. Patrick Dmale[null]003349127.7333[null]Q[null][null][null]
7430O'Connor, Mr. Mauricemale[null]003710607.75[null]Q[null][null][null]
7530O'Connor, Mr. Patrickmale[null]003667137.75[null]Q[null][null][null]
7630Odahl, Mr. Nils Martinmale23.00072679.225[null]S[null][null][null]
7730O'Donoghue, Ms. Bridgetfemale[null]003648567.75[null]Q[null][null][null]
7831O'Driscoll, Miss. Bridgetfemale[null]00143117.75[null]QD[null][null]
7931O'Dwyer, Miss. Ellen "Nellie"female[null]003309597.8792[null]Q[null][null][null]
8031Ohman, Miss. Velinfemale22.0003470857.775[null]SC[null][null]
8131O'Keefe, Mr. Patrickmale[null]003684027.75[null]QB[null][null]
8231O'Leary, Miss. Hanora "Norah"female[null]003309197.8292[null]Q13[null][null]
8331Olsen, Master. Artur Karlmale9.001C 173683.1708[null]S13[null][null]
8430Olsen, Mr. Henry Margidomale28.000C 400122.525[null]S[null]173[null]
8530Olsen, Mr. Karl Siegwart Andreasmale42.00145798.4042[null]S[null][null][null]
8630Olsen, Mr. Ole Martinmale[null]00Fa 2653027.3125[null]S[null][null][null]
8730Olsson, Miss. Elinafemale31.0003504077.8542[null]S[null][null][null]
8830Olsson, Mr. Nils Johan Goranssonmale28.0003474647.8542[null]S[null][null][null]
8931Olsson, Mr. Oscar Wilhelmmale32.0003470797.775[null]SA[null][null]
9030Olsvigen, Mr. Thor Andersonmale20.00065639.225[null]S[null]89Oslo, Norway Cameron, WI
9130Oreskovic, Miss. Jelkafemale23.0003150858.6625[null]S[null][null][null]
9230Oreskovic, Miss. Marijafemale20.0003150968.6625[null]S[null][null][null]
9330Oreskovic, Mr. Lukamale20.0003150948.6625[null]S[null][null][null]
9430Osen, Mr. Olaf Elonmale16.00075349.2167[null]S[null][null][null]
9531Osman, Mrs. Marafemale31.0003492448.6833[null]S[null][null][null]
9630O'Sullivan, Miss. Bridget Maryfemale[null]003309097.6292[null]Q[null][null][null]
9730Palsson, Master. Gosta Leonardmale2.03134990921.075[null]S[null]4[null]
9830Palsson, Master. Paul Folkemale6.03134990921.075[null]S[null][null][null]
9930Palsson, Miss. Stina Violafemale3.03134990921.075[null]S[null][null][null]
10030Palsson, Miss. Torborg Danirafemale8.03134990921.075[null]S[null][null][null]
Rows: 1-100 | Columns: 14

Note

vastorbit offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the vastorbit environment.

Let’s select few categorical features

data = data.select(["pclass", "sex", "survived", "embarked"])
data
123
pclass
Integer
Abc
sex
Varchar(20)
123
survived
Integer
Abc
embarked
Varchar(20)
13male1Q
23female1Q
33female1Q
43male1Q
53female1Q
63male0Q
73female1Q
83female1Q
93female0Q
103male0Q
113male0S
123female0S
133female0Q
143female0S
153female0S
163male0S
173male0Q
183male1S
193male0S
203male0S
213male0S
223male0S
233male0S
243female1Q
253male0S
263male1S
273female1S
283male0S
293female1Q
303male0Q
313male0Q
323male0S
333male0Q
343male1S
353male1C
363male1C
373female1C
383female1C
393male0S
403female1Q
413female1Q
423male0S
433female1Q
443female1Q
453female1Q
463male0S
473male0S
483female1C
493female1C
503male1C
513female1C
523male0S
533male0S
543male0C
553female0Q
563male0S
573male1C
583female1C
593female0S
603male0S
613female1S
623female1S
633male0S
643male0S
653male1S
663male0S
673male0C
683female1S
693male0S
703male0Q
713male0Q
723female1Q
733male0Q
743male0Q
753male0Q
763male0S
773female0Q
783female1Q
793female1Q
803female1S
813male1Q
823female1Q
833male1S
843male0S
853male0S
863male0S
873female0S
883male0S
893male1S
903male0S
913female0S
923female0S
933male0S
943male0S
953female1S
963female0Q
973male0S
983male0S
993female0S
1003female0S
Rows: 1-100 | Columns: 4

Let’s apply encoding on all the VastColumns of the datasets

data.one_hot_encode()
123
pclass
Integer
Abc
sex
Varchar(20)
123
survived
Integer
Abc
embarked
Varchar(20)
123
pclass_1
Integer
123
pclass_2
Integer
123
sex_female
Integer
123
embarked_C
Integer
123
embarked_Q
Integer
13male1Q00001
23female1Q00101
33female1Q00101
43male1Q00001
53female1Q00101
63male0Q00001
73female1Q00101
83female1Q00101
93female0Q00101
103male0Q00001
113male0S00000
123female0S00100
133female0Q00101
143female0S00100
153female0S00100
163male0S00000
173male0Q00001
183male1S00000
193male0S00000
203male0S00000
213male0S00000
223male0S00000
233male0S00000
243female1Q00101
253male0S00000
263male1S00000
273female1S00100
283male0S00000
293female1Q00101
303male0Q00001
313male0Q00001
323male0S00000
333male0Q00001
343male1S00000
353male1C00010
363male1C00010
373female1C00110
383female1C00110
393male0S00000
403female1Q00101
413female1Q00101
423male0S00000
433female1Q00101
443female1Q00101
453female1Q00101
463male0S00000
473male0S00000
483female1C00110
493female1C00110
503male1C00010
513female1C00110
523male0S00000
533male0S00000
543male0C00010
553female0Q00101
563male0S00000
573male1C00010
583female1C00110
593female0S00100
603male0S00000
613female1S00100
623female1S00100
633male0S00000
643male0S00000
653male1S00000
663male0S00000
673male0C00010
683female1S00100
693male0S00000
703male0Q00001
713male0Q00001
723female1Q00101
733male0Q00001
743male0Q00001
753male0Q00001
763male0S00000
773female0Q00101
783female1Q00101
793female1Q00101
803female1S00100
813male1Q00001
823female1Q00101
833male1S00000
843male0S00000
853male0S00000
863male0S00000
873female0S00100
883male0S00000
893male1S00000
903male0S00000
913female0S00100
923female0S00100
933male0S00000
943male0S00000
953female1S00100
963female0Q00101
973male0S00000
983male0S00000
993female0S00100
1003female0S00100
Rows: 1-100 of 1309 | Columns: 9

Let’s apply encoding on two specific VastColumns viz. “pclass” and “embarked”

data = data.select(["pclass", "sex", "survived", "embarked"])
data.one_hot_encode(columns = ["pclass", "embarked"])
123
pclass
Integer
Abc
sex
Varchar(20)
123
survived
Integer
Abc
embarked
Varchar(20)
123
pclass_1
Integer
123
pclass_2
Integer
123
embarked_C
Integer
123
embarked_Q
Integer
13male1Q0001
23female1Q0001
33female1Q0001
43male1Q0001
53female1Q0001
63male0Q0001
73female1Q0001
83female1Q0001
93female0Q0001
103male0Q0001
113male0S0000
123female0S0000
133female0Q0001
143female0S0000
153female0S0000
163male0S0000
173male0Q0001
183male1S0000
193male0S0000
203male0S0000
213male0S0000
223male0S0000
233male0S0000
243female1Q0001
253male0S0000
263male1S0000
273female1S0000
283male0S0000
293female1Q0001
303male0Q0001
313male0Q0001
323male0S0000
333male0Q0001
343male1S0000
353male1C0010
363male1C0010
373female1C0010
383female1C0010
393male0S0000
403female1Q0001
413female1Q0001
423male0S0000
433female1Q0001
443female1Q0001
453female1Q0001
463male0S0000
473male0S0000
483female1C0010
493female1C0010
503male1C0010
513female1C0010
523male0S0000
533male0S0000
543male0C0010
553female0Q0001
563male0S0000
573male1C0010
583female1C0010
593female0S0000
603male0S0000
613female1S0000
623female1S0000
633male0S0000
643male0S0000
653male1S0000
663male0S0000
673male0C0010
683female1S0000
693male0S0000
703male0Q0001
713male0Q0001
723female1Q0001
733male0Q0001
743male0Q0001
753male0Q0001
763male0S0000
773female0Q0001
783female1Q0001
793female1Q0001
803female1S0000
813male1Q0001
823female1Q0001
833male1S0000
843male0S0000
853male0S0000
863male0S0000
873female0S0000
883male0S0000
893male1S0000
903male0S0000
913female0S0000
923female0S0000
933male0S0000
943male0S0000
953female1S0000
963female0Q0001
973male0S0000
983male0S0000
993female0S0000
1003female0S0000
Rows: 1-100 of 1309 | Columns: 8

Let’s apply encoding on all features having cardinality less than 3

data = data.select(["pclass", "sex", "survived", "embarked"])
data.one_hot_encode(
    max_cardinality = 3,
    drop_first = False,
)
123
pclass
Integer
Abc
sex
Varchar(20)
123
survived
Integer
Abc
embarked
Varchar(20)
123
sex_female
Integer
123
sex_male
Integer
13male1Q01
23female1Q10
33female1Q10
43male1Q01
53female1Q10
63male0Q01
73female1Q10
83female1Q10
93female0Q10
103male0Q01
113male0S01
123female0S10
133female0Q10
143female0S10
153female0S10
163male0S01
173male0Q01
183male1S01
193male0S01
203male0S01
213male0S01
223male0S01
233male0S01
243female1Q10
253male0S01
263male1S01
273female1S10
283male0S01
293female1Q10
303male0Q01
313male0Q01
323male0S01
333male0Q01
343male1S01
353male1C01
363male1C01
373female1C10
383female1C10
393male0S01
403female1Q10
413female1Q10
423male0S01
433female1Q10
443female1Q10
453female1Q10
463male0S01
473male0S01
483female1C10
493female1C10
503male1C01
513female1C10
523male0S01
533male0S01
543male0C01
553female0Q10
563male0S01
573male1C01
583female1C10
593female0S10
603male0S01
613female1S10
623female1S10
633male0S01
643male0S01
653male1S01
663male0S01
673male0C01
683female1S10
693male0S01
703male0Q01
713male0Q01
723female1Q10
733male0Q01
743male0Q01
753male0Q01
763male0S01
773female0Q10
783female1Q10
793female1Q10
803female1S10
813male1Q01
823female1Q10
833male1S01
843male0S01
853male0S01
863male0S01
873female0S10
883male0S01
893male1S01
903male0S01
913female0S10
923female0S10
933male0S01
943male0S01
953female1S10
963female0Q10
973male0S01
983male0S01
993female0S10
1003female0S10
Rows: 1-100 of 1309 | Columns: 6

See also

VastFrame.decode() : User Defined Encoding.
VastFrame.label_encode() : Label Encoding.
VastFrame.mean_encode() : Mean Encoding.
VastFrame.discretize() : Discretization.