Loading...

vastorbit.VastFrame.filter

VastFrame.filter(conditions: None | list | str = None, *args, **kwargs) VastFrame

Filters the VastFrame using the input expressions.

Parameters:
  • conditions (SQLExpression, optional) – List of expressions. For example, to keep only the records where the VastColumn ‘column’ is greater than 5 and less than 10, you can write: ['"column" > 5', '"column" < 10'].

  • force_filter (bool, optional) – Default Value: True When set to True, the VastFrame will be modified even if no filtering occurred. This parameter can be used to enforce filtering and ensure pipeline consistency.

  • raise_error (bool, optional) – Default Value: False If set to True and the input filtering is incorrect, an error is raised.

Returns:

self

Return type:

VastFrame

Examples

We import vastorbit:

import vastorbit as vo

Hint

By assigning an alias to vastorbit, we mitigate the risk of code collisions with other libraries. This precaution is necessary because vastorbit uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions from vastorbit are used as intended without interfering with functions from other libraries.

For this example, we will use the Titanic dataset:

from vastorbit.datasets import load_titanic

vdf = load_titanic()
123
pclass
Integer
123
survived
Integer
Abc
name
Varchar(164)
Abc
sex
Varchar(20)
123
age
Double
123
sibsp
Integer
123
parch
Integer
Abc
ticket
Varchar(36)
123
fare
Double
Abc
cabin
Varchar(30)
Abc
embarked
Varchar(20)
Abc
boat
Varchar(100)
123
body
Integer
Abc
home.dest
Varchar(100)
131McCormack, Mr. Thomas Josephmale[null]003672287.75[null]Q[null][null][null]
231McCoy, Miss. Agnesfemale[null]2036722623.25[null]Q16[null][null]
331McCoy, Miss. Aliciafemale[null]2036722623.25[null]Q16[null][null]
431McCoy, Mr. Bernardmale[null]2036722623.25[null]Q16[null][null]
531McDermott, Miss. Brigdet Deliafemale[null]003309327.7875[null]Q13[null][null]
630McEvoy, Mr. Michaelmale[null]003656815.5[null]Q[null][null][null]
731McGovern, Miss. Maryfemale[null]003309317.8792[null]Q13[null][null]
831McGowan, Miss. Anna "Annie"female15.0003309238.0292[null]Q[null][null][null]
930McGowan, Miss. Katherinefemale35.00092327.75[null]Q[null][null][null]
1030McMahon, Mr. Martinmale[null]003703727.75[null]Q[null][null][null]
1130McNamee, Mr. Nealmale24.01037656616.1[null]S[null][null][null]
1230McNamee, Mrs. Neal (Eileen O'Leary)female19.01037656616.1[null]S[null]53[null]
1330McNeill, Miss. Bridgetfemale[null]003703687.75[null]Q[null][null][null]
1430Meanwell, Miss. (Marion Ogden)female[null]00SOTON/O.Q. 3920878.05[null]S[null][null][null]
1530Meek, Mrs. Thomas (Annie Louise Rowley)female[null]003430958.05[null]S[null][null][null]
1630Meo, Mr. Alfonzomale55.500A.5. 112068.05[null]S[null]201[null]
1730Mernagh, Mr. Robertmale[null]003687037.75[null]Q[null][null][null]
1831Midtsjo, Mr. Karl Albertmale21.0003455017.775[null]S15[null][null]
1930Miles, Mr. Frankmale[null]003593068.05[null]S[null][null][null]
2030Mineff, Mr. Ivanmale24.0003492337.8958[null]S[null][null][null]
2130Minkoff, Mr. Lazarmale21.0003492117.8958[null]S[null][null][null]
2230Mionoff, Mr. Stoytchomale28.0003492077.8958[null]S[null][null][null]
2330Mitkoff, Mr. Mitomale[null]003492217.8958[null]S[null][null][null]
2431Mockler, Miss. Helen Mary "Ellie"female[null]003309807.8792[null]Q16[null][null]
2530Moen, Mr. Sigurd Hansenmale25.0003481237.65F G73S[null]309[null]
2631Moor, Master. Meiermale6.00139209612.475E121S14[null][null]
2731Moor, Mrs. (Beila)female27.00139209612.475E121S14[null][null]
2830Moore, Mr. Leonard Charlesmale[null]00A4. 545108.05[null]S[null][null][null]
2931Moran, Miss. Berthafemale[null]1037111024.15[null]Q16[null][null]
3030Moran, Mr. Daniel Jmale[null]1037111024.15[null]Q[null][null][null]
3130Moran, Mr. Jamesmale[null]003308778.4583[null]Q[null][null][null]
3230Morley, Mr. Williammale34.0003645068.05[null]S[null][null][null]
3330Morrow, Mr. Thomas Rowanmale[null]003726227.75[null]Q[null][null][null]
3431Moss, Mr. Albert Johanmale[null]003129917.775[null]SB[null][null]
3531Moubarek, Master. Geriosmale[null]11266115.2458[null]CC[null][null]
3631Moubarek, Master. Halim Gonios ("William George")male[null]11266115.2458[null]CC[null][null]
3731Moubarek, Mrs. George (Omine "Amenia" Alexander)female[null]02266115.2458[null]CC[null][null]
3831Moussa, Mrs. (Mantoura Boulos)female[null]0026267.2292[null]C[null][null][null]
3930Moutal, Mr. Rahamin Haimmale[null]003747468.05[null]S[null][null][null]
4031Mullens, Miss. Katherine "Katie"female[null]00358527.7333[null]Q16[null][null]
4131Mulvihill, Miss. Bertha Efemale24.0003826537.75[null]Q15[null][null]
4230Murdlin, Mr. Josephmale[null]00A./5. 32358.05[null]S[null][null][null]
4331Murphy, Miss. Katherine "Kate"female[null]1036723015.5[null]Q16[null][null]
4431Murphy, Miss. Margaret Janefemale[null]1036723015.5[null]Q16[null][null]
4531Murphy, Miss. Norafemale[null]003656815.5[null]Q16[null][null]
4630Myhrman, Mr. Pehr Fabian Oliver Malkolmmale18.0003470787.75[null]S[null][null][null]
4730Naidenoff, Mr. Penkomale22.0003492067.8958[null]S[null][null][null]
4831Najib, Miss. Adele Kiamie "Jane"female15.00026677.225[null]CC[null][null]
4931Nakid, Miss. Maria ("Mary")female1.002265315.7417[null]CC[null][null]
5031Nakid, Mr. Sahidmale20.011265315.7417[null]CC[null][null]
5131Nakid, Mrs. Said (Waika "Mary" Mowad)female19.011265315.7417[null]CC[null][null]
5230Nancarrow, Mr. William Henrymale33.000A./5. 33388.05[null]S[null][null][null]
5330Nankoff, Mr. Minkomale[null]003492187.8958[null]S[null][null][null]
5430Nasr, Mr. Mustafamale[null]0026527.2292[null]C[null][null][null]
5530Naughton, Miss. Hannahfemale[null]003652377.75[null]Q[null][null][null]
5630Nenkoff, Mr. Christomale[null]003492347.8958[null]S[null][null][null]
5731Nicola-Yarred, Master. Eliasmale12.010265111.2417[null]CC[null][null]
5831Nicola-Yarred, Miss. Jamilafemale14.010265111.2417[null]CC[null][null]
5930Nieminen, Miss. Manta Josefinafemale29.00031012977.925[null]S[null][null][null]
6030Niklasson, Mr. Samuelmale28.0003636118.05[null]S[null][null][null]
6131Nilsson, Miss. Berta Oliviafemale18.0003470667.775[null]SD[null][null]
6231Nilsson, Miss. Helmina Josefinafemale26.0003474707.8542[null]S13[null][null]
6330Nilsson, Mr. August Ferdinandmale21.0003504107.8542[null]S[null][null][null]
6430Nirva, Mr. Iisakki Antino Aijomale41.000SOTON/O2 31012727.125[null]S[null][null]Finland Sudbury, ON
6531Niskanen, Mr. Juhamale39.000STON/O 2. 31012897.925[null]S9[null][null]
6630Nosworthy, Mr. Richard Catermale21.000A/4. 398867.8[null]S[null][null][null]
6730Novel, Mr. Mansouermale28.50026977.2292[null]C[null]181[null]
6831Nysten, Miss. Anna Sofiafemale22.0003470817.75[null]S13[null][null]
6930Nysveen, Mr. Johan Hansenmale61.0003453646.2375[null]S[null][null][null]
7030O'Brien, Mr. Thomasmale[null]1037036515.5[null]Q[null][null][null]
7130O'Brien, Mr. Timothymale[null]003309797.8292[null]Q[null][null][null]
7231O'Brien, Mrs. Thomas (Johanna "Hannah" Godfrey)female[null]1037036515.5[null]Q[null][null][null]
7330O'Connell, Mr. Patrick Dmale[null]003349127.7333[null]Q[null][null][null]
7430O'Connor, Mr. Mauricemale[null]003710607.75[null]Q[null][null][null]
7530O'Connor, Mr. Patrickmale[null]003667137.75[null]Q[null][null][null]
7630Odahl, Mr. Nils Martinmale23.00072679.225[null]S[null][null][null]
7730O'Donoghue, Ms. Bridgetfemale[null]003648567.75[null]Q[null][null][null]
7831O'Driscoll, Miss. Bridgetfemale[null]00143117.75[null]QD[null][null]
7931O'Dwyer, Miss. Ellen "Nellie"female[null]003309597.8792[null]Q[null][null][null]
8031Ohman, Miss. Velinfemale22.0003470857.775[null]SC[null][null]
8131O'Keefe, Mr. Patrickmale[null]003684027.75[null]QB[null][null]
8231O'Leary, Miss. Hanora "Norah"female[null]003309197.8292[null]Q13[null][null]
8331Olsen, Master. Artur Karlmale9.001C 173683.1708[null]S13[null][null]
8430Olsen, Mr. Henry Margidomale28.000C 400122.525[null]S[null]173[null]
8530Olsen, Mr. Karl Siegwart Andreasmale42.00145798.4042[null]S[null][null][null]
8630Olsen, Mr. Ole Martinmale[null]00Fa 2653027.3125[null]S[null][null][null]
8730Olsson, Miss. Elinafemale31.0003504077.8542[null]S[null][null][null]
8830Olsson, Mr. Nils Johan Goranssonmale28.0003474647.8542[null]S[null][null][null]
8931Olsson, Mr. Oscar Wilhelmmale32.0003470797.775[null]SA[null][null]
9030Olsvigen, Mr. Thor Andersonmale20.00065639.225[null]S[null]89Oslo, Norway Cameron, WI
9130Oreskovic, Miss. Jelkafemale23.0003150858.6625[null]S[null][null][null]
9230Oreskovic, Miss. Marijafemale20.0003150968.6625[null]S[null][null][null]
9330Oreskovic, Mr. Lukamale20.0003150948.6625[null]S[null][null][null]
9430Osen, Mr. Olaf Elonmale16.00075349.2167[null]S[null][null][null]
9531Osman, Mrs. Marafemale31.0003492448.6833[null]S[null][null][null]
9630O'Sullivan, Miss. Bridget Maryfemale[null]003309097.6292[null]Q[null][null][null]
9730Palsson, Master. Gosta Leonardmale2.03134990921.075[null]S[null]4[null]
9830Palsson, Master. Paul Folkemale6.03134990921.075[null]S[null][null][null]
9930Palsson, Miss. Stina Violafemale3.03134990921.075[null]S[null][null][null]
10030Palsson, Miss. Torborg Danirafemale8.03134990921.075[null]S[null][null][null]
Rows: 1-100 | Columns: 14

Note

vastorbit offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the vastorbit environment.

Using filter, we can create custom filters:

vdf.filter("sex = 'female' AND pclass = 1")
123
pclass
Integer
123
survived
Integer
Abc
name
Varchar(164)
Abc
sex
Varchar(20)
123
age
Double
123
sibsp
Integer
123
parch
Integer
Abc
ticket
Varchar(36)
123
fare
Double
Abc
cabin
Varchar(30)
Abc
embarked
Varchar(20)
Abc
boat
Varchar(100)
123
body
Integer
Abc
home.dest
Varchar(100)
131McCormack, Mr. Thomas Josephmale[null]003672287.75[null]Q[null][null][null]
231McCoy, Miss. Agnesfemale[null]2036722623.25[null]Q16[null][null]
331McCoy, Miss. Aliciafemale[null]2036722623.25[null]Q16[null][null]
431McCoy, Mr. Bernardmale[null]2036722623.25[null]Q16[null][null]
531McDermott, Miss. Brigdet Deliafemale[null]003309327.7875[null]Q13[null][null]
630McEvoy, Mr. Michaelmale[null]003656815.5[null]Q[null][null][null]
731McGovern, Miss. Maryfemale[null]003309317.8792[null]Q13[null][null]
831McGowan, Miss. Anna "Annie"female15.0003309238.0292[null]Q[null][null][null]
930McGowan, Miss. Katherinefemale35.00092327.75[null]Q[null][null][null]
1030McMahon, Mr. Martinmale[null]003703727.75[null]Q[null][null][null]
1130McNamee, Mr. Nealmale24.01037656616.1[null]S[null][null][null]
1230McNamee, Mrs. Neal (Eileen O'Leary)female19.01037656616.1[null]S[null]53[null]
1330McNeill, Miss. Bridgetfemale[null]003703687.75[null]Q[null][null][null]
1430Meanwell, Miss. (Marion Ogden)female[null]00SOTON/O.Q. 3920878.05[null]S[null][null][null]
1530Meek, Mrs. Thomas (Annie Louise Rowley)female[null]003430958.05[null]S[null][null][null]
1630Meo, Mr. Alfonzomale55.500A.5. 112068.05[null]S[null]201[null]
1730Mernagh, Mr. Robertmale[null]003687037.75[null]Q[null][null][null]
1831Midtsjo, Mr. Karl Albertmale21.0003455017.775[null]S15[null][null]
1930Miles, Mr. Frankmale[null]003593068.05[null]S[null][null][null]
2030Mineff, Mr. Ivanmale24.0003492337.8958[null]S[null][null][null]
2130Minkoff, Mr. Lazarmale21.0003492117.8958[null]S[null][null][null]
2230Mionoff, Mr. Stoytchomale28.0003492077.8958[null]S[null][null][null]
2330Mitkoff, Mr. Mitomale[null]003492217.8958[null]S[null][null][null]
2431Mockler, Miss. Helen Mary "Ellie"female[null]003309807.8792[null]Q16[null][null]
2530Moen, Mr. Sigurd Hansenmale25.0003481237.65F G73S[null]309[null]
2631Moor, Master. Meiermale6.00139209612.475E121S14[null][null]
2731Moor, Mrs. (Beila)female27.00139209612.475E121S14[null][null]
2830Moore, Mr. Leonard Charlesmale[null]00A4. 545108.05[null]S[null][null][null]
2931Moran, Miss. Berthafemale[null]1037111024.15[null]Q16[null][null]
3030Moran, Mr. Daniel Jmale[null]1037111024.15[null]Q[null][null][null]
3130Moran, Mr. Jamesmale[null]003308778.4583[null]Q[null][null][null]
3230Morley, Mr. Williammale34.0003645068.05[null]S[null][null][null]
3330Morrow, Mr. Thomas Rowanmale[null]003726227.75[null]Q[null][null][null]
3431Moss, Mr. Albert Johanmale[null]003129917.775[null]SB[null][null]
3531Moubarek, Master. Geriosmale[null]11266115.2458[null]CC[null][null]
3631Moubarek, Master. Halim Gonios ("William George")male[null]11266115.2458[null]CC[null][null]
3731Moubarek, Mrs. George (Omine "Amenia" Alexander)female[null]02266115.2458[null]CC[null][null]
3831Moussa, Mrs. (Mantoura Boulos)female[null]0026267.2292[null]C[null][null][null]
3930Moutal, Mr. Rahamin Haimmale[null]003747468.05[null]S[null][null][null]
4031Mullens, Miss. Katherine "Katie"female[null]00358527.7333[null]Q16[null][null]
4131Mulvihill, Miss. Bertha Efemale24.0003826537.75[null]Q15[null][null]
4230Murdlin, Mr. Josephmale[null]00A./5. 32358.05[null]S[null][null][null]
4331Murphy, Miss. Katherine "Kate"female[null]1036723015.5[null]Q16[null][null]
4431Murphy, Miss. Margaret Janefemale[null]1036723015.5[null]Q16[null][null]
4531Murphy, Miss. Norafemale[null]003656815.5[null]Q16[null][null]
4630Myhrman, Mr. Pehr Fabian Oliver Malkolmmale18.0003470787.75[null]S[null][null][null]
4730Naidenoff, Mr. Penkomale22.0003492067.8958[null]S[null][null][null]
4831Najib, Miss. Adele Kiamie "Jane"female15.00026677.225[null]CC[null][null]
4931Nakid, Miss. Maria ("Mary")female1.002265315.7417[null]CC[null][null]
5031Nakid, Mr. Sahidmale20.011265315.7417[null]CC[null][null]
5131Nakid, Mrs. Said (Waika "Mary" Mowad)female19.011265315.7417[null]CC[null][null]
5230Nancarrow, Mr. William Henrymale33.000A./5. 33388.05[null]S[null][null][null]
5330Nankoff, Mr. Minkomale[null]003492187.8958[null]S[null][null][null]
5430Nasr, Mr. Mustafamale[null]0026527.2292[null]C[null][null][null]
5530Naughton, Miss. Hannahfemale[null]003652377.75[null]Q[null][null][null]
5630Nenkoff, Mr. Christomale[null]003492347.8958[null]S[null][null][null]
5731Nicola-Yarred, Master. Eliasmale12.010265111.2417[null]CC[null][null]
5831Nicola-Yarred, Miss. Jamilafemale14.010265111.2417[null]CC[null][null]
5930Nieminen, Miss. Manta Josefinafemale29.00031012977.925[null]S[null][null][null]
6030Niklasson, Mr. Samuelmale28.0003636118.05[null]S[null][null][null]
6131Nilsson, Miss. Berta Oliviafemale18.0003470667.775[null]SD[null][null]
6231Nilsson, Miss. Helmina Josefinafemale26.0003474707.8542[null]S13[null][null]
6330Nilsson, Mr. August Ferdinandmale21.0003504107.8542[null]S[null][null][null]
6430Nirva, Mr. Iisakki Antino Aijomale41.000SOTON/O2 31012727.125[null]S[null][null]Finland Sudbury, ON
6531Niskanen, Mr. Juhamale39.000STON/O 2. 31012897.925[null]S9[null][null]
6630Nosworthy, Mr. Richard Catermale21.000A/4. 398867.8[null]S[null][null][null]
6730Novel, Mr. Mansouermale28.50026977.2292[null]C[null]181[null]
6831Nysten, Miss. Anna Sofiafemale22.0003470817.75[null]S13[null][null]
6930Nysveen, Mr. Johan Hansenmale61.0003453646.2375[null]S[null][null][null]
7030O'Brien, Mr. Thomasmale[null]1037036515.5[null]Q[null][null][null]
7130O'Brien, Mr. Timothymale[null]003309797.8292[null]Q[null][null][null]
7231O'Brien, Mrs. Thomas (Johanna "Hannah" Godfrey)female[null]1037036515.5[null]Q[null][null][null]
7330O'Connell, Mr. Patrick Dmale[null]003349127.7333[null]Q[null][null][null]
7430O'Connor, Mr. Mauricemale[null]003710607.75[null]Q[null][null][null]
7530O'Connor, Mr. Patrickmale[null]003667137.75[null]Q[null][null][null]
7630Odahl, Mr. Nils Martinmale23.00072679.225[null]S[null][null][null]
7730O'Donoghue, Ms. Bridgetfemale[null]003648567.75[null]Q[null][null][null]
7831O'Driscoll, Miss. Bridgetfemale[null]00143117.75[null]QD[null][null]
7931O'Dwyer, Miss. Ellen "Nellie"female[null]003309597.8792[null]Q[null][null][null]
8031Ohman, Miss. Velinfemale22.0003470857.775[null]SC[null][null]
8131O'Keefe, Mr. Patrickmale[null]003684027.75[null]QB[null][null]
8231O'Leary, Miss. Hanora "Norah"female[null]003309197.8292[null]Q13[null][null]
8331Olsen, Master. Artur Karlmale9.001C 173683.1708[null]S13[null][null]
8430Olsen, Mr. Henry Margidomale28.000C 400122.525[null]S[null]173[null]
8530Olsen, Mr. Karl Siegwart Andreasmale42.00145798.4042[null]S[null][null][null]
8630Olsen, Mr. Ole Martinmale[null]00Fa 2653027.3125[null]S[null][null][null]
8730Olsson, Miss. Elinafemale31.0003504077.8542[null]S[null][null][null]
8830Olsson, Mr. Nils Johan Goranssonmale28.0003474647.8542[null]S[null][null][null]
8931Olsson, Mr. Oscar Wilhelmmale32.0003470797.775[null]SA[null][null]
9030Olsvigen, Mr. Thor Andersonmale20.00065639.225[null]S[null]89Oslo, Norway Cameron, WI
9130Oreskovic, Miss. Jelkafemale23.0003150858.6625[null]S[null][null][null]
9230Oreskovic, Miss. Marijafemale20.0003150968.6625[null]S[null][null][null]
9330Oreskovic, Mr. Lukamale20.0003150948.6625[null]S[null][null][null]
9430Osen, Mr. Olaf Elonmale16.00075349.2167[null]S[null][null][null]
9531Osman, Mrs. Marafemale31.0003492448.6833[null]S[null][null][null]
9630O'Sullivan, Miss. Bridget Maryfemale[null]003309097.6292[null]Q[null][null][null]
9730Palsson, Master. Gosta Leonardmale2.03134990921.075[null]S[null]4[null]
9830Palsson, Master. Paul Folkemale6.03134990921.075[null]S[null][null][null]
9930Palsson, Miss. Stina Violafemale3.03134990921.075[null]S[null][null][null]
10030Palsson, Miss. Torborg Danirafemale8.03134990921.075[null]S[null][null][null]
Rows: 1-100 | Columns: 14

Note

Similarly, the same can be done in a pandas-like way:

vdf.filter((vdf["sex"] == "female") && (vdf["pclass"] == 1))

Or:

vdf = vdf[(vdf["sex"] == "female") && (vdf["pclass"] == 1)]

Warning

Ensure to use the && operator and correctly place parentheses. The and operator is specific to Python, and its behavior cannot be changed.

See also

VastFrame.balance() : Balances the VastFrame.
VastFrame.drop() : Drops the VastFrame input columns.