Loading...

vastorbit.VastColumn.dropna

VastColumn.dropna() VastFrame

Filters the VastFrame where the VastColumn is missing.

Returns:

self._parent

Return type:

VastFrame

Examples

We import vastorbit:

import vastorbit as vo

Hint

By assigning an alias to vastorbit, we mitigate the risk of code collisions with other libraries. This precaution is necessary because vastorbit uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions from vastorbit are used as intended without interfering with functions from other libraries.

For this example, we will use the Titanic dataset:

from vastorbit.datasets import load_titanic

vdf = load_titanic()
123
pclass
Integer
123
survived
Integer
Abc
name
Varchar(164)
Abc
sex
Varchar(20)
123
age
Double
123
sibsp
Integer
123
parch
Integer
Abc
ticket
Varchar(36)
123
fare
Double
Abc
cabin
Varchar(30)
Abc
embarked
Varchar(20)
Abc
boat
Varchar(100)
123
body
Integer
Abc
home.dest
Varchar(100)
131McCormack, Mr. Thomas Josephmale[null]003672287.75[null]Q[null][null][null]
231McCoy, Miss. Agnesfemale[null]2036722623.25[null]Q16[null][null]
331McCoy, Miss. Aliciafemale[null]2036722623.25[null]Q16[null][null]
431McCoy, Mr. Bernardmale[null]2036722623.25[null]Q16[null][null]
531McDermott, Miss. Brigdet Deliafemale[null]003309327.7875[null]Q13[null][null]
630McEvoy, Mr. Michaelmale[null]003656815.5[null]Q[null][null][null]
731McGovern, Miss. Maryfemale[null]003309317.8792[null]Q13[null][null]
831McGowan, Miss. Anna "Annie"female15.0003309238.0292[null]Q[null][null][null]
930McGowan, Miss. Katherinefemale35.00092327.75[null]Q[null][null][null]
1030McMahon, Mr. Martinmale[null]003703727.75[null]Q[null][null][null]
1130McNamee, Mr. Nealmale24.01037656616.1[null]S[null][null][null]
1230McNamee, Mrs. Neal (Eileen O'Leary)female19.01037656616.1[null]S[null]53[null]
1330McNeill, Miss. Bridgetfemale[null]003703687.75[null]Q[null][null][null]
1430Meanwell, Miss. (Marion Ogden)female[null]00SOTON/O.Q. 3920878.05[null]S[null][null][null]
1530Meek, Mrs. Thomas (Annie Louise Rowley)female[null]003430958.05[null]S[null][null][null]
1630Meo, Mr. Alfonzomale55.500A.5. 112068.05[null]S[null]201[null]
1730Mernagh, Mr. Robertmale[null]003687037.75[null]Q[null][null][null]
1831Midtsjo, Mr. Karl Albertmale21.0003455017.775[null]S15[null][null]
1930Miles, Mr. Frankmale[null]003593068.05[null]S[null][null][null]
2030Mineff, Mr. Ivanmale24.0003492337.8958[null]S[null][null][null]
2130Minkoff, Mr. Lazarmale21.0003492117.8958[null]S[null][null][null]
2230Mionoff, Mr. Stoytchomale28.0003492077.8958[null]S[null][null][null]
2330Mitkoff, Mr. Mitomale[null]003492217.8958[null]S[null][null][null]
2431Mockler, Miss. Helen Mary "Ellie"female[null]003309807.8792[null]Q16[null][null]
2530Moen, Mr. Sigurd Hansenmale25.0003481237.65F G73S[null]309[null]
2631Moor, Master. Meiermale6.00139209612.475E121S14[null][null]
2731Moor, Mrs. (Beila)female27.00139209612.475E121S14[null][null]
2830Moore, Mr. Leonard Charlesmale[null]00A4. 545108.05[null]S[null][null][null]
2931Moran, Miss. Berthafemale[null]1037111024.15[null]Q16[null][null]
3030Moran, Mr. Daniel Jmale[null]1037111024.15[null]Q[null][null][null]
3130Moran, Mr. Jamesmale[null]003308778.4583[null]Q[null][null][null]
3230Morley, Mr. Williammale34.0003645068.05[null]S[null][null][null]
3330Morrow, Mr. Thomas Rowanmale[null]003726227.75[null]Q[null][null][null]
3431Moss, Mr. Albert Johanmale[null]003129917.775[null]SB[null][null]
3531Moubarek, Master. Geriosmale[null]11266115.2458[null]CC[null][null]
3631Moubarek, Master. Halim Gonios ("William George")male[null]11266115.2458[null]CC[null][null]
3731Moubarek, Mrs. George (Omine "Amenia" Alexander)female[null]02266115.2458[null]CC[null][null]
3831Moussa, Mrs. (Mantoura Boulos)female[null]0026267.2292[null]C[null][null][null]
3930Moutal, Mr. Rahamin Haimmale[null]003747468.05[null]S[null][null][null]
4031Mullens, Miss. Katherine "Katie"female[null]00358527.7333[null]Q16[null][null]
4131Mulvihill, Miss. Bertha Efemale24.0003826537.75[null]Q15[null][null]
4230Murdlin, Mr. Josephmale[null]00A./5. 32358.05[null]S[null][null][null]
4331Murphy, Miss. Katherine "Kate"female[null]1036723015.5[null]Q16[null][null]
4431Murphy, Miss. Margaret Janefemale[null]1036723015.5[null]Q16[null][null]
4531Murphy, Miss. Norafemale[null]003656815.5[null]Q16[null][null]
4630Myhrman, Mr. Pehr Fabian Oliver Malkolmmale18.0003470787.75[null]S[null][null][null]
4730Naidenoff, Mr. Penkomale22.0003492067.8958[null]S[null][null][null]
4831Najib, Miss. Adele Kiamie "Jane"female15.00026677.225[null]CC[null][null]
4931Nakid, Miss. Maria ("Mary")female1.002265315.7417[null]CC[null][null]
5031Nakid, Mr. Sahidmale20.011265315.7417[null]CC[null][null]
5131Nakid, Mrs. Said (Waika "Mary" Mowad)female19.011265315.7417[null]CC[null][null]
5230Nancarrow, Mr. William Henrymale33.000A./5. 33388.05[null]S[null][null][null]
5330Nankoff, Mr. Minkomale[null]003492187.8958[null]S[null][null][null]
5430Nasr, Mr. Mustafamale[null]0026527.2292[null]C[null][null][null]
5530Naughton, Miss. Hannahfemale[null]003652377.75[null]Q[null][null][null]
5630Nenkoff, Mr. Christomale[null]003492347.8958[null]S[null][null][null]
5731Nicola-Yarred, Master. Eliasmale12.010265111.2417[null]CC[null][null]
5831Nicola-Yarred, Miss. Jamilafemale14.010265111.2417[null]CC[null][null]
5930Nieminen, Miss. Manta Josefinafemale29.00031012977.925[null]S[null][null][null]
6030Niklasson, Mr. Samuelmale28.0003636118.05[null]S[null][null][null]
6131Nilsson, Miss. Berta Oliviafemale18.0003470667.775[null]SD[null][null]
6231Nilsson, Miss. Helmina Josefinafemale26.0003474707.8542[null]S13[null][null]
6330Nilsson, Mr. August Ferdinandmale21.0003504107.8542[null]S[null][null][null]
6430Nirva, Mr. Iisakki Antino Aijomale41.000SOTON/O2 31012727.125[null]S[null][null]Finland Sudbury, ON
6531Niskanen, Mr. Juhamale39.000STON/O 2. 31012897.925[null]S9[null][null]
6630Nosworthy, Mr. Richard Catermale21.000A/4. 398867.8[null]S[null][null][null]
6730Novel, Mr. Mansouermale28.50026977.2292[null]C[null]181[null]
6831Nysten, Miss. Anna Sofiafemale22.0003470817.75[null]S13[null][null]
6930Nysveen, Mr. Johan Hansenmale61.0003453646.2375[null]S[null][null][null]
7030O'Brien, Mr. Thomasmale[null]1037036515.5[null]Q[null][null][null]
7130O'Brien, Mr. Timothymale[null]003309797.8292[null]Q[null][null][null]
7231O'Brien, Mrs. Thomas (Johanna "Hannah" Godfrey)female[null]1037036515.5[null]Q[null][null][null]
7330O'Connell, Mr. Patrick Dmale[null]003349127.7333[null]Q[null][null][null]
7430O'Connor, Mr. Mauricemale[null]003710607.75[null]Q[null][null][null]
7530O'Connor, Mr. Patrickmale[null]003667137.75[null]Q[null][null][null]
7630Odahl, Mr. Nils Martinmale23.00072679.225[null]S[null][null][null]
7730O'Donoghue, Ms. Bridgetfemale[null]003648567.75[null]Q[null][null][null]
7831O'Driscoll, Miss. Bridgetfemale[null]00143117.75[null]QD[null][null]
7931O'Dwyer, Miss. Ellen "Nellie"female[null]003309597.8792[null]Q[null][null][null]
8031Ohman, Miss. Velinfemale22.0003470857.775[null]SC[null][null]
8131O'Keefe, Mr. Patrickmale[null]003684027.75[null]QB[null][null]
8231O'Leary, Miss. Hanora "Norah"female[null]003309197.8292[null]Q13[null][null]
8331Olsen, Master. Artur Karlmale9.001C 173683.1708[null]S13[null][null]
8430Olsen, Mr. Henry Margidomale28.000C 400122.525[null]S[null]173[null]
8530Olsen, Mr. Karl Siegwart Andreasmale42.00145798.4042[null]S[null][null][null]
8630Olsen, Mr. Ole Martinmale[null]00Fa 2653027.3125[null]S[null][null][null]
8730Olsson, Miss. Elinafemale31.0003504077.8542[null]S[null][null][null]
8830Olsson, Mr. Nils Johan Goranssonmale28.0003474647.8542[null]S[null][null][null]
8931Olsson, Mr. Oscar Wilhelmmale32.0003470797.775[null]SA[null][null]
9030Olsvigen, Mr. Thor Andersonmale20.00065639.225[null]S[null]89Oslo, Norway Cameron, WI
9130Oreskovic, Miss. Jelkafemale23.0003150858.6625[null]S[null][null][null]
9230Oreskovic, Miss. Marijafemale20.0003150968.6625[null]S[null][null][null]
9330Oreskovic, Mr. Lukamale20.0003150948.6625[null]S[null][null][null]
9430Osen, Mr. Olaf Elonmale16.00075349.2167[null]S[null][null][null]
9531Osman, Mrs. Marafemale31.0003492448.6833[null]S[null][null][null]
9630O'Sullivan, Miss. Bridget Maryfemale[null]003309097.6292[null]Q[null][null][null]
9730Palsson, Master. Gosta Leonardmale2.03134990921.075[null]S[null]4[null]
9830Palsson, Master. Paul Folkemale6.03134990921.075[null]S[null][null][null]
9930Palsson, Miss. Stina Violafemale3.03134990921.075[null]S[null][null][null]
10030Palsson, Miss. Torborg Danirafemale8.03134990921.075[null]S[null][null][null]
Rows: 1-100 | Columns: 14

In the above dataset, notice that the first and last entries are identical i.e. duplicates.

Note

vastorbit offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the vastorbit environment.

We can see the count of each column to check if any column has missing values.

vdf.count()
count
"pclass"1309.0
"survived"1309.0
"name"1309.0
"sex"1309.0
"age"1046.0
"sibsp"1309.0
"parch"1309.0
"ticket"1309.0
"fare"1308.0
"cabin"295.0
"embarked"1307.0
"boat"486.0
"body"121.0
"home.dest"745.0
Rows: 1-14 | Columns: 2

From the above table, we can see that there are a lot of missing values in “boat” column.

Using dropna, we can filter the entire dataset to drop the rows where “boats” does not have a value:

vdf["boat"].dropna()
123
pclass
Integer
123
survived
Integer
Abc
name
Varchar(164)
Abc
sex
Varchar(20)
123
age
Double
123
sibsp
Integer
123
parch
Integer
Abc
ticket
Varchar(36)
123
fare
Double
Abc
cabin
Varchar(30)
Abc
embarked
Varchar(20)
Abc
boat
Varchar(100)
123
body
Integer
Abc
home.dest
Varchar(100)
131McCoy, Miss. Agnesfemale[null]2036722623.25[null]Q16[null][null]
231McCoy, Miss. Aliciafemale[null]2036722623.25[null]Q16[null][null]
331McCoy, Mr. Bernardmale[null]2036722623.25[null]Q16[null][null]
431McDermott, Miss. Brigdet Deliafemale[null]003309327.7875[null]Q13[null][null]
531McGovern, Miss. Maryfemale[null]003309317.8792[null]Q13[null][null]
631Midtsjo, Mr. Karl Albertmale21.0003455017.775[null]S15[null][null]
731Mockler, Miss. Helen Mary "Ellie"female[null]003309807.8792[null]Q16[null][null]
831Moor, Master. Meiermale6.00139209612.475E121S14[null][null]
931Moor, Mrs. (Beila)female27.00139209612.475E121S14[null][null]
1031Moran, Miss. Berthafemale[null]1037111024.15[null]Q16[null][null]
1131Moss, Mr. Albert Johanmale[null]003129917.775[null]SB[null][null]
1231Moubarek, Master. Geriosmale[null]11266115.2458[null]CC[null][null]
1331Moubarek, Master. Halim Gonios ("William George")male[null]11266115.2458[null]CC[null][null]
1431Moubarek, Mrs. George (Omine "Amenia" Alexander)female[null]02266115.2458[null]CC[null][null]
1531Mullens, Miss. Katherine "Katie"female[null]00358527.7333[null]Q16[null][null]
1631Mulvihill, Miss. Bertha Efemale24.0003826537.75[null]Q15[null][null]
1731Murphy, Miss. Katherine "Kate"female[null]1036723015.5[null]Q16[null][null]
1831Murphy, Miss. Margaret Janefemale[null]1036723015.5[null]Q16[null][null]
1931Murphy, Miss. Norafemale[null]003656815.5[null]Q16[null][null]
2031Najib, Miss. Adele Kiamie "Jane"female15.00026677.225[null]CC[null][null]
2131Nakid, Miss. Maria ("Mary")female1.002265315.7417[null]CC[null][null]
2231Nakid, Mr. Sahidmale20.011265315.7417[null]CC[null][null]
2331Nakid, Mrs. Said (Waika "Mary" Mowad)female19.011265315.7417[null]CC[null][null]
2431Nicola-Yarred, Master. Eliasmale12.010265111.2417[null]CC[null][null]
2531Nicola-Yarred, Miss. Jamilafemale14.010265111.2417[null]CC[null][null]
2631Nilsson, Miss. Berta Oliviafemale18.0003470667.775[null]SD[null][null]
2731Nilsson, Miss. Helmina Josefinafemale26.0003474707.8542[null]S13[null][null]
2831Niskanen, Mr. Juhamale39.000STON/O 2. 31012897.925[null]S9[null][null]
2931Nysten, Miss. Anna Sofiafemale22.0003470817.75[null]S13[null][null]
3031O'Driscoll, Miss. Bridgetfemale[null]00143117.75[null]QD[null][null]
3131Ohman, Miss. Velinfemale22.0003470857.775[null]SC[null][null]
3231O'Keefe, Mr. Patrickmale[null]003684027.75[null]QB[null][null]
3331O'Leary, Miss. Hanora "Norah"female[null]003309197.8292[null]Q13[null][null]
3431Olsen, Master. Artur Karlmale9.001C 173683.1708[null]S13[null][null]
3531Olsson, Mr. Oscar Wilhelmmale32.0003470797.775[null]SA[null][null]
3631Persson, Mr. Ernst Ulrikmale25.0103470837.775[null]S15[null][null]
3731Peter, Master. Michael Jmale[null]11266822.3583[null]CC[null][null]
3831Peter, Miss. Annafemale[null]11266822.3583F E69CD[null][null]
3931Peter, Mrs. Catherine (Catherine Rizk)female[null]02266822.3583[null]CD[null][null]
4031Pickard, Mr. Berk (Berk Trembisky)male32.000SOTON/O.Q. 3920788.05E10S9[null][null]
4131Riordan, Miss. Johanna "Hannah"female[null]003349157.7208[null]Q13[null][null]
4231Roth, Miss. Sarah Afemale[null]003427128.05[null]SC[null][null]
4331Ryan, Mr. Edwardmale[null]003831627.75[null]Q14[null][null]
4431Salkjelsvik, Miss. Anna Kristinefemale21.0003431207.65[null]SC[null][null]
4531Sandstrom, Miss. Beatrice Irenefemale1.011PP 954916.7G6S13[null][null]
4631Sandstrom, Mrs. Hjalmar (Agnes Charlotta Bengtsson)female24.002PP 954916.7G6S13[null][null]
4731Sandstrom, Miss. Marguerite Rutfemale4.011PP 954916.7G6S13[null][null]
4831Sap, Mr. Juliusmale25.0003457689.5[null]S11[null][null]
4931Sheerlinck, Mr. Jan Baptistmale29.0003457799.5[null]S11[null][null]
5031Sjoblom, Miss. Anna Sofiafemale18.00031012657.4958[null]S16[null][null]
5131Smyth, Miss. Juliafemale[null]003354327.7333[null]Q13[null][null]
5231Stanley, Miss. Amy Zillah Elsiefemale23.000CA. 23147.55[null]SC[null][null]
5331Stranden, Mr. Juhomale31.000STON/O 2. 31012887.925[null]S9[null][null]
5431Sunderland, Mr. Victor Francismale16.000SOTON/OQ 3920898.05[null]SB[null][null]
5531Sundman, Mr. Johan Julianmale44.000STON/O 2. 31012697.925[null]S15[null][null]
5631Svensson, Mr. Johan Cervinmale14.00075389.225[null]S13[null][null]
5731Tenglin, Mr. Gunnar Isidormale25.0003500337.7958[null]S13 15[null][null]
5831Thomas, Master. Assad Alexandermale0.420126258.5167[null]C16[null][null]
5931Thomas, Mrs. Alexander (Thamine "Thelma")female16.01126258.5167[null]C14[null][null]
6031Thorneycroft, Mrs. Percival (Florence Kate White)female[null]1037656416.1[null]S10[null][null]
6131Tornquist, Mr. William Henrymale25.000LINE0.0[null]S15[null][null]
6231Touma, Master. Georges Youssefmale7.011265015.2458[null]CC[null][null]
6331Touma, Miss. Maria Yousseffemale9.011265015.2458[null]CC[null][null]
6431Touma, Mrs. Darwis (Hanne Youssef Razi)female29.002265015.2458[null]CC[null][null]
6531Turja, Miss. Anna Sofiafemale18.00041389.8417[null]S15[null][null]
6631Turkula, Mrs. (Hedwig)female63.00041349.5875[null]S15[null][null]
6731Vartanian, Mr. Davidmale22.00026587.225[null]C13 15[null][null]
6831Whabee, Mrs. George Joseph (Shawneene Abi-Saab)female38.00026887.2292[null]CC[null][null]
6930Yasbeck, Mr. Antonimale27.010265914.4542[null]CC[null][null]
7011Allen, Miss. Elisabeth Waltonfemale29.00024160211.3375B5S2[null]St Louis, MO
7111Allison, Master. Hudson Trevormale0.9212113781151.55C22 C26S11[null]Montreal, PQ / Chesterville, ON
7211Anderson, Mr. Harrymale48.0001995226.55E12S3[null]New York, NY
7311Andrews, Miss. Kornelia Theodosiafemale63.0101350277.9583D7S10[null]Hudson, NY
7411Appleton, Mrs. Edward Dale (Charlotte Lamson)female53.0201176951.4792C101SD[null]Bayside, Queens, NY
7511Astor, Mrs. John Jacob (Madeleine Talmadge Force)female18.010PC 17757227.525C62 C64C4[null]New York, NY
7611Aubart, Mme. Leontine Paulinefemale24.000PC 1747769.3B35C9[null]Paris, France
7711Barber, Miss. Ellen "Nellie"female26.0001987778.85[null]S6[null][null]
7811Barkworth, Mr. Algernon Henry Wilsonmale80.0002704230.0A23SB[null]Hessle, Yorks
7911Baxter, Mrs. James (Helene DeLaudeniere Chaput)female50.001PC 17558247.5208B58 B60C6[null]Montreal, PQ
8011Bazzani, Miss. Albinafemale32.0001181376.2917D15C8[null][null]
8110Beattie, Mr. Thomsonmale36.0001305075.2417C6CA[null]Winnipeg, MN
8211Beckwith, Mr. Richard Leonardmale37.0111175152.5542D35S5[null]New York, NY
8311Beckwith, Mrs. Richard Leonard (Sallie Monypeny)female47.0111175152.5542D35S5[null]New York, NY
8411Behr, Mr. Karl Howellmale26.00011136930.0C148C5[null]New York, NY
8511Bidois, Miss. Rosaliefemale42.000PC 17757227.525[null]C4[null][null]
8611Bird, Miss. Ellenfemale29.000PC 17483221.7792C97S8[null][null]
8711Bishop, Mr. Dickinson Hmale25.0101196791.0792B49C7[null]Dowagiac, MI
8811Bishop, Mrs. Dickinson H (Helen Walton)female19.0101196791.0792B49C7[null]Dowagiac, MI
8911Bissette, Miss. Ameliafemale35.000PC 17760135.6333C99S8[null][null]
9011Bjornstrom-Steffansson, Mr. Mauritz Hakanmale28.00011056426.55C52SD[null]Stockholm, Sweden / Washington, DC
9111Blank, Mr. Henrymale40.00011227731.0A31C7[null]Glen Ridge, NJ
9211Bonnell, Miss. Carolinefemale30.00036928164.8667C7S8[null]Youngstown, OH
9311Bonnell, Miss. Elizabethfemale58.00011378326.55C103S8[null]Birkdale, England Cleveland, Ohio
9411Bowen, Miss. Grace Scottfemale45.000PC 17608262.375[null]C4[null]Cooperstown, NY
9511Bowerman, Miss. Elsie Edithfemale22.00111350555.0E33S6[null]St Leonards-on-Sea, England Ohio
9611Bradley, Mr. George ("George Arthur Brayton")male[null]0011142726.55[null]S9[null]Los Angeles, CA
9711Brown, Mrs. James Joseph (Margaret Tobin)female44.000PC 1761027.7208B4C6[null]Denver, CO
9811Brown, Mrs. John Murray (Caroline Lane Lamson)female59.0201176951.4792C101SD[null]Belmont, MA
9911Bucknell, Mrs. William Robert (Emma Eliza Ward)female60.0001181376.2917D15C8[null]Philadelphia, PA
10011Burns, Miss. Elizabeth Margaretfemale41.00016966134.5E40C3[null][null]
Rows: 1-100 of 486 | Columns: 14

Now again, if we look at the count, we will notice that the total count has decreased based on the “boats” column.

vdf.count()
count
"pclass"486.0
"survived"486.0
"name"486.0
"sex"486.0
"age"417.0
"sibsp"486.0
"parch"486.0
"ticket"486.0
"fare"486.0
"cabin"193.0
"embarked"484.0
"boat"486.0
"body"0.0
"home.dest"341.0
Rows: 1-14 | Columns: 2

See also

VastFrame.dropna() : Drops the VastFrame missing values.
VastFrame.drop() : Drops the VastFrame input columns.