Best Practices

Optimize your VAST Orbit workflow for maximum performance and efficiency.

---

Restrict to Essential Columns

VAST Orbit executes queries directly in the VAST DataBase. Limiting operations to essential columns significantly improves performance, especially with large datasets.

Load specific columns:

from vastorbit.datasets import load_titanic
import vastorbit as vo

load_titanic()

	123 pclass Integer 100%	123 survived Integer 100%	Abc name Varchar(164) 100%	Abc sex Varchar(20) 100%	123 age Double 79%	123 sibsp Integer 100%	123 parch Integer 100%	Abc ticket Varchar(36) 100%	123 fare Double 99%	Abc cabin Varchar(30) 22%	Abc embarked Varchar(20) 99%	Abc boat Varchar(100) 37%	123 body Integer 9%	Abc home.dest Varchar(100) 56%
1	3	1	McCormack, Mr. Thomas Joseph	male	[null]	0	0	367228	7.75	[null]	Q	[null]	[null]	[null]
2	3	1	McCoy, Miss. Agnes	female	[null]	2	0	367226	23.25	[null]	Q	16	[null]	[null]
3	3	1	McCoy, Miss. Alicia	female	[null]	2	0	367226	23.25	[null]	Q	16	[null]	[null]
4	3	1	McCoy, Mr. Bernard	male	[null]	2	0	367226	23.25	[null]	Q	16	[null]	[null]
5	3	1	McDermott, Miss. Brigdet Delia	female	[null]	0	0	330932	7.7875	[null]	Q	13	[null]	[null]
6	3	0	McEvoy, Mr. Michael	male	[null]	0	0	36568	15.5	[null]	Q	[null]	[null]	[null]
7	3	1	McGovern, Miss. Mary	female	[null]	0	0	330931	7.8792	[null]	Q	13	[null]	[null]
8	3	1	McGowan, Miss. Anna "Annie"	female	15.0	0	0	330923	8.0292	[null]	Q	[null]	[null]	[null]
9	3	0	McGowan, Miss. Katherine	female	35.0	0	0	9232	7.75	[null]	Q	[null]	[null]	[null]
10	3	0	McMahon, Mr. Martin	male	[null]	0	0	370372	7.75	[null]	Q	[null]	[null]	[null]
11	3	0	McNamee, Mr. Neal	male	24.0	1	0	376566	16.1	[null]	S	[null]	[null]	[null]
12	3	0	McNamee, Mrs. Neal (Eileen O'Leary)	female	19.0	1	0	376566	16.1	[null]	S	[null]	53	[null]
13	3	0	McNeill, Miss. Bridget	female	[null]	0	0	370368	7.75	[null]	Q	[null]	[null]	[null]
14	3	0	Meanwell, Miss. (Marion Ogden)	female	[null]	0	0	SOTON/O.Q. 392087	8.05	[null]	S	[null]	[null]	[null]
15	3	0	Meek, Mrs. Thomas (Annie Louise Rowley)	female	[null]	0	0	343095	8.05	[null]	S	[null]	[null]	[null]
16	3	0	Meo, Mr. Alfonzo	male	55.5	0	0	A.5. 11206	8.05	[null]	S	[null]	201	[null]
17	3	0	Mernagh, Mr. Robert	male	[null]	0	0	368703	7.75	[null]	Q	[null]	[null]	[null]
18	3	1	Midtsjo, Mr. Karl Albert	male	21.0	0	0	345501	7.775	[null]	S	15	[null]	[null]
19	3	0	Miles, Mr. Frank	male	[null]	0	0	359306	8.05	[null]	S	[null]	[null]	[null]
20	3	0	Mineff, Mr. Ivan	male	24.0	0	0	349233	7.8958	[null]	S	[null]	[null]	[null]

Use the usecols parameter to load only required columns:

import vastorbit as vo

titanic = vo.VastFrame(
    "titanic",
    usecols=["survived", "pclass", "age", "parch", "sibsp"],
)
titanic.head(100)

	123 pclass Integer 100%	123 survived Integer 100%	Abc name Varchar(164) 100%	Abc sex Varchar(20) 100%	123 age Double 79%	123 sibsp Integer 100%	123 parch Integer 100%	Abc ticket Varchar(36) 100%	123 fare Double 99%	Abc cabin Varchar(30) 22%	Abc embarked Varchar(20) 99%	Abc boat Varchar(100) 37%	123 body Integer 9%	Abc home.dest Varchar(100) 56%
1	3	1	McCormack, Mr. Thomas Joseph	male	[null]	0	0	367228	7.75	[null]	Q	[null]	[null]	[null]
2	3	1	McCoy, Miss. Agnes	female	[null]	2	0	367226	23.25	[null]	Q	16	[null]	[null]
3	3	1	McCoy, Miss. Alicia	female	[null]	2	0	367226	23.25	[null]	Q	16	[null]	[null]
4	3	1	McCoy, Mr. Bernard	male	[null]	2	0	367226	23.25	[null]	Q	16	[null]	[null]
5	3	1	McDermott, Miss. Brigdet Delia	female	[null]	0	0	330932	7.7875	[null]	Q	13	[null]	[null]
6	3	0	McEvoy, Mr. Michael	male	[null]	0	0	36568	15.5	[null]	Q	[null]	[null]	[null]
7	3	1	McGovern, Miss. Mary	female	[null]	0	0	330931	7.8792	[null]	Q	13	[null]	[null]
8	3	1	McGowan, Miss. Anna "Annie"	female	15.0	0	0	330923	8.0292	[null]	Q	[null]	[null]	[null]
9	3	0	McGowan, Miss. Katherine	female	35.0	0	0	9232	7.75	[null]	Q	[null]	[null]	[null]
10	3	0	McMahon, Mr. Martin	male	[null]	0	0	370372	7.75	[null]	Q	[null]	[null]	[null]
11	3	0	McNamee, Mr. Neal	male	24.0	1	0	376566	16.1	[null]	S	[null]	[null]	[null]
12	3	0	McNamee, Mrs. Neal (Eileen O'Leary)	female	19.0	1	0	376566	16.1	[null]	S	[null]	53	[null]
13	3	0	McNeill, Miss. Bridget	female	[null]	0	0	370368	7.75	[null]	Q	[null]	[null]	[null]
14	3	0	Meanwell, Miss. (Marion Ogden)	female	[null]	0	0	SOTON/O.Q. 392087	8.05	[null]	S	[null]	[null]	[null]
15	3	0	Meek, Mrs. Thomas (Annie Louise Rowley)	female	[null]	0	0	343095	8.05	[null]	S	[null]	[null]	[null]
16	3	0	Meo, Mr. Alfonzo	male	55.5	0	0	A.5. 11206	8.05	[null]	S	[null]	201	[null]
17	3	0	Mernagh, Mr. Robert	male	[null]	0	0	368703	7.75	[null]	Q	[null]	[null]	[null]
18	3	1	Midtsjo, Mr. Karl Albert	male	21.0	0	0	345501	7.775	[null]	S	15	[null]	[null]
19	3	0	Miles, Mr. Frank	male	[null]	0	0	359306	8.05	[null]	S	[null]	[null]	[null]
20	3	0	Mineff, Mr. Ivan	male	24.0	0	0	349233	7.8958	[null]	S	[null]	[null]	[null]
21	3	0	Minkoff, Mr. Lazar	male	21.0	0	0	349211	7.8958	[null]	S	[null]	[null]	[null]
22	3	0	Mionoff, Mr. Stoytcho	male	28.0	0	0	349207	7.8958	[null]	S	[null]	[null]	[null]
23	3	0	Mitkoff, Mr. Mito	male	[null]	0	0	349221	7.8958	[null]	S	[null]	[null]	[null]
24	3	1	Mockler, Miss. Helen Mary "Ellie"	female	[null]	0	0	330980	7.8792	[null]	Q	16	[null]	[null]
25	3	0	Moen, Mr. Sigurd Hansen	male	25.0	0	0	348123	7.65	F G73	S	[null]	309	[null]
26	3	1	Moor, Master. Meier	male	6.0	0	1	392096	12.475	E121	S	14	[null]	[null]
27	3	1	Moor, Mrs. (Beila)	female	27.0	0	1	392096	12.475	E121	S	14	[null]	[null]
28	3	0	Moore, Mr. Leonard Charles	male	[null]	0	0	A4. 54510	8.05	[null]	S	[null]	[null]	[null]
29	3	1	Moran, Miss. Bertha	female	[null]	1	0	371110	24.15	[null]	Q	16	[null]	[null]
30	3	0	Moran, Mr. Daniel J	male	[null]	1	0	371110	24.15	[null]	Q	[null]	[null]	[null]
31	3	0	Moran, Mr. James	male	[null]	0	0	330877	8.4583	[null]	Q	[null]	[null]	[null]
32	3	0	Morley, Mr. William	male	34.0	0	0	364506	8.05	[null]	S	[null]	[null]	[null]
33	3	0	Morrow, Mr. Thomas Rowan	male	[null]	0	0	372622	7.75	[null]	Q	[null]	[null]	[null]
34	3	1	Moss, Mr. Albert Johan	male	[null]	0	0	312991	7.775	[null]	S	B	[null]	[null]
35	3	1	Moubarek, Master. Gerios	male	[null]	1	1	2661	15.2458	[null]	C	C	[null]	[null]
36	3	1	Moubarek, Master. Halim Gonios ("William George")	male	[null]	1	1	2661	15.2458	[null]	C	C	[null]	[null]
37	3	1	Moubarek, Mrs. George (Omine "Amenia" Alexander)	female	[null]	0	2	2661	15.2458	[null]	C	C	[null]	[null]
38	3	1	Moussa, Mrs. (Mantoura Boulos)	female	[null]	0	0	2626	7.2292	[null]	C	[null]	[null]	[null]
39	3	0	Moutal, Mr. Rahamin Haim	male	[null]	0	0	374746	8.05	[null]	S	[null]	[null]	[null]
40	3	1	Mullens, Miss. Katherine "Katie"	female	[null]	0	0	35852	7.7333	[null]	Q	16	[null]	[null]
41	3	1	Mulvihill, Miss. Bertha E	female	24.0	0	0	382653	7.75	[null]	Q	15	[null]	[null]
42	3	0	Murdlin, Mr. Joseph	male	[null]	0	0	A./5. 3235	8.05	[null]	S	[null]	[null]	[null]
43	3	1	Murphy, Miss. Katherine "Kate"	female	[null]	1	0	367230	15.5	[null]	Q	16	[null]	[null]
44	3	1	Murphy, Miss. Margaret Jane	female	[null]	1	0	367230	15.5	[null]	Q	16	[null]	[null]
45	3	1	Murphy, Miss. Nora	female	[null]	0	0	36568	15.5	[null]	Q	16	[null]	[null]
46	3	0	Myhrman, Mr. Pehr Fabian Oliver Malkolm	male	18.0	0	0	347078	7.75	[null]	S	[null]	[null]	[null]
47	3	0	Naidenoff, Mr. Penko	male	22.0	0	0	349206	7.8958	[null]	S	[null]	[null]	[null]
48	3	1	Najib, Miss. Adele Kiamie "Jane"	female	15.0	0	0	2667	7.225	[null]	C	C	[null]	[null]
49	3	1	Nakid, Miss. Maria ("Mary")	female	1.0	0	2	2653	15.7417	[null]	C	C	[null]	[null]
50	3	1	Nakid, Mr. Sahid	male	20.0	1	1	2653	15.7417	[null]	C	C	[null]	[null]
51	3	1	Nakid, Mrs. Said (Waika "Mary" Mowad)	female	19.0	1	1	2653	15.7417	[null]	C	C	[null]	[null]
52	3	0	Nancarrow, Mr. William Henry	male	33.0	0	0	A./5. 3338	8.05	[null]	S	[null]	[null]	[null]
53	3	0	Nankoff, Mr. Minko	male	[null]	0	0	349218	7.8958	[null]	S	[null]	[null]	[null]
54	3	0	Nasr, Mr. Mustafa	male	[null]	0	0	2652	7.2292	[null]	C	[null]	[null]	[null]
55	3	0	Naughton, Miss. Hannah	female	[null]	0	0	365237	7.75	[null]	Q	[null]	[null]	[null]
56	3	0	Nenkoff, Mr. Christo	male	[null]	0	0	349234	7.8958	[null]	S	[null]	[null]	[null]
57	3	1	Nicola-Yarred, Master. Elias	male	12.0	1	0	2651	11.2417	[null]	C	C	[null]	[null]
58	3	1	Nicola-Yarred, Miss. Jamila	female	14.0	1	0	2651	11.2417	[null]	C	C	[null]	[null]
59	3	0	Nieminen, Miss. Manta Josefina	female	29.0	0	0	3101297	7.925	[null]	S	[null]	[null]	[null]
60	3	0	Niklasson, Mr. Samuel	male	28.0	0	0	363611	8.05	[null]	S	[null]	[null]	[null]
61	3	1	Nilsson, Miss. Berta Olivia	female	18.0	0	0	347066	7.775	[null]	S	D	[null]	[null]
62	3	1	Nilsson, Miss. Helmina Josefina	female	26.0	0	0	347470	7.8542	[null]	S	13	[null]	[null]
63	3	0	Nilsson, Mr. August Ferdinand	male	21.0	0	0	350410	7.8542	[null]	S	[null]	[null]	[null]
64	3	0	Nirva, Mr. Iisakki Antino Aijo	male	41.0	0	0	SOTON/O2 3101272	7.125	[null]	S	[null]	[null]	Finland Sudbury, ON
65	3	1	Niskanen, Mr. Juha	male	39.0	0	0	STON/O 2. 3101289	7.925	[null]	S	9	[null]	[null]
66	3	0	Nosworthy, Mr. Richard Cater	male	21.0	0	0	A/4. 39886	7.8	[null]	S	[null]	[null]	[null]
67	3	0	Novel, Mr. Mansouer	male	28.5	0	0	2697	7.2292	[null]	C	[null]	181	[null]
68	3	1	Nysten, Miss. Anna Sofia	female	22.0	0	0	347081	7.75	[null]	S	13	[null]	[null]
69	3	0	Nysveen, Mr. Johan Hansen	male	61.0	0	0	345364	6.2375	[null]	S	[null]	[null]	[null]
70	3	0	O'Brien, Mr. Thomas	male	[null]	1	0	370365	15.5	[null]	Q	[null]	[null]	[null]
71	3	0	O'Brien, Mr. Timothy	male	[null]	0	0	330979	7.8292	[null]	Q	[null]	[null]	[null]
72	3	1	O'Brien, Mrs. Thomas (Johanna "Hannah" Godfrey)	female	[null]	1	0	370365	15.5	[null]	Q	[null]	[null]	[null]
73	3	0	O'Connell, Mr. Patrick D	male	[null]	0	0	334912	7.7333	[null]	Q	[null]	[null]	[null]
74	3	0	O'Connor, Mr. Maurice	male	[null]	0	0	371060	7.75	[null]	Q	[null]	[null]	[null]
75	3	0	O'Connor, Mr. Patrick	male	[null]	0	0	366713	7.75	[null]	Q	[null]	[null]	[null]
76	3	0	Odahl, Mr. Nils Martin	male	23.0	0	0	7267	9.225	[null]	S	[null]	[null]	[null]
77	3	0	O'Donoghue, Ms. Bridget	female	[null]	0	0	364856	7.75	[null]	Q	[null]	[null]	[null]
78	3	1	O'Driscoll, Miss. Bridget	female	[null]	0	0	14311	7.75	[null]	Q	D	[null]	[null]
79	3	1	O'Dwyer, Miss. Ellen "Nellie"	female	[null]	0	0	330959	7.8792	[null]	Q	[null]	[null]	[null]
80	3	1	Ohman, Miss. Velin	female	22.0	0	0	347085	7.775	[null]	S	C	[null]	[null]
81	3	1	O'Keefe, Mr. Patrick	male	[null]	0	0	368402	7.75	[null]	Q	B	[null]	[null]
82	3	1	O'Leary, Miss. Hanora "Norah"	female	[null]	0	0	330919	7.8292	[null]	Q	13	[null]	[null]
83	3	1	Olsen, Master. Artur Karl	male	9.0	0	1	C 17368	3.1708	[null]	S	13	[null]	[null]
84	3	0	Olsen, Mr. Henry Margido	male	28.0	0	0	C 4001	22.525	[null]	S	[null]	173	[null]
85	3	0	Olsen, Mr. Karl Siegwart Andreas	male	42.0	0	1	4579	8.4042	[null]	S	[null]	[null]	[null]
86	3	0	Olsen, Mr. Ole Martin	male	[null]	0	0	Fa 265302	7.3125	[null]	S	[null]	[null]	[null]
87	3	0	Olsson, Miss. Elina	female	31.0	0	0	350407	7.8542	[null]	S	[null]	[null]	[null]
88	3	0	Olsson, Mr. Nils Johan Goransson	male	28.0	0	0	347464	7.8542	[null]	S	[null]	[null]	[null]
89	3	1	Olsson, Mr. Oscar Wilhelm	male	32.0	0	0	347079	7.775	[null]	S	A	[null]	[null]
90	3	0	Olsvigen, Mr. Thor Anderson	male	20.0	0	0	6563	9.225	[null]	S	[null]	89	Oslo, Norway Cameron, WI
91	3	0	Oreskovic, Miss. Jelka	female	23.0	0	0	315085	8.6625	[null]	S	[null]	[null]	[null]
92	3	0	Oreskovic, Miss. Marija	female	20.0	0	0	315096	8.6625	[null]	S	[null]	[null]	[null]
93	3	0	Oreskovic, Mr. Luka	male	20.0	0	0	315094	8.6625	[null]	S	[null]	[null]	[null]
94	3	0	Osen, Mr. Olaf Elon	male	16.0	0	0	7534	9.2167	[null]	S	[null]	[null]	[null]
95	3	1	Osman, Mrs. Mara	female	31.0	0	0	349244	8.6833	[null]	S	[null]	[null]	[null]
96	3	0	O'Sullivan, Miss. Bridget Mary	female	[null]	0	0	330909	7.6292	[null]	Q	[null]	[null]	[null]
97	3	0	Palsson, Master. Gosta Leonard	male	2.0	3	1	349909	21.075	[null]	S	[null]	4	[null]
98	3	0	Palsson, Master. Paul Folke	male	6.0	3	1	349909	21.075	[null]	S	[null]	[null]	[null]
99	3	0	Palsson, Miss. Stina Viola	female	3.0	3	1	349909	21.075	[null]	S	[null]	[null]	[null]
100	3	0	Palsson, Miss. Torborg Danira	female	8.0	3	1	349909	21.075	[null]	S	[null]	[null]	[null]

Inspect generated SQL:

Note

Enable SQL output with set_option() to see generated queries.

# Turn on SQL generation
vo.set_option("sql_on", True)

titanic.avg()

# Turn off SQL generation
vo.set_option("sql_on", False)

Restrict operations to specific columns:

titanic.avg(columns=["age", "survived"])

	avg
"age"	29.881137667304014
"survived"	0.3819709702062643

Exclude columns:

titanic.numcol(exclude_columns=["parch", "sibsp"])

Note

Use get_columns() to list all columns.

Combine with other operations:

titanic.corr(columns=titanic.numcol(exclude_columns=["parch", "sibsp"]))

Tip

For datasets > 1TB, selecting essential columns can reduce query time by 10-100x.

---

Save Complex Relations

VastFrame works like a SQL view. Heavy transformations (joins, window functions, complex operations) can slow down each method call. Save complex relations as tables to improve performance.

Example transformation chain:

titanic = vo.VastFrame("titanic")
titanic["sex"].label_encode()["boat"].fillna(method="0ifnull")["name"].str_extract(
    r' ([A-Za-z]+)\.').eval("family_size", expr="parch + sibsp + 1").drop(
    columns=["cabin", "body", "ticket", "home.dest"])["fare"].fill_outliers().fillna()

print(titanic.current_relation())

View query plan:

print(titanic.explain())

Save as table:

vo.drop("titanic_clean", method="table")

titanic.to_db(
    "titanic_clean",
    relation_type="table",
    inplace=True,
)

print(titanic.current_relation())

Warning

For very large datasets, save tables only after thorough exploration to avoid unnecessary storage.

---

Use Built-in Help

Quick reference for any function:

help(vo.connect)

---

Close Connections

Close connections when finished to reduce database concurrency:

# Connect
vo.connect("vast_connection")

# Do work...

# Close when done
vo.close_connection()

Important

Always close connections in multi-user environments.

---

Consider Time Complexity

Some methods are computationally expensive. For example, Kendall correlation uses cross joins (O(n²)) vs Pearson (O(n)).

Performance comparison:

import time

titanic = vo.VastFrame("titanic")

start_time = time.time()
x = titanic.corr(method="pearson", show=False)
print(f"Pearson: {time.time() - start_time:.2f}s")

start_time = time.time()
x = titanic.corr(method="kendall", show=False)
print(f"Kendall: {time.time() - start_time:.2f}s")

Tip

Choose methods appropriate for your dataset size. Use Pearson for large datasets unless Kendall is required.

---

Limit Plot Elements

Keep visualizations readable by limiting categories:

Too many categories:

titanic.bar(["name", "survived"])  # 1000+ categories - unreadable

Optimal categories:

titanic.hist(["pclass", "survived"])  # 3 categories - clear

Check cardinality:

titanic.nunique()

	approx_unique
"pclass"	3.0
"survived"	2.0
"name"	1288.0
"sex"	2.0
"age"	98.0
"sibsp"	7.0
"parch"	8.0
"ticket"	939.0
"fare"	285.0
"cabin"	186.0
"embarked"	3.0
"boat"	27.0
"body"	121.0
"home.dest"	361.0

---

Filter Unnecessary Data

Filtering reduces computation and improves performance:

Filter rows:

titanic.filter("boat IS NOT NULL")

	123 pclass Integer 100%	123 survived Integer 100%	Abc name Varchar(164) 100%	Abc sex Varchar(20) 100%	123 age Double 85%	123 sibsp Integer 100%	123 parch Integer 100%	Abc ticket Varchar(36) 100%	123 fare Double 100%	Abc cabin Varchar(30) 39%	Abc embarked Varchar(20) 99%	Abc boat Varchar(100) 100%	123 body Integer 0%	Abc home.dest Varchar(100) 70%
1	3	1	McCoy, Miss. Agnes	female	[null]	2	0	367226	23.25	[null]	Q	16	[null]	[null]
2	3	1	McCoy, Miss. Alicia	female	[null]	2	0	367226	23.25	[null]	Q	16	[null]	[null]
3	3	1	McCoy, Mr. Bernard	male	[null]	2	0	367226	23.25	[null]	Q	16	[null]	[null]
4	3	1	McDermott, Miss. Brigdet Delia	female	[null]	0	0	330932	7.7875	[null]	Q	13	[null]	[null]
5	3	1	McGovern, Miss. Mary	female	[null]	0	0	330931	7.8792	[null]	Q	13	[null]	[null]
6	3	1	Midtsjo, Mr. Karl Albert	male	21.0	0	0	345501	7.775	[null]	S	15	[null]	[null]
7	3	1	Mockler, Miss. Helen Mary "Ellie"	female	[null]	0	0	330980	7.8792	[null]	Q	16	[null]	[null]
8	3	1	Moor, Master. Meier	male	6.0	0	1	392096	12.475	E121	S	14	[null]	[null]
9	3	1	Moor, Mrs. (Beila)	female	27.0	0	1	392096	12.475	E121	S	14	[null]	[null]
10	3	1	Moran, Miss. Bertha	female	[null]	1	0	371110	24.15	[null]	Q	16	[null]	[null]
11	3	1	Moss, Mr. Albert Johan	male	[null]	0	0	312991	7.775	[null]	S	B	[null]	[null]
12	3	1	Moubarek, Master. Gerios	male	[null]	1	1	2661	15.2458	[null]	C	C	[null]	[null]
13	3	1	Moubarek, Master. Halim Gonios ("William George")	male	[null]	1	1	2661	15.2458	[null]	C	C	[null]	[null]
14	3	1	Moubarek, Mrs. George (Omine "Amenia" Alexander)	female	[null]	0	2	2661	15.2458	[null]	C	C	[null]	[null]
15	3	1	Mullens, Miss. Katherine "Katie"	female	[null]	0	0	35852	7.7333	[null]	Q	16	[null]	[null]
16	3	1	Mulvihill, Miss. Bertha E	female	24.0	0	0	382653	7.75	[null]	Q	15	[null]	[null]
17	3	1	Murphy, Miss. Katherine "Kate"	female	[null]	1	0	367230	15.5	[null]	Q	16	[null]	[null]
18	3	1	Murphy, Miss. Margaret Jane	female	[null]	1	0	367230	15.5	[null]	Q	16	[null]	[null]
19	3	1	Murphy, Miss. Nora	female	[null]	0	0	36568	15.5	[null]	Q	16	[null]	[null]
20	3	1	Najib, Miss. Adele Kiamie "Jane"	female	15.0	0	0	2667	7.225	[null]	C	C	[null]	[null]

Drop columns:

titanic.drop(["name", "body"])

	123 pclass Integer 100%	123 survived Integer 100%	Abc sex Varchar(20) 100%	123 age Double 85%	123 sibsp Integer 100%	123 parch Integer 100%	Abc ticket Varchar(36) 100%	123 fare Double 100%	Abc cabin Varchar(30) 39%	Abc embarked Varchar(20) 99%	Abc boat Varchar(100) 100%	Abc home.dest Varchar(100) 70%
1	3	1	female	[null]	2	0	367226	23.25	[null]	Q	16	[null]
2	3	1	female	[null]	2	0	367226	23.25	[null]	Q	16	[null]
3	3	1	male	[null]	2	0	367226	23.25	[null]	Q	16	[null]
4	3	1	female	[null]	0	0	330932	7.7875	[null]	Q	13	[null]
5	3	1	female	[null]	0	0	330931	7.8792	[null]	Q	13	[null]
6	3	1	male	21.0	0	0	345501	7.775	[null]	S	15	[null]
7	3	1	female	[null]	0	0	330980	7.8792	[null]	Q	16	[null]
8	3	1	male	6.0	0	1	392096	12.475	E121	S	14	[null]
9	3	1	female	27.0	0	1	392096	12.475	E121	S	14	[null]
10	3	1	female	[null]	1	0	371110	24.15	[null]	Q	16	[null]
11	3	1	male	[null]	0	0	312991	7.775	[null]	S	B	[null]
12	3	1	male	[null]	1	1	2661	15.2458	[null]	C	C	[null]
13	3	1	male	[null]	1	1	2661	15.2458	[null]	C	C	[null]
14	3	1	female	[null]	0	2	2661	15.2458	[null]	C	C	[null]
15	3	1	female	[null]	0	0	35852	7.7333	[null]	Q	16	[null]
16	3	1	female	24.0	0	0	382653	7.75	[null]	Q	15	[null]
17	3	1	female	[null]	1	0	367230	15.5	[null]	Q	16	[null]
18	3	1	female	[null]	1	0	367230	15.5	[null]	Q	16	[null]
19	3	1	female	[null]	0	0	36568	15.5	[null]	Q	16	[null]
20	3	1	female	15.0	0	0	2667	7.225	[null]	C	C	[null]

Verify relation:

print(titanic.current_relation())

---

Optimize Resource Usage

Control query concurrency for large datasets with many columns.

Generate test dataset:

from vastorbit.datasets import gen_dataset

vo.drop("test_dataset", method="table")

features_ranges = {}
for i in range(20):
    features_ranges[f"x{i}"] = {"type": float, "range": [0, 1]}

vdf = gen_dataset(
    features_ranges,
    nrows=10000,
).to_db(
    "test_dataset",
    relation_type="table",
    inplace=True,
)
vdf.head(100)

	123 x0 Double 100%	123 x1 Double 100%	123 x2 Double 100%	123 x3 Double 100%	123 x4 Double 100%	123 x5 Double 100%	123 x6 Double 100%	123 x7 Double 100%	123 x8 Double 100%	123 x9 Double 100%	123 x10 Double 100%	123 x11 Double 100%	123 x12 Double 100%	123 x13 Double 100%	123 x14 Double 100%	123 x15 Double 100%	123 x16 Double 100%	123 x17 Double 100%	123 x18 Double 100%	123 x19 Double 100%
1	0.14350886334607715	0.14044753950014888	0.19025645010865588	0.051623291471994404	0.35352990652033467	0.8301849388025972	0.859345078875869	0.3562859034104793	0.8063753730225969	0.31270105109989976	0.5550510197553127	0.2234873786893059	0.6577927771313732	0.8043982366240008	0.045796173666069584	0.9596184379073176	0.5396441175566526	0.3289317276328343	0.5760427250779837	0.7230085984479506
2	0.5380939234520807	0.28899493980245017	0.7085455752111903	0.23244720637752148	0.44686088189083095	0.6440852064376195	0.5783827516639957	0.3688063013599784	0.9932471470372578	0.838603919014869	0.4988390944172897	0.8740481383992994	0.11015992147691211	0.4514613301892946	0.8595469337586313	0.4793556405709477	0.426541406692436	0.8626318535262203	0.8985789773720951	0.18793844920616531
3	0.18082036352487874	0.8197925259949336	0.7776608627768629	0.9207965978630458	0.8844858099367702	0.30680244531115464	0.7220513763279636	0.44418225308924697	0.3196631423078993	0.11095189306696518	0.48464252288862675	0.807673969858335	0.974870591623373	0.6493174537781536	0.4328925040350112	0.74221137937457	0.8370847790546778	0.7168079413305701	0.9699454014760016	0.4646854275589223
4	0.9357480909617392	0.6077707614578217	0.10899193777465621	0.7911196673417376	0.09086667714700791	0.0702877447841963	0.07799561762146334	0.4530925549343373	0.7646487202775384	0.7637321208261905	0.5624555121192995	0.9164216825793566	0.3395426972160911	0.5689686698754401	0.2596081874054489	0.36183629736520095	0.9201455112955025	0.5342284467512745	0.2909200545115136	0.0014577112764556643
5	0.504249255349948	0.5573097168795871	0.6088711839057737	0.8707522129946711	0.1591386194220371	0.19402771648223094	0.23137269554681072	0.3302137225993135	0.673620835673669	0.0287666765293908	0.867852604103231	0.26471080448049	0.07264930186381535	0.08637952736838272	0.09447785602739434	0.4290745474387002	0.964844002127972	0.9513812610168726	0.2579854524705668	0.10476109652642063
6	0.6921788050954798	0.10349049476240058	0.475203674318485	0.6346340069099524	0.10404835836232118	0.8683933225854986	0.98795829518756	0.7546673721520909	0.7060069271362632	0.8895699321959076	0.8235382284211363	0.6742333221047468	0.043946323931581155	0.5619810995358139	0.5798462187863542	0.49873258978824464	0.7313576208013673	0.11862793965311691	0.9854228684444801	0.2729342509405739
7	0.35791349679593465	0.7545152097136493	0.4512707251377446	0.7130702200178571	0.5536897910526456	0.6798622981029886	0.9982929715788451	0.46036073148472767	0.6469571009210228	0.024319050002041043	0.4661369295091189	0.9113385043179467	0.012476662765559698	0.9744501665785715	0.8256764196960391	0.8274164752849957	0.5553810750304061	0.5555895975895278	0.4255701640320846	0.5334403534815733
8	0.05841849986941405	0.9877328758646011	0.14961027929627546	0.7671671825730674	0.5856808242289444	0.6953417236572833	0.40153191211376027	0.4292757435877286	0.9614682656473181	0.29406548391207266	0.671387578745432	0.41255418990497217	0.7284889227327022	0.14986950776979124	0.12329901694945888	0.15562152088989412	0.06818303351771571	0.9984887157167831	0.8433017371320994	0.4290224171145396
9	0.42021709160044574	0.49217684620477986	0.9335307699225869	0.49105967650926097	0.670542996320435	0.3169593903507558	0.12772299180877944	0.6806911193343569	0.5154191299541389	0.25928336720029554	0.06972437500855866	0.03126840403363118	0.14418784457014744	0.856669274493498	0.7567734098691009	0.4035226593843807	0.06994872349250059	0.7276113380322857	0.08134861884259414	0.4587005056909246
10	0.42365419657675585	0.20600494724206175	0.4972663606185811	0.5811943592250152	0.5526905085689117	0.4180084951681965	0.8793046305161403	0.6353259710200148	0.14786408955668495	0.8064251574173882	0.6544429714813418	0.32321572566190115	0.7769533776314639	0.1474551695500368	0.6186066644362062	0.7201076084251806	0.07300880556297606	0.9348270793643986	0.7027444747078407	0.532400544711062
11	0.5569537595854189	0.567850325591512	0.9982362793236409	0.6545488545936283	0.3501492186299885	0.9053612322295658	0.12428123340727049	0.7747321040379918	0.3269704760594725	0.5750106918076928	0.5150092279006812	0.33259660743807773	0.9978352889523873	0.7087574903861267	0.9387238586358172	0.9206914682285077	0.06702766677293148	0.2776815472902867	0.596890983523906	0.01260230634946602
12	0.7237212654524301	0.43351315578674565	0.8584176358913788	0.7251369834037051	0.5280883647424798	0.4911752955650225	0.4619401302414573	0.975290743392889	0.22973382229327832	0.8626408141992109	0.2497934816143672	0.20419700478810454	0.13956510262749988	0.804821469231423	0.8495063517777123	0.33133101485785454	0.5118756434354912	0.7267205023171833	0.6195228048237811	0.004444009491327838
13	0.5021577826744671	0.1341728513654895	0.6388866555986067	0.5954580474069499	0.5778973267480363	0.4503184001706153	0.6670945240850071	0.14539277244443316	0.3172194403920535	0.45277066053218185	0.128316031242097	0.38666444685090473	0.7413689238676371	0.2508081886433112	0.776590119301604	0.9016012895451401	0.1382480331074245	0.11042811588632695	0.6380773295170671	0.38325285937649933
14	0.38553430915073106	0.047259598699944005	0.34680268922908053	0.3488040565190249	0.060650600409254296	0.8530379709903559	0.6363394336474154	0.7709481121165269	0.9639807295263154	0.6616276551899799	0.9503107263382989	0.14009619168974086	0.4922970071872572	0.6923715185280167	0.6668650113931203	0.5958796448567266	0.7141720254020307	0.8122655934961713	0.15125377892813074	0.008705885395445367
15	0.34041745274214485	0.7085255946636035	0.0018841521548299545	0.3272234482279711	0.33214008607398926	0.31125567883043803	0.23099570045451756	0.20805142053025338	0.3609638836138621	0.7657463144946846	0.6898225092543315	0.06540089322627074	0.4038397207645852	0.6083217125104387	0.41513994905428764	0.8762696810895143	0.1611112461415376	0.4159455770643007	0.656188181142059	0.18703508752717568
16	0.05170415988394217	0.6329251839944118	0.9752497755696246	0.18134072911177979	0.5929200251833431	0.8979651765928935	0.26695706236418304	0.7817921906691913	0.7563616370240901	0.7573357861084444	0.2780743008628892	0.2810754371733103	0.020028849413875283	0.2789155083082253	0.9011674644038233	0.08368737745712329	0.7437101758402093	0.15265426820942551	0.1327333480622136	0.885050142754828
17	0.2558186438147626	0.19071246869086778	0.9653008190622716	0.3842403806423019	0.12920537991006942	0.9655938736450056	0.2689352059208968	0.6195842512623219	0.6660845887064746	0.7603241677020018	0.6480815519336428	0.007990714868077231	0.7024867250897958	0.9764239144668476	0.07922055185174626	0.3391905081686952	0.3336012661598531	0.7791604950271833	0.2967313505634954	0.6484439320417712
18	0.695400593496913	0.41521560661095536	0.6432387964237309	0.6632807776109513	0.583178881933409	0.9244277993326704	0.5633973030098189	0.5162608281634801	0.22931478869434474	0.2471837722291812	0.8230334301255755	0.39696241402260024	0.5805149504873716	0.952542832371542	0.9907199184332236	0.7867574929580476	0.22948955509304292	0.5172036075333792	0.05541696200178303	0.592025384203108
19	0.3556771406357141	0.8367656557592105	0.5697957285066386	0.7768748518587356	0.4258082455655867	0.19297662861922327	0.032195207521402924	0.7699453809354294	0.31306639645392886	0.7903732486335572	0.3620078224788268	0.7724216721107677	0.07895012833405257	0.3185572728930285	0.4105708601482658	0.4035679441628546	0.49419516094214455	0.5644597115557418	0.7252103298992486	0.22896081037643
20	0.3154147038482422	0.43993539469137377	0.7333191994059989	0.8499440280018885	0.6961479339240529	0.3849410981254473	0.13047086109745354	0.8208808306826452	0.17680251400311298	0.739673555836553	0.5644930968892298	0.5112449842365744	0.8617106714551758	0.3534742217913408	0.31496045874157697	0.32615168681889617	0.951299034860253	0.717542633560538	0.8900593867240942	0.22928274115086533
21	0.3142610703365296	0.8222055359739925	0.5439063837420384	0.9754470958104651	0.35255669801750233	0.57622764276761	0.5269388761165517	0.29947017876836346	0.8633176318657575	0.18988163829920013	0.22759419880543097	0.5730855335572491	0.09882351620733731	0.502675830174828	0.5097612597020211	0.4222450494030293	0.7611109002910329	0.43099665009423627	0.8893931631760552	0.3066928674151179
22	0.9648583409875955	0.010041701785117985	0.8787347960346888	0.4831831225253185	0.17545707486456297	0.3414318018777137	0.2589815035397114	0.6287844699498428	0.3777741396472891	0.46171246736688143	0.8658228850718003	0.1433617769696559	0.8447975266346031	0.5430844284065646	0.8872767383522496	0.3521839171378003	0.7872711889520382	0.5327245894037881	0.49620030151563144	0.08916413139422841
23	0.5613734660279045	0.3907973593909737	0.7355760266117742	0.024827740497853856	0.9936434343377387	0.7859990371955093	0.4117542597794136	0.3843708548090454	0.8577209814469707	0.42121256163780196	0.9211411293448889	0.7817005335269143	0.03249869893720403	0.4777752146413171	0.34629661531772404	0.9970227320180047	0.26627956730471314	0.9913019632347512	0.3471726073252368	0.7007176456287205
24	0.1031083832296561	0.899767995714402	0.14126509608513982	0.6658948716774272	0.7124119925088382	0.37101412056022476	0.80059159593984	0.3762172929942187	0.880479095402597	0.06326380277113985	0.4622934504136744	0.45308897187835817	0.2838292536361342	0.10169239671105124	0.45572226830726925	0.9528375897946882	0.8111908120202753	0.37266992612312366	0.4757939103755142	0.6368607260413189
25	0.5892852852174122	0.3194443194840242	0.7139282869553902	0.4263615826586612	0.05204422753781679	0.061196128671790695	0.6374662981076653	0.35512677052212216	0.9432974292798667	0.02189700544857065	0.04883031257502779	0.22312641046477721	0.39499865409394386	0.3866255713703094	0.6783949820367944	0.6174327065370988	0.6008265030520343	0.008700016319844295	0.4636005988703681	0.3942452008913737
26	0.2733164868887762	0.3903371103466817	0.30209740782053174	0.857919541666539	0.3249991349328252	0.00023736663701123817	0.5746008155164305	0.7087107182975567	0.16216740654755657	0.07765456047757091	0.1207341905849807	0.5837765002088507	0.3330162392159487	0.40739190940614745	0.7897145709434044	0.8814272682767966	0.61845703827075	0.4470086964949167	0.5226691114226577	0.6583262867806537
27	0.12383653440269948	0.3727136088578681	0.9851210720016097	0.3572561429395007	0.3346422406148286	0.9546085876855176	0.7286020131150022	0.08684800263368042	0.8797703265231774	0.7899765029183907	0.08230145737191075	0.29077534600370103	0.9594374380452095	0.9776713921339555	0.13063835837552784	0.7649487659932884	0.6194952638882902	0.5525289381362352	0.26042701972377047	0.7675157195267935
28	0.4646719994232038	0.09490903815907925	0.4188926300997988	0.9908845229923435	0.7717513210699114	0.44555546923248224	0.014210307404966982	0.04907179023290553	0.16089261833958346	0.16807040653969996	0.482153428590946	0.982124992831741	0.1292313527122516	0.4215039082009905	0.26437529533968185	0.9006025812895414	0.13559024782451934	0.7100018317792435	0.5486739599556103	0.9284675138737672
29	0.179946439300261	0.638237514024015	0.06969983248024458	0.6946193930844496	0.47608641180199185	0.5589826433245725	0.0871193161801026	0.5920648734350927	0.39810852298128807	0.4350326550847563	0.9985630462888658	0.7355130806949602	0.06699030904523584	0.6155158957325491	0.5506502025975132	0.601488040635999	0.7476338280904052	0.8398075410100655	0.18997046268816864	0.18995656445942666
30	0.021108384031964023	0.7755626676747976	0.7100167319342867	0.6240554664838377	0.15736302784110878	0.09674650407725294	0.4628059389281325	0.035168110239398476	0.2411634089782111	0.19131505627876189	0.022750711438807136	0.6969769533432134	0.24483092239780768	0.011131053772078459	0.48939114890042856	0.7764662026158362	0.12155598836596004	0.02268220027405321	0.5030322889253254	0.3850817452616643
31	0.9024153065955062	0.7146698680096983	0.44307298573290566	0.6450966321226407	0.6271202660073165	0.8955939075452652	0.19855788852808898	0.5065621816420511	0.06740986041356278	0.4274804237712383	0.049722904944947355	0.5991588633012072	0.34571948061902924	0.8322611019905081	0.5853898337267834	0.7801589117278738	0.11275585339707528	0.5429221856976781	0.8194620301296969	0.040493006249402996
32	0.8775068069157782	0.7380677935408616	0.06194204398072889	0.028242102046642237	0.37660325036875775	0.7847912118843937	0.8703109664655323	0.9051038171000944	0.8790764421363647	0.2119765105791911	0.6297525029421048	0.4950379418980768	0.42069345415243564	0.09009385530108083	0.6470673205198992	0.1420201806021123	0.8906294508700282	0.41401559340038363	0.2270714645043883	0.08416126886697584
33	0.6145420633861272	0.5465187908586919	0.3181033926610184	0.022432745920474395	0.9827179357572252	0.38551548038199634	0.4188899180615183	0.5658377798814945	0.3255494079797663	0.861487238609705	0.8471205458791015	0.4463247474795926	0.5071606939970186	0.664096824053838	0.11794899842345552	0.8206345091604106	0.36912660527184515	0.1029740079084136	0.6487415150376583	0.22435349398595839
34	0.032980963574750555	0.4958120019424539	0.06585754756569473	0.4476140782021054	0.6080309473365363	0.5028357354552916	0.5182705504453368	0.7269423581472505	0.3774118692715389	0.4781527857850256	0.6726679623657121	0.2826753608815791	0.3171741893804342	0.8951178789763757	0.29517269405480795	0.5314687127276073	0.6615576920126741	0.18119748221862642	0.6972812004023111	0.9969844352732846
35	0.15089116402474845	0.4052372934945636	0.8654047227581535	0.3440255870687078	0.33146232803701503	0.37806587990177754	0.4246259526064805	0.9787242453886702	0.7020066691765506	0.7753047792992384	0.8294316046900218	0.7228951310671874	0.3489345508814925	0.1379591461434525	0.1534574325645539	0.5300772493840933	0.6162666757050802	0.34062023459505486	0.2017400707032213	0.1999500505384827
36	0.7419481043774484	0.5464148853693653	0.10478855122871811	0.7639604137418342	0.9732579864832958	0.05316646240168732	0.5403518978456466	0.2578728711000122	0.8767699968359299	0.18947026141967138	0.29920601535331093	0.7206911553083747	0.1786469297061276	0.1907737938967099	0.0019539070336891395	0.5654380913479276	0.9699170721963147	0.8043848962708982	0.10457804377797941	0.736109888631213
37	0.4449179197724239	0.13198294695306834	0.49066470226080694	0.7530116772775907	0.8996104252489412	0.008339851154862243	0.34273005884348706	0.8742736274793491	0.7658088623391879	0.8419613778145986	0.8805748081867283	0.40510366906205664	0.2872017482772583	0.7989875029363456	0.765248060672041	0.6877417171352976	0.9126632298039942	0.8408465390544363	0.5156442437625631	0.3611210102972927
38	0.03587985655175008	0.49894715404934253	0.31786758741502785	0.33866274190031687	0.9067193964163401	0.26641050651372655	0.09636420422257896	0.16016622573001205	0.8372016019631228	0.5340565164178742	0.1886598332513566	0.08524656591466173	0.2945297639933536	0.77231941932841	0.41749883208906746	0.8406100480747389	0.03899481228268398	0.3609124695119277	0.503502935218866	0.18280201972786936
39	0.6027583262759932	0.204244194283975	0.08749040302685518	0.4634848638361224	0.41358355091675747	0.9042540892274422	0.023957723396251795	0.9019235436195239	0.17375779610367603	0.3662778722288129	0.3988397153890685	0.32715872323483397	0.04039767994999721	0.0682527758935384	0.8737259649240323	0.5911076076406825	0.6422920135269792	0.9541811146351693	0.49236361117104055	0.26062645266706086
40	0.9161443719759288	0.08708644972804569	0.446918216309956	0.23835949687604863	0.8091144242160989	0.6200676547641488	0.8237684072203317	0.28594064639693795	0.23702041869532076	0.3378951543708951	0.22052740303849572	0.887269356708196	0.5912688229688208	0.19386206804050288	0.9947875374933673	0.1492973665137891	0.8092175493213952	0.1508320901340784	0.03299871831336165	0.46170008179986954
41	0.39756304134460796	0.38949924231249367	0.5612454776851588	0.7554752080030772	0.43369022114068123	0.33665411871741835	0.775475899677045	0.597732170722272	0.43551358254810724	0.007927033709481002	0.5334482784344432	0.4036270994893342	0.8291377458098059	0.9685110237029599	0.14064673593487975	0.4995825965346936	0.8700082401762774	0.5192136118456453	0.8264197601233512	0.7520630960786994
42	0.8182836443637974	0.4069127609235439	0.13108989376676117	0.6675606000585693	0.9292355180880529	0.2953400728024228	0.8284586725098891	0.6921014684770554	0.7438701068559814	0.9437528720979849	0.6260304527223456	0.48893548410762655	0.6267694035928543	0.6409048662955376	0.2617723091551195	0.31315282511112985	0.3061005386071941	0.02260990574649524	0.37335472003416126	0.7248272538626606
43	0.9737031822918089	0.6240768880731392	0.07464609982905801	0.868344617732499	0.14652150938852404	0.7137660850505212	0.4253075523496045	0.2532894750195511	0.40532156853782275	0.6691163799349515	0.716416464761014	0.08793251157904014	0.7165537120683562	0.747764353142304	0.8468175373451263	0.29294423947176973	0.28686005758422395	0.2899449650486484	0.4800501697681805	0.18127100201194302
44	0.7380074412130034	0.6666061741838555	0.4811708069191707	0.3718268041358601	0.45129558250308377	0.24498960976590267	0.169048583039191	0.633469057650817	0.6871249434059226	0.15711330460037887	0.8760363005621545	0.9655095232455082	0.062327635530100234	0.6696626397332314	0.892515114286895	0.3355574300511257	0.8277681366436759	0.5244506026070468	0.7890162958726853	0.3651812620181666
45	0.5779501131976644	0.8105085942059874	0.6667284824116623	0.9551445640914836	0.0038720660882276547	0.016951993803560383	0.7023357100282578	0.06789953755185629	0.3634479881716781	0.9057342439955164	0.8152313106683842	0.6271071698569012	0.8122792875148843	0.7642262968179608	0.2594643768653644	0.9798429008585119	0.8212219568179951	0.7799864241166415	0.33080843064131404	0.13308881727965038
46	0.4846379625356688	0.9526297377719923	0.45831463345991996	0.1850525183993449	0.7221698923431605	0.568724194504923	0.5970758898923474	0.7563410422725698	0.5649980871636913	0.5616431285761508	0.8404942535150917	0.4381991966889667	0.27449695966276855	0.37782397894795516	0.5522450246116273	0.7803094401105836	0.5165515016243336	0.8882909114781925	0.017154861139300537	0.4175227008134901
47	0.8223234763011489	0.26229620043019575	0.941576025326022	0.42558896791265755	0.48075251182842094	0.9933941818593306	0.04683442156505635	0.673090207776233	0.1130235456021077	0.5332485234447188	0.5747781315869244	0.48282766524026544	0.878431086350019	0.6398713663853465	0.5685298169901897	0.2714510510179178	0.877626322726724	0.6537753547075881	0.9654207362074432	0.35383100321973227
48	0.21670859438845302	0.8899233293631648	0.7246252202051743	0.5804096072215141	0.37926986458996803	0.24013668097734453	0.1541393641824551	0.7198204058362108	0.6499983820175039	0.8706769556781017	0.13406336329259017	0.35140317195330706	0.14944181921672184	0.3205788185366055	0.9293778986039293	0.8461152382238873	0.9861936796806154	0.9269817332567596	0.506740636311365	0.8405342958856693
49	0.01716117410293505	0.0230040299490869	0.09596025756202275	0.17503334963570738	0.4878454564969703	0.2489359458555086	0.8005406162648367	0.39156916650498874	0.538729983943377	0.5396816491450722	0.32573904462943126	0.32016658117429864	0.8462874492810547	0.03864232786761346	0.8735310659388128	0.09104553669893267	0.3385814161984432	0.9551055825780187	0.4079821554649148	0.6423409578329223
50	0.5120444381175684	0.8085286137221855	0.49071992905161155	0.419731486213466	0.7997282365767096	0.6255594556554768	0.4006414039364352	0.1495280543840427	0.4340960516326329	0.9880310753586105	0.40654777253037144	0.6744226116961456	0.8712120254283799	0.6772878644477925	0.9582604511872183	0.11891180668546786	0.07600900456986481	0.6794564464662218	0.7631541197522429	0.19267280971294976
51	0.6522683814276826	0.7271957547873726	0.4539231869221738	0.1121582055333118	0.08744512711125108	0.15491909370852286	0.5657958315187026	0.06416992461599935	0.7845036679643731	0.8566908984959167	0.12947136408290516	0.5475495239176731	0.6697135042521729	0.40228998053932274	0.17362046111098606	0.9999831588151833	0.039299674760361336	0.25168225356610274	0.1087462353957861	0.12017160196531596
52	0.2459351588385208	0.511807663757042	0.5686543217323504	0.905965418050743	0.2491261523047631	0.9253987063941123	0.9008032393232656	0.9347387141202079	0.31838277326803566	0.9133769430576334	0.41835594721166436	0.9591389915525982	0.44330880111794513	0.3383909727497223	0.5689252762549545	0.6554435175827393	0.005893918414282218	0.8795702156372799	0.6948645976939926	0.1414317471565556
53	0.9997331761344462	0.6198178607425685	0.18470838149925395	0.529464049513948	0.9600790499428878	0.3159933429860443	0.8140379127239816	0.17888648204334234	0.2617016017289472	0.09684082093118751	0.32419381485790066	0.31752585128778366	0.373368433434218	0.7984545223804898	0.27346155218632373	0.7471543604782979	0.4063016559605037	0.15205156490921867	0.8198687348173522	0.41085721600485337
54	0.7973239242868995	0.4480682742657688	0.784578269182037	0.4708222118535349	0.08512882311134817	0.7086576393254655	0.8784233107310712	0.500342353357634	0.9315584254959988	0.7039231180809215	0.4228801004093622	0.917737489462148	0.5324539065495639	0.20836293547061058	0.9991466653437738	0.1782047126407177	0.3998296742570868	0.6945486354329496	0.9152435827556086	0.8276063606621294
55	0.5757357424429469	0.03061511610856882	0.3380633011233757	0.31603421188692804	0.6129640818351084	0.07742167942253553	0.21442359550339862	0.31137299925558304	0.6757065703444058	0.494730312818889	0.33514117237103347	0.4997137553559087	0.7385589235557543	0.3241404689243079	0.7272213128210818	0.2543177304450214	0.5502072484042488	0.8986529950870038	0.9676823439671159	0.13184684006121394
56	0.006310124097108183	0.8678540046828439	0.3362175441869475	0.09806330340377323	0.6187104564498104	0.5038757123659853	0.9063021358445668	0.7773103271939429	0.6497865110428772	0.7019986215511255	0.19175128118264928	0.3206984307889207	0.6663105128383909	0.5649673383330337	0.767601783375747	0.3140581472272409	0.04707354253444518	0.3176743463600733	0.7398715202896172	0.5531232044411556
57	0.26550678635795255	0.2580010813412619	0.4922661326667014	0.07089671480658499	0.7627349600309823	0.5967769707912983	0.16401588847415394	0.0674356598346505	0.7515954573305128	0.802495619886949	0.013691807064267203	0.8503635416465095	0.5144823629796218	0.32445723753369915	0.15886656421067968	0.6936766167883636	0.8067010537855812	0.5519955770476025	0.7143337853587045	0.16395199434029895
58	0.23376477566209453	0.8541462966041894	0.41336806293093264	0.09056520441508131	0.1417154792652331	0.9608095724423654	0.21739864220037364	0.9240509958710844	0.4491033632700755	0.8100353999806463	0.1618983423788204	0.25770709606580844	0.04336743989916925	0.09153510975097012	0.3206067529026654	0.769811934592043	0.05873885019254277	0.06466512917965106	0.9991802072899102	0.4801491742945285
59	0.9013917916403374	0.7779855164666639	0.23817856890141353	0.41259199183463924	0.8877663804554704	0.905644911811142	0.609517386226013	0.06668560659042555	0.21203678629658174	0.7813521282558131	0.5767250643296705	0.9326802616252111	0.9256258235010498	0.34996600371457265	0.1106495419550787	0.7031112091896989	0.46398449993272994	0.20180945234363845	0.41304664039608363	0.4765060134285598
60	0.9766253607897473	0.7862337474140335	0.9217116502729986	0.8578999017984307	0.5114933171363636	0.9464451402632166	0.8418250004074066	0.8283987045367684	0.156825160786775	0.9176290007333568	0.04710829362748281	0.45599474642961124	0.544216366675112	0.6904710781492776	0.2250796677766852	0.303288187444127	0.680590156826393	0.6422713709792284	0.7907905682125986	0.5943153791526478
61	0.3977845465117693	0.4119907357000674	0.07217729908351012	0.251108582577238	0.23671459854406462	0.5285176691230729	0.05225328383442995	0.04997357480247677	0.3557648548726029	0.6465976184654627	0.8930142162103513	0.5671082207137215	0.23982416858506994	0.7101791449313607	0.8716018297910919	0.7091164026836146	0.3753948774748378	0.19733527327249867	0.33626725334296315	0.25350262892063835
62	0.4241115062909603	0.7954567044975994	0.661187789580807	0.8638772378803462	0.2449096722954437	0.28722313037337854	0.6288458429451831	0.7506651448211111	0.2598711396092893	0.6735325933668421	0.6345778277472749	0.8344098512816338	0.5370964452821466	0.17768746741770025	0.6251663021112626	0.7051291815526352	0.6984355567079811	0.5468646735215269	0.4020530856913169	0.9397945488697867
63	0.23853596277986444	0.8382328646044527	0.38119431542780813	0.8130352847495486	0.30899724891818137	0.1598278485813971	0.8384119543516412	0.5648471810117568	0.2649962048723695	0.7367325876158497	0.48268552195051717	0.6132887031264043	0.8302420234233127	0.33226832724127764	0.6152585226943201	0.7570766616898505	0.6221413000333939	0.12178359139329598	0.7934279312276081	0.9426796192232688
64	0.44606445425316255	0.8300369617776752	0.22401890657375634	0.5271139387183006	0.33221861560464916	0.36804538169834433	0.7936632171288115	0.21763703668536627	0.8155951004685661	0.2686640585782031	0.6477278589403904	0.7215349989158036	0.7612283254573243	0.2658502546739241	0.7786554920068796	0.7061400207183932	0.3049687579301422	0.9670769025749686	0.8462377894512283	0.37879192461970523
65	0.700546634543613	0.9278329694989534	0.47412629488253266	0.4020057303710247	0.18787934214112145	0.6909219401037797	0.7772355205497162	0.7058384318410866	0.7464433500491638	0.6674068242649127	0.48647341018122425	0.38136684942104915	0.7727110315136795	0.5920242418984631	0.26073315410259956	0.37052048714537333	0.7819422826558284	0.4058749577570391	0.5509094743726707	0.8942054100074796
66	0.01869665640298057	0.18109620406922755	0.7551111254368962	0.8074770935824025	0.3809991864554362	0.018016901595707013	0.24826040610446343	0.020493510348584043	0.14794146001450303	0.04429396775747363	0.9853741349253279	0.2102756590419772	0.25791929708163663	0.7333381520943312	0.4619180233811435	0.9326873784021985	0.15456266593139112	0.21815465887451257	0.6024838410958	0.28522661262277726
67	0.9041018765673086	0.718615803143289	0.8938849365764725	0.09067271203153693	0.13526667736316822	0.9895537061120732	0.4468356622132983	0.08990594778920757	0.0720252515936739	0.8902400831009473	0.8591973832753385	0.43031709370313986	0.09355155038259921	0.8133127646175222	0.7539349249545367	0.4315279001896659	0.3625248220313301	0.9244972594488006	0.5892249349111193	0.6485775402155215
68	0.6067288656869094	0.9835732505651372	0.6323615066160806	0.6217495471704892	0.6803222815483426	0.9822849491192323	0.15986857761550133	0.5086599201076802	0.6785615267593044	0.2001985099332909	0.9334900154368078	0.305549004024663	0.7316006028972483	0.9995786429467625	0.029919382088194668	0.5658116178961734	0.5225149032922077	0.8477011052447088	0.05181491132664984	0.4541604051895779
69	0.017554381885381498	0.9344192855508778	0.8044008907391932	0.5240101293514041	0.6396887164135312	0.7983914412174692	0.27799437595983356	0.951857987091272	0.0909958061916224	0.909160777434857	0.7314617340137003	0.7327105475190747	0.4298102473517197	0.7613531947340779	0.19807143883412048	0.8917689418582014	0.6422220160848441	0.5236143096812881	0.9743559422500704	0.5278365307180093
70	0.2749888818696057	0.6276362823616598	0.9870734684376657	0.496728584429995	0.8721871086658856	0.2913786807915776	0.3485777229172812	0.6916619401922053	0.36351886411991974	0.3215328730981175	0.7895257605732074	0.37789070337365616	0.3178308121268797	0.6069190939505679	0.3119820031679592	0.09375775247658014	0.8762981233833297	0.057181748258230636	0.4645325189880135	0.9196370644901942
71	0.5236702245522357	0.9007842001009312	0.3233880196718365	0.41559684165845723	0.18371195262419737	0.9642271736408659	0.5575285938294413	0.925149478444962	0.9225946432125145	0.3076289706701021	0.9376060833310949	0.8787936717897505	0.16669279065142795	0.3871084233923272	0.14958838065547964	0.331275719961338	0.12468470027521295	0.3983842116829611	0.1827656589626927	0.9919749688316316
72	0.39643265342549205	0.20076831054308253	0.24256915115517697	0.4869431947450866	0.27839495379040324	0.6348235124453082	0.07650739205415158	0.17104630085349448	0.13582796032473898	0.5512130507148107	0.8510387665200174	0.3172213606659512	0.3972382068346684	0.5553095297944817	0.800654435939996	0.24228584041686407	0.7798826013879877	0.6899486493704793	0.9146881703194578	0.6025220597751698
73	0.08087176428968657	0.9458761579266945	0.9848349638762753	0.08014826429051713	0.28559983147675116	0.4002435532237101	0.007464902222456549	0.18147870091124174	0.476409837250966	0.1522525948124862	0.4769494964061489	0.8828520888889894	0.6754353933607239	0.03666137533287894	0.08122753168425045	0.30313844916363375	0.48522174640618587	0.6442113341104734	0.7241874487370813	0.07167881053003466
74	0.015129792967800704	0.02098307177283054	0.6138635808947814	0.8272016896948566	0.5607194166466742	0.07751419639707247	0.01325964918774658	0.11359907092582411	0.4484346325920934	0.7791877883037618	0.14810686523183114	0.9445030280298191	0.7105062240367345	0.49237519350831005	0.29509118276222157	0.5942196810309537	0.22818700055807561	0.07503381776115259	0.13934124108209955	0.19453001683904503
75	0.7000780066321787	0.8367129192727312	0.6998901099555656	0.3648384877017212	0.9453857339598045	0.4269851692833182	0.03792781800666678	0.19000620938967516	0.2699378794612467	0.1425910075675857	0.04829745222255688	0.4905703318385052	0.3033290237533336	0.2043440770077013	0.7191537387374333	0.7900136105731954	0.5908534155370169	0.6913140803538851	0.5911774460707087	0.9863872194598333
76	0.08190329683941111	0.09054679872875271	0.17727968374282332	0.6249238723311823	0.6974182968077393	0.2086742969307629	0.934143106565787	0.4908539358756694	0.6271906924341936	0.3028967391303238	0.24870439345887818	0.5298841838014025	0.19333562488188794	0.8488035585484659	0.6751892305696924	0.8428595024190206	0.7579267286970359	0.5996415232856638	0.3044662365448473	0.3905152194739152
77	0.6776424745633244	0.29696043649832893	0.4560154132068325	0.9910375868314591	0.4360201575376528	0.6726292752039874	0.09267637831380715	0.007097730405700942	0.9234997429413871	0.9619225527962498	0.4796335064912517	0.4487176066087134	0.31735675885731873	0.672870231408215	0.14198624504520552	0.5052276531366257	0.7572447836042745	0.40208047085526333	0.8099013240504238	0.5287269745798895
78	0.44225114598661786	0.5279461103126765	0.7399849654388475	0.5806482834426003	0.1627057923284143	0.03686185961070487	0.7654234002533338	0.4872548099176208	0.5613931446359499	0.432122575214024	0.8255308095948689	0.6274708770711199	0.6298007327365552	0.1578785941645765	0.699538635866396	0.6911023647863886	0.9962420906245593	0.7217002763695485	0.8589945234297267	0.6719906437738582
79	0.17697372383670062	0.8674590553439081	0.529425250587824	0.6162039735466626	0.6863352812490704	0.9744536450121255	0.3550609487702512	0.8098815274942647	0.023322883361294955	0.3733664706811466	0.27373324285932377	0.4582302070371209	0.7729949988352237	0.7682750375837328	0.06115465016123112	0.0406841973175982	0.8678949971380445	0.2494161199289765	0.6085474526946957	0.5881168703624854
80	0.011611729980218422	0.08207536628007184	0.9210262023913344	0.41473514784133825	0.9268955491404234	0.2533786826106943	0.5107043628304399	0.06954983261522307	0.25086738839599587	0.4616853016318333	0.6865341028325397	0.6480849921158698	0.5323796601248852	0.4549192307387334	0.3490263849749601	0.20674392955339604	0.8741368664870762	0.31094333175195976	0.19415824327306963	0.3424342306839766
81	0.7704519696990781	0.9902344569015695	0.8522428617414849	0.5207331508971594	0.8630175818005498	0.4749014303322069	0.9124997005480382	0.8642094679790472	0.012287382945524561	0.2838486897074407	0.021034873312360647	0.5984910933823088	0.9491274016754198	0.17148712150686118	0.4610119450090343	0.6751568024727438	0.18637974786347844	0.04186091275501458	0.2719560862654691	0.5339287224293189
82	0.3298994392134892	0.16204544674385069	0.7262033619238779	0.7059088213010583	0.3490706438667772	0.7164517602980391	0.21029018667003796	0.9366771473866903	0.9840322261714243	0.1391040919843476	0.7102692941527611	0.5621597436960307	0.07062429286023952	0.18644307477619093	0.6116230472260179	0.8423959044936289	0.5445894168269256	0.7926987626269801	0.3101819750945447	0.219071044421289
83	0.5892827608478225	0.5193045625758179	0.38763246573267707	0.4976369423866608	0.31162173019718153	0.8380539330742309	0.25728762147573137	0.10117818664739309	0.05036011967067988	0.6748362076194759	0.37151036285316097	0.8289878292223917	0.48919849965959394	0.5994087256820143	0.14498208054775896	0.3579291403427951	0.614141943551914	0.1628841887387542	0.4235486668588595	0.5808539734664543
84	0.3813436985886851	0.7133496651982354	0.43935738769662946	0.5136188184190625	0.2985987325654794	0.7412459002743745	0.17948962509188526	0.5344322609393534	0.3483681014504064	0.29237494571971845	0.7574000254523414	0.6433951747913288	0.3737422386153403	0.09417276333945301	0.7194308055484093	0.3737409045651702	0.8713359636933236	0.37089162387098396	0.17950609617099844	0.5956660824503904
85	0.7346945424952972	0.6450799231285262	0.6591837739911214	0.7755546359274778	0.18279895031153537	0.8042181196236557	0.6675651017905985	0.528543412795393	0.9547155255504695	0.23760620504286323	0.41350188959559997	0.5012026410688379	0.7755587741368869	0.5138036589703245	0.46932298848089327	0.13426960176890146	0.42122215944165486	0.07571276918338932	0.17231516630337906	0.5660135459113731
86	0.29378528980212426	0.6446739465699235	0.24371487956580418	0.6310566529056564	0.7815924241338255	0.8353672150017016	0.3441403455719705	0.6870038492785833	0.6497276229746867	0.6126132843697685	0.5405246230620818	0.5027367126495569	0.1944487754713099	0.18394295643986558	0.7231921873414605	0.3172065509048766	0.6628114720746744	0.7566342193104999	0.016759300374374786	0.7611756444675928
87	0.81304599931994	0.9940284031373012	0.7113187693004773	0.10298077890185975	0.9425259777754139	0.37353373495296094	0.29101261620110574	0.7911160805662795	0.7964461695297924	0.028794253620687105	0.0986954306296225	0.6720820497719819	0.5136062689645471	0.4450536494530254	0.8677185800279847	0.6024463322018272	0.8635968189280063	0.7398483356957032	0.693684396589267	0.829936232436218
88	0.0728623127327156	0.028708132253140284	0.0708502545944455	0.21643458446195785	0.5427023425542618	0.8987205959003289	0.1018827371005061	0.5129420960587476	0.40850152861694167	0.8766878555981447	0.2925573661738671	0.449571992270562	0.18092141659938488	0.3995811890273674	0.6082989682731488	0.24513594729008092	0.8241413899087651	0.9039586303548335	0.2741221165182186	0.24813216487454937
89	0.19851246517162602	0.3972234056776941	0.8818102847988184	0.2303656296485147	0.2569915948646504	0.06266326534728595	0.9542418809607445	0.1974132295469394	0.7048934190656649	0.8350032817591528	0.22380542146143068	0.7941212491366109	0.7114393602229974	0.6879671404096384	0.5587210462634503	0.45907585462920963	0.6152795131473064	0.9607139268903352	0.029096153306568673	0.0013640584916936316
90	0.7940854283160026	0.47887830466349945	0.8693309543747946	0.41196216592170154	0.4729843410191823	0.04678697161886569	0.7033867937560432	0.9503759823382367	0.9324154069439409	0.9651687314863564	0.38428311075528176	0.8329894632555254	0.3995092777154897	0.21052607779475185	0.9026332736105159	0.19299537678768808	0.9498780275456365	0.07999277869396915	0.9654485282974876	0.8462958305605167
91	0.24517806810298537	0.826243190141097	0.9957467472000759	0.5453391954594949	0.3223586185987155	0.5277324671104756	0.7818037212552453	0.8112936415796143	0.39373903333028226	0.6768008265380066	0.2132743995800641	0.11475920684378815	0.5556241366580877	0.8015372714771155	0.08238491168343731	0.5102626707889729	0.5435782239249368	0.6336618798683767	0.2691689732207374	0.3621227878583483
92	0.02236861866273665	0.5436064387247257	0.5722933278233278	0.5493993228574683	0.6162620724084704	0.4288889285981069	0.04471584520977989	0.08763937690412316	0.9206370439880247	0.3055508100770692	0.2320083396828312	0.717554423931977	0.19798116115491138	0.791467489869348	0.6747669852653363	0.3328598239693862	0.5970057957114228	0.34317023239418243	0.6551503517770585	0.6042211294847113
93	0.09912836289345539	0.27012455450787287	0.5611192103501427	0.49893004414349085	0.39826061607109164	0.9723573036616572	0.43433574618858417	0.01726087117793451	0.014724006131290768	0.0742115692183406	0.3325299699935661	0.05256345043384436	0.14905910719506954	0.37413042119198237	0.5742653910903226	0.35018287471874676	0.9188355212894345	0.917021690741368	0.7142273766718106	0.7744688404247526
94	0.4911906035709752	0.3294335686948545	0.6701515477365343	0.5287022726275609	0.20225037346063268	0.5157765425235628	0.6886445054946418	0.27315536440306887	0.9468303860308537	0.15052645712412027	0.7319303674300504	0.8373341280834737	0.9443063985732353	0.7809226723517547	0.5818944273022325	0.4961031669131183	0.8815683918922314	0.5239233979403347	0.5654692730035298	0.13603614007985154
95	0.446550919478505	0.6794794638043248	0.46794948867977404	0.6226828556484018	0.9864686884774462	0.5960751479294906	0.003966283687273919	0.20320652210383128	0.230556647818416	0.1051013053783133	0.27626187561630333	0.038161064265256006	0.3934227404671591	0.2470546969831896	0.44799375656189633	0.2918371679974916	0.8129034750136948	0.2343881167688635	0.09294582205266688	0.2739243192674218
96	0.4802453939440505	0.9457082991104431	0.3322288974388069	0.367196287237064	0.9545079310706601	0.442821130596707	0.12082255943311115	0.08911679343053691	0.9189695348050696	0.2677785427594942	0.5220650019814844	0.1581497449310555	0.9369194057914237	0.9325591365215669	0.006304368481577449	0.4594667297368017	0.6557210040290873	0.987699374833239	0.4680324958621931	0.5566350455760088
97	0.3551641188336926	0.4660105514883047	0.4338498474182587	0.6211336799625706	0.884193844002649	0.1723933479095282	0.03447676291747026	0.6489588865655753	0.9726271209859496	0.059075181976192104	0.022284402878888665	0.7664250142221117	0.7624504474399308	0.769097317913216	0.0008070791648451259	0.4973769063020004	0.8317013295336368	0.30923191270410455	0.23323607614748199	0.26919851225985447
98	0.35549028207271627	0.9233041900066532	0.6457950153344254	0.6516649788482108	0.7974812051479646	0.6676909917390083	0.8388491951907738	0.25095880968204765	0.46346615780275624	0.5919274367786479	0.9500948497316042	0.40774446053975366	0.7043459989245003	0.22343774429144803	0.022728601451654673	0.861562434654931	0.7303290026724144	0.4575375664787418	0.9414696791931231	0.18765774999734008
99	0.35075064466208694	0.07577942694717599	0.8778006563204881	0.6400607854756564	0.3708114130261989	0.6577264778834104	0.9459252952821685	0.6695281719118779	0.950526922448457	0.465303520259736	0.36795191358507184	0.01768108661633161	0.13465416860762258	0.4229888848006067	0.2018717841737725	0.5391985983893003	0.8390020628653354	0.8502697295224114	0.5623634884060347	0.1451841093098123
100	0.5928652419057733	0.9310823811916091	0.1420868794139939	0.5924701460124143	0.8673163995063917	0.9047868181360221	0.7212711492063613	0.21309667809064803	0.8864751582194511	0.44985390519538515	0.006387964700910187	0.39137879712552537	0.038452062250765695	0.23685824009344858	0.01105510068440907	0.7687960097547921	0.7234617508948908	0.7166994752939646	0.32249985229700795	0.7116625604553003

Enable monitoring:

vo.set_option("sql_on", True)
vo.set_option("cache", False)

Single query (all columns):

display(vdf.avg(ncols_block=20))

Split into batches:

display(vdf.avg(ncols_block=5))  # 4 queries of 5 columns each

Parallel execution:

vdf.avg(ncols_block=5, processes=4)  # 4 concurrent workers

Tip

Use ncols_block and processes to balance query load on shared databases.