Linear Regression

Linear regression is one of the most popular regression algorithms and produces good predictions for well-prepared data. Its optimization function computes coefficients to express a response column as a linear relationship of its predictors.

You must verify the Gauss-Markov assumptions when using linear regression algorithms:

• Linearity: the parameters we are estimating using the OLS method must be linear.

• Non-Collinearity: the regressors being calculated aren’t perfectly correlated with each other.

• Exogeneity: the regressors aren’t correlated with the error term.

• Homoscedasticity: no matter what the values of our regressors might be, the error of the variance is constant.

To create a good linear regression model, it’s important to:

• Impute missing values.

• Encode categorical features (linear regression only accepts numerical variables).

• Compute the correlation matrix to retrieve highly-correlated predictors.

• Decompose the data (optional).

• Normalize the data (optional, but recommended).

Example without decomposition

Let’s use the africa_education dataset to compute a linear regression model of students’ performance in school.

from vastorbit.datasets import load_africa_education

africa = load_africa_education()
africa = africa.select(
    [
        "(zralocp + zmalocp) / 2 AS student_score",
        "zraloct AS teacher_score",
        "XNUMYRS AS teacher_year_teaching",
        "numstu AS number_students_school",
        "PENGLISH AS english_at_home",
        "PTRAVEL AS travel_distance",
        "PTRAVEL2 AS means_of_travel",
        "PMOTHER AS m_education",
        "PFATHER AS f_education",
        "PLIGHT AS source_of_lighting",
        "PABSENT AS days_absent",
        "PREPEAT AS repeated_grades",
        "zpsit AS sitting_place",
        "PAGE AS age",
        "zpses AS socio_eco_statut",
        "country_long AS country",
    ],
)
africa.head(100)

	123 student_score Decimal(23,18) 99%	123 teacher_score Decimal(11,7) 89%	123 teacher_year_teaching Decimal(3,1) 99%	123 number_students_school Integer 100%	Abc english_at_home Varchar(16) 100%	Abc travel_distance Varchar(11) 100%	Abc means_of_travel Varchar(13) 100%	Abc m_education Varchar(34) 100%	Abc f_education Varchar(34) 99%	Abc source_of_lighting Varchar(12) 100%	123 days_absent Integer 100%	Abc repeated_grades Varchar(10) 100%	Abc sitting_place Varchar(27) 100%	123 age Integer 100%	123 socio_eco_statut Decimal(3,1) 99%	Abc country Varchar(12) 100%
1	508.4821854	812.3311897	2.0	15	NEVER	>0.5-1KM	WALK	I Do Not Have a Mother	I Do Not Know	ELECTRIC	0	NEVER	I have my own sitting place	13	10.0	Botswana
2	499.890896	740.6072335	2.0	18	NEVER	>0.5-1KM	WALK	Completed Some Primary	No School, No Adult Education	CANDLE	0	NEVER	I have my own sitting place	14	5.0	Botswana
3	512.3963035	781.8103573	20.0	23	SOMETIMES	>0.5-1KM	WALK	Completed All Secondary	Completed All Secondary	ELECTRIC	0	ONCE	I have my own sitting place	14	12.0	Botswana
4	638.0964246	767.5990947	2.0	23	SOMETIMES	>4KM-4.5KM	BUS/TRUCK/VAN	Completed Training After Secondary	Completed Training After Secondary	ELECTRIC	0	NEVER	I have my own sitting place	11	14.0	Botswana
5	476.61604135	740.6072335	20.0	19	NEVER	>3-3.5KM	WALK	Completed Some Secondary	Completed All Secondary	PARAFFIN/OIL	0	ONCE	I have my own sitting place	13	8.0	Botswana
6	363.50642795	796.6892631	3.0	15	NEVER	>1.5-2KM	WALK	Completed All Secondary	Some Training After Primary	PARAFFIN/OIL	0	ONCE	I have my own sitting place	12	9.0	Botswana
7	486.87185345	740.6072335	23.0	25	NEVER	>4KM-4.5KM	WALK	Completed All Primary	Completed All Secondary	CANDLE	0	ONCE	I have my own sitting place	14	7.0	Botswana
8	574.16180765	796.6892631	17.0	25	SOMETIMES	>0.5-1KM	WALK	Completed Some Secondary	Completed All Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	12.0	Botswana
9	464.89787835	703.2192138	17.0	23	NEVER	UP TO 0.5KM	WALK	Completed Some Primary	Completed Some Secondary	CANDLE	0	NEVER	I have my own sitting place	12	5.0	Botswana
10	518.8684453	767.5990947	22.0	24	SOMETIMES	>0.5-1KM	WALK	Completed Some Secondary	I Do Not Know	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	8.0	Botswana
11	473.3024924	812.3311897	18.0	25	SOMETIMES	>2-2.5KM	WALK	No School, Some Adult Education	No School, No Adult Education	PARAFFIN/OIL	2	ONCE	I have my own sitting place	15	3.0	Botswana
12	523.5419478	715.3321692	13.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed All Secondary	Completed University Degree	ELECTRIC	0	NEVER	I have my own sitting place	11	13.0	Botswana
13	438.3829052	727.8266349	25.0	25	NEVER	UP TO 0.5KM	WALK	I Do Not Know	I Do Not Have a Father	PARAFFIN/OIL	0	ONCE	I have my own sitting place	11	5.0	Botswana
14	431.55012475	781.8103573	2.0	24	SOMETIMES	>5KM	WALK	No School, No Adult Education	Completed Some Primary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	13	4.0	Botswana
15	468.7501092	740.6072335	9.0	25	NEVER	>0.5-1KM	WALK	Completed Some Secondary	Completed All Primary	ELECTRIC	0	ONCE	I have my own sitting place	12	10.0	Botswana
16	462.11156245	846.0948606	2.0	24	NEVER	>0.5-1KM	WALK	I Do Not Know	I Do Not Know	PARAFFIN/OIL	0	NEVER	I have my own sitting place	13	4.0	Botswana
17	625.32189785	796.6892631	9.0	25	NEVER	UP TO 0.5KM	WALK	I Do Not Know	Completed University Degree	ELECTRIC	0	NEVER	I have my own sitting place	13	13.0	Botswana
18	510.49975695	767.5990947	12.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed All Primary	Some Training After Primary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	8.0	Botswana
19	392.6647144	767.5990947	6.0	25	NEVER	>1-1.5KM	OTHER	I Do Not Know	I Do Not Know	CANDLE	0	NEVER	I have my own sitting place	12	3.0	Botswana
20	604.00500395	767.5990947	16.0	25	SOMETIMES	>4KM-4.5KM	WALK	I Do Not Have a Mother	I Do Not Have a Father	ELECTRIC	2	THREE PLUS	I have my own sitting place	12	10.0	Botswana
21	523.57632325	753.8647201	17.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed All Secondary	I Do Not Know	ELECTRIC	0	NEVER	I have my own sitting place	12	11.0	Botswana
22	408.61576405	753.8647201	20.0	24	NEVER	UP TO 0.5KM	WALK	No School, Some Adult Education	No School, Some Adult Education	PARAFFIN/OIL	0	TWICE	I have my own sitting place	15	6.0	Botswana
23	506.12832545	727.8266349	26.0	25	SOMETIMES	>2.5-3KM	WALK	Completed Some Primary	Completed All Primary	ELECTRIC	0	NEVER	I have my own sitting place	13	9.0	Botswana
24	612.54879395	812.3311897	2.0	23	SOMETIMES	UP TO 0.5KM	WALK	Completed All Primary	Completed All Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	7.0	Botswana
25	[null]	796.6892631	17.0	24	SOMETIMES	>1-1.5KM	WALK	No School, Some Adult Education	No School, Some Adult Education	CANDLE	0	NEVER	I have my own sitting place	13	5.0	Botswana
26	471.31550095	753.8647201	22.0	24	SOMETIMES	>3.5-4KM	BICYCLE	Completed Some Primary	Completed Some Primary	ELECTRIC	0	ONCE	I have my own sitting place	13	9.0	Botswana
27	501.8363334	753.8647201	7.0	24	SOMETIMES	UP TO 0.5KM	WALK	Completed Training After Secondary	I Do Not Know	ELECTRIC	0	NEVER	I have my own sitting place	12	13.0	Botswana
28	498.8611914	906.5642598	16.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed Some Secondary	Completed Training After Secondary	ELECTRIC	0	ONCE	I have my own sitting place	12	12.0	Botswana
29	423.1570224	846.0948606	7.0	24	SOMETIMES	>1-1.5KM	WALK	Completed All Secondary	I Do Not Know	ELECTRIC	0	NEVER	I have my own sitting place	12	12.0	Botswana
30	450.387201	812.3311897	10.0	24	NEVER	>0.5-1KM	WALK	Completed Some Primary	Completed All Primary	CANDLE	3	ONCE	I have my own sitting place	14	4.0	Botswana
31	519.147368	781.8103573	12.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed All Secondary	Completed All Secondary	ELECTRIC	0	NEVER	I have my own sitting place	14	12.0	Botswana
32	590.32581285	624.6280703	12.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed Some Secondary	Completed University Degree	CANDLE	0	NEVER	I have my own sitting place	12	9.0	Botswana
33	510.49975695	846.0948606	22.0	25	NEVER	>0.5-1KM	WALK	Completed Some Secondary	I Do Not Have a Father	CANDLE	0	NEVER	I have my own sitting place	14	7.0	Botswana
34	643.49873135	646.2787858	22.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed All Secondary	Completed All Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	12.0	Botswana
35	495.3200764	767.5990947	15.0	24	NEVER	>0.5-1KM	WALK	Completed Some Primary	No School, No Adult Education	ELECTRIC	3	NEVER	I have my own sitting place	13	8.0	Botswana
36	459.9863116	781.8103573	7.0	25	NEVER	UP TO 0.5KM	WALK	Completed Some Secondary	Completed All Primary	ELECTRIC	0	TWICE	I have my own sitting place	13	10.0	Botswana
37	487.99064305	740.6072335	13.0	25	SOMETIMES	>1-1.5KM	WALK	Completed Some Primary	Completed Some Primary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	5.0	Botswana
38	566.4337237	715.3321692	15.0	25	SOMETIMES	>1.5-2KM	WALK	I Do Not Have a Mother	I Do Not Know	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	4.0	Botswana
39	552.117438	691.3923912	3.0	25	SOMETIMES	>1.5-2KM	WALK	No School, Some Adult Education	No School, Some Adult Education	CANDLE	0	ONCE	I have my own sitting place	14	4.0	Botswana
40	468.7501092	691.3923912	15.0	25	SOMETIMES	UP TO 0.5KM	WALK	No School, No Adult Education	Some Training After Primary	ELECTRIC	0	THREE PLUS	I have my own sitting place	14	4.0	Botswana
41	538.25214075	781.8103573	10.0	24	SOMETIMES	>3-3.5KM	WALK	Completed All Primary	Completed All Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	7.0	Botswana
42	577.2088944	846.0948606	10.0	24	SOMETIMES	UP TO 0.5KM	WALK	Completed Training After Secondary	Completed All Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	13	9.0	Botswana
43	430.1331224	781.8103573	14.0	25	NEVER	>0.5-1KM	WALK	Completed All Primary	I Do Not Have a Father	ELECTRIC	0	ONCE	I have my own sitting place	13	9.0	Botswana
44	532.00740605	646.2787858	26.0	25	SOMETIMES	>1.5-2KM	WALK	Completed Some Primary	Completed Some Primary	ELECTRIC	0	NEVER	I have my own sitting place	11	10.0	Botswana
45	488.90656515	864.5981152	17.0	24	SOMETIMES	>0.5-1KM	WALK	Completed All Secondary	Completed Some Secondary	PARAFFIN/OIL	2	ONCE	I have my own sitting place	14	7.0	Botswana
46	504.18288805	884.6274115	18.0	24	SOMETIMES	>1.5-2KM	WALK	Completed Some Secondary	Some Training After Primary	PARAFFIN/OIL	2	NEVER	I have my own sitting place	12	7.0	Botswana
47	498.21906805	753.8647201	4.0	25	SOMETIMES	>5KM	CAR	Completed University Degree	Completed University Degree	ELECTRIC	0	NEVER	I have my own sitting place	12	14.0	Botswana
48	518.8684453	753.8647201	11.0	23	NEVER	>0.5-1KM	WALK	Completed Training After Secondary	Completed Training After Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	11	11.0	Botswana
49	536.69074345	753.8647201	20.0	24	NEVER	UP TO 0.5KM	WALK	Completed All Primary	Completed All Primary	ELECTRIC	0	NEVER	I have my own sitting place	12	10.0	Botswana
50	477.86961185	864.5981152	6.0	25	NEVER	UP TO 0.5KM	WALK	Completed All Primary	I Do Not Know	CANDLE	1	NEVER	I have my own sitting place	13	6.0	Botswana
51	569.53207295	864.5981152	8.0	25	SOMETIMES	>0.5-1KM	WALK	Completed Some Primary	Completed All Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	10.0	Botswana
52	534.32951535	679.7563239	17.0	23	NEVER	>0.5-1KM	WALK	Completed Some Secondary	Completed All Secondary	ELECTRIC	0	NEVER	I have my own sitting place	13	10.0	Botswana
53	428.54585685	767.5990947	4.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed Some Primary	Completed Some Primary	ELECTRIC	0	ONCE	I have my own sitting place	13	9.0	Botswana
54	574.3841869	679.7563239	5.0	23	SOMETIMES	UP TO 0.5KM	WALK	Completed All Primary	No School, No Adult Education	PARAFFIN/OIL	1	ONCE	I have my own sitting place	13	6.0	Botswana
55	588.42325725	679.7563239	19.0	25	ALL THE TIME	>0.5-1KM	CAR	Completed University Degree	Completed University Degree	ELECTRIC	1	ONCE	I have my own sitting place	12	13.0	Botswana
56	687.60827855	906.5642598	9.0	25	SOMETIMES	>5KM	CAR	I Do Not Know	Some Training After Primary	ELECTRIC	1	NEVER	I have my own sitting place	12	11.0	Botswana
57	713.8913859	958.6404302	20.0	23	SOMETIMES	>1.5-2KM	CAR	Completed Training After Secondary	Completed Training After Secondary	ELECTRIC	0	NEVER	I have my own sitting place	11	14.0	Botswana
58	685.5165495	812.3311897	24.0	24	SOMETIMES	>5KM	CAR	Completed Training After Secondary	Completed Training After Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	15.0	Botswana
59	730.6463148	767.5990947	1.0	25	SOMETIMES	>4KM-4.5KM	BUS/TRUCK/VAN	Completed University Degree	Completed University Degree	ELECTRIC	0	NEVER	I have my own sitting place	12	12.0	Kenya
60	655.9907931	781.8103573	27.0	25	SOMETIMES	>0.5-1KM	WALK	Completed All Secondary	Completed Training After Secondary	ELECTRIC	0	NEVER	I have my own sitting place	14	11.0	Kenya
61	678.03411965	767.5990947	2.0	24	SOMETIMES	UP TO 0.5KM	WALK	I Do Not Know	I Do Not Know	ELECTRIC	0	ONCE	I have my own sitting place	16	8.0	Kenya
62	664.3520506	846.0948606	22.0	23	SOMETIMES	UP TO 0.5KM	WALK	No School, No Adult Education	I Do Not Have a Father	PARAFFIN/OIL	0	NEVER	I have my own sitting place	16	3.0	Kenya
63	558.4746904	781.8103573	4.0	21	SOMETIMES	>0.5-1KM	WALK	Completed Some Secondary	Some Training After Primary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	4.0	Kenya
64	369.37215045	740.6072335	4.0	40	SOMETIMES	>1-1.5KM	WALK	No School, Some Adult Education	No School, Some Adult Education	NO LIGHTING	0	NEVER	I have my own sitting place	13	2.0	Kenya
65	672.3160803	[null]	3.0	32	SOMETIMES	UP TO 0.5KM	WALK	No School, Some Adult Education	I Do Not Have a Father	GAS	5	NEVER	I have my own sitting place	15	2.0	Kenya
66	659.65783435	703.2192138	3.0	42	SOMETIMES	>1-1.5KM	WALK	No School, Some Adult Education	Completed Some Primary	GAS	0	NEVER	I have my own sitting place	15	3.0	Kenya
67	649.74605845	781.8103573	18.0	23	ALL THE TIME	>3-3.5KM	WALK	Completed Some Secondary	Completed All Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	11	6.0	Kenya
68	639.2043353	828.7361371	27.0	23	SOMETIMES	>1.5-2KM	WALK	Completed All Primary	Some Training After Primary	ELECTRIC	1	ONCE	I have my own sitting place	14	7.0	Kenya
69	756.7965389	[null]	19.0	22	SOMETIMES	>1.5-2KM	WALK	Completed Some Primary	Completed Some Primary	PARAFFIN/OIL	1	NEVER	I have my own sitting place	11	4.0	Kenya
70	625.94792415	753.8647201	29.0	23	SOMETIMES	>1-1.5KM	WALK	Completed All Secondary	Completed All Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	7.0	Kenya
71	606.58769385	715.3321692	18.0	24	SOMETIMES	>0.5-1KM	WALK	Completed All Secondary	Completed All Secondary	PARAFFIN/OIL	4	ONCE	I have my own sitting place	11	8.0	Kenya
72	436.4143824	781.8103573	28.0	25	SOMETIMES	>2-2.5KM	WALK	Completed All Primary	Completed All Primary	PARAFFIN/OIL	1	ONCE	I have my own sitting place	13	5.0	Kenya
73	430.1331224	767.5990947	9.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed All Primary	Completed All Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	14	6.0	Kenya
74	612.8324285	740.6072335	1.0	25	SOMETIMES	>1-1.5KM	WALK	Completed All Primary	Completed Some Secondary	PARAFFIN/OIL	1	ONCE	I have my own sitting place	13	5.0	Kenya
75	556.7909405	781.8103573	19.0	25	NEVER	UP TO 0.5KM	WALK	Completed University Degree	Completed University Degree	FIRE	0	NEVER	I have my own sitting place	13	8.0	Kenya
76	801.8048367	796.6892631	4.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed Training After Secondary	Completed Training After Secondary	ELECTRIC	2	TWICE	I have my own sitting place	13	12.0	Kenya
77	643.926602	767.5990947	1.0	23	NEVER	>1-1.5KM	WALK	Some Training After Primary	I Do Not Know	PARAFFIN/OIL	0	ONCE	I have my own sitting place	13	5.0	Kenya
78	653.7518235	828.7361371	21.0	22	ALL THE TIME	>0.5-1KM	WALK	Completed All Primary	Completed All Primary	PARAFFIN/OIL	0	ONCE	I have my own sitting place	15	5.0	Kenya
79	545.70392675	668.4063893	21.0	21	SOMETIMES	>0.5-1KM	WALK	Completed All Primary	Completed Some Primary	ELECTRIC	0	NEVER	I have my own sitting place	14	7.0	Kenya
80	419.0689409	864.5981152	18.0	16	ALL THE TIME	>1-1.5KM	WALK	Completed All Primary	Completed All Secondary	FIRE	1	NEVER	I have my own sitting place	13	6.0	Kenya
81	544.4968754	864.5981152	6.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed All Primary	Completed All Primary	PARAFFIN/OIL	0	TWICE	I have my own sitting place	14	3.0	Kenya
82	697.6229267	781.8103573	7.0	21	SOMETIMES	UP TO 0.5KM	WALK	No School, No Adult Education	Completed All Primary	PARAFFIN/OIL	2	TWICE	I have my own sitting place	13	4.0	Kenya
83	713.8913859	781.8103573	1.0	21	SOMETIMES	>1-1.5KM	WALK	Completed Some Primary	I Do Not Know	PARAFFIN/OIL	3	NEVER	I have my own sitting place	14	3.0	Kenya
84	620.13842915	781.8103573	16.0	21	MOST OF THE TIME	UP TO 0.5KM	WALK	No School, No Adult Education	Completed Some Primary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	14	2.0	Kenya
85	653.37429785	796.6892631	2.0	25	SOMETIMES	>2-2.5KM	WALK	Completed Some Primary	Completed All Primary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	15	5.0	Kenya
86	545.70392675	767.5990947	20.0	20	SOMETIMES	>1-1.5KM	WALK	Completed Some Primary	Completed Some Primary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	13	3.0	Kenya
87	580.68518045	812.3311897	1.0	23	SOMETIMES	>2-2.5KM	WALK	Completed Some Primary	Completed Some Primary	PARAFFIN/OIL	0	ONCE	I have my own sitting place	14	3.0	Kenya
88	512.77417745	864.5981152	31.0	19	SOMETIMES	>1-1.5KM	WALK	Completed All Secondary	Completed All Secondary	PARAFFIN/OIL	0	ONCE	I have my own sitting place	12	6.0	Kenya
89	620.55908965	864.5981152	20.0	23	MOST OF THE TIME	>1-1.5KM	WALK	Completed All Secondary	Completed All Secondary	ELECTRIC	0	ONCE	I have my own sitting place	12	9.0	Kenya
90	537.2691123	812.3311897	15.0	23	NEVER	>1.5-2KM	WALK	Completed All Primary	Completed All Primary	PARAFFIN/OIL	2	NEVER	I have my own sitting place	14	4.0	Kenya
91	615.66339735	846.0948606	22.0	25	SOMETIMES	>0.5-1KM	WALK	Completed Some Secondary	Completed Some Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	13	9.0	Kenya
92	517.10655305	668.4063893	19.0	21	SOMETIMES	>0.5-1KM	WALK	Completed Some Primary	Completed Some Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	13	6.0	Kenya
93	488.9603572	828.7361371	14.0	24	SOMETIMES	>1-1.5KM	WALK	Completed Some Primary	I Do Not Have a Father	PARAFFIN/OIL	0	TWICE	I have my own sitting place	15	4.0	Kenya
94	681.81298295	884.6274115	10.0	22	SOMETIMES	>0.5-1KM	WALK	Completed All Secondary	Some Training After Primary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	11	7.0	Kenya
95	547.58700195	[null]	6.0	25	MOST OF THE TIME	>0.5-1KM	WALK	Completed Some Primary	Completed All Primary	PARAFFIN/OIL	0	ONCE	I have my own sitting place	15	5.0	Kenya
96	431.55012475	796.6892631	6.0	25	MOST OF THE TIME	>1-1.5KM	WALK	Completed All Primary	Some Training After Primary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	13	4.0	Kenya
97	463.9812291	828.7361371	1.0	25	MOST OF THE TIME	>3.5-4KM	BICYCLE	Completed Some Secondary	Completed Some Secondary	PARAFFIN/OIL	2	TWICE	I have my own sitting place	14	6.0	Kenya
98	495.59890395	740.6072335	6.0	25	SOMETIMES	UP TO 0.5KM	WALK	No School, No Adult Education	Completed Some Secondary	PARAFFIN/OIL	0	TWICE	I have my own sitting place	15	3.0	Kenya
99	553.94425435	781.8103573	25.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed Some Secondary	Completed Some Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	16	7.0	Kenya
100	557.51591795	846.0948606	15.0	21	NEVER	>2-2.5KM	WALK	Completed Some Secondary	Some Training After Primary	PARAFFIN/OIL	1	ONCE	I have my own sitting place	13	5.0	Kenya

First, let’s look for missing values.

africa.count_percent()

	123 student_score Decimal(23,18) 100%	123 teacher_score Decimal(11,7) 100%	123 teacher_year_teaching Decimal(3,1) 100%	123 number_students_school Integer 100%	Abc english_at_home Varchar(16) 100%	Abc travel_distance Varchar(11) 100%	Abc means_of_travel Varchar(13) 100%	Abc m_education Varchar(34) 100%	Abc f_education Varchar(34) 100%	Abc source_of_lighting Varchar(12) 100%	123 days_absent Integer 100%	Abc repeated_grades Varchar(10) 100%	Abc sitting_place Varchar(27) 100%	123 age Integer 100%	123 socio_eco_statut Decimal(3,1) 100%	Abc country Varchar(12) 100%
1	409.34074155	727.8266349	12.0	25	SOMETIMES	>2-2.5KM	WALK	No School, No Adult Education	I Do Not Know	CANDLE	0	TWICE	I have my own sitting place	13	4.0	South Africa
2	418.54468005	703.2192138	11.0	24	MOST OF THE TIME	>0.5-1KM	WALK	No School, Some Adult Education	Completed Some Secondary	ELECTRIC	1	ONCE	I have my own sitting place	15	10.0	South Africa
3	457.34268235	703.2192138	11.0	24	SOMETIMES	>0.5-1KM	WALK	Completed All Secondary	Completed Training After Secondary	ELECTRIC	5	NEVER	I have my own sitting place	12	12.0	South Africa
4	411.3052167	703.2192138	11.0	24	NEVER	>0.5-1KM	OTHER	I Do Not Know	No School, No Adult Education	ELECTRIC	0	THREE PLUS	I have my own sitting place	13	4.0	South Africa
5	430.619181	703.2192138	11.0	24	SOMETIMES	>0.5-1KM	WALK	Completed All Secondary	I Do Not Have a Father	ELECTRIC	5	ONCE	I have my own sitting place	13	11.0	South Africa
6	482.43679495	703.2192138	11.0	24	SOMETIMES	>0.5-1KM	WALK	I Do Not Have a Mother	Completed All Secondary	ELECTRIC	1	NEVER	I have my own sitting place	12	11.0	South Africa
7	384.113366	679.7563239	15.0	22	SOMETIMES	>4.5-5KM	WALK	Completed All Secondary	I Do Not Have a Father	ELECTRIC	2	THREE PLUS	I have my own sitting place	14	11.0	South Africa
8	519.13142985	679.7563239	15.0	22	NEVER	>2-2.5KM	WALK	Completed All Secondary	I Do Not Know	CANDLE	4	NEVER	I have my own sitting place	12	10.0	South Africa
9	439.6800531	679.7563239	15.0	22	NEVER	>2-2.5KM	WALK	Completed All Secondary	I Do Not Have a Father	CANDLE	2	NEVER	I have my own sitting place	12	11.0	South Africa
10	385.53865385	679.7563239	15.0	22	ALL THE TIME	>0.5-1KM	WALK	I Do Not Know	I Do Not Know	GAS	5	ONCE	I have my own sitting place	12	5.0	South Africa
11	436.8841869	679.7563239	15.0	22	NEVER	UP TO 0.5KM	WALK	Completed Some University	I Do Not Know	ELECTRIC	2	NEVER	I have my own sitting place	12	13.0	South Africa
12	656.75362455	864.5981152	14.0	18	SOMETIMES	>5KM	CAR	Completed All Secondary	Completed Some Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	12.0	South Africa
13	643.9412439	864.5981152	14.0	18	SOMETIMES	>1-1.5KM	WALK	Completed Some University	Completed All Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	14.0	South Africa
14	644.39741815	864.5981152	14.0	18	SOMETIMES	>3.5-4KM	CAR	Completed All Secondary	Completed All Primary	ELECTRIC	0	NEVER	I have my own sitting place	13	11.0	South Africa
15	687.60827855	864.5981152	14.0	18	SOMETIMES	>0.5-1KM	CAR	Completed Some Secondary	Completed All Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	13.0	South Africa
16	618.89140445	864.5981152	14.0	18	SOMETIMES	>0.5-1KM	CAR	Completed University Degree	Completed Some Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	12.0	South Africa
17	436.4143824	715.3321692	17.0	24	SOMETIMES	>0.5-1KM	WALK	Completed Some Primary	I Do Not Know	ELECTRIC	0	ONCE	I have my own sitting place	13	10.0	South Africa
18	528.06517375	715.3321692	17.0	24	SOMETIMES	>3-3.5KM	WALK	Completed Some Secondary	Completed Some Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	9.0	South Africa
19	535.91289125	715.3321692	17.0	24	SOMETIMES	>3-3.5KM	WALK	I Do Not Know	I Do Not Know	PARAFFIN/OIL	0	ONCE	I have my own sitting place	13	8.0	South Africa
20	507.1368738	884.6274115	12.0	21	SOMETIMES	UP TO 0.5KM	WALK	Completed Some Primary	Completed Some Primary	ELECTRIC	0	NEVER	I have my own sitting place	12	9.0	South Africa

We’ll simply drop the missing values to avoid adding bias to the data.

africa.dropna()

	123 student_score Decimal(23,18) 100%	123 teacher_score Decimal(11,7) 100%	123 teacher_year_teaching Decimal(3,1) 100%	123 number_students_school Integer 100%	Abc english_at_home Varchar(16) 100%	Abc travel_distance Varchar(11) 100%	Abc means_of_travel Varchar(13) 100%	Abc m_education Varchar(34) 100%	Abc f_education Varchar(34) 100%	Abc source_of_lighting Varchar(12) 100%	123 days_absent Integer 100%	Abc repeated_grades Varchar(10) 100%	Abc sitting_place Varchar(27) 100%	123 age Integer 100%	123 socio_eco_statut Decimal(3,1) 100%	Abc country Varchar(12) 100%
1	409.34074155	727.8266349	12.0	25	SOMETIMES	>2-2.5KM	WALK	No School, No Adult Education	I Do Not Know	CANDLE	0	TWICE	I have my own sitting place	13	4.0	South Africa
2	418.54468005	703.2192138	11.0	24	MOST OF THE TIME	>0.5-1KM	WALK	No School, Some Adult Education	Completed Some Secondary	ELECTRIC	1	ONCE	I have my own sitting place	15	10.0	South Africa
3	457.34268235	703.2192138	11.0	24	SOMETIMES	>0.5-1KM	WALK	Completed All Secondary	Completed Training After Secondary	ELECTRIC	5	NEVER	I have my own sitting place	12	12.0	South Africa
4	411.3052167	703.2192138	11.0	24	NEVER	>0.5-1KM	OTHER	I Do Not Know	No School, No Adult Education	ELECTRIC	0	THREE PLUS	I have my own sitting place	13	4.0	South Africa
5	430.619181	703.2192138	11.0	24	SOMETIMES	>0.5-1KM	WALK	Completed All Secondary	I Do Not Have a Father	ELECTRIC	5	ONCE	I have my own sitting place	13	11.0	South Africa
6	482.43679495	703.2192138	11.0	24	SOMETIMES	>0.5-1KM	WALK	I Do Not Have a Mother	Completed All Secondary	ELECTRIC	1	NEVER	I have my own sitting place	12	11.0	South Africa
7	384.113366	679.7563239	15.0	22	SOMETIMES	>4.5-5KM	WALK	Completed All Secondary	I Do Not Have a Father	ELECTRIC	2	THREE PLUS	I have my own sitting place	14	11.0	South Africa
8	519.13142985	679.7563239	15.0	22	NEVER	>2-2.5KM	WALK	Completed All Secondary	I Do Not Know	CANDLE	4	NEVER	I have my own sitting place	12	10.0	South Africa
9	439.6800531	679.7563239	15.0	22	NEVER	>2-2.5KM	WALK	Completed All Secondary	I Do Not Have a Father	CANDLE	2	NEVER	I have my own sitting place	12	11.0	South Africa
10	385.53865385	679.7563239	15.0	22	ALL THE TIME	>0.5-1KM	WALK	I Do Not Know	I Do Not Know	GAS	5	ONCE	I have my own sitting place	12	5.0	South Africa
11	436.8841869	679.7563239	15.0	22	NEVER	UP TO 0.5KM	WALK	Completed Some University	I Do Not Know	ELECTRIC	2	NEVER	I have my own sitting place	12	13.0	South Africa
12	656.75362455	864.5981152	14.0	18	SOMETIMES	>5KM	CAR	Completed All Secondary	Completed Some Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	12.0	South Africa
13	643.9412439	864.5981152	14.0	18	SOMETIMES	>1-1.5KM	WALK	Completed Some University	Completed All Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	14.0	South Africa
14	644.39741815	864.5981152	14.0	18	SOMETIMES	>3.5-4KM	CAR	Completed All Secondary	Completed All Primary	ELECTRIC	0	NEVER	I have my own sitting place	13	11.0	South Africa
15	687.60827855	864.5981152	14.0	18	SOMETIMES	>0.5-1KM	CAR	Completed Some Secondary	Completed All Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	13.0	South Africa
16	618.89140445	864.5981152	14.0	18	SOMETIMES	>0.5-1KM	CAR	Completed University Degree	Completed Some Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	12.0	South Africa
17	436.4143824	715.3321692	17.0	24	SOMETIMES	>0.5-1KM	WALK	Completed Some Primary	I Do Not Know	ELECTRIC	0	ONCE	I have my own sitting place	13	10.0	South Africa
18	528.06517375	715.3321692	17.0	24	SOMETIMES	>3-3.5KM	WALK	Completed Some Secondary	Completed Some Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	9.0	South Africa
19	535.91289125	715.3321692	17.0	24	SOMETIMES	>3-3.5KM	WALK	I Do Not Know	I Do Not Know	PARAFFIN/OIL	0	ONCE	I have my own sitting place	13	8.0	South Africa
20	507.1368738	884.6274115	12.0	21	SOMETIMES	UP TO 0.5KM	WALK	Completed Some Primary	Completed Some Primary	ELECTRIC	0	NEVER	I have my own sitting place	12	9.0	South Africa

We need to encode the categorical columns to dummies to retain linearity.

africa.one_hot_encode(max_cardinality = 20)

	123 student_score Decimal(23,18) 100%	123 teacher_score Decimal(11,7) 100%	123 teacher_year_teaching Decimal(3,1) 100%	123 number_students_school Integer 100%	Abc english_at_home Varchar(16) 100%	Abc travel_distance Varchar(11) 100%	Abc means_of_travel Varchar(13) 100%	Abc m_education Varchar(34) 100%	Abc f_education Varchar(34) 100%	Abc source_of_lighting Varchar(12) 100%	123 days_absent Integer 100%	Abc repeated_grades Varchar(10) 100%	Abc sitting_place Varchar(27) 100%	123 age Integer 100%	123 socio_eco_statut Decimal(3,1) 100%	Abc country Varchar(12) 100%	123 english_at_home_ALL_THE_TIME Bool 100%	123 english_at_home_MOST_OF_THE_TIME Bool 100%	123 english_at_home_NEVER Bool 100%	travel_distance_>0.5-1KM	travel_distance_>1-1.5KM	travel_distance_>1.5-2KM	travel_distance_>2-2.5KM	travel_distance_>2.5-3KM	travel_distance_>3-3.5KM	...	123 socio_eco_statut_3.0 Bool 100%	123 socio_eco_statut_4.0 Bool 100%	123 socio_eco_statut_5.0 Bool 100%	123 socio_eco_statut_6.0 Bool 100%	123 socio_eco_statut_7.0 Bool 100%	123 socio_eco_statut_8.0 Bool 100%	123 socio_eco_statut_9.0 Bool 100%	123 socio_eco_statut_10.0 Bool 100%	123 socio_eco_statut_11.0 Bool 100%	123 socio_eco_statut_12.0 Bool 100%	123 socio_eco_statut_13.0 Bool 100%	123 socio_eco_statut_14.0 Bool 100%	123 country_Botswana Bool 100%	123 country_Kenya Bool 100%	123 country_Lesotho Bool 100%	123 country_Malawi Bool 100%	123 country_Mozambique Bool 100%	123 country_Namibia Bool 100%	123 country_Seychelles Bool 100%	123 country_South_Africa Bool 100%	123 country_Swaziland Bool 100%	123 country_Tanzania Bool 100%	123 country_Uganda Bool 100%	123 country_Zambia Bool 100%	123 country_Zanzibar Bool 100%
1	618.89140445	864.5981152	9.0	25	SOMETIMES	>1.5-2KM	WALK	Completed Training After Secondary	Completed Training After Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	14.0	Botswana	0	0	0	0	0	1	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0
2	480.2333067	864.5981152	9.0	25	SOMETIMES	>3-3.5KM	WALK	Completed Some Primary	I Do Not Know	ELECTRIC	0	ONCE	I have my own sitting place	12	10.0	Botswana	0	0	0	0	0	0	0	0	1	...	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
3	445.80907615	864.5981152	9.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed Some Primary	No School, Some Adult Education	PARAFFIN/OIL	0	ONCE	I have my own sitting place	13	6.0	Botswana	0	0	0	0	0	0	0	0	0	...	0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
4	582.5511155	781.8103573	15.0	25	SOMETIMES	>1-1.5KM	WALK	Completed Training After Secondary	Completed University Degree	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	13.0	Botswana	0	0	0	0	1	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0
5	433.75709155	646.2787858	14.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed All Secondary	Completed All Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	8.0	Botswana	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
6	402.9024375	646.2787858	14.0	25	NEVER	>2-2.5KM	WALK	Completed All Primary	Completed All Secondary	ELECTRIC	0	ONCE	I have my own sitting place	14	8.0	Botswana	0	0	1	0	0	0	1	0	0	...	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
7	645.3928111	781.8103573	15.0	25	MOST OF THE TIME	>0.5-1KM	WALK	Completed All Secondary	Completed All Secondary	ELECTRIC	2	ONCE	I have my own sitting place	13	14.0	Botswana	0	1	0	1	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0
8	534.7037205	781.8103573	15.0	25	SOMETIMES	>3.5-4KM	WALK	Completed Training After Secondary	Completed Training After Secondary	PARAFFIN/OIL	0	NEVER	I have my own sitting place	12	11.0	Botswana	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
9	448.50248125	781.8103573	15.0	25	SOMETIMES	>2-2.5KM	WALK	Completed Some Secondary	I Do Not Know	ELECTRIC	0	NEVER	I have my own sitting place	12	10.0	Botswana	0	0	0	0	0	0	1	0	0	...	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
10	596.5705475	864.5981152	8.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed Some Primary	Completed Some Primary	CANDLE	0	NEVER	I have my own sitting place	12	5.0	Botswana	0	0	0	0	0	0	0	0	0	...	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
11	561.55982055	864.5981152	8.0	25	SOMETIMES	UP TO 0.5KM	WALK	Completed Some Primary	Completed All Primary	CANDLE	0	NEVER	I have my own sitting place	12	6.0	Botswana	0	0	0	0	0	0	0	0	0	...	0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
12	365.26664425	781.8103573	4.0	25	MOST OF THE TIME	>1.5-2KM	WALK	I Do Not Know	I Do Not Have a Father	CANDLE	0	ONCE	I have my own sitting place	16	4.0	Botswana	0	1	0	0	0	1	0	0	0	...	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
13	509.855262	781.8103573	4.0	25	SOMETIMES	>3-3.5KM	WALK	Completed All Primary	Completed All Primary	CANDLE	0	NEVER	I have my own sitting place	12	4.0	Botswana	0	0	0	0	0	0	0	0	1	...	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
14	495.3200764	781.8103573	13.0	25	SOMETIMES	>2.5-3KM	WALK	I Do Not Have a Mother	No School, No Adult Education	ELECTRIC	0	ONCE	I have my own sitting place	14	6.0	Botswana	0	0	0	0	0	0	0	1	0	...	0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
15	459.9863116	812.3311897	14.0	24	SOMETIMES	>3-3.5KM	CAR	I Do Not Have a Mother	I Do Not Have a Father	ELECTRIC	0	NEVER	I have my own sitting place	12	11.0	Botswana	0	0	0	0	0	0	0	0	1	...	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
16	586.986174	812.3311897	14.0	24	SOMETIMES	>5KM	CAR	Completed Some Primary	Completed Some Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	10.0	Botswana	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
17	776.69753635	884.6274115	15.0	24	MOST OF THE TIME	>5KM	CAR	Completed All Secondary	Completed Training After Secondary	ELECTRIC	0	NEVER	I have my own sitting place	11	13.0	Botswana	0	1	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0
18	510.4680063	884.6274115	15.0	24	SOMETIMES	>5KM	CAR	Completed University Degree	Completed Some University	ELECTRIC	0	NEVER	I have my own sitting place	12	15.0	Botswana	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
19	600.0593246	828.7361371	10.0	24	SOMETIMES	>4.5-5KM	CAR	Completed Some University	Completed University Degree	ELECTRIC	0	NEVER	I have my own sitting place	11	14.0	Botswana	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0
20	607.15746125	828.7361371	10.0	24	SOMETIMES	>4KM-4.5KM	CAR	Completed Training After Secondary	Completed Training After Secondary	ELECTRIC	0	NEVER	I have my own sitting place	12	14.0	Botswana	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0

Linear regression can only handle numerical columns, so we’ll drop the categorical columns.

africa.drop(
    columns = [
        "english_at_home",
        "travel_distance",
        "means_of_travel",
        "m_education",
        "f_education",
        "source_of_lighting",
        "repeated_grades",
        "sitting_place",
        "country",
    ],
)

	123 student_score Decimal(23,18) 100%	123 teacher_score Decimal(11,7) 100%	123 teacher_year_teaching Decimal(3,1) 100%	123 number_students_school Integer 100%	123 days_absent Integer 100%	123 age Integer 100%	123 socio_eco_statut Decimal(3,1) 100%	123 english_at_home_ALL_THE_TIME Bool 100%	123 english_at_home_MOST_OF_THE_TIME Bool 100%	123 english_at_home_NEVER Bool 100%	travel_distance_>0.5-1KM	travel_distance_>1-1.5KM	travel_distance_>1.5-2KM	travel_distance_>2-2.5KM	travel_distance_>2.5-3KM	travel_distance_>3-3.5KM	travel_distance_>3.5-4KM	travel_distance_>4.5-5KM	travel_distance_>4KM-4.5KM	travel_distance_>5KM	123 means_of_travel_BICYCLE Bool 100%	123 means_of_travel_BUS_TRUCK_VAN Bool 100%	123 means_of_travel_CAR Bool 100%	123 means_of_travel_OTHER Bool 100%	123 means_of_travel_TRAIN Bool 100%	...	123 socio_eco_statut_3.0 Bool 100%	123 socio_eco_statut_4.0 Bool 100%	123 socio_eco_statut_5.0 Bool 100%	123 socio_eco_statut_6.0 Bool 100%	123 socio_eco_statut_7.0 Bool 100%	123 socio_eco_statut_8.0 Bool 100%	123 socio_eco_statut_9.0 Bool 100%	123 socio_eco_statut_10.0 Bool 100%	123 socio_eco_statut_11.0 Bool 100%	123 socio_eco_statut_12.0 Bool 100%	123 socio_eco_statut_13.0 Bool 100%	123 socio_eco_statut_14.0 Bool 100%	123 country_Botswana Bool 100%	123 country_Kenya Bool 100%	123 country_Lesotho Bool 100%	123 country_Malawi Bool 100%	123 country_Mozambique Bool 100%	123 country_Namibia Bool 100%	123 country_Seychelles Bool 100%	123 country_South_Africa Bool 100%	123 country_Swaziland Bool 100%	123 country_Tanzania Bool 100%	123 country_Uganda Bool 100%	123 country_Zambia Bool 100%	123 country_Zanzibar Bool 100%
1	618.89140445	864.5981152	9.0	25	0	12	14.0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0
2	480.2333067	864.5981152	9.0	25	0	12	10.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
3	445.80907615	864.5981152	9.0	25	0	13	6.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
4	582.5511155	781.8103573	15.0	25	0	12	13.0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0
5	433.75709155	646.2787858	14.0	25	0	12	8.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
6	402.9024375	646.2787858	14.0	25	0	14	8.0	0	0	1	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
7	645.3928111	781.8103573	15.0	25	2	13	14.0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0
8	534.7037205	781.8103573	15.0	25	0	12	11.0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
9	448.50248125	781.8103573	15.0	25	0	12	10.0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
10	596.5705475	864.5981152	8.0	25	0	12	5.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
11	561.55982055	864.5981152	8.0	25	0	12	6.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
12	365.26664425	781.8103573	4.0	25	0	16	4.0	0	1	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
13	509.855262	781.8103573	4.0	25	0	12	4.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	...	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
14	495.3200764	781.8103573	13.0	25	0	14	6.0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	...	0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
15	459.9863116	812.3311897	14.0	24	0	12	11.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	...	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
16	586.986174	812.3311897	14.0	24	0	12	10.0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	1	0	0	...	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
17	776.69753635	884.6274115	15.0	24	0	11	13.0	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	1	0	0	...	0	0	0	0	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0
18	510.4680063	884.6274115	15.0	24	0	12	15.0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	1	0	0	...	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0
19	600.0593246	828.7361371	10.0	24	0	11	14.0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	...	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0
20	607.15746125	828.7361371	10.0	24	0	12	14.0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	...	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0

Let’s look at the correlation between the response column and the predictors. We’ll look to keep columns with correlations coefficients greater than 20% (the top 10 features).

x = africa.corr(focus = "student_score", show = False)
africa = africa.select(columns = x["index"][0:12])
africa.head(100)

	123 student_score Decimal(23,18) 100%	123 socio_eco_statut Decimal(3,1) 100%	123 source_of_lighting_ELECTRIC Integer 100%	123 socio_eco_statut_14.0 Integer 100%	123 means_of_travel_CAR Integer 100%	123 english_at_home_NEVER Integer 100%	123 repeated_grades_NEVER Integer 100%	123 age Integer 100%	123 country_Tanzania Integer 100%	123 teacher_score Decimal(11,7) 100%	123 source_of_lighting_CANDLE Integer 100%	123 f_education_Completed_University_Degree Integer 100%
1	508.4821854	10.0	1	0	0	1	1	13	0	812.3311897	0	0
2	499.890896	5.0	0	0	0	1	1	14	0	740.6072335	1	0
3	512.3963035	12.0	1	0	0	0	0	14	0	781.8103573	0	0
4	638.0964246	14.0	1	1	0	0	1	11	0	767.5990947	0	0
5	476.61604135	8.0	0	0	0	1	0	13	0	740.6072335	0	0
6	363.50642795	9.0	0	0	0	1	0	12	0	796.6892631	0	0
7	486.87185345	7.0	0	0	0	1	0	14	0	740.6072335	1	0
8	574.16180765	12.0	1	0	0	0	1	12	0	796.6892631	0	0
9	464.89787835	5.0	0	0	0	1	1	12	0	703.2192138	1	0
10	518.8684453	8.0	0	0	0	0	1	12	0	767.5990947	0	0
11	473.3024924	3.0	0	0	0	0	0	15	0	812.3311897	0	0
12	523.5419478	13.0	1	0	0	0	1	11	0	715.3321692	0	1
13	438.3829052	5.0	0	0	0	1	0	11	0	727.8266349	0	0
14	431.55012475	4.0	0	0	0	0	1	13	0	781.8103573	0	0
15	468.7501092	10.0	1	0	0	1	0	12	0	740.6072335	0	0
16	462.11156245	4.0	0	0	0	1	1	13	0	846.0948606	0	0
17	625.32189785	13.0	1	0	0	1	1	13	0	796.6892631	0	1
18	510.49975695	8.0	0	0	0	0	1	12	0	767.5990947	0	0
19	392.6647144	3.0	0	0	0	1	1	12	0	767.5990947	1	0
20	604.00500395	10.0	1	0	0	0	0	12	0	767.5990947	0	0
21	523.57632325	11.0	1	0	0	0	1	12	0	753.8647201	0	0
22	408.61576405	6.0	0	0	0	1	0	15	0	753.8647201	0	0
23	506.12832545	9.0	1	0	0	0	1	13	0	727.8266349	0	0
24	612.54879395	7.0	0	0	0	0	1	12	0	812.3311897	0	0
25	471.31550095	9.0	1	0	0	0	0	13	0	753.8647201	0	0
26	501.8363334	13.0	1	0	0	0	1	12	0	753.8647201	0	0
27	498.8611914	12.0	1	0	0	0	0	12	0	906.5642598	0	0
28	423.1570224	12.0	1	0	0	0	1	12	0	846.0948606	0	0
29	450.387201	4.0	0	0	0	1	0	14	0	812.3311897	1	0
30	519.147368	12.0	1	0	0	0	1	14	0	781.8103573	0	0
31	590.32581285	9.0	0	0	0	0	1	12	0	624.6280703	1	1
32	510.49975695	7.0	0	0	0	1	1	14	0	846.0948606	1	0
33	643.49873135	12.0	1	0	0	0	1	12	0	646.2787858	0	0
34	495.3200764	8.0	1	0	0	1	1	13	0	767.5990947	0	0
35	459.9863116	10.0	1	0	0	1	0	13	0	781.8103573	0	0
36	487.99064305	5.0	0	0	0	0	1	12	0	740.6072335	0	0
37	566.4337237	4.0	0	0	0	0	1	12	0	715.3321692	0	0
38	552.117438	4.0	0	0	0	0	0	14	0	691.3923912	1	0
39	468.7501092	4.0	1	0	0	0	0	14	0	691.3923912	0	0
40	538.25214075	7.0	0	0	0	0	1	12	0	781.8103573	0	0
41	577.2088944	9.0	0	0	0	0	1	13	0	846.0948606	0	0
42	430.1331224	9.0	1	0	0	1	0	13	0	781.8103573	0	0
43	532.00740605	10.0	1	0	0	0	1	11	0	646.2787858	0	0
44	488.90656515	7.0	0	0	0	0	0	14	0	864.5981152	0	0
45	504.18288805	7.0	0	0	0	0	1	12	0	884.6274115	0	0
46	498.21906805	14.0	1	1	1	0	1	12	0	753.8647201	0	1
47	518.8684453	11.0	0	0	0	1	1	11	0	753.8647201	0	0
48	536.69074345	10.0	1	0	0	1	1	12	0	753.8647201	0	0
49	477.86961185	6.0	0	0	0	1	1	13	0	864.5981152	1	0
50	569.53207295	10.0	1	0	0	0	1	12	0	864.5981152	0	0
51	534.32951535	10.0	1	0	0	1	1	13	0	679.7563239	0	0
52	428.54585685	9.0	1	0	0	0	0	13	0	767.5990947	0	0
53	574.3841869	6.0	0	0	0	0	0	13	0	679.7563239	0	0
54	588.42325725	13.0	1	0	1	0	0	12	0	679.7563239	0	1
55	687.60827855	11.0	1	0	1	0	1	12	0	906.5642598	0	0
56	713.8913859	14.0	1	1	1	0	1	11	0	958.6404302	0	0
57	685.5165495	15.0	1	0	1	0	1	12	0	812.3311897	0	0
58	730.6463148	12.0	1	0	0	0	1	12	0	767.5990947	0	1
59	655.9907931	11.0	1	0	0	0	1	14	0	781.8103573	0	0
60	678.03411965	8.0	1	0	0	0	0	16	0	767.5990947	0	0
61	664.3520506	3.0	0	0	0	0	1	16	0	846.0948606	0	0
62	558.4746904	4.0	0	0	0	0	1	12	0	781.8103573	0	0
63	369.37215045	2.0	0	0	0	0	1	13	0	740.6072335	0	0
64	659.65783435	3.0	0	0	0	0	1	15	0	703.2192138	0	0
65	649.74605845	6.0	0	0	0	0	1	11	0	781.8103573	0	0
66	639.2043353	7.0	1	0	0	0	0	14	0	828.7361371	0	0
67	625.94792415	7.0	0	0	0	0	1	12	0	753.8647201	0	0
68	606.58769385	8.0	0	0	0	0	0	11	0	715.3321692	0	0
69	436.4143824	5.0	0	0	0	0	0	13	0	781.8103573	0	0
70	430.1331224	6.0	0	0	0	0	1	14	0	767.5990947	0	0
71	612.8324285	5.0	0	0	0	0	0	13	0	740.6072335	0	0
72	556.7909405	8.0	0	0	0	1	1	13	0	781.8103573	0	1
73	801.8048367	12.0	1	0	0	0	0	13	0	796.6892631	0	0
74	643.926602	5.0	0	0	0	1	0	13	0	767.5990947	0	0
75	653.7518235	5.0	0	0	0	0	0	15	0	828.7361371	0	0
76	545.70392675	7.0	1	0	0	0	1	14	0	668.4063893	0	0
77	419.0689409	6.0	0	0	0	0	1	13	0	864.5981152	0	0
78	544.4968754	3.0	0	0	0	0	0	14	0	864.5981152	0	0
79	697.6229267	4.0	0	0	0	0	0	13	0	781.8103573	0	0
80	713.8913859	3.0	0	0	0	0	1	14	0	781.8103573	0	0
81	620.13842915	2.0	0	0	0	0	1	14	0	781.8103573	0	0
82	653.37429785	5.0	0	0	0	0	1	15	0	796.6892631	0	0
83	545.70392675	3.0	0	0	0	0	1	13	0	767.5990947	0	0
84	580.68518045	3.0	0	0	0	0	0	14	0	812.3311897	0	0
85	512.77417745	6.0	0	0	0	0	0	12	0	864.5981152	0	0
86	620.55908965	9.0	1	0	0	0	0	12	0	864.5981152	0	0
87	537.2691123	4.0	0	0	0	1	1	14	0	812.3311897	0	0
88	615.66339735	9.0	0	0	0	0	1	13	0	846.0948606	0	0
89	517.10655305	6.0	0	0	0	0	1	13	0	668.4063893	0	0
90	488.9603572	4.0	0	0	0	0	0	15	0	828.7361371	0	0
91	681.81298295	7.0	0	0	0	0	1	11	0	884.6274115	0	0
92	431.55012475	4.0	0	0	0	0	1	13	0	796.6892631	0	0
93	463.9812291	6.0	0	0	0	0	0	14	0	828.7361371	0	0
94	495.59890395	3.0	0	0	0	0	0	15	0	740.6072335	0	0
95	553.94425435	7.0	0	0	0	0	1	16	0	781.8103573	0	0
96	557.51591795	5.0	0	0	0	1	0	13	0	846.0948606	0	0
97	640.024659	6.0	0	0	0	0	0	14	0	884.6274115	0	0
98	459.9863116	2.0	0	0	0	0	0	15	0	812.3311897	1	0
99	586.65127695	4.0	0	0	0	0	0	15	0	812.3311897	0	0
100	573.5674997	9.0	0	0	0	0	1	11	0	846.0948606	0	0

Let’s examine the correlation matrix to see if we have any independent predictors.

africa.corr()

Some of these features are highly-correlated, like socioeconomic status and having an electric lighting. We’ll drop the lighting column to avoid unexpected results while computing the linear regression.

africa["source_of_lighting_ELECTRIC"].drop()

	123 student_score Decimal(23,18) 100%	123 socio_eco_statut Decimal(3,1) 100%	123 socio_eco_statut_14.0 Integer 100%	123 means_of_travel_CAR Integer 100%	123 english_at_home_NEVER Integer 100%	123 repeated_grades_NEVER Integer 100%	123 age Integer 100%	123 country_Tanzania Integer 100%	123 teacher_score Decimal(11,7) 100%	123 source_of_lighting_CANDLE Integer 100%	123 f_education_Completed_University_Degree Integer 100%
1	508.4821854	10.0	0	0	1	1	13	0	812.3311897	0	0
2	499.890896	5.0	0	0	1	1	14	0	740.6072335	1	0
3	512.3963035	12.0	0	0	0	0	14	0	781.8103573	0	0
4	638.0964246	14.0	1	0	0	1	11	0	767.5990947	0	0
5	476.61604135	8.0	0	0	1	0	13	0	740.6072335	0	0
6	363.50642795	9.0	0	0	1	0	12	0	796.6892631	0	0
7	486.87185345	7.0	0	0	1	0	14	0	740.6072335	1	0
8	574.16180765	12.0	0	0	0	1	12	0	796.6892631	0	0
9	464.89787835	5.0	0	0	1	1	12	0	703.2192138	1	0
10	518.8684453	8.0	0	0	0	1	12	0	767.5990947	0	0
11	473.3024924	3.0	0	0	0	0	15	0	812.3311897	0	0
12	523.5419478	13.0	0	0	0	1	11	0	715.3321692	0	1
13	438.3829052	5.0	0	0	1	0	11	0	727.8266349	0	0
14	431.55012475	4.0	0	0	0	1	13	0	781.8103573	0	0
15	468.7501092	10.0	0	0	1	0	12	0	740.6072335	0	0
16	462.11156245	4.0	0	0	1	1	13	0	846.0948606	0	0
17	625.32189785	13.0	0	0	1	1	13	0	796.6892631	0	1
18	510.49975695	8.0	0	0	0	1	12	0	767.5990947	0	0
19	392.6647144	3.0	0	0	1	1	12	0	767.5990947	1	0
20	604.00500395	10.0	0	0	0	0	12	0	767.5990947	0	0

Let’s normalize the dataset to follow the Gaussian-Markov assumptions.

africa.normalize(columns = africa.get_columns(exclude_columns = ["student_score"]))

	123 student_score Decimal(23,18) 100%	123 socio_eco_statut Real 100%	123 socio_eco_statut_14.0 Real 100%	123 means_of_travel_CAR Real 100%	123 english_at_home_NEVER Real 100%	123 repeated_grades_NEVER Real 100%	123 age Real 100%	123 country_Tanzania Real 100%	123 teacher_score Real 100%	123 source_of_lighting_CANDLE Real 100%	123 f_education_Completed_University_Degree Real 100%
1	394.6525909	1.433742650904167	-0.1798854581	-0.2396043034	2.02297214381	0.80707982649	-0.80723920987	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
2	553.94425435	1.433742650904167	-0.1798854581	4.1733141292	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
3	711.012538	2.0189437329058677	5.5587818787	-0.2396043034	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
4	552.6380938	1.433742650904167	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
5	473.37592285	1.1411421099033165	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-0.80723920987	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
6	567.91707295	1.7263431919050174	-0.1798854581	4.1733141292	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
7	562.54417665	2.0189437329058677	5.5587818787	-0.2396043034	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
8	470.89351185	1.433742650904167	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
9	495.0852689	1.1411421099033165	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
10	337.41711175	0.8485415689024662	-0.1798854581	-0.2396043034	2.02297214381	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
11	672.19391705	2.0189437329058677	5.5587818787	4.1733141292	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
12	566.4240782	1.433742650904167	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.3392338760315992	-0.521244728	-0.258861602
13	672.3160803	1.1411421099033165	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-0.80723920987	-0.2832577574	1.101928632197215	-0.521244728	-0.258861602
14	606.58769385	0.8485415689024662	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.101928632197215	-0.521244728	-0.258861602
15	713.8913859	1.7263431919050174	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.101928632197215	-0.521244728	3.8628518743
16	665.9588279	1.7263431919050174	-0.1798854581	4.1733141292	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.101928632197215	-0.521244728	-0.258861602
17	601.76862675	1.433742650904167	-0.1798854581	4.1733141292	-0.49429451003	0.80707982649	-0.80723920987	-0.2832577574	1.101928632197215	-0.521244728	-0.258861602
18	616.6632499	1.7263431919050174	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-0.80723920987	-0.2832577574	1.101928632197215	-0.521244728	3.8628518743
19	545.70392675	0.8485415689024662	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-0.80723920987	-0.2832577574	1.101928632197215	-0.521244728	-0.258861602
20	558.59580965	1.433742650904167	-0.1798854581	-0.2396043034	-0.49429451003	0.80707982649	-1.40536158075	-0.2832577574	1.101928632197215	-0.521244728	-0.258861602

We can use a cross-validation to test our model.

from vastorbit.machine_learning.vast import LinearRegression
from vastorbit.machine_learning.model_selection import cross_validate

cross_validate(
    LinearRegression(),
    input_relation = africa,
    X = africa.get_columns(exclude_columns = ["student_score"]),
    y = "student_score",
)

	explained_variance	max_error	median_absolute_error	mean_absolute_error	mean_squared_error	root_mean_squared_error	r2	r2_adj	aic	bic	time
1-fold	0.27642002833993773	371.7970956	48.704937	57.8362245	13664.883502701427	73.02250807933852	0.27634065342313585	0.275114320010872	56319.60575884116	56393.052130218195	0.355010986328125
2-fold	0.27602050991272475	385.7734318	47.087543	56.8832757	13451.911786982977	71.96488668556192	0.2760044573710837	0.2747773462818821	56217.232858746196	56290.67735480488	0.34036874771118164
3-fold	0.29088408634668095	386.6387509	46.790024	57.5590349	14005.838111695775	73.05664525485433	0.29077433722071944	0.28957510546086274	56589.42053734675	56662.891258814154	0.32038116455078125
avg	0.28110820819978116	381.4030927666666	47.52750133333333	57.42617836666667	13707.544467126727	72.6813466732516	0.2810398160049797	0.27982225725120563	56375.419718311365	56448.87358127907	0.338586966196696
std	0.006914513668157029	6.801645901824671	0.8413859402574358	0.4002216033585845	228.14258950157995	0.5068053683997836	0.006884714199883666	0.006897677095658918	156.98686904605265	156.99856344005812	0.014193594755068688

The model isn’t bad. We’re just using a few variables to get a median absolute error of 47; that is, our score has a distance of 47 from the true value. This seems high, but if we keep in mind that the final score is over 1000, our predictions are quite good.

Let’s compare the importance of our features.

model = LinearRegression()
model.fit(
    input_relation = africa,
    X = africa.get_columns(exclude_columns = ["student_score"]),
    y = "student_score",
)

model.features_importance()

The following factors seem to have the greatest influence on a student’s performance:

• Having a good teacher.

• Being of good socio-economic status.

• Tanzanian teachers tend to overrate their students.

• Age (younger students tend to perform better).

• Being able to get to school by car.

Let’s add the prediction to the VastFrame to see how our model performs its estimations.

model.predict(africa, name = "estimated_student_score")
africa.boxplot(["estimated_student_score", "student_score"])

africa.describe(columns = ["student_score", "estimated_student_score"])

	count	mean	std	min	approx_25%	approx_50%	approx_75%	max
"student_score"	17865.0	504.1966737665631	86.29385920170321	117.53299985	441.20236	494.76453	556.62067	918.5515125
"estimated_student_score"	17865.0	504.1966738	46.27915086017581	389.798936	471.03687	497.74542	531.69556	689.2642057

Our model has trouble catching outliers: exceptionally well-performing and struggling students.

Let’s draw a residual plot.

africa["residual"] = africa["student_score"] - africa["estimated_student_score"]
africa.scatter(["residual", "student_score"])

We see a high heteroscedasticity, indicating that we can’t trust the p-value of the coefficients.

model.coef_

Let’s look at the model’s analysis of variance (ANOVA) table.

model.report("anova")

	Df	SS	MS	F	p_value
Regression	10	38260397.14471028	3826039.714471028	275.65405034960395	0.0
Residual	17854	247811026.08697397	13879.86031628621
Total	17864	133026600.74813513

According to the ANOVA table, at least one of our variables is influencing the prediction.

We can also see that a student’s estimated score and true score skew heavily from a normal distribution.

africa["estimated_student_score"].hist()

from vastorbit.machine_learning.model_selection.statistical_tests import jarque_bera

jarque_bera(africa, "estimated_student_score")

Our model doesn’t verify the basic hypothesis and therefore isn’t stable enough to be put into production. Let’s look at a second technique.

Example with decomposition

Let’s look at the same dataset, but use decomposition techniques to filter out unimportant information. We don’t have to normalize our data or look at correlations with these types of methods.

We’ll begin by repeating the data preparation process of the previous section and export the resulting VastFrame to VAST.

africa = load_africa_education()
africa = africa.select(
    [
        "(zralocp + zmalocp) / 2 AS student_score",
        "zraloct AS teacher_score",
        "XNUMYRS AS teacher_year_teaching",
        "numstu AS number_students_school",
        "PENGLISH AS english_at_home",
        "PTRAVEL AS travel_distance",
        "PTRAVEL2 AS means_of_travel",
        "PMOTHER AS m_education",
        "PFATHER AS f_education",
        "PLIGHT AS source_of_lighting",
        "PABSENT AS days_absent",
        "PREPEAT AS repeated_grades",
        "zpsit AS sitting_place",
        "PAGE AS age",
        "zpses AS socio_eco_statut",
        "country_long AS country",
    ],
)
africa.dropna()
africa.one_hot_encode(max_cardinality = 20)
africa.drop(
    columns = [
        "english_at_home",
        "travel_distance",
        "means_of_travel",
        "m_education",
        "f_education",
        "source_of_lighting",
        "repeated_grades",
        "sitting_place",
        "country",
    ],
)

	123 student_score Decimal(23,18) 100%	123 teacher_score Decimal(11,7) 100%	123 teacher_year_teaching Decimal(3,1) 100%	123 number_students_school Integer 100%	123 days_absent Integer 100%	123 age Integer 100%	123 socio_eco_statut Decimal(3,1) 100%	123 english_at_home_ALL_THE_TIME Bool 100%	123 english_at_home_MOST_OF_THE_TIME Bool 100%	123 english_at_home_NEVER Bool 100%	travel_distance_>0.5-1KM	travel_distance_>1-1.5KM	travel_distance_>1.5-2KM	travel_distance_>2-2.5KM	travel_distance_>2.5-3KM	travel_distance_>3-3.5KM	travel_distance_>3.5-4KM	travel_distance_>4.5-5KM	travel_distance_>4KM-4.5KM	travel_distance_>5KM	123 means_of_travel_BICYCLE Bool 100%	123 means_of_travel_BUS_TRUCK_VAN Bool 100%	123 means_of_travel_CAR Bool 100%	123 means_of_travel_OTHER Bool 100%	123 means_of_travel_TRAIN Bool 100%	...	123 socio_eco_statut_3.0 Bool 100%	123 socio_eco_statut_4.0 Bool 100%	123 socio_eco_statut_5.0 Bool 100%	123 socio_eco_statut_6.0 Bool 100%	123 socio_eco_statut_7.0 Bool 100%	123 socio_eco_statut_8.0 Bool 100%	123 socio_eco_statut_9.0 Bool 100%	123 socio_eco_statut_10.0 Bool 100%	123 socio_eco_statut_11.0 Bool 100%	123 socio_eco_statut_12.0 Bool 100%	123 socio_eco_statut_13.0 Bool 100%	123 socio_eco_statut_14.0 Bool 100%	123 country_Botswana Bool 100%	123 country_Kenya Bool 100%	123 country_Lesotho Bool 100%	123 country_Malawi Bool 100%	123 country_Mozambique Bool 100%	123 country_Namibia Bool 100%	123 country_Seychelles Bool 100%	123 country_South_Africa Bool 100%	123 country_Swaziland Bool 100%	123 country_Tanzania Bool 100%	123 country_Uganda Bool 100%	123 country_Zambia Bool 100%	123 country_Zanzibar Bool 100%
1	417.89208995	691.3923912	1.0	19	1	13	4.0	0	0	1	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	...	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
2	380.3882634	691.3923912	1.0	19	0	13	2.0	0	0	1	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
3	533.9601487	727.8266349	5.0	24	2	12	5.0	0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
4	462.11156245	727.8266349	5.0	24	0	12	7.0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
5	394.6525909	727.8266349	5.0	24	0	11	6.0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
6	413.12320985	727.8266349	5.0	24	10	12	3.0	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
7	462.0710837	727.8266349	5.0	24	0	11	5.0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
8	397.036715	727.8266349	5.0	24	8	12	5.0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
9	365.22920145	796.6892631	4.0	24	3	11	5.0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
10	517.10655305	796.6892631	4.0	24	5	11	4.0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
11	497.7289933	796.6892631	4.0	24	5	13	6.0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
12	504.14250455	796.6892631	4.0	24	6	13	2.0	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
13	543.1042547	796.6892631	4.0	24	0	12	7.0	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
14	555.7062418	846.0948606	15.0	25	0	11	12.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
15	553.94425435	846.0948606	15.0	25	0	12	10.0	1	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	0	...	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
16	645.4134935	846.0948606	15.0	25	2	11	13.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0
17	713.8913859	846.0948606	15.0	25	2	12	14.0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0
18	575.6524936	846.0948606	15.0	25	0	12	12.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
19	736.76449035	812.3311897	10.0	25	2	12	11.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
20	615.3610104	812.3311897	10.0	25	0	11	12.0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	...	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0

Let’s create our principal component analysis (PCA) model.

from vastorbit.machine_learning.vast import PCA

model = PCA()
model.fit(
    africa,
    africa.get_columns(exclude_columns = ["student_score"]),
)
africa_pca = model.transform()
africa_pca.head(100)

	123 student_score Decimal(23, 18) 100%	123 teacher_score Decimal(11, 7) 100%	123 teacher_year_teaching Decimal(3, 1) 100%	123 number_students_school Integer 100%	123 days_absent Integer 100%	123 age Integer 100%	123 socio_eco_statut Decimal(3, 1) 100%	123 english_at_home_ALL_THE_TIME Integer 100%	123 english_at_home_MOST_OF_THE_TIME Integer 100%	123 english_at_home_NEVER Integer 100%	travel_distance_>0.5-1KM	travel_distance_>1-1.5KM	travel_distance_>1.5-2KM	travel_distance_>2-2.5KM	travel_distance_>2.5-3KM	travel_distance_>3-3.5KM	travel_distance_>3.5-4KM	travel_distance_>4.5-5KM	travel_distance_>4KM-4.5KM	travel_distance_>5KM	123 means_of_travel_BICYCLE Integer 100%	123 means_of_travel_BUS_TRUCK_VAN Integer 100%	123 means_of_travel_CAR Integer 100%	123 means_of_travel_OTHER Integer 100%	123 means_of_travel_TRAIN Integer 100%	...	123 col71 Double 100%	123 col72 Double 100%	123 col73 Double 100%	123 col74 Double 100%	123 col75 Double 100%	123 col76 Double 100%	123 col77 Double 100%	123 col78 Double 100%	123 col79 Double 100%	123 col80 Double 100%	123 col81 Double 100%	123 col82 Double 100%	123 col83 Double 100%	123 col84 Double 100%	123 col85 Double 100%	123 col86 Double 100%	123 col87 Double 100%	123 col88 Double 100%	123 col89 Double 100%	123 col90 Double 100%	123 col91 Double 100%	123 col92 Double 100%	123 col93 Double 100%	123 col94 Double 100%	123 col95 Double 100%
1	459.30500675	691.3923912	33.0	25	2	15	8.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.16468424673036466	0.005478813736449143	-0.0758141924105899	0.021555810038108872	0.01926481540419621	0.07527774297467825	-0.09285981592076925	-0.21931697246016343	-0.0077398547524008005	0.015901026742319392	0.01983286479075019	0.0017061478850020162	0.024705060906892166	-0.0011996385299720895	-0.005038044053941804	-0.018920656312102487	-0.003469793572097145	-0.0005652200162196022	-0.0013034927445658994	-0.003999780398782298	-0.0010247333029025933	-9.70868546023751e-05	-0.00037388397990500326	2.0714492708818413e-12	3.918824062221148e-12
2	415.95588695	691.3923912	33.0	25	4	15	7.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.07537360186060986	0.0016268675098148342	-0.08895757508013444	0.1549386692403849	-0.273207472068721	-0.021710613859685143	-0.19796146652302496	0.006228130981422693	-0.014702254116133783	0.004473906895853603	0.01757522588104989	0.005809926723884975	-0.16235458466713956	-0.003568406129030553	-0.014275902171451488	-0.22379394864655733	0.004179761370909981	-0.008403765827305109	0.000805630985773145	-0.004670888588003413	0.000848435591817498	-0.00042046636046948486	-0.00014244407822225827	-9.853331336526696e-12	-1.8643558455633323e-11
3	626.91634205	715.3321692	7.0	25	2	14	7.0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.0051722375148657695	-0.1611443611596413	0.05166270590153258	-0.015275467505789965	0.05747089459971502	0.0763006828391417	0.008677951677576016	0.019373706631950014	0.011664045192633522	-0.002258566484757832	-0.006571165836629698	0.005442516305628063	0.008833271871696215	-0.008694826817393171	-0.019157256973551296	0.00068496814466607	-0.00509903663689935	-0.0016616715290981737	0.0018245555795430933	-0.002535093348303169	-0.0006830647117833293	-0.0007193113966225306	-0.00038745245195814576	2.1321507147532218e-12	4.032941111364806e-12
4	430.21879075	715.3321692	7.0	25	1	12	9.0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	...	-0.3906827346992055	-0.11845788676922266	0.10735395849067106	0.005550834217690411	0.03489416134110126	-0.028331746350337247	0.18416119134219014	0.13030234091168028	-0.02080826411923034	-0.009568255044144991	0.0012068436051373022	-0.018801823400206343	0.034119876129537506	0.007689805075527398	0.03989289248697943	-0.015989416059230176	-0.002078944173679842	0.005893286345246452	-0.0026977557436544015	-0.004225928935381707	-0.00018471020101292504	-0.0005350477678121061	-0.0002841851681450016	1.8623751933866886e-12	3.523161392926454e-12
5	459.11697145	715.3321692	7.0	25	0	18	6.0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.003773250004018947	0.012687363620875771	-0.17177062429041917	-0.6552869212182855	-0.6144920912179278	-0.31896967026763484	0.00937361491996097	0.031311071517202124	-0.08121544963765495	0.42425362063088196	0.10954737651142184	0.0058041355896119	0.03357914815591162	0.03346109492496352	-0.02402207362984553	0.009076190797223879	-0.039390941191182464	0.0020914800725588944	-0.013922060719186889	-0.0005520332204358156	-0.003335592151863145	-0.0017254339361437612	-8.973114020465088e-05	2.1298539408710285e-12	4.026363039943902e-12
6	532.25568495	715.3321692	7.0	25	2	13	9.0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.016830316578898708	-0.13430061151271336	0.04934381283933908	-0.025569690889973642	0.04535109379922118	0.07290425280948647	0.03556920599901546	0.029644638504764866	-0.0027165575278407603	0.010233767879623856	-0.013595699611988192	-6.410437404176778e-05	0.0043545254402171645	0.005019313941555477	0.00519644051246247	0.001618901442989909	0.0016127399687157306	-0.0017389597108269255	0.00018286855908229748	-0.001162968335591439	-0.0006796017700717988	-0.0009428522230134318	7.527225512566338e-06	7.21565524433109e-13	1.3645547661477619e-12
7	456.26620545	657.24721	6.0	22	4	13	6.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.041653211614760155	0.0033270274029735735	-0.05967706660144537	0.0025477327080951585	0.015283462093164783	0.08259937331981884	-0.17385160705536354	-0.026147654452512385	0.029363883349448235	-0.006664291310790635	0.012648807448858247	0.0023812651499744222	0.005159177816571993	-0.00446370539714488	-0.002154593578806086	0.004743152377648132	-0.0010671662209808612	-0.005282835590130057	0.002045032147531458	0.00021491941019512553	0.0001560834798091177	-0.001040639542542261	-0.0004218036214945535	2.251576017733369e-12	4.259260074934644e-12
8	450.7787366	657.24721	6.0	22	9	16	6.0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.0489783907082303	-0.005207169892764915	0.024876055815879802	0.09338265773399662	0.0722807150681085	0.044866043455231486	0.001432956941705648	0.014382440682134514	0.004034944894302877	0.008758227581122201	0.015217139453011508	0.0008103074477957378	0.009653941285250033	-0.0004582574634411983	-2.3563281169636935e-05	0.005103493394347334	-0.003242558434747671	0.0037872031026271857	0.0024765318008756207	-0.00040496342586612794	0.00019310755938481105	-0.0011001583003489327	-0.0005350971385986912	3.26112956960064e-12	6.168621632684988e-12
9	436.4143824	657.24721	6.0	22	0	12	6.0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	0.08030268893285135	0.06122328230744093	0.0028766510290793293	0.029363809467961317	0.03556140429039695	-0.1249977007696803	0.022886554435036555	-0.00429403480895865	0.0029364058408168324	-0.0018334676335048014	0.0024726100496145712	-0.010105751167071507	0.01854314916670819	0.015306207764636431	0.030257581059947317	-0.004227950194250579	-0.0024908820727741004	0.002755364685206303	-0.0013831727364342118	0.0008832648761786889	0.0006124536547563527	-0.0006939298518193096	0.00039702671346004156	-1.0288277087021964e-12	-1.948052366895837e-12
10	337.41711175	657.24721	6.0	22	0	16	7.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.11644579445829246	-0.12052789815680735	0.017945615999993877	0.08237789031794979	0.0800028540263768	0.07599385390899198	-0.19849904241536345	0.018364361336375996	0.047524626950484115	0.036427768693216345	-0.007712299566523633	0.02095250010469227	-0.16145026094828843	-0.00822746536658499	-0.03690530806967643	-0.21817124385381956	-0.0029375440821105925	-0.009918126833507443	0.0009850495584282674	-0.0015982452108470654	-0.0006758620057912403	-0.000739053275009882	-0.0004102845391956915	-8.130556635852417e-12	-1.538283343230924e-11
11	385.53865385	657.24721	6.0	22	0	13	6.0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	...	-0.03136611568082949	-0.022719779942325023	0.03669736881213648	0.0036264189897251093	-0.004338580246844779	-0.0014545548267584105	0.17081441694915175	0.06140954718425331	0.04675654553135986	0.0024552003937192528	-0.011280234228801901	-0.0202025670433349	0.004861467205633548	-0.002473697729883553	0.009884976719695916	0.0049364786463717255	0.0034492689989346534	-0.0008574304440151384	-0.0028143957061212855	-0.008545531212583326	4.805565982733718e-05	-0.0008259696545435234	-0.0004144690569000735	2.9018336432563233e-12	5.4887210524045424e-12
12	438.7512602	753.8647201	5.0	25	3	14	3.0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	...	-0.07875244245759995	-0.05338150143582993	0.048297498010531646	0.040630196912923945	0.06339773516808284	-0.20151983550113706	0.13485456032503362	0.020060747022352384	-0.002843119509374119	0.0004318155186491085	-0.0052649022835595406	-0.018002080996165893	0.010877490997256547	-6.110697177508879e-05	0.009612297848165237	-0.003205388649360373	0.0023443045369324926	0.009779832475616741	-0.005473384032225864	-0.008142661730804045	-0.0008840712570197127	-0.0005422215790630742	-0.00012408670929420681	9.769844743124581e-13	1.846009919670365e-12
13	435.52195685	812.3311897	17.0	25	4	17	5.0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.032784104474714325	-0.010206009546066607	-0.03812582499801118	0.4804335481196948	-0.1598460478682447	0.08865887743375088	0.02286593392387765	-0.022670525753225004	-0.0329392739495994	0.035249942024752776	0.016426378559827343	-0.005682431969099963	0.009536918851968253	0.012240429853446455	0.014104845327895089	0.016717382416431118	-0.005149174598006863	0.0005880043942150925	-0.0020110545652857106	-0.002515101258817299	-0.0005332590208827346	-0.0007600799983012576	-0.00028988157286297204	3.225519166085788e-12	6.0994963716142645e-12
14	494.23537775	753.8647201	5.0	25	0	11	4.0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	...	-0.15800052150314461	-0.010416737703068422	0.16793128602713014	0.007182141243171044	0.017910555252913695	-0.19034832645001926	0.2291455495392692	-0.0122387799356431	-0.11284325755007908	-0.02018205126879639	-0.011427184744255053	-0.025992544478301516	0.029021449241139155	0.0030597543049494177	0.07582502630966424	0.008769105626701273	0.0015612438007139968	-0.0013708981775329583	-0.003907098987073816	-0.004728937829043918	-0.0018701905612353838	-0.0010435687660657557	-0.0005076503745884708	3.5784590656141253e-12	6.76968250042924e-12
15	435.52195685	753.8647201	5.0	25	0	16	6.0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.07014089505164044	-0.06422820330373764	0.015286257473907808	0.0942412092916324	0.037714699888280585	0.09808951301305764	-0.14426403301719196	-0.05286725463408856	0.024900192934127534	0.025080156029805036	0.0033213021180746646	0.007062988053119987	0.020186891430883265	-0.00759685271604932	-0.02662592263271663	-0.006784941946718988	-0.010478573021682857	0.001803229327913365	-0.0010632602937147408	-0.0017590669607800116	-0.0012737195303157169	-0.0013565486741837517	-4.663428304623387e-05	1.2437433115542676e-12	2.3525908080190162e-12
16	494.23537775	753.8647201	5.0	25	0	12	8.0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	...	-0.1357647453839869	-0.05974880969871952	0.034151314387008774	0.03626067582199873	0.028134862708115355	-0.10498016932012746	0.09406079517510238	0.03127265220829989	-0.03767569895875409	-0.013667269933142628	0.005444599975844679	-0.014376676019203742	0.013601480785774108	-0.002568793288013862	0.02641889104321525	0.012874855240395834	0.007026279955879994	0.008739274214101934	-0.0004413487990810776	-0.003977192745325572	-0.0025411108899540203	-0.0021727028710087277	-0.0005095743916107135	3.292028464154739e-12	6.2273940640510825e-12
17	438.76115885	753.8647201	5.0	25	3	13	7.0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	...	-0.11045825509403083	-0.04853939887873455	-0.0055294628750236705	0.11726149775620334	-0.29580379796906026	-0.02941711822316746	0.08462145347119279	0.04587384936578884	-0.015756162980939917	-0.010775351519049362	0.00032388061547458495	-0.012358208456956296	0.02629141395381259	-0.0005140239091867209	0.02479578215277707	0.004597130884029962	-0.001542142982164582	0.0016305481040705795	-0.0017088256225630527	-0.004074872796632865	-2.4375667360536868e-05	-0.001370929339435872	-0.0004025302374928495	2.7413162105772374e-12	5.184131365598631e-12
18	421.22448725	812.3311897	17.0	25	0	13	5.0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	...	-0.06654824178453587	-0.014105026460516801	0.07022155466859367	0.11146787193641185	-0.2840622489801758	-0.04504430181231047	0.2093582522624388	0.059953528654162064	-0.040434304705447295	-0.013503206743414826	-0.007152175272696561	-0.03475190713826719	0.0086922440979715	0.016855032421239223	0.04975977330274105	0.0073296014216262815	0.0067269374372440965	-0.0011815416004087745	-0.0023713179717299677	-0.00439765839718997	0.000584251485739427	-0.0005511416782386868	-0.00026632065860043776	2.416277603436511e-12	4.570441710949268e-12
19	510.4680063	812.3311897	17.0	25	0	13	3.0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	...	-0.03948307275763732	0.05666581265387954	-0.004272059013649224	-0.012384928071443953	0.028971551654994515	-0.008135855245246204	0.06923561825795604	-0.00879638949421498	-0.043271933695230365	-0.00510312627811122	-0.014071999817269087	0.00015326308818698284	0.005586745810369347	0.004458172503489226	0.03522855364387491	0.007533646106314009	0.0016726067470384442	-0.005697813872269879	-0.0010694423966225253	-0.0008431418215113873	-0.0001465817067331073	-0.00027884716206095345	-0.00031940297877513314	1.6067307294554625e-12	3.039388648839946e-12
20	459.30500675	691.3923912	3.0	25	1	13	6.0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	0.004755697859827513	0.05539150935563374	-0.0020229887786840824	0.009074135458471305	0.02331716871170792	0.016525640337449552	0.03702286160474551	0.006356028055911276	-0.007266421724117915	0.0008306870065927475	0.006152994165094912	-0.004037559170170139	0.006729126862196101	0.005289787862001856	0.014767695147822153	0.009921000795102292	0.0024274317968676134	-0.006473162962495159	-0.0011265677218465057	0.0003376794693903119	-0.00043733284016424885	-0.0008369306695025334	-5.6628101059418647e-05	1.2787847257689991e-12	2.4183715222280568e-12
21	484.62067635	691.3923912	3.0	25	0	15	5.0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	...	0.021145325349354982	0.01615765855468512	-0.017046878303068468	0.1318580481132251	-0.2778164354230367	-0.024190674042382686	0.07944495941879356	-0.0004386590945539316	-0.0076146601298805894	0.0004041024391831049	0.009757869305531998	-0.014403372156386774	0.01421485797573695	0.002795441734209987	0.00533446473594555	0.007957343265753637	0.00017935484508958634	-0.007264822324282704	-0.0022400218137460575	-0.0018805892190828298	0.0003691770043263346	-0.00020626663725460684	-0.0002869013232464748	1.9399641702967033e-12	3.668135703260787e-12
22	447.4437773	691.3923912	3.0	25	1	13	5.0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	...	0.24309399320245545	-0.07173891621933713	0.05411241512890049	-0.05049731514976593	0.05792950712795401	0.013881506267369115	0.07732187221879097	0.010898507158601758	-0.002437390280339959	0.001876281592168301	-0.01998034860798406	-0.005720368577968872	0.03482787104522949	-0.0015172729558490315	-0.0025311624493116087	-0.01813624127459842	-0.005763386205230253	-0.0036575527468563516	0.0007553014120965058	-0.0008470669798185777	0.00105876695595803	-0.0009698985614557346	-0.00032966694442988694	1.7186967214889096e-12	3.2513857353921195e-12
23	482.43679495	691.3923912	3.0	25	3	12	5.0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	...	-0.017610117401381114	0.0160202188497332	-0.006872411323888182	-0.007300313490473242	0.01721048040324291	0.006812740359449037	0.04715502466862601	0.007823419563733034	-0.025297843167015728	-0.019192181822170577	0.005471414829812336	-0.012340062274619161	0.017040065089713917	-0.0021925869103670346	0.019956131287682324	-0.0046570801245786496	0.0037511850346758145	-0.0007781496387101897	0.0018972692040604747	0.0002145209681818441	0.0001808979250373362	-0.0012996551698436234	-0.00045158607492884733	2.4922168583208718e-12	4.71463192627246e-12
24	490.83732915	691.3923912	3.0	25	2	17	5.0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.020482584981483944	0.015666686108757735	0.016348776822029648	0.3664160979807561	0.12767240825930484	0.14431420110744242	0.0104238181052136	-0.0407459441243225	-0.0036653089221956193	0.050041680536435554	0.02502978609351133	0.0007661097572850413	0.012610674554399744	-0.0006833254842627599	-0.015388860681514478	-0.004376662085654772	-0.01208493276129685	-0.0036616364966486235	-0.004009322786396907	-0.00033147504682683274	-0.0005783780682036698	-0.0006805180234210337	-0.0002985153630006852	2.4203021619007773e-12	4.578546339029031e-12
25	445.80907615	691.3923912	3.0	25	2	19	3.0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	...	0.03343875294842417	-0.12784546990327342	0.0026230881172726516	-0.5158485616631041	-0.2011676131659857	-0.1628571308948707	0.1295901857743768	0.0055167026911098465	0.05376172369110834	-0.8220130619066143	-0.19202193157178896	-0.06313429842548592	-0.03547441163612081	0.4678656717287773	-0.14032754147383392	-0.002113059783893901	-0.2027162679440671	-0.0007332734876423377	-0.052817517594531224	-0.008085275062305948	-0.011239287771631684	-0.0026151048141847245	0.00039802056704819823	1.751947901076433e-12	3.3146684477957535e-12
26	464.56443645	691.3923912	3.0	25	9	18	3.0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	...	-0.032132184207210764	0.014880611678249004	-0.07716586488263139	-0.7799031838044772	-0.2582546936222958	-0.34648866930119965	0.13021473667571243	0.029718319470756357	-0.09215127809488727	0.42187112840151725	0.07804790338188193	0.01492947598921524	0.024225367745547857	0.0378197494712843	0.009214561431697339	0.01412626204493795	-0.0394035287987289	-0.0023255873897955353	-0.013135488527860829	0.0018024969493055315	-0.0032003126822884843	-0.0006973613902841031	-0.0003449119680132318	2.687116510293818e-12	5.0827263700869305e-12
27	431.5787757	691.3923912	3.0	25	2	17	3.0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	...	-0.05212491836465165	0.07206016198745437	0.08882614569042868	0.13675425084993859	0.6092374446957894	0.12859179362531886	0.16160756006364452	-0.012268237471164605	-0.016479268363028043	0.01284417238533214	-0.02170398477261736	-0.007156683342710907	0.003506466605161404	0.009500270141570113	0.01490845281554788	0.007426539981045274	-0.004375607296477075	-0.007721391962894524	-0.00468673840149064	-0.010082848804428142	-0.0018710852371411934	-0.00091703178642334	0.0005095989631967503	-2.1994696853592317e-13	-4.157364260960039e-13
28	480.2333067	703.2192138	10.0	17	1	15	8.0	0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.5552886852291143	0.20500512384103778	-0.02375556382804564	0.036277082542409585	0.044925913110535245	-0.0531190542954476	0.014428834776373599	-0.2028546712084298	0.06927548875278444	0.011791943053149649	-0.011131919601928185	-0.012682465386092781	0.03005281798707815	0.0061369475917518665	0.02175036032929796	0.033764643542197825	-0.0009425379540720919	0.014015368283954069	0.0008455951637803286	2.2763043735939863e-05	-0.0003685432055816491	-0.00033522430654752246	-0.0003024760420800789	3.267326001856829e-12	6.179321407084813e-12
29	506.3606662	703.2192138	10.0	17	2	15	6.0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	...	0.06770658583645177	-0.13319939358071525	0.15089831402793957	0.020667501280245505	0.09431957025998007	-0.10806087588229732	0.17257686964746558	0.04604093568100537	0.07133553339703128	0.009582008027697186	-0.02773460821359031	-0.02434589287062459	0.017637048010162117	0.014672257538223534	0.017570610679060475	-0.003909483810937624	-0.0024690481797005662	0.008479953860906093	-0.0016838850213922423	-0.0015717472081644424	0.0005482981576839521	-0.0010409522894188092	-0.00017511178530102732	2.0748146344252363e-12	3.9261654119714815e-12
30	459.30500675	781.8103573	7.0	25	1	13	8.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.08351096744122888	-0.058171657762645136	-0.10052037217257839	0.1072205310313986	-0.30819194434265473	0.03199101538968283	-0.1399578483415807	-0.005562484460656329	-0.014298927557182402	-0.012038750991617827	0.021760949880026914	0.007223068277938191	-0.23488459947627263	-0.014501968005455123	0.0010229079598907242	0.1732505159053241	0.012231623702061115	-0.0035317418642951013	-0.00131129568034405	-0.0007725693684899765	0.0005694223848523711	-0.000640468769182998	-0.00017776126366322382	6.034568765712282e-13	1.13790967456444e-12
31	447.4437773	781.8103573	7.0	25	0	14	9.0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.1357813725183744	-0.08177278108028944	0.009970200311767097	-0.01135983241644128	0.021181965358531167	0.07534955561975905	0.040225180262699264	0.04181024038937415	0.025280628460115785	0.010871470403331923	0.0004030053101815712	-0.010285551529597082	0.014263064194462664	-0.007147588121863552	-0.01880817100281958	-0.006817490264433373	0.0015104254223745816	0.0027767841320013153	-0.003257741892114748	-0.001769597922683725	-0.0008991923012059931	-0.0006893214044523464	0.00044103607773160567	-1.3112944670189758e-12	-2.4823680141784053e-12
32	445.7273602	781.8103573	7.0	25	3	14	8.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	...	0.06790072728041403	-0.008029720872619241	0.03661537493108631	0.016631182613022644	0.05889983941687656	0.001313813890860768	0.13267949991233616	0.009620765955150434	-0.03140974245964954	-0.0034030711063378506	0.011765407739930657	-0.014801126430971147	0.0031740051777227004	0.00012409584925390793	0.014794490469131822	0.008780091280931828	0.000859460567374689	0.0025848702144351376	0.003387116550791048	0.00017075185119509837	-0.0018253193101965936	3.095133484902002e-05	-0.0003515808789691173	1.772660499379596e-12	3.352110719301237e-12
33	450.7787366	781.8103573	7.0	25	0	14	6.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.003638029024072799	-0.11547542020722211	-0.03609107387294646	-0.02723806415299112	0.03376757277297411	0.10252770267707685	-0.15643807494926326	-0.026040256507240362	-0.005445341519061399	-0.002014680055412852	-0.009534066890969816	0.013592538058078106	0.007857800968097219	-0.0003479866737093949	-0.003068875746317302	0.001925306420306699	0.0029232481716421925	0.0031485468482475975	-0.0012615645118069414	-0.00039784202773263073	-0.0005067905082561486	-0.001397255430046544	-0.00047830676930388536	2.981728067665911e-12	5.6405440510220326e-12
34	518.60185665	781.8103573	7.0	25	1	12	6.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.021742751330194876	-0.006020465252159525	-0.10539194581394853	0.11714125982819298	-0.3353593138751243	0.04019172661121704	-0.14611206328676687	-0.010784120400604483	-0.03041665061287483	-0.019671072213044264	0.0033262713557511373	-0.004274614815262521	0.0016666744603602065	0.0021803329793691276	0.025248996165723472	0.009024014264063828	0.005932826326751229	0.00014356769599847656	-0.0009120040748067665	-0.0012226873655334547	0.0002973159535903176	-0.0008732380777210542	-0.00030224171103132016	1.9114314385638368e-12	3.6136098749638847e-12
35	444.7408847	753.8647201	15.0	25	2	15	3.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.02652450210862127	0.003789934034533474	-0.10004651083816826	0.122930804996265	-0.2965710071926247	0.06434471967500789	-0.17422252734227878	-0.03854831238564055	-0.01959413702835905	-0.012381584772653234	0.010613061661071604	0.003984110499683759	-0.003553949856702025	-0.0012585991127231004	-0.0014844310999724726	0.0030061512836122768	0.0039857472551706385	-0.005339486030415107	-0.0028211327616092163	-0.0014057763712064563	0.0005125767320088865	-0.00023853338645501792	-0.00038746500126384453	1.6799499379294916e-12	3.1765011923811527e-12
36	418.54468005	753.8647201	15.0	25	9	16	2.0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	0.05438497167662245	0.05755322975414292	0.022575651912257078	0.11265165927229198	0.10348351908616876	-0.03258713795886082	0.014387260932349343	-0.007606952390871741	-0.04607144739222228	0.018603924464602486	-0.0058988296504176915	0.003612618450798244	0.006382354549672731	-0.0043015373194194035	0.0009753274111995013	-0.01722344382639465	-0.005901543267261479	-0.006099297236308103	0.003908545592120988	0.0017698903050029514	0.0014397258753260716	-0.0006656199848942036	0.0008116278921446069	-3.604961459466794e-12	-6.822056345090545e-12
37	484.60224	753.8647201	15.0	25	0	17	2.0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	...	0.09565996114713289	0.018426524177155037	0.10100509437689664	0.3989934177582488	0.17530693798938224	0.01644991935038834	0.17489123295088982	0.011079841100071958	-0.025339782951757463	0.04375162030637533	-0.006976129178177235	-0.004957515470485162	-0.0028007063956856223	-0.0010006173346658385	-0.003203563231958601	-0.019021954407382895	-0.011609251907446473	-0.005175014875860072	-0.0012149430712480173	-0.0014849497839536548	0.0003928148847265786	-0.0008591223413912698	0.0005501469511352014	-2.0524533374065905e-12	-3.882879665273521e-12
38	453.0102116	753.8647201	15.0	25	0	13	6.0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.01952648122465593	-0.13040127943623805	0.04673641529669254	-0.019735216524988032	0.020359870622315257	0.06726949382026082	0.01259458696170268	0.02471621503669287	0.004055080495576593	-0.0056034693799946215	-0.017127455705366543	0.005497551441053347	0.004656163586320719	-0.00045034500728613437	0.00144974555437123	0.009617372175765655	0.00020132002634350673	-0.0012775989556691898	-0.0010448675129645516	-0.002675387623734372	-0.0010156498869533723	-0.0010869724770980718	-0.0004786477762381606	3.007957086622681e-12	5.690670620583858e-12
39	457.34268235	753.8647201	15.0	25	0	14	5.0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	...	0.03840486473636607	-0.14764777226489023	0.020345258331901553	0.10448969817739893	-0.27114219217372654	0.022227200346708834	0.05046833667194062	0.0034386054141785534	0.009048760253702843	-0.0069403835551185514	-0.015660975870212014	-0.004535969866760742	0.01209814676152562	-0.003222679205852549	-0.0011936468669929041	-0.0022037393161692376	0.004189881795994113	-0.00425144788088973	-0.0014879142640804746	-0.0010050774826267986	-0.0003276436904897629	-0.0015423033065821135	-0.0004557677315042046	2.8292528130214537e-12	5.35145585319745e-12
40	384.113366	753.8647201	15.0	25	0	12	4.0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.03165028817492944	-0.01350111247552693	-0.09799154620132164	0.08632875855240227	-0.34211386456799875	0.03486980305273868	-0.15350452755317626	-0.031852562207265483	-0.017831713189734297	-0.018997617412366986	0.0026623370238096217	-0.008213436583081898	0.008094423262756507	0.0057794060601953565	0.020933124501442454	0.008772838731085156	0.006041217219052467	0.0013756712873552023	-0.001912096003305062	-0.003518208335432603	0.0008871557117846341	-3.071829124732866e-05	-0.0002356375249674147	1.6174998927943266e-12	3.057485284141336e-12
41	424.1239538	753.8647201	15.0	25	0	18	4.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.33832416682780697	0.03448935438262712	-0.20946882212108692	-1.010750272995323	0.12405116866561595	-0.16402274083742469	-0.1751624007360347	-0.054947486209678695	-0.1141184129179454	0.4234314250438701	0.09792353823418214	0.01929225279643821	0.043198913921277414	0.03697186712022234	-0.007000301742577392	0.01805802082251868	-0.04248688222039378	-0.00032855109355968605	-0.016824978857039763	-0.010439243381161357	-0.005824524698664188	-0.001062666247911803	0.0007604785976352747	-6.232493600188191e-13	-1.1799306895211144e-12
42	442.27302135	753.8647201	15.0	25	2	13	8.0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	...	-0.04028905428402547	-0.05504256444562646	0.05676302151642853	0.006130811538246194	0.04967869286224057	-0.04525609694300086	0.18678735904782187	0.0716032829504075	-0.027324073197432942	-0.021220214396666288	-0.008455851085534111	-0.011101137539877714	-0.0070848426286479055	0.005969816478796214	0.031066492131610793	0.007635788036092976	0.006429324228830764	0.006455383153977935	0.001826633006510948	-0.004139809531107559	-0.00021379309867442084	-0.0008314143418062295	-0.00021958200749371898	1.3508035055976522e-12	2.555303654527762e-12
43	450.59161785	753.8647201	15.0	25	0	12	11.0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	...	0.0986571200444837	-0.16166197393836118	0.15455598326771763	-0.0012769883085920774	0.056955379958751114	0.023106334965826886	-0.001530629768904715	-0.03908999193154349	-0.17068846551890487	-0.014975440120428422	-0.0008552559519388481	0.017621890965711637	0.016611957175719814	-0.0038256805756238742	-0.007404424939269256	0.010171604809257682	-0.0012852326802628342	0.025575896147203875	-0.001570830745674718	0.0025806280315542347	-0.000710824781880378	-0.0009496321959895315	-0.00020741107559096232	1.6993987901804056e-12	3.2142930106670455e-12
44	523.57632325	767.5990947	4.0	25	0	13	9.0	1	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	...	0.2796166897360024	0.017872366168714324	0.05557771923781573	0.015219477904852756	0.0444842731227525	-0.02687965391277419	0.06643336379572565	-0.12105928679192358	0.05520560052710311	0.008347677809937252	-0.008324850339237366	-0.011974996851198674	0.02793221931832582	-0.0038725924967825897	-0.005030270102575302	0.027244511134724123	-0.0013036078515528625	0.02654244917007796	-0.0005083822466163484	-0.005495509463069274	-0.0013634721459708703	-0.0013531422435382572	-9.074075059450296e-05	2.9703222608113636e-12	5.620233908565296e-12
45	441.1832625	767.5990947	4.0	25	4	14	6.0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	...	0.36135770168925824	0.03245161661390441	0.07036724308702644	0.10959143715651071	0.1363144216211305	-0.2783291778791942	-0.1300639976118408	0.9213390419463517	-0.041524698172269954	-0.007210859553791961	-0.009402680875742048	-0.026899985363526228	0.06426528666247569	0.003560875630780891	0.0017708129915407333	0.0476945382031438	-0.005570513083231569	0.01464542770204279	-0.0031201274172463446	-0.006623593887558269	0.0013546306530644885	-0.0012057654262263681	-5.244000541331257e-05	2.6828282738612033e-12	5.0715825064772555e-12
46	479.52518475	767.5990947	4.0	25	0	14	8.0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	-0.017305581956371042	-0.14647064913675717	0.009790101114025434	-0.03435630507135447	0.058162624220630635	0.02078341436848771	0.09836079341394341	0.03015440202045791	0.025469232798502585	0.006572353427425961	-0.040162094771393105	-0.014110208814956138	0.002916646537010565	0.0037370653110068708	0.006185452770993678	0.0027095055251013015	0.005176295151777664	0.005001696836123903	0.0006840723011423438	-0.005746443822304472	-0.0004219553525586483	-0.0006448042419376471	-0.0004174194675092076	2.295409010524351e-12	4.341097389637326e-12
47	589.27915745	767.5990947	4.0	25	1	12	7.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.03163014203932511	-0.13398277356283317	-0.030075795615135854	-0.035138395389320766	-0.006934285528461873	0.08215775180904546	-0.15254040608335315	-0.03319118680060656	-0.025217939008732683	-0.0038059114217203266	-0.019469874254360753	0.01601832266652528	0.009833973541302854	-0.0002654761808646082	0.020052134109314237	-0.0062732954602868285	-0.001284910226235551	0.0027588066488978946	-0.0009190347518462126	-0.0020445911457853258	-9.484705723874161e-06	-0.0007150959980443028	-0.0002512849292267378	1.4648927991656949e-12	2.7707007990928502e-12
48	462.0710837	767.5990947	4.0	25	0	12	5.0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.05440930801634934	0.00819576773342134	-0.0192358929261325	-0.003175993838161033	0.005845077364850579	-0.029235573178972973	-0.0022658637562627646	-0.0323083191378244	-0.021983433135659168	-0.009195580278977357	-0.013730229944971084	-0.002729445983539373	0.02268130088565771	0.013159384810951488	0.03711414118736593	0.009955717818735376	-0.0012055382460718959	0.0030448900996122973	0.0003751963486016154	-0.0011314499949037948	0.0005784439594039414	-0.00041223264918222144	-0.00022644284448150847	2.03358372684822e-12	3.846215476410662e-12
49	481.63339065	767.5990947	4.0	25	2	15	6.0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	0.008207836044124204	0.11136917042007778	-0.0910426418113032	-0.13367995032905844	0.5485915764000862	-0.1587817147641679	0.0026156991982504057	0.056182503957901385	-0.02040567850164396	-0.008237302309278795	-0.01776326119570714	-0.04414084000290336	-0.15480690778001935	0.02519072556758177	0.04474328518769417	-0.21380094875918199	0.008112961518385621	0.0006062592696702406	-0.0024789244131209138	-0.012604248075779718	-0.0006059718201949779	-0.0016349779106290979	0.001100248726428038	-1.3558124753019131e-11	-2.5651647009550315e-11
50	525.78591475	691.3923912	18.0	25	0	12	2.0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	...	0.051981393202090324	0.017448890457875548	0.04439548856764467	0.018238893119437244	0.009512706367366303	-0.09406132798169063	0.14980325463246674	0.03736213280200329	-0.015170925495149758	0.0003440545612937355	-0.02705174877089634	-0.012326121768614343	0.004765212313430472	-0.005255601137139099	0.012197593134387964	-0.015854032208286933	-0.0018736136101650568	-0.00464079015512463	5.6321177273674694e-05	-0.008119347672044369	0.0017813502211397089	-0.00037001944797217755	0.0005587395121964543	-2.47827937850157e-12	-4.6906501824151694e-12
51	428.98821145	691.3923912	18.0	25	0	18	10.0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	...	-0.0027007312242205415	-0.07995042292114136	0.03640032422496761	-0.76753332843243	-0.26853235652504903	-0.3323458271121235	0.21396200398234058	0.033755862871819024	-0.0617547774174096	0.4559764563026648	0.08595508453778694	-0.011138866314243495	0.0471219935437781	0.03474566598071774	-0.02724652396954017	0.01628211375694322	-0.05106695614453096	-0.00291508934723571	-0.01665481322268492	-0.0012918950665804876	-0.004280641916562705	-0.0011742979072911748	0.00021903927445877117	1.4853399847770335e-12	2.8088837975225757e-12
52	433.75709155	691.3923912	18.0	25	2	13	5.0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	...	-0.03990953132812222	-0.18034470270148084	0.13125550955070558	-0.03073777978388506	0.03819075153646367	0.004649301205847026	0.15717169566733444	0.03500311815184899	0.02751906174899111	-0.010538126898850801	-0.03620012651420055	-0.012142237602648006	0.015440036623186538	0.00958020407099582	0.013853403868434517	0.002161890388189981	0.0011454599342365522	0.00019295266118503178	-0.0023409397542040254	-0.0032383923251734304	0.0006286090910252436	-0.000598215235500335	-0.00038029018440736886	2.637461785517458e-12	4.990716636921121e-12
53	508.05662265	691.3923912	18.0	25	5	12	7.0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	...	-0.01123284209940479	-0.14982653602379475	0.15917643788777341	-0.02352533789726481	0.054915849877068455	-0.03350013169336633	0.18763695962513507	0.03188039931306606	-0.014589929377013202	0.004373027760403334	-0.018116325847368074	-0.0048380315107611396	-0.21634402565363772	-0.0037536926090544083	0.025661444877145625	0.18264361040152288	0.004925855582450378	-0.002892662280251356	-0.000782913154689691	-0.0004417136024493637	0.0008012371429299684	-0.00040642686963348153	-0.0004220867983382964	3.2191839559515208e-12	6.08904639739498e-12
54	607.15746125	691.3923912	18.0	25	2	12	11.0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	...	0.4607145837595272	0.18120133918037853	0.000524510599047733	0.008289211424478355	0.02303820026566958	-0.031918807099505404	0.18518227911102328	-0.33624603424640503	-0.041007836806548105	0.019705931724566025	-0.012652822935672187	-0.032036914281595344	0.036291609817296476	0.002936650742957063	0.026799405606667923	0.023490709535822038	0.0008195294877619915	0.017348930700842036	0.0022214495534337323	-0.002730207626149034	-0.0008794675303793319	0.00034230990456889543	-0.0003906444810496894	3.8559792599391565e-12	7.291892379934704e-12
55	644.39741815	691.3923912	18.0	25	0	14	4.0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	...	-0.029750985537749665	-0.0007223620889820204	0.010163795308791298	0.10860920977131376	-0.31038700991174517	-0.05342892982456845	0.17521791377549326	0.015095374671466533	0.011984746240102834	-0.0009090475737689925	-0.0032734746591221043	-0.019848980166745772	0.014927505458163348	-0.005222290694062251	0.005079486084035951	-0.012053663498827149	0.004801375453630357	-0.0037554066332615875	-0.006860149018670534	-0.010583806530105992	0.0004295484621438917	-0.00016818293121613267	-0.00025121885279509874	1.669749763890748e-12	3.1554902216401216e-12
56	470.89351185	691.3923912	18.0	25	0	12	8.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	...	-0.010633210909815213	-0.15307668964436458	0.08111902314863059	-0.017027842604347237	0.009333557007516424	0.021892689805845963	0.11935005197846209	0.03306257509521846	-0.015443680373117346	0.003908523220214469	-0.02584943694704637	-0.0020255600988271097	0.009631381581096618	0.007558510549698583	0.03113771630638658	0.002093129312105365	0.000523011437307038	-0.0004413757122853237	0.00012599759405182692	-0.00031280936369239466	-0.0017016445481700191	-6.656250276169038e-05	-0.00037036508331715493	1.8711372816638475e-12	3.5386559430138788e-12
57	499.890896	691.3923912	18.0	25	3	12	5.0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	...	-0.09529887413971448	0.0022072867164837926	0.016227611796461726	-0.007382095559746457	0.014687554740562016	-0.03189882181922928	0.07158078443746416	-0.0028160745278344628	-0.034887031987989765	-0.004791118902197688	0.008762596087388351	-0.0035265553385185272	-0.20938795841256116	-0.01446478764344995	0.01914446248085168	0.1701560448675271	0.004937252406370543	-0.003630560314983866	-0.00040963339360206236	0.0017241679786483347	0.001117216294716699	-0.00047366628974190975	-0.00026232608830458186	1.271082553535662e-12	2.402509210763725e-12
58	428.98821145	691.3923912	18.0	25	3	14	4.0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	-0.04174095559716845	0.042856563212067995	-0.0266242654273178	-0.030137365774325187	0.042530332435998665	0.018068031617439886	0.07511497314943652	0.013796076651225148	0.014986844397549895	-0.0022488928932225738	-0.0021557571510919266	-0.027164726491719245	0.009856679315543968	-0.0023124963437158095	0.0033072745119965673	-0.0024651934065779437	0.0056637391038452025	-0.0005397049842691007	-0.000443693414576558	-0.0053833187982001715	0.0006362869459070998	5.203906493135785e-05	-0.000437611202948346	2.1387634806436454e-12	4.044778864364872e-12
59	457.34268235	703.2192138	3.0	25	1	15	4.0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	0.09864532002456562	-0.10909445245794173	0.04657473270846408	0.005262175038184837	0.08253075556839655	-0.0796919331239354	0.04630879766827669	-0.018041593930073426	0.0235894799296913	0.015894266166497625	-0.036755915304830473	-0.0230351984713826	0.023076199533736526	0.0046270949971015315	0.004826790644184576	-0.014379242942301767	-0.0003384410776939288	0.0024988392088716015	-0.00021602500032812783	-0.0026342484647216826	0.0007321067584511534	-0.00016364614965134726	-0.00012429966372209524	7.440458059582925e-13	1.407111002460425e-12
60	404.57186145	703.2192138	3.0	25	2	15	5.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	...	-0.03421731679261354	0.012026487660629689	0.011867312987462336	0.009985421690512015	0.0427668614286131	0.009642584326490122	0.11337829613131507	0.021557146371435274	-0.006314217537563243	-0.004640574205857262	0.010385859961759294	-0.017473737703008616	0.015539366783208344	-0.00022559962669783202	0.006789254812651175	-0.0007239943278797018	0.002013697922469858	0.002966297422030374	-0.0004988499706203884	-0.0007215473692500686	-0.001133849489888464	0.00013692571103253416	-0.00044915792031438324	2.3644024326109006e-12	4.4726865736310226e-12
61	427.09602845	703.2192138	3.0	25	1	16	7.0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.06489693062042061	0.001142692079443384	0.016200643172292667	0.0914772561682746	0.054016469598274726	0.036888467616153456	0.025250517573359627	0.00225802036417374	-0.022358953245652742	0.015460139656833134	0.011425418095632297	0.004327232327638833	0.012317589098274559	-0.0035176248047308688	-0.001011045383723281	-0.009057077977176446	-0.007143260489630942	0.0016998700234957932	-0.0005937218875587087	-0.0022274691719620585	-0.00020992226236822402	-0.0006015501149852663	-0.0002200798967269472	1.4236896471642938e-12	2.692846409491011e-12
62	438.3829052	703.2192138	3.0	25	1	11	6.0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.14944809385324717	0.022877325085778413	-0.02013627468355792	-0.03182552689729703	-0.04018343589109851	0.02037076065665844	0.06502025290758402	-0.17878200455300222	-0.08871324850951079	-0.018023596196242227	-0.0032304424004127846	-0.0038495549240009207	0.022528087460329096	0.0015389082545747626	0.054091347533618786	-0.017812373788530927	0.003975729026610965	-0.0021073047053030308	0.001851801543115697	-0.00028233015598510457	-0.0007776361118991754	-0.0001254271179433476	-0.0004835563371271639	2.262567225677158e-12	4.279688178587747e-12
63	390.59366675	703.2192138	3.0	25	0	11	8.0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	...	0.4656379270829644	0.13451593682952837	0.010766678600927584	-0.02509533610581905	-0.03478625786302601	-0.08734424232020245	0.24056126167246675	-0.34102687884546534	-0.02566484557771046	-0.000388618637530667	-0.02718721756170613	-0.04194618481868223	0.045401489455771715	0.004159615494981755	0.04197145038055013	0.02090628036599658	0.001765633984731005	0.01984532667494228	-0.000545187937225939	-0.009373685017707831	-0.0004906400814120108	0.0005815726095792352	-0.0004531596259562385	4.6066157920377715e-12	8.71146257049864e-12
64	510.65265825	703.2192138	3.0	25	0	12	4.0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.0186945613685261	0.043853647734151756	0.0010415524815251846	-0.01414792892538175	-0.0007037949661651687	0.012333875222585604	-0.0028441910632645236	-0.010695648326581772	-0.03691709526445481	-0.012257473714192769	0.0031757179139722744	-0.0018547287220060805	0.014459920480579553	-0.0007648310388957478	0.025226245336758366	-0.0016490860311258275	0.0048257472764621865	-0.002295743136175909	0.0005061428362768693	0.00020558403508482846	0.00020730734957616437	-0.0003285332361746532	-0.0002909649995606066	1.4277627778858873e-12	2.700104492514498e-12
65	424.1239538	703.2192138	3.0	25	0	11	5.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	...	0.09273678974056733	0.03427168298082001	0.045446590250574785	-0.027545177106944714	-0.0017261360528756022	-0.05725207609936558	0.12601205580537114	-0.032959087133170466	-0.10272118036165355	-0.028628258941626985	-0.00316782632484359	-0.015048078025275526	0.01988741794069309	0.005552611995239841	0.06362320033875012	0.008375228288530398	0.0015276494230613094	-0.004110276931708251	0.0013241066997055709	0.0014626230981340574	-0.0007714598309741818	0.0002723368351844217	-0.00037068365173981415	2.1549311034397492e-12	4.07539326426891e-12
66	442.96103005	646.2787858	5.0	25	4	13	6.0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.031212183245990562	0.03653922503011705	0.02469505982521611	0.04234824711863415	0.04690036951210719	-0.044100817289881654	-0.06213741227995721	0.033097671102906914	-0.01107012815997885	0.007025397793408906	0.00497046240003958	0.005492608486859432	-0.14769714734840197	-0.0010840174624396066	0.0005144430765055656	-0.2191335751565404	0.0033624316896385307	-0.016385489637292465	0.0028076882798070777	0.0004985093152567778	3.7222869307227294e-05	-0.0010926742553728328	-0.0004852309293448174	-8.063339570605267e-12	-1.5255907185018968e-11
67	468.03205725	646.2787858	5.0	25	0	14	8.0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.04605183885086779	-0.09636816213798498	-0.008541763139235036	0.11740096061205486	-0.3143142686080176	0.03002399247369771	-0.11516045119190405	-0.04952392796570764	0.048840890281925765	0.0015449236003254457	0.0023855057972827585	0.003668591320924319	0.019221153594331505	-0.007889981749969683	-0.027182035624656323	-0.005426048075589773	-0.002573056375130271	-0.007066198896678493	-0.004933064427136829	-0.005385167496538082	-0.0001969008827889368	-0.0004683829664569261	8.045514327425333e-05	3.063730701587854e-14	5.5386714403681326e-14
68	418.54468005	646.2787858	5.0	25	0	16	6.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.03492274659196175	0.036677921984791105	-0.028833334456217274	0.10016641343216474	0.06258704040375837	0.08986716860406951	-0.16090620257039237	-0.05275709050525462	-0.004950314080573117	0.02438937077512773	0.012227280179744685	0.007720159114165047	0.015700209578165716	0.0052327306910088175	-0.0013491893174920784	0.0036549812544993654	-0.007766751689624858	-0.0072875073179975314	0.0004451045974484281	0.0019998271800406443	-0.0006712969867652162	-0.0011616691000570249	-0.0004662855421732524	3.0334783384012516e-12	5.738660010823281e-12
69	475.2331927	646.2787858	5.0	25	3	16	5.0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	...	0.019551460662484046	-0.229603324220589	0.17821276590779783	0.03654323708160109	0.06970535695247082	0.09660210017572267	0.1082355779118758	-0.004270784478047255	0.13188408789510275	-0.11625005504970784	0.8759663892574925	-0.3372166938071019	0.026927067980637237	0.03821799596056634	0.06756325941138212	-0.0029691006120362335	-0.015760611664810076	0.004750740322916818	0.0023983778728498585	-0.008156868948248987	-0.0017237657904038111	0.00011498429925792254	-0.00027550397337348855	1.8032679603897309e-12	3.4117574512992234e-12
70	519.2366424	646.2787858	5.0	25	1	11	7.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.10158677840844126	-0.12760836369502482	-0.0013395472935929008	-0.06279864476770779	-0.007211710816040249	0.05105783339383048	-0.16238444663166146	-0.07201823377604002	-0.07778633888940784	-0.021760241943493314	-0.015697843887118847	0.015168228746343777	0.01224401984294899	-0.0003135565452552151	0.031243636292031376	0.011420921521840922	-0.0007108271529276321	-0.005701098809708637	0.0007421030599307216	0.0003760173089324398	0.0005096964296207992	-0.0004939668862510159	-0.00035681690211871895	2.3217906851470063e-12	4.3914598815918846e-12
71	466.5061422	646.2787858	5.0	25	2	14	4.0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.014691120500544094	0.008114937585839224	-0.0441337689932197	0.01921040819080584	0.06083166020551214	-0.037498882526800136	-0.19589136573631535	-0.05855070875171493	-0.01763225947975823	-0.0011404624409981276	0.020027915647391606	0.002952577183800882	0.015201847070889573	0.001272978351952718	0.00016243771852484865	-0.01129449258292261	-0.0050682038565869674	0.0009696501881276253	0.0012733061138925645	0.0005807350203387437	0.0006031874555168003	-0.0003273481100155727	-2.074822670048853e-05	-1.5134906340189631e-13	-2.8731137772248894e-13
72	491.2592231	646.2787858	5.0	25	2	11	8.0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.03988236788822677	-0.09119702733537118	-0.002090161882722489	-0.04392254129750606	-0.04363592207857585	0.0569865355000596	-0.16730482360127777	-0.06559872130908395	-0.05085460154733568	-0.015094450841197668	-0.006198455927338439	0.007827059773147179	0.01535776674659025	-0.0019136377957918167	0.023769898941837966	0.009084658069777098	-0.0017053774483671914	-0.00823941678857225	-0.0003283611813022998	0.0004957693018158209	6.2234653411460295e-06	-0.0002261083107582148	9.420436605776785e-06	2.770235517558165e-13	5.224575408634848e-13
73	468.7501092	646.2787858	5.0	25	1	13	7.0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.030393701158966887	0.02102633805558665	-0.05486483243688609	0.0004023436128865723	0.011690420131833119	0.036659839529203556	-0.16361640664046753	-0.07799330078971053	-0.028502114798011437	0.0005177371365202995	0.005745841558916127	0.01210316098480099	0.010512759925599214	0.0034280993719394692	0.020895298913906887	0.007238886885976681	-0.001974170716560752	-0.007591952322481764	0.0003297092564760397	-5.3250428385112993e-05	0.00031161541393138825	-0.0002838315388291291	-0.00022728490292889906	1.55934502298568e-12	2.9495854839355786e-12
74	435.52195685	828.7361371	10.0	13	2	14	6.0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.27549945491936595	-0.06171555363284987	-0.09403155300683652	0.01609383433491936	0.005611455383192783	0.11165325950436268	-0.1561662699850619	-0.019339405311779724	0.07225614904582922	0.01235485798479522	0.004604728224684917	0.008073168527496863	0.0028657150801521263	0.008073658213193918	0.03437673068394036	0.02344323799201388	0.0015906750537626638	0.0044285984769987345	-0.0019766278338399614	-0.0009645684697767241	-0.001146904698469845	-0.0005050103852581908	-0.0005672893556208609	3.6310420036179343e-12	6.86886704989169e-12
75	454.86994825	828.7361371	10.0	13	0	12	5.0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.1036449715300576	-0.07841980304918146	-0.033421174270485444	-0.029575782914279175	-0.0028879051432971663	0.05481217028838057	-0.18477489869489871	-0.08609627358400228	0.05624695895619925	0.009996719240214636	-0.037537384583687224	0.019243459952446917	0.011595904342138112	0.017963066948294276	0.06716434274641918	0.01604334093149333	0.0011385624352092895	-0.002865982415694456	-0.0006400799840133574	0.0029190044281953396	0.0002975639446733942	-0.00058269592640134	-0.0002827458691570138	2.4538031416688413e-12	4.6415792512521245e-12
76	529.23396095	828.7361371	10.0	13	0	12	3.0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	0.05325115522263791	0.05264784595991618	-0.02261367666121624	0.002253252438681487	0.03273323664459265	-0.11114144664930525	0.0557280677356853	0.008085878240214745	0.08628517088067453	-0.0008272031788096675	-0.030124604979576173	-0.028190175615084506	0.006499361569014903	0.01539842759280472	0.054888016267879716	-0.013002833775817171	-0.0026053379586430474	0.0040881878586115205	-0.00032065107966889337	-0.0038682552550205447	0.0014259880240898826	0.0001535969602447373	4.77240623758853e-05	-5.887075461651689e-13	-1.1150103981561584e-12
77	461.4261462	727.8266349	7.0	25	1	15	8.0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	0.27790754558989783	0.025829098876642736	0.008741428080314414	0.19262849030349424	-0.2030762996657255	-0.23871313590804816	0.09306260783724571	-0.09012988009112509	-0.004011029034281634	0.003433878288247646	-0.00735041310737886	-0.03434870653149891	0.04037915143125854	0.010665224385526103	0.01481941300313115	0.006696717084037994	-0.006473326252980958	0.013804072334089854	-0.0022703904012670354	-0.005896138144844643	0.0012397915793013884	-0.00013097781952871705	9.18061878377518e-05	9.21646795428798e-13	1.7401223986967406e-12
78	475.7982482	727.8266349	7.0	25	3	12	5.0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.0960681603383793	0.014818637557131944	-0.007929049698273051	-0.009358465657320299	-0.006636255718319561	0.004035231965844874	0.01576346931948602	-0.0049665331845741715	-0.040654868970216314	-0.016733151769924345	0.0029049825476455578	-0.00024784332327460346	0.013947514176172775	0.003651744369525624	0.03225534286752295	-0.004194577870552283	-0.001941174235254382	0.00011214912613134558	-0.000823181590924061	-0.002960432356015994	4.7573099434815186e-05	-0.00028271561207615756	-0.00029804627629765034	1.4876870656400301e-12	2.8134582633287265e-12
79	486.2592043	691.3923912	10.0	25	0	15	6.0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.059450148339634794	0.04030739841027383	-0.0016912727783393276	0.0367141908707001	0.033181917828124484	0.006207493962625154	0.042447505559537996	-0.007992388675174619	-0.010095244209091804	0.004657550266286534	0.006593889704699088	-0.008202171639205863	0.01683745334281065	-0.0026178197745043217	-0.0035950314507552823	-0.010612514307167021	-0.00038598521480321285	-0.0010261140659795656	-0.002131865646294728	-5.0700333183731025e-05	-0.00050621311529167	-0.0012709541327111417	-5.15118626409934e-06	6.152044239505431e-13	1.1634031707673952e-12
80	486.87185345	691.3923912	10.0	25	2	14	5.0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.02539284742712384	-0.1298401337345198	-0.021739668029167235	-0.029379425702894764	0.04073402877876994	0.0947875269208716	-0.17402856248237564	-0.056976322261053994	-0.0037827883534017562	-0.00015271356983815145	-0.011130950995305644	0.015008149149571012	0.01447561771494112	-0.0017950570322771	-0.003707043703828506	-0.004932341099522425	0.0011298930885831665	-0.0028505752341727187	0.0006657095297692095	0.0012521624902571116	0.00042043336714058844	-0.0009253179432638593	-0.00035151204460514857	2.00380199421265e-12	3.790315747120786e-12
81	415.95588695	691.3923912	10.0	25	2	14	4.0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.17831560484305303	0.6705744188432026	-0.20063085855648854	0.22407409516596258	-0.2754853649314031	-0.1791869068813513	0.07702211761932276	0.026197174613729942	-0.0049647641362632975	-0.004785569702276303	0.02018134210038059	-0.02282390076860953	0.020408378190060868	0.0037276852376885924	0.009057945447500727	-0.011558118819257019	0.0040457275611536685	-0.004176553314727538	-0.0019109718601462977	-0.006384874063193498	0.0009453926924564484	-0.0004044914950447635	0.0004058507684522498	-1.6761571209977017e-12	-3.1753345316799085e-12
82	497.5443413	691.3923912	10.0	25	5	13	3.0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	...	0.05986760132750858	0.02931819751834473	0.014634105800084932	0.017004388598133614	0.061646082062534045	-0.07902981125960058	0.011092607055313099	-0.01437100606968542	-0.013296540128432012	-0.013833008396762953	0.002875010286828175	-0.0072763375040187424	0.010654347408203929	4.9127753192636237e-05	0.009768047658517338	-0.015242519356134613	-0.0015573754220387871	-0.004917458675220909	0.0006586340930871349	0.0012792733155852771	0.0007181618762836259	-0.0007382892447283587	-0.00015406507632870854	2.965287825196997e-13	5.613708578765963e-13
83	489.53970635	727.8266349	8.0	25	2	14	7.0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.00011672248256128096	-0.09892522258848548	0.044128157264667935	-0.018745112156521977	0.06004966736605087	0.053361514129168966	0.02790714661537368	0.0012260109447545188	-0.006145171789764899	0.013504326035326518	-0.015415107599454317	0.006595016774991201	0.014471524319252058	0.00808636548798572	0.013061158630743959	0.0014811175450270514	-0.0006120231045971769	-0.003676767637482775	-0.0006568348091468098	0.0002073780732470209	-0.00039380686341292635	-0.0007486250674861796	-2.4725119741946496e-05	1.0827124007262924e-12	2.0474182541251763e-12
84	427.07559925	727.8266349	8.0	25	0	13	8.0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	...	0.04054544516984523	-0.15201268547661934	0.0760548457381218	-0.026263232030820753	0.010166342202358949	-0.005861343933044758	0.15978270878525766	0.016361048615658107	0.006044801760622543	0.013142323505345133	-0.03591391179092696	-0.005048339798945771	0.014773703583052588	0.004523632752101499	0.017721581155467556	0.005434568190875288	0.004274974609789181	-0.003936365438430778	-0.005867725363493256	-0.007128120260998166	-0.00020210381845417582	-0.0005801792494921887	-0.0003525030442350038	2.8751674739836078e-12	5.437525893181515e-12
85	444.1151116	727.8266349	8.0	25	6	16	9.0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	1	0	0	...	-0.07801507219444716	-0.009253885388691565	-0.00615425318914789	0.09130274454528432	0.10074418131607396	0.03821469544411175	-0.08188077566179101	-0.018401721479718386	-0.032767497397339564	0.020706432697381868	0.018028551725915404	0.015290259486310246	0.009537938749498221	0.006288835760620448	-0.0005127914977127342	0.0004576358829518326	-0.0005477363422623165	0.03143600946024602	0.0013993498487956072	-0.001164209567914395	-0.0010426244098391431	-0.0016073252433940356	-0.0005741759122522362	3.390611064571025e-12	6.414265415777208e-12
86	466.5061422	727.8266349	8.0	25	2	13	10.0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	0.00819679343661955	-0.004392874930981602	-0.011434703993350316	-0.011462230064223094	0.035869248635910414	0.0151901679191427	0.13080005488699248	0.046797907498356085	0.009750835057670586	0.0035537710120234003	-0.007207200914095308	-0.02642005399824844	-0.004646473802095516	0.0013182578322135999	0.011411604065287363	0.006909895948587966	-0.0001559885084432186	-0.0025706674568632282	-0.0006834413250363856	-0.007484433779126518	0.00017810524342801022	-2.8017210855468475e-05	-0.0003035605914477715	1.4991206280701933e-12	2.834432804876762e-12
87	431.55012475	727.8266349	8.0	25	1	15	5.0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	...	-0.05280162042402116	-0.07336896362500596	0.07061037125737896	-0.24376191054633756	0.4762688589864843	0.050624335064015674	0.15058668367396832	0.001700733841688475	0.013062129419207435	-0.012345701135585477	-0.04480110384466191	-0.014380093906034163	0.027407604554570163	0.0013798697812961933	0.003433191132611908	-0.010475414298787556	0.00192342923032504	-0.0007498386921018666	-0.00555612858014564	-0.01821020813298125	-0.000705643925117344	-0.0012454530190507958	0.0006517038789463247	-6.839924372536228e-13	-1.2947277502673556e-12
88	491.2592231	727.8266349	8.0	25	1	15	11.0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.0025709873991726973	-0.16399680099533673	0.03452789941192422	-0.010930046352582226	0.05349315194841713	0.09340625406416216	0.007789007557482345	0.005351996670817552	-0.023836153038355602	0.014129738051261371	-0.011040232777424448	0.004821958287173406	-0.001310801535087384	-0.0012190083235995432	-0.01925901301466189	0.0011976955647824865	0.0006456618008409165	-0.0014159513199412134	-0.0007466558288214625	0.0002898626964970252	-0.0004573576071982203	-0.0007323687678753941	-0.0003061258359134008	1.7317201579848055e-12	3.274897310023771e-12
89	457.13415415	727.8266349	8.0	25	0	13	6.0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.02954778209463537	-0.10854937181038492	-0.01655960413627202	-0.026951456429412567	0.015810078615245114	0.10750859421883031	-0.17114998643962775	-0.03706950240140492	0.004303188784727425	0.002363546689378186	-0.021920056992191135	0.011456603487938472	0.013814955339652806	0.002958701573193369	0.00860493853138587	0.00020098469172254572	0.0021868618301749405	-0.005658896302036519	-0.0007060109393456333	0.0013408604554760143	-0.0003407717551022339	-0.0013479182684668326	-0.000445930373969871	2.658833578741492e-12	5.03053201014174e-12
90	365.26664425	603.2634876	21.0	25	1	20	7.0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.1922954283045703	0.03885923731091782	-0.07201691159041024	-0.46756908586879875	-0.16095555733024028	-0.2041346249999883	0.00791607688636934	-0.014755308872877505	0.04530106211837997	-0.45699775049262314	-0.06638664004362885	-0.0406814595963082	0.05607170824534977	-0.2625738069141465	0.06691436298421481	-0.011259036307520755	0.3396393247370937	-0.00149020514447683	0.9413971558650825	0.02282223551600636	-0.04898088589585995	-0.009748770527607323	0.0031843693428763534	-3.5864260857896216e-11	-6.786798854322435e-11
91	448.38439795	603.2634876	21.0	25	0	16	6.0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	...	0.09740413272037328	-0.041070279972802846	0.06506318208173223	0.24697346364489067	-0.19977990420747962	-0.1769479711219683	0.1653397951567878	0.05354185182045876	0.008686112636440039	-0.0039821425213104715	-0.0074504910381996805	-0.02202907379540803	0.011217114533702699	0.011276328322234483	0.005311294150839708	0.008634513894435569	0.0004840872216152635	0.006121242377405118	-0.0022780554590293694	-0.0015739138156466489	0.0007770377316698692	-0.0008552956534194662	-4.234430073015514e-06	1.267502084281246e-12	2.3958617504037817e-12
92	443.61551785	603.2634876	21.0	25	2	14	6.0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.07958149428404976	-0.1463509007896501	0.07603511189459139	0.004426210530018103	0.05417250077794631	0.02284740665693508	0.021425796258406477	-0.003323777036113587	0.020490489745521798	0.012458650801435605	-0.005030813476817294	0.01289093362904089	-0.21591747021885668	-0.01726998120302917	-0.023541623393078626	0.17278433231354656	0.004816395470931019	-0.004709984317687976	-0.00213770083320349	0.002687897283344667	-0.00022841252709713015	-0.0014711398806327549	-0.00023247683134785011	1.7903477399406556e-12	3.385153732071643e-12
93	468.15210195	603.2634876	21.0	25	3	16	6.0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	...	0.2930597656110591	-0.05301591418461097	0.0442785694189874	0.04346240470555008	0.09396857026360424	0.029820930293969847	0.09138192330342026	0.012586677197960895	0.033759234250708654	0.03400143087722589	-0.041687115577648294	-0.013708817254902859	0.02538350187669282	0.008955710253941837	-0.000589820624467597	-0.007185658117597725	-0.004892012223601566	-0.0022149047226520274	0.00021346013275763904	-0.0026406500060411923	0.0011671974458028732	-0.0007317555688187872	-0.0005375078147200251	3.427329956387026e-12	6.483453126893067e-12
94	479.52518475	603.2634876	21.0	25	0	15	7.0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	...	-0.020968148243828404	0.10297581258574247	0.031817207630675186	-0.20734173650504537	0.497925547069368	0.036631415767832395	0.08822714519493655	-0.0062830363360482045	-0.017048022755928163	-0.012142944952637982	-0.00980323456725125	-0.012992580829011878	0.02549651991857317	0.005608348360239289	0.013561481342435356	0.016778614297061095	-0.0033066376835062003	-0.010759648900916172	-0.001708177858003797	-0.012154076220523363	-0.00048718969853247575	-0.0006127047811783112	0.0007205856906141126	-1.021451664482343e-12	-1.9322871999461598e-12
95	464.56443645	603.2634876	21.0	25	0	16	5.0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	1	0	...	-0.03376519609235181	0.019393594595332308	0.028799064546378282	0.08273611475199415	0.07290692895150927	-0.02227421704889874	0.164433043977812	0.012521471874281875	0.01635518782562264	-0.09719601806251912	0.321522192710887	0.9266859670666646	0.053778992501164254	0.015741494979075073	0.013104794738495752	-0.0032389010819291607	-0.009303275202500929	0.019910203687903014	-0.0013256680090554622	0.004903649342337187	-0.005155974111508614	-0.0033611007791492513	0.004015472841624888	-1.831022624311e-11	-3.4650518102286104e-11
96	480.2333067	603.2634876	21.0	25	3	13	7.0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.03701852738318585	0.0025285450605780227	0.05123755919857204	0.029446540276251962	0.02905584761511141	-0.027777260649174778	0.019469089709004173	-0.030684965844874585	-0.01716035715097257	-0.00010976677559208813	0.007517829718353757	0.0043038493885319325	0.024031631472570543	-0.001774888316911596	0.012054181464628095	-0.0031989844049226395	-0.00897511098863969	-0.007499254564152446	-0.00012452669332775853	-0.003971039637471647	0.0006126194486591168	-0.0007022969504712454	-0.00022654617129758784	1.6239600030188642e-12	3.0722373725810427e-12
97	416.3602296	603.2634876	21.0	25	1	14	6.0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	...	0.003659645872171321	-0.1108987643522605	0.060627589040711756	-0.030129832680246264	0.03672445838694542	0.05160613041495227	0.07931081646818805	0.019458129113045234	0.027184109292410927	0.0030288405770601753	-0.016927425891387363	-0.0031284897109152233	0.014046454863173564	-0.009502224742826134	-0.02341807001042025	-5.229081143893227e-05	0.0013015961603376088	-0.004387738382194295	-0.002023603764252411	-0.00018474157013696548	-2.8982084277439117e-05	-0.0009021886937620327	-0.00044242986556775374	2.4194001056932693e-12	4.5768116155530545e-12
98	423.1570224	603.2634876	21.0	25	2	18	4.0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	0.06406410569120104	0.0317927426432336	-0.12410842166020304	-0.7735709291609583	-0.26130524113072073	-0.3872803363881008	-0.029529264391786013	0.005595048566616991	-0.07662733555600809	0.4415771498876703	0.11355871864821093	0.01069942951219153	0.053369309337763	0.02788190307729485	-0.03468251307822731	0.0016707130668811919	-0.04606457041634784	0.004519604380332425	-0.01222465896278576	0.000472180949301555	-0.004368994548437792	-0.0017869858286736172	-0.00016416205018677534	1.986246592635765e-12	3.756426189294103e-12
99	457.13415415	603.2634876	21.0	25	2	14	3.0	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.047414704467353894	0.044349648712309696	-0.0029018680595119823	-0.013640326152784253	0.022236549499701386	0.045544503884491064	0.023944470176322553	0.004034011584863306	-0.008041097390678391	-0.00587256024608387	0.0024621686415475918	0.001424740857437639	-0.001625920018573836	-0.0026329810995889195	-0.0026689259419935447	-0.0019093007094792805	0.005889133320923832	-0.007814446442867372	-0.0032672891676502986	0.0008071229456056093	0.00041803975155164745	-5.066179192061553e-06	-7.502750996637078e-05	1.3998733604755255e-13	2.638310872770545e-13
100	433.75709155	603.2634876	21.0	25	2	19	5.0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	...	-0.35212595914117123	0.6663957844391095	-0.27322541967410147	-0.4037969384904917	-0.23810548341801	-0.20863589145061068	0.023877943646167386	0.020554817451712174	0.02681149207107368	-0.8262805937212061	-0.14595655503534863	-0.06528501249308948	-0.028293857436145334	0.4808101829722664	-0.11355371845268833	0.0072013306166490215	-0.20255786123491606	-0.003939946415383466	-0.04834939270325894	-0.006792792213135571	-0.010593134503787382	-0.0027374305182770037	0.0005933297360025191	6.074467405659749e-13	1.149955594381623e-12

We can verify the Gauss-Markov assumptions with our PCA model.

africa_pca.corr(
    columns =
        [
            "student_score",
            "col1",
            "col2",
            "col3",
            "col4",
            "col5",
            "col6",
            "col7",
        ],
)

Let’s use a cross-validation to test our linear regression model.

cross_validate(
    LinearRegression(),
    input_relation = africa_pca,
    X = africa_pca.get_columns(exclude_columns = ["student_score"]),
    y = "student_score",
)

	explained_variance	max_error	median_absolute_error	mean_absolute_error	mean_squared_error	root_mean_squared_error	r2	r2_adj	aic	bic	time
1-fold	0.4050283580653491	309.5975513807107	41.770274529561064	51.35840950946751	55938.20660447109	65.93962875689486	0.4049841420550828	0.3850080894715151	65042.126054857654	66306.1559023096	2.4776298999786377
2-fold	0.40432552897486995	350.2654719034392	40.689067229790766	51.15568327098015	54646.0161634645	66.16116738953453	0.4043044923247504	0.3843021248057493	64893.05078866574	66157.04341820814	2.244778871536255
3-fold	0.3884946697531727	320.53434146119037	42.60785984472052	52.14689981929701	50581.979841890185	66.90413087544111	0.3884620884583734	0.36797791556653947	64587.81828186497	65852.3313152073	2.1981658935546875
avg	0.39928285226446397	326.79912158178007	41.68906720135745	51.55366419991489	53722.067536608585	66.33497567395683	0.3992502409460689	0.379096043281268	64840.99837512946	66105.17687857502	2.30685822168986
std	0.007633791264516083	17.18343276492881	0.7854456261226151	0.42756742705728124	2282.1849299664073	0.4124908677537966	0.0076334202256617585	0.007866984574286298	189.08724347349425	188.8681946310379	0.12224406463642548

As you can see, we’ve created a much more accurate model here than in our first attempt. This example emphasizes the importance of filtering noise from the data.

Conclusion

We’ve seen two techniques that can help us create powerful linear regression models. While the first method normalized the data and looked for correlations, the second method applied a PCA model. The second one allows us to confirm the Gauss-Markov assumptions - an essential part of using linear models.