Booking

This example uses the expedia dataset to predict, based on site activity, whether a user is likely to make a booking. You can download the dataset here.

• cnt: Number of similar events in the context of the same user session.

• user_location_city: The ID of the city in which the customer is located.

• is_package: 1 if the click/booking was generated as a part of a package (i.e. combined with a flight), 0 otherwise.

• user_id: ID of the user.

• srch_children_cnt: The number of (extra occupancy) children specified in the hotel room.

• channel: marketing ID of a marketing channel.

• hotel_cluster: ID of a hotel cluster.

• srch_destination_id: ID of the destination where the hotel search was performed.

• is_mobile: 1 if the user is on a mobile device, 0 otherwise.

• srch_adults_cnt: The number of adults specified in the hotel room.

• user_location_country: The ID of the country in which the customer is located.

• srch_destination_type_id: ID of the destination where the hotel search was performed.

• srch_rm_cnt: The number of hotel rooms specified in the search.

• posa_continent: ID of the continent associated with the site_name.

• srch_ci: Check-in date.

• user_location_region: The ID of the region in which the customer is located.

• hotel_country: Hotel’s country.

• srch_co: Check-out date.

• is_booking: 1 if a booking, 0 if a click.

• orig_destination_distance: Physical distance between a hotel and a customer at the time of search. A null means the distance could not be calculated.

• hotel_continent: Hotel continent.

• site_name: ID of the Expedia point of sale (i.e. Expedia.com, Expedia.co.uk, Expedia.co.jp, …).

We will follow the data science cycle (Data Exploration - Data Preparation - Data Modeling - Model Evaluation - Model Deployment) to solve this problem.

Initialization

This example uses the following version of vastorbit:

import vastorbit as vo

vo.__version__

Connect to VAST. This example uses an existing connection called VASTDSN. For details on how to create a connection, see the Connection tutorial. You can skip the below cell if you already have an established connection.

vo.connect("VASTDSN")

Let’s create a VastFrame of the dataset.

expedia = vo.read_csv("expedia.csv", parse_nrows = 1000)
expedia

	📅 date_time Timestamp(3) 100%	123 site_name Integer 100%	123 posa_continent Integer 100%	123 user_location_country Integer 100%	123 user_location_region Integer 100%	123 user_location_city Integer 100%	123 orig_destination_distance Double 37%	123 user_id Integer 100%	123 is_mobile Integer 100%	123 is_package Integer 100%	123 channel Integer 100%	📅 srch_ci Date 99%	📅 srch_co Date 99%	123 srch_adults_cnt Integer 100%	123 srch_children_cnt Integer 100%	123 srch_rm_cnt Integer 100%	123 srch_destination_id Integer 100%	123 srch_destination_type_id Integer 100%	123 is_booking Integer 100%	123 cnt Integer 100%	123 hotel_continent Integer 100%	123 hotel_country Integer 100%	123 hotel_market Integer 100%	123 hotel_cluster Integer 100%
1	2014-09-30 17:23:11	24	2	3	50	5703	[null]	2854	0	0	0	2014-10-06	2014-10-14	1	0	1	8803	1	0	1	3	151	69	58
2	2014-09-30 17:27:05	28	1	68	335	22273	[null]	2901	0	0	5	2015-01-16	2015-01-17	1	0	5	22074	6	0	1	6	204	1463	82
3	2014-09-30 17:30:46	23	1	1	365	19156	132.1213	4143	0	0	9	2014-10-10	2014-10-12	2	0	1	14039	6	0	1	6	105	35	8
4	2014-09-30 17:37:12	37	1	69	585	40970	[null]	1832	1	1	9	2014-10-20	2014-10-24	1	0	1	21958	6	0	1	6	68	275	68
5	2014-09-30 17:43:00	24	2	3	64	3169	[null]	3195	0	0	5	2014-10-01	2014-10-05	1	0	1	12209	6	0	2	6	70	19	77
6	2014-09-30 18:05:02	37	1	69	761	41949	[null]	3045	0	0	9	2014-10-01	2014-10-02	1	0	1	22799	6	0	1	6	77	20	5
7	2014-09-30 18:45:41	2	3	66	462	49272	874.7298	258	1	1	9	2015-05-28	2015-05-31	1	0	1	8250	1	0	1	2	50	628	1
8	2014-09-30 18:47:08	34	3	205	354	12328	137.6445	3466	1	0	9	2014-10-17	2014-10-19	4	1	1	41556	1	0	2	2	198	395	42
9	2014-09-30 18:51:20	34	3	205	354	12328	141.1502	3466	1	0	9	2014-10-17	2014-10-19	4	1	1	41556	1	0	1	2	198	395	91
10	2014-09-30 19:05:14	2	3	66	258	16835	1994.997	3618	0	1	9	2014-12-26	2015-01-02	2	2	1	11439	1	0	1	4	163	1503	65
11	2014-09-30 19:06:14	2	3	66	258	16835	1995.603	3618	0	1	9	2014-12-26	2015-01-02	2	2	1	11439	1	0	4	4	163	1503	65
12	2014-09-30 19:11:52	2	3	66	258	16835	1990.3352	3618	0	1	9	2014-12-27	2015-01-03	2	2	1	11439	1	0	1	4	163	1503	52
13	2014-09-30 19:12:49	2	3	66	258	16835	1990.6724	3618	0	1	9	2014-12-27	2015-01-03	2	2	1	11439	1	0	1	4	163	1503	52
14	2014-09-30 19:13:48	2	3	66	258	16835	1993.3471	3618	0	1	9	2014-12-27	2015-01-03	2	2	1	11439	1	0	1	4	163	1503	87
15	2014-09-30 19:33:21	2	3	66	174	24103	5447.5333	718	0	1	9	2014-12-24	2015-01-01	3	0	1	8253	1	0	2	6	70	19	95
16	2014-09-30 19:43:15	2	3	66	189	55926	45.9667	1674	1	0	9	2014-12-20	2014-12-21	1	2	1	8267	1	0	1	2	50	675	69
17	2014-09-30 19:46:20	37	1	69	558	33389	[null]	2667	1	0	6	2014-11-28	2014-11-30	4	0	2	12245	6	0	2	6	204	27	11
18	2014-09-30 20:41:30	24	2	3	50	5703	[null]	1305	0	1	2	2014-12-29	2015-01-02	1	0	1	8220	1	0	2	3	182	46	58
19	2014-09-30 20:49:05	2	3	66	174	49258	1899.2891	2089	0	0	9	2014-10-11	2014-10-12	2	0	1	8216	1	1	1	2	50	350	11
20	2014-09-30 20:53:02	2	3	66	174	47287	5935.4694	3460	0	0	2	2014-10-20	2014-10-22	1	0	1	8293	1	0	1	6	46	39	43

Warning

This example uses a sample dataset. For the full analysis, you should consider using the complete dataset.

Data Exploration and Preparation

Sessionization is the process of gathering clicks for a certain period of time. We usually consider that after 30 minutes of inactivity, the user session ends (date_time - lag(date_time) > 30 minutes). For these kinds of use cases, aggregating sessions with meaningful statistics is the key for making accurate predictions.

We start by using the sessionize() method to create the variable session_id. We can then use this variable to aggregate the data.

expedia.sessionize(
    ts = "date_time",
    by = ["user_id"],
    session_threshold = "30 minutes",
    name = "session_id",
)

	📅 date_time Timestamp(3) 100%	123 site_name Integer 100%	123 posa_continent Integer 100%	123 user_location_country Integer 100%	123 user_location_region Integer 100%	123 user_location_city Integer 100%	123 orig_destination_distance Double 37%	123 user_id Integer 100%	123 is_mobile Integer 100%	123 is_package Integer 100%	123 channel Integer 100%	📅 srch_ci Date 99%	📅 srch_co Date 99%	123 srch_adults_cnt Integer 100%	123 srch_children_cnt Integer 100%	123 srch_rm_cnt Integer 100%	123 srch_destination_id Integer 100%	123 srch_destination_type_id Integer 100%	123 is_booking Integer 100%	123 cnt Integer 100%	123 hotel_continent Integer 100%	123 hotel_country Integer 100%	123 hotel_market Integer 100%	123 hotel_cluster Integer 100%	123 session_id Bigint 100%
1	2014-02-08 19:20:31	2	3	66	442	30104	1658.1959	8	1	1	0	2014-03-16	2014-03-19	1	0	1	12273	6	0	1	2	50	663	39	1
2	2014-03-26 10:39:48	2	3	66	442	30104	450.4073	8	0	1	9	2014-04-21	2014-04-24	2	0	1	8266	1	0	1	2	50	411	48	2
3	2014-03-26 11:53:51	2	3	66	442	30104	453.5987	8	0	1	9	2014-04-29	2014-05-02	2	0	1	8266	1	0	1	2	50	411	10	3
4	2014-03-26 11:56:10	2	3	66	442	30104	454.1026	8	0	1	9	2014-04-29	2014-05-02	2	0	1	8266	1	0	1	2	50	411	18	3
5	2014-03-26 12:01:10	2	3	66	442	30104	453.5987	8	0	1	9	2014-04-28	2014-05-01	2	0	1	8266	1	0	1	2	50	411	10	3
6	2014-03-26 12:04:01	2	3	66	442	30104	454.1026	8	0	1	9	2014-04-28	2014-05-01	2	0	1	8266	1	0	1	2	50	411	18	3
7	2014-03-26 12:14:31	2	3	66	442	30104	454.1026	8	0	1	9	2014-04-13	2014-04-16	2	0	1	8266	1	0	1	2	50	411	18	3
8	2014-03-26 12:19:10	2	3	66	442	30104	453.7973	8	0	1	9	2014-04-13	2014-04-16	2	0	1	8266	1	0	1	2	50	411	4	3
9	2014-04-06 23:04:59	2	3	66	442	30104	453.49	8	0	1	5	2014-04-21	2014-04-22	2	0	1	8266	1	0	1	2	50	411	72	4
10	2014-04-06 23:13:49	2	3	66	442	30104	453.343	8	0	1	5	2014-04-21	2014-04-22	2	0	1	8266	1	0	4	2	50	411	97	4
11	2014-04-19 12:30:50	2	3	66	442	30104	1318.2732	8	0	1	9	2014-05-05	2014-05-07	1	0	1	11569	1	0	1	2	50	623	89	5
12	2014-04-19 13:11:53	2	3	66	442	30104	1318.1185	8	0	1	9	2014-05-12	2014-05-14	1	0	1	11569	1	0	1	2	50	623	79	6
13	2014-04-21 09:53:49	2	3	66	442	30104	1318.1185	8	0	1	9	2014-05-12	2014-05-15	1	0	1	11569	1	0	1	2	50	623	79	7
14	2014-08-13 20:31:04	2	3	66	337	6085	1069.0775	8	0	1	9	2014-10-05	2014-10-07	2	0	1	11353	1	0	1	2	50	699	59	8
15	2014-08-13 20:53:48	2	3	66	337	6085	1066.7572	8	0	1	9	2014-10-05	2014-10-07	2	0	1	11353	1	0	1	2	50	699	10	8
16	2014-08-16 21:34:33	2	3	66	337	6085	7.4843	8	0	0	9	2014-08-20	2014-08-27	2	0	1	8267	1	0	1	2	50	674	23	9
17	2014-08-16 21:35:14	2	3	66	337	6085	7.4843	8	0	0	9	2014-08-20	2014-09-03	2	0	1	8267	1	0	1	2	50	674	23	9
18	2014-08-16 21:43:25	2	3	66	337	6085	16.7311	8	0	0	9	2014-08-20	2014-08-27	2	0	1	8267	1	0	1	2	50	675	23	9
19	2014-08-18 06:11:31	2	3	66	337	6085	7.4843	8	0	0	9	2014-08-19	2014-08-26	2	0	1	8267	1	0	1	2	50	674	23	10
20	2014-09-11 21:37:22	2	3	66	337	1794	20.6983	8	0	0	2	2014-09-12	2014-09-14	2	0	1	8267	1	0	1	2	50	676	76	11

The duration of the trip should also influence/be indicative of the user’s behavior on the site, so we’ll take that into account.

expedia["trip_duration"] = expedia["srch_co"] - expedia["srch_ci"]

If a user looks at the same hotel several times, then it might mean that they’re looking to book that hotel during the session.

expedia.analytic(
    "mode",
    columns = "hotel_cluster",
    by = [
        "user_id",
        "session_id",
    ],
    name = "mode_hotel_cluster",
    add_count = True,
)

	📅 date_time Timestamp(3) 100%	123 site_name Integer 100%	123 posa_continent Integer 100%	123 user_location_country Integer 100%	123 user_location_region Integer 100%	123 user_location_city Integer 100%	123 orig_destination_distance Double 37%	123 user_id Integer 100%	123 is_mobile Integer 100%	123 is_package Integer 100%	123 channel Integer 100%	📅 srch_ci Date 99%	📅 srch_co Date 99%	123 srch_adults_cnt Integer 100%	123 srch_children_cnt Integer 100%	123 srch_rm_cnt Integer 100%	123 srch_destination_id Integer 100%	123 srch_destination_type_id Integer 100%	123 is_booking Integer 100%	123 cnt Integer 100%	123 hotel_continent Integer 100%	123 hotel_country Integer 100%	123 hotel_market Integer 100%	123 hotel_cluster Integer 100%	123 session_id Bigint 100%	📅 trip_duration Interval day to second 99%	123 mode_hotel_cluster Integer 100%	123 mode_hotel_cluster_count Bigint 100%
1	2014-08-27 10:03:53	2	3	66	318	31646	5380.6319	7	0	1	2	2015-01-30	2015-02-05	2	0	1	792	1	0	1	6	208	1480	64	1	6 days, 0:00:00	64	64
2	2014-08-27 10:01:09	2	3	66	318	31646	5380.6319	7	0	0	2	2014-10-31	2014-11-01	2	0	1	792	1	0	1	6	208	1480	64	1	1 day, 0:00:00	64	64
3	2014-08-27 09:58:02	2	3	66	318	31646	5378.6918	7	0	0	2	2015-01-30	2015-02-05	2	0	1	792	1	0	2	6	208	1480	64	1	6 days, 0:00:00	64	64
4	2014-08-27 09:51:32	2	3	66	318	31646	5378.6918	7	0	0	2	2015-01-29	2015-02-08	2	0	1	792	1	0	2	6	208	1480	64	1	10 days, 0:00:00	64	64
5	2014-08-27 09:45:42	2	3	66	318	31646	5378.6918	7	0	0	2	2014-10-31	2014-11-01	2	0	1	792	1	0	5	6	208	1480	64	1	1 day, 0:00:00	64	64
6	2014-08-27 09:58:55	2	3	66	318	31646	5375.9045	7	0	0	2	2014-10-31	2014-11-01	2	0	1	792	1	0	1	6	208	1480	86	1	1 day, 0:00:00	64	64
7	2014-09-10 12:20:39	2	3	66	318	31646	3936.1062	7	0	0	2	2014-10-12	2014-10-13	1	0	1	22616	6	0	1	6	204	1452	5	2	1 day, 0:00:00	5	5
8	2014-09-24 11:57:49	2	3	66	318	878	830.2591	7	0	1	9	2014-12-04	2014-12-07	2	0	1	19535	1	0	1	2	50	593	5	3	3 days, 0:00:00	5	5
9	2014-09-24 11:36:08	2	3	66	318	878	830.2591	7	0	0	9	2014-12-04	2014-12-07	2	0	1	19535	1	1	1	2	50	593	5	3	3 days, 0:00:00	5	5
10	2014-09-24 11:23:59	2	3	66	318	878	830.2591	7	0	0	9	2014-12-04	2014-12-07	2	0	1	19535	1	0	2	2	50	593	5	3	3 days, 0:00:00	5	5
11	2014-10-06 18:28:52	2	3	66	318	33705	826.9443	7	0	0	0	2014-12-04	2014-12-07	2	0	1	19535	1	0	1	2	50	593	91	4	3 days, 0:00:00	91	91
12	2014-10-06 18:32:36	2	3	66	318	33705	826.9994	7	0	0	0	2014-12-04	2014-12-07	2	0	1	19535	1	0	1	2	50	593	5	4	3 days, 0:00:00	91	91
13	2014-10-29 05:57:44	2	3	66	318	8031	862.7599	7	0	0	2	2014-11-11	2014-11-12	2	0	1	19535	1	0	2	2	50	593	32	5	1 day, 0:00:00	32	32
14	2014-10-29 05:58:44	2	3	66	318	8031	862.7337	7	0	0	2	2014-11-11	2014-11-12	2	0	1	19535	1	0	1	2	50	593	7	5	1 day, 0:00:00	32	32
15	2014-10-31 07:14:03	2	3	66	318	8031	863.0788	7	0	1	4	2014-12-04	2014-12-07	2	0	1	19535	1	0	6	2	50	593	90	6	3 days, 0:00:00	90	90
16	2014-11-06 10:14:59	2	3	66	318	41478	[null]	7	0	1	2	2014-12-04	2014-12-07	2	0	1	19535	1	0	1	2	50	593	32	7	3 days, 0:00:00	32	32
17	2014-11-06 10:44:50	2	3	66	318	41478	[null]	7	0	1	2	2014-12-04	2014-12-07	2	0	1	19535	1	0	1	2	50	593	28	7	3 days, 0:00:00	32	32
18	2014-11-06 10:45:59	2	3	66	318	41478	[null]	7	0	1	2	2014-12-04	2014-12-07	2	0	1	19535	1	0	1	2	50	593	42	7	3 days, 0:00:00	32	32
19	2014-11-06 10:47:25	2	3	66	318	41478	[null]	7	0	1	2	2014-12-04	2014-12-07	2	0	1	19535	1	0	1	2	50	593	76	7	3 days, 0:00:00	32	32
20	2014-11-06 11:27:14	2	3	66	318	41478	[null]	7	0	1	2	2014-12-04	2014-12-07	2	0	1	19535	1	0	1	2	50	593	33	8	3 days, 0:00:00	33	33

We can now aggregate the session and get some useful statistics out of it:

• end_session_date_time: Date and time when the session ends.

• session_duration: Session duration.

• is_booking: 1 if the user booked during the session, 0 otherwise.

• trip_duration: Trip duration.

• orig_destination_distance: Average of the physical distances between the hotels and the customer.

• srch_family_cnt: The number of people specified in the hotel room.

import vastorbit.sql.functions as fun

expedia = expedia.groupby(
    columns = [
        "user_id",
        "session_id",
        "mode_hotel_cluster_count",
    ],
    expr = [
        fun.max(expedia["date_time"])._as("end_session_date_time"),
        "EXTRACT(SECOND FROM MAX(date_time) - MIN(date_time)) AS session_duration",
        fun.max(expedia["is_booking"])._as("is_booking"),
        "EXTRACT(DAY FROM AVG(trip_duration)) AS trip_duration",
        fun.avg(expedia["orig_destination_distance"])._as("avg_distance"),
        fun.sum(expedia["cnt"])._as("nb_click_session"),
        fun.median(expedia["srch_children_cnt"] + expedia["srch_adults_cnt"])._as("srch_family_cnt"),
    ],
)
expedia

	123 user_id Integer 100%	123 session_id Bigint 100%	123 mode_hotel_cluster_count Integer 100%	📅 end_session_date_time Timestamp(3) 100%	123 session_duration Bigint 100%	123 is_booking Integer 100%	123 trip_duration Bigint 99%	123 avg_distance Double 38%	123 nb_click_session Bigint 100%	123 srch_family_cnt Bigint 100%
1	9	1	70	2014-01-26 23:45:07	16	0	5	2353.9244999999996	9	1
2	9	2	87	2014-01-27 00:36:29	0	1	4	2354.1447	1	1
3	15	1	40	2014-09-12 23:11:45	0	0	3	2366.3264	2	2
4	15	2	54	2014-11-14 12:50:33	0	0	2	190.9165	1	2
5	15	3	42	2014-11-17 14:04:53	0	0	2	364.5643	1	2
6	15	4	54	2014-11-18 10:38:06	0	0	2	184.2075	1	2
7	15	5	54	2014-11-19 09:25:32	0	0	2	190.9165	2	2
8	15	6	49	2014-11-21 07:57:35	0	0	4	7146.5761	1	1
9	15	7	54	2014-11-21 14:28:19	32	0	3	303.6503	5	2
10	15	8	54	2014-11-24 14:30:18	24	0	2	148.4997666666667	7	2
11	15	9	54	2014-11-24 16:43:20	56	0	2	184.71529999999998	3	2
12	15	10	54	2014-12-03 23:42:48	10	0	2	185.0873	2	2
13	15	11	33	2014-12-08 22:49:04	0	0	1	77.5291	1	2
14	15	12	54	2014-12-10 22:13:11	17	1	3	187.3865	4	2
15	15	13	48	2014-12-12 11:17:20	0	0	1	82.6471	2	2
16	15	14	91	2014-12-15 10:35:47	0	0	3	748.0281	1	2
17	15	15	54	2014-12-15 20:38:21	0	0	2	184.2075	1	2
18	15	16	72	2014-12-17 11:54:40	9	0	1	365.225475	4	2
19	15	17	54	2014-12-20 11:40:25	0	0	3	187.3865	1	2
20	26	1	59	2014-08-07 08:28:48	12	1	1	13.387300000000002	3	2

Let’s look at the missing values.

expedia.count_percent()

	count	percent
"user_id"	46987.0	100.0
"session_id"	46987.0	100.0
"mode_hotel_cluster_count"	46987.0	100.0
"end_session_date_time"	46987.0	100.0
"session_duration"	46987.0	100.0
"is_booking"	46987.0	100.0
"nb_click_session"	46987.0	100.0
"srch_family_cnt"	46987.0	100.0
"trip_duration"	46872.0	99.755
"avg_distance"	18219.0	38.775

Let’s impute the missing values for avg_distance and trip_duration.

expedia["avg_distance" ].fillna(method = "avg")
expedia["trip_duration"].fillna(method = "avg")

	123 user_id Integer 100%	123 session_id Bigint 100%	123 mode_hotel_cluster_count Integer 100%	📅 end_session_date_time Timestamp(3) 100%	123 session_duration Bigint 100%	123 is_booking Integer 100%	123 trip_duration Real 100%	123 avg_distance Real 100%	123 nb_click_session Bigint 100%	123 srch_family_cnt Bigint 100%
1	13	1	64	2014-07-09 12:04:53	47	1	4.0	1979.8650360133345	8	1
2	13	2	64	2014-07-10 05:42:46	0	0	6.0	1979.8650360133345	7	1
3	13	3	64	2014-07-11 06:55:20	21	0	6.0	1979.8650360133345	12	1
4	13	4	64	2014-07-11 07:40:33	0	0	6.0	1979.8650360133345	1	1
5	13	5	64	2014-07-14 09:36:15	52	1	2.0	1979.8650360133345	3	2
6	13	6	64	2014-07-15 09:01:43	0	0	3.0	1979.8650360133345	2	2
7	13	7	8	2014-07-16 07:02:12	0	0	0.0	1979.8650360133345	1	2
8	13	8	64	2014-07-17 13:33:44	36	0	2.0	1979.8650360133345	5	2
9	13	9	64	2014-07-17 20:53:38	0	0	3.0	1979.8650360133345	1	2
10	13	10	64	2014-07-17 22:16:18	0	0	3.0	1979.8650360133345	1	2
11	13	11	64	2014-07-20 15:27:20	0	0	3.0	1979.8650360133345	1	2
12	13	12	64	2014-07-21 07:26:26	0	0	3.0	1979.8650360133345	1	2
13	13	13	64	2014-07-22 06:47:22	9	0	3.0	1979.8650360133345	2	2
14	13	14	46	2014-08-18 13:14:50	45	0	4.0	4880.027433333333	14	1
15	13	15	82	2014-08-18 15:05:22	18	0	3.0	4881.4641	6	2
16	13	16	82	2014-08-19 10:23:12	48	0	3.0	4880.41435	10	2
17	13	17	58	2014-08-19 14:14:09	15	0	3.0	4925.1201	2	2
18	13	18	25	2014-08-21 04:07:52	50	0	2.0	1979.8650360133345	10	2
19	13	19	5	2014-08-23 04:16:45	22	1	5.0	1979.8650360133345	25	2
20	13	20	46	2014-08-23 04:50:09	0	0	5.0	1979.8650360133345	1	2

We can then look at the links between the variables. We will use Spearman’s rank correleation coefficient to get all the monotonic relationships.

expedia.corr(method = "spearman")

We can see huge links between some of the variables (mode_hotel_cluster_count and session_duration) and our response variable (is_booking). A logistic regression would work well in this case because the response and predictors have a monotonic relationship.

Machine Learning

Let’s create our LogisticRegression model.

from vastorbit.machine_learning.vast import LogisticRegression

model_logit = LogisticRegression(
    max_iter = 1000,
)
model_logit.fit(
    expedia,
    [
        "avg_distance",
        "session_duration",
        "nb_click_session",
        "mode_hotel_cluster_count",
        "session_id",
        "srch_family_cnt",
        "trip_duration",
    ],
    "is_booking",
)

None of our coefficients are rejected (pvalue = 0). Let’s look at their importance.

model_logit.features_importance()

It looks like there are two main predictors: mode_hotel_cluster_count and trip_duration. According to our model, users likely to make a booking during a particular session will tend to:

• look at the same hotel many times.

• look for a shorter trip duration.

• not click as much (spend more time at the same web page).

Let’s add our prediction to the VastFrame.

model_logit.predict_proba(
    expedia,
    name = "booking_prob_logit",
    pos_label = 1,
)

	123 user_id Integer 100%	123 session_id Bigint 100%	123 mode_hotel_cluster_count Integer 100%	📅 end_session_date_time Timestamp(3) 100%	123 session_duration Bigint 100%	123 is_booking Integer 100%	123 trip_duration Real 100%	123 avg_distance Real 100%	123 nb_click_session Bigint 100%	123 srch_family_cnt Bigint 100%	123 booking_prob_logit Double 100%
1	7	1	64	2014-08-27 10:03:53	11	0	4.0	5378.87395	12	2	0.16251318890936553
2	7	2	5	2014-09-10 12:20:39	0	0	1.0	3936.1062	1	1	0.2070044481824488
3	7	3	5	2014-09-24 11:57:49	50	1	3.0	830.2590999999999	4	2	0.38692915542922157
4	7	4	91	2014-10-06 18:32:36	44	0	3.0	826.97185	2	2	0.30547100062432875
5	7	5	32	2014-10-29 05:58:44	0	0	1.0	862.7468	3	2	0.1923126340027882
6	7	6	90	2014-10-31 07:14:03	0	0	3.0	863.0788	6	2	0.13575038197798783
7	7	7	32	2014-11-06 10:47:25	26	0	3.0	1979.8650360133345	4	2	0.2395448571436353
8	7	8	33	2014-11-06 11:27:47	33	0	3.0	1979.8650360133345	4	2	0.2716918969593289
9	7	9	73	2014-11-07 11:03:11	6	0	3.0	1979.8650360133345	3	2	0.14598914360410956
10	7	10	32	2014-11-10 16:27:13	0	0	5.0	1979.8650360133345	1	2	0.09530501381912733
11	7	11	77	2014-11-12 11:40:28	43	0	1.0	1979.8650360133345	2	2	0.3749680936647942
12	7	12	20	2014-11-20 07:56:48	6	0	7.0	5457.39715	2	2	0.07629242631229817
13	7	13	50	2014-11-20 09:02:53	0	0	3.0	866.3349	1	2	0.12846550135449503
14	7	14	32	2014-12-01 15:07:49	0	0	3.0	862.3329	1	2	0.13179467441684575
15	7	15	91	2014-12-18 11:04:14	27	0	2.0	616.5975000000001	2	2	0.2467497435557924
16	7	16	6	2014-12-18 12:16:29	0	0	2.0	628.2538	1	2	0.1598834412013379
17	7	17	6	2014-12-18 13:19:09	0	0	2.0	628.2538	1	2	0.15923153625324338
18	14	1	41	2014-03-16 16:43:38	45	1	2.0	181.06066	19	2	0.4665229486982773
19	14	2	45	2014-04-27 19:44:21	57	0	2.0	2266.2161	7	1	0.4970769412396686
20	14	3	55	2014-05-27 18:24:42	13	0	2.0	2477.112775	4	2	0.20864304603563083

While analyzing the following boxplot (prediction partitioned by is_booking), we can notice that the cutoff is around 0.22 because most of the positive predictions have a probability between 0.23 and 0.5. Most of the negative predictions are between 0.05 and 0.2.

expedia["booking_prob_logit"].boxplot(by = "is_booking")

Let’s confirm our hypothesis by computing the best cutoff.

model_logit.score(metric = "best_cutoff")

Let’s look at the efficiency of our model with a cutoff of 0.22.

model_logit.report(cutoff = 0.22)

	value
auc	0.7086731231019248
prc_auc	0.7480492842247632
accuracy	0.6786770808947156
log_loss	0.47390987496962017
precision	0.3551469766402146
recall	0.6410127092999798
f1_score	0.4570627157652474
mcc	0.2770804435239417
informedness	0.32976193380298735
markedness	0.2328151442406683
csi	0.2962289656458304

ROC Curve:

model_logit.roc_curve()

We’re left with an excellent model. With this, we can predict whether a user will book a hotel during a specific session and make adjustments to our site accordingly. For example, to influence a user to make a booking, we could propose new hotels.

Conclusion

We’ve solved our problem in a pandas-like way, all without ever loading data into memory!