vastorbit.machine_learning.model_selection.statistical_tests.tsa.adfuller

vastorbit.machine_learning.model_selection.statistical_tests.tsa.adfuller(input_relation: Annotated[str | VastFrame, ''], column: str, ts: str, by: Annotated[str | list[str], 'STRING representing one column or a list of columns'] | None = None, p: int = 1, with_trend: bool = False, regresults: bool = False) → TableSample

Augmented Dickey Fuller test (Time Series stationarity).

Parameters:

• input_relation (SQLRelation) – Input relation.

• column (str) – Input VastColumn to test.

• ts (str) – VastColumn used as timeline to order the data. It can be a numerical or type date like (date, datetime, timestamp…) VastColumn.

• by (SQLColumns, optional) – VastColumns used in the partition.

• p (int, optional) – Number of lags to consider in the test.

• with_trend (bool, optional) – Adds a trend in the Regression.

• regresults (bool, optional) – If True, the full regression results are returned.

Returns:

result of the test.

Return type:

TableSample

Examples

Initialization

Let’s try this test on a dummy dataset that has the following elements:

• A value of interest

• Time-stamp data

Before we begin we can import the necessary libraries:

import vastorbit as vo
import numpy as np

Example 1: Trend

Now we can create the dummy dataset:

# Initialization
N = 100  # Number of Rows

# VastFrame
vdf = vo.VastFrame(
    {
        "year": list(range(N)),
        "X": [x + np.random.normal(0, 5) for x in range(N)],
    }
)

We can visually inspect the trend by drawing the appropriate graph:

vdf["X"].plot(ts="year")

Though the increasing trend is obvious, we can test its adfuller score by first importing the function:

from vastorbit.machine_learning.model_selection.statistical_tests import adfuller

And then simply applying it on the VastFrame:

adfuller(vdf, column="X", ts="year")

In the above context, the high p-value is evidence of lack of stationarity.

Note

A p_value in statistics represents the probability of obtaining results as extreme as, or more extreme than, the observed data, assuming the null hypothesis is true. A smaller p-value typically suggests stronger evidence against the null hypothesis i.e. the test data does not have a trend with respect to time in the current case.

However, small is a relative term. And the choice for the threshold value which determines a “small” should be made before analyzing the data.

Generally a p-value less than 0.05 is considered the threshold to reject the null hypothesis. But it is not always the case - read more

Example 2: Stationary

We can contrast the results with a dataset that has barely any trend:

vdf = vo.VastFrame(
    {
        "year": list(range(N)),
        "X": [np.random.normal(0, 5) for x in range(N)],
    }
)

We can visually inspect the absence of trend by drawing the appropriate graph:

vdf["X"].plot(ts="year")

Now we can perform the test on this dataset:

adfuller(vdf, column="X", ts="year")

Note

Notice the low p-value which proves that there is stationarity.

For more information, see Mann-Kendall Test.