What Is Stationarity?

A drifting mean leaves no single value to forecast; a growing variance widens intervals without bound. Differencing removes a stochastic trend; a log stabilizes the variance.

Unit Roots and Differencing

AR(1): \(y_t = \phi_1\,y_{t-1} + \varepsilon_t\). Give it one unit shock and watch what is left of it:

Below one, each period keeps the same share \(\phi_1\), so the shock halves every \(\ln 0.5 / \ln \phi_1\) periods. At \(\phi_1 = 1\), a unit root, it never fades: a random walk. Differencing, \(\Delta y_t = y_t - y_{t-1}\), removes it. \(d\) counts first differences; \(D\) counts seasonal ones, \(\Delta_{12}y_t = y_t - y_{t-12}\).

Testing for a Unit Root

Rule of thumb: a visible trend or growing variance means the series is likely non-stationary. Then test formally.

ADF (\(H_0\): unit root) and KPSS (\(H_0\): stationary) test opposite nulls, so run both, and be suspicious when they disagree.

Lab 1’s TX_1 HOUSEHOLD: ADF −1.38 on the levels (a unit root), −8.51 after one difference (stationary). So \(d = 1\).

Why the Rule Exists: Spurious Regression

Differencing is not bookkeeping. Skip it and the statistics actively lie to you.

Two independent random walks — no shared cause, no relationship at all. Their changes correlate at −0.13, which is the truth. But their levels correlate at +0.62, and that is what OLS sees.

ACF and PACF: Model Identification

The autocorrelation function, \(\rho_k = \mathrm{Corr}(y_t, y_{t-k})\), measures how \(y_t\) relates to its own past. The partial autocorrelation controls for the lags in between, which separates a direct effect from one inherited through them.

Read together, they identify the model:

What Those Patterns Actually Look Like

Exact values, not a sample. Read the pair: the sharp cut-off gives the order, the slow decay says which side it is on. The AR(1) ACF is a staircase: each lag keeps \(\phi\) of the one before.

Difference First, Then Read the ACF

Lab 1’s TX_1 HOUSEHOLD series, before and after one difference:

The levels’ ACF crawls down whatever the process is: it only says you have not differenced yet. The change has one negative spike at lag 1, the MA(1) signature. So ARIMA(0,1,1), which is simple exponential smoothing: Lab 1’s \(\alpha = 0.62\) implies \(\rho_1 = -0.33\); the data show \(-0.29\).

ARIMA(\(p\),\(d\),\(q\))

Familiar models are corners of this space: ARIMA(0,1,0) is a random walk, ARIMA(1,0,0) an AR(1), and ARIMA(0,1,1) simple exponential smoothing.

A forecast by hand. ARIMA(1,1,0) with \(\hat{\phi}_1 = 0.5\), \(y_{100} = 120\) and \(y_{99} = 118\). The last change was 2, so the next is half of it: \(\hat{y}_{101|100} = 120 + 0.5 \times 2 = \mathbf{121}\).

Python: pmdarima.auto_arima() searches orders; statsmodels.SARIMAX fits the order you choose.

SARIMA: Adding Seasonal Terms

Seasonal data also correlate at lags \(m, 2m, 3m, \ldots\), and SARIMA adds AR and MA terms there: two ARIMA models at different time scales, \((p,d,q)\) for the short run and \((P,D,Q)[m]\) for the season.

CA_1 FOODS is daily, so \(m = 7\) here (monthly data use \(m = 12\)). One seasonal difference, \(D = 1\), removes the peaks; the spike left at lag 7 points to one seasonal MA term, \(Q = 1\).

SARIMA in Practice: A Four-Step Workflow

Practical workflow:

1. ADF / KPSS → choose \(d\)
2. Seasonal unit root test → choose \(D\)
3. ACF / PACF on the differenced series → guide \(p, q, P, Q\)
4. Minimize AIC, then check the residual ACF with a Ljung-Box test

Ljung-Box (\(H_0\): the residuals are white noise) is the one people skip. Here a large \(p\)-value is the good outcome: nothing left to model.

Monthly retail: start from SARIMA(1,1,1)(0,1,1)[12] and make anything fancier beat it.

Why Use Multiple Series?

Every model so far — ETS, ARIMA — uses only \(y_t\)’s own history. But business variables are linked: consumer sentiment today moves retail spending next month, and advertising moves sales over the following weeks.

Granger causality (Granger 1969) is the formal test of exactly that claim. It comes right after the VAR.

VAR or ARIMAX: Which Way Does Influence Run?

The modeling choice turns on whether the influence runs one way or both:

The VAR(\(p\)) Model

For two stationary series and \(p = 1\), each equation regresses on the lags of both:

\[\begin{cases} y_{1t} = c_1 + a_{11}\,y_{1,t-1} + a_{12}\,y_{2,t-1} + \varepsilon_{1t}\\ y_{2t} = c_2 + a_{21}\,y_{1,t-1} + a_{22}\,y_{2,t-1} + \varepsilon_{2t} \end{cases}\]

The cross terms carry the value: \(a_{12}\) is how much \(y_{2,t-1}\) predicts \(y_{1t}\), what a univariate AR throws away. If \(a_{12} = a_{21} = 0\), it is two separate ARs.

Python: statsmodels.tsa.api.VAR; var.fit(maxlags=p, ic='bic') picks \(p\) by BIC.

Granger Causality Test

The test: fit the \(y\)-equation with and without the \(x\)-lags; an \(F\)-test of \(H_0\): all \(x\)-lag coefficients are zero.

Not structural causality. It tests predictive content: umbrella sales Granger-cause rain, because shoppers read the forecast. Say “predictive of,” never “causes.”

Granger on Real Data: The COVID Window

Does unemployment Granger-cause retail sales? Lab 2’s monthly FRED data:

Both directions are significant in both samples; one shock decides which looks stronger. With COVID: unemployment → retail. Without its 17 months: retail → unemployment.

ARIMAX: Exogenous Variables with ARIMA Errors

When one variable clearly drives another, use ARIMAX.

Use it when the direction is clear (advertising → sales), you have \(x_t\) or can forecast it, and adding it cuts cross-validated error. Check that last one.

The catch: forecasting \(y_{t+h}\) needs \(x_{t+h}\), a forecast of your own predictor. Without one, use a VAR, which forecasts every series it contains.

Python: SARIMAX(y, exog=X, order=(p,d,q)).fit()

Lecture 2: Key Takeaways

Next: Generalized Additive Models (GAMs) — flexible, interpretable nonlinear forecasting with Prophet and pyGAM.