import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import math
import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader
from sklearn.metrics import mean_squared_error
import warnings
warnings.filterwarnings('ignore')
from pathlib import Path
def data_path(name):
for base in (Path.cwd(), *Path.cwd().parents):
p = base / "data" / "processed" / name
if p.exists():
return p
raise FileNotFoundError(
f"Could not find data/processed/{name}. "
"Run `python scripts/prep_m5.py` from the repository root first."
)
RANDOM_STATE = 42
torch.manual_seed(RANDOM_STATE)
np.random.seed(RANDOM_STATE)
weekly = (pd.read_csv(data_path("m5_weekly.csv"), parse_dates=["week_start"])
.sort_values(["store_id", "cat_id", "week_start"]))
weekly["series"] = weekly.store_id + "_" + weekly.cat_id
# The four channels every window carries through time
CHANNELS = ["units", "snap_days", "event_days", "avg_price"]
CUTOFF = pd.Timestamp("2015-05-17") # same split as Homework 3 and 4
print(f"{weekly.series.nunique()} series | "
f"{weekly.week_start.min().date()} to {weekly.week_start.max().date()} | "
f"{len(weekly):,} rows")22 Homework 5: Deep Learning for Time Series
Due Week 12. Brings the five architectures together and sets them against every method in the course.
Your numbers will not match mine, and that is fine
Every result in this course depends on a random seed, and often also on your library versions, your hardware, and how many CPU threads your machine uses. Two people running identical, correct code will get different numbers — sometimes noticeably different.
What is graded is that you used the right model and the right procedure, and that your interpretation follows from the numbers you actually obtained. Quote your own output.
What is not graded is matching any particular value. Do not tune your way toward a number you saw on a slide or heard from a classmate — that is the one way to turn this into a wrong answer.
One caveat worth knowing. The broad ordering of models usually survives this variation. If yours comes out backwards from what the lecture found, say so in a sentence and offer a reason: either you have found something real about your run, or there is a bug worth hunting. Both are worth more marks than quietly reporting a number you do not believe.
22.1 Overview
This part continues Homework 5. See Part 1 for the full framing.
Learning objectives:
- Compare five architectures — FFN, CNN, RNN, LSTM and Transformer — on one held-out block and one metric
- Place those results beside every earlier homework in a single course-wide table
- Explain an ordering that contradicts the reputations of the methods, from the data rather than from the architecture
- Write a reflection that says which model you would ship, and why
22.2 Dataset and Setup
We construct fixed-length sliding windows over each store-category series. Each input is a window of past weeks; the target is the next week’s units. Part A (FFN, CNN) uses a 26-week window; Part B (RNN, LSTM, Transformer) uses 52 weeks, so the sequence models get a full annual cycle.
Building the windows. For each series, slide a window of length W along the history: the window is the input, the units in week W+1 is the target. A window belongs to the training set if its target week falls before the cutoff.
Two mistakes that will not raise an error
Do not shuffle. DataLoader(..., shuffle=False) for every model here. Shuffling destroys the ordering the sequence models exist to exploit, and it will not warn you.
Standardize using training statistics only. Compute the mean and standard deviation on the training windows, then apply them to the test windows. Computing them over the full dataset leaks the test period into training — the same error as scaling outside a CV fold in Homework 4.
22.3 Initial Prompt for Your AI Assistant
Any assistant works — ChatGPT, Claude, Gemini, or Copilot, in an editor or in a browser tab. Paste the block below as your first message, before you ask anything else.
Initial Prompt for Your AI Assistant
Paste the following as your first message to your assistant. Save it as INITIAL_PROMPT.md in your submission folder.
I am working on Homework 5 Part 3 for ECON 8310: Business Forecasting at the
University of Nebraska at Omaha.
Dataset: M5/Walmart — weekly aggregated, 30 store-category series, at
`data/processed/m5_weekly.csv`. Columns: store_id, cat_id, state_id,
week_start, units (target), avg_price, snap_days, event_days, and lagged
units. I build sliding windows over each series: 26 weeks for Part A and
52 weeks for Part B, with four channels (units, snap_days, event_days,
avg_price). Temporal split at 2015-05-17, matching Homework 3 and 4.
This homework covers four neural network architectures for time series
forecasting: FFN, 1D CNN, LSTM, and Transformer encoder (all in PyTorch).
I will ask you questions one at a time. For each:
1. Write clean, well-commented PyTorch code that runs end-to-end
2. Explain what the output means so that I understand it — but do NOT draft the written
answers I submit. The interpretation and business-recommendation sections are mine to
write, in my own words.
My prompt budget for Part 3 is 3 (this context prompt is excluded). I will label
each question before asking.
Repository rules — these apply to every prompt in this thread:
- If you cannot write files or run git — a browser chat cannot — then the file and commit
rules below are mine to carry out, not yours. Tell me what to write and what to commit,
and I will do it. Everything else in this list still applies to you.
- My submission folder is `submissions/hw05_part3_<my_name>/`. Put everything there.
- Keep a `PROMPT_LOG.md` in that folder. For every prompt I send, append a block:
### Prompt <n> — YYYY-MM-DD HH:MM
<my prompt, verbatim>
Numbered from 1, in order. Never edit or delete an earlier entry. If your tool cannot write
files, I will paste them in myself.
- After finishing the work for each prompt, make exactly one git commit:
`hw05_part3 prompt <prompt_id>: <short summary>`
- If a prompt would take me past my budget of 3, do not do the work. Reply exactly
PROMPT_LIMIT_REACHED, and still log that prompt.
- Never modify `AGENTS.md`, anything in `assignments/`, or anything in `policy/`.
Confirm by responding with: HW05P3_CONTEXT_ACK22.4 Part C: Comparison
How the code cells work
A cell that is empty except for a comment is where you write. The comment states the task; your code goes underneath it.
Some cells already contain setup code — a train/test split, a rename, a cutoff date — and end with a comment marking where your part begins. Leave the setup as it is and write below the comment. It is there so that everyone’s results are computed on the same basis.
Every cell runs as the document stands, so you can render at any point, finished or not. Doing that early and often is how you avoid finding a problem at the deadline.
The Your response blocks are written answers, not code. They are yours to write, in your own words — your assistant explains output to you, it does not draft these.
22.4.1 Q1 — Architecture Comparison (Suggested prompts: 2)
| Model | Window | Parameters | Test RMSE |
|---|---|---|---|
| FFN | 26 | ||
| CNN (best window from Part 1 Q4) | |||
| Vanilla RNN | 52 | ||
| LSTM | 52 | ||
| Transformer | 52 | ||
| Best tree model (HW3/HW4) | — | — |
Plot actual vs predicted for one series over the test period, all five neural models on one axes.
# Comparison table and actual-vs-predicted plot
Do not assume an ordering
Earlier versions of this assignment asked whether your results matched a “theoretical expectation” that sequence models beat convolutional ones, which beat feedforward ones. That expectation is not what Lecture 9 found. On this panel, at a fixed training budget, the vanilla RNN beat both the LSTM and the Transformer — and every neural model lost to a regularized linear model. Note the qualifier: Lecture 9 Part 1 also showed the RNN’s margin over the LSTM shrinking to nothing once both are trained to convergence, so how long you trained is part of any ranking you report.
Report your ranking, whatever it is, and explain it in terms of this problem: window length, the amount of training data, and where the predictive signal actually sits.
Your response: Which architecture won, and by how much over the runner-up? Is that margin large enough to matter operationally? Name one architecture that underperformed relative to its parameter count and give a reason grounded in the structure of this data.
22.4.2 Q2 — Full Course Comparison (Suggested prompts: 1)
Build the complete comparison spanning every homework. Fill in from your own records.
| Model | Category | Test RMSE |
|---|---|---|
| Holt-Winters | Classical TS | |
| ARIMA | Classical TS | |
| Prophet | GAM | |
| pyGAM | GAM | |
| Decision Tree | Tree | |
| Random Forest | Tree | |
| XGBoost (tuned) | Tree | |
| LASSO / Elastic Net | Regularized linear | |
| FFN / CNN / RNN / LSTM / Transformer | Neural Net |
Sort ascending and plot as a horizontal bar chart, colored by category.
Only compare what is comparable
Homework 1 and 2 used different frequencies and different series. An RMSE on weekly units for one store is not comparable to an RMSE on daily units, and neither is comparable to the 30-series panel. Group your table by what was actually forecast, and say plainly which rows can be compared and which cannot.
Your response: Does moving from classical → tree → neural consistently reduce RMSE? What property of this dataset explains the answer?
22.4.3 Q3 — Reflection (Suggested prompts: 0 — write yourself)
No code required. In 200–250 words: Looking at the full comparison table, reflect on the arc of methods in this course.
Answer all three:
- Where did increasing model complexity pay off, and where did it not?
- Neural networks are often described as “data hungry.” Your 52-week training set holds roughly 5,900 windows across 30 series. Is that enough? What would you expect to change with 10× more data — and which of the five architectures would benefit most?
- If you were advising a mid-sized retailer with a small data science team, which two models from the entire course would you recommend they invest in, and why those two specifically?
Your response:
22.5 Submission Checklist
Not graded, but worth doing: commit locally once per prompt, hw05_part3 prompt <id>: <summary>. Nobody sees these — they are your own record of which prompt produced what, and they make it far easier to retrace your steps when something breaks. - [ ] REPORT.md with a clear business recommendation