Time Series Basics
When order matters.
HomeWorth's sales team asks a new question. How many valuations will we get next month? This is different from everything so far. Up to now, we could shuffle our rows. Shuffle a time series, and you destroy it. Yesterday really does help to predict today.
- How many next month?
- Shuffling destroys it
- Yesterday predicts today
The parts of a time series
Most time series are made of a few simple parts. A trend, the long slow direction. Seasonality, a repeating pattern, such as weekends or the summer. Sometimes longer cycles, like the economy. And noise, the random part that you cannot predict. If you can separate these in your head, the data suddenly makes much more sense.
Daily sales
Here are two years of daily sales, which we simulate with a known trend, a weekly rhythm, and a yearly wave, so that we know what the truth is. Pandas has excellent support for dates. Grouping by day of the week reveals the rhythm immediately. Run it. Some days are clearly stronger than others. Whenever you get a time series, group it by hour, by weekday, and by month. It takes seconds, and reveals so much.
import numpy as np
import pandas as pd
rng = np.random.default_rng(1)
days = pd.date_range("2024-01-01", periods=730, freq="D")
trend = np.linspace(100, 160, 730)
weekly = 12 * np.sin(2 * np.pi * days.dayofweek / 7)
yearly = 20 * np.sin(2 * np.pi * days.dayofyear / 365)
sales = pd.Series(trend + weekly + yearly + rng.normal(0, 4, 730), index=days)
print(sales.head(3).round(1).to_string())
print()
print("Mean by weekday:")
print(sales.groupby(sales.index.dayofweek).mean().round(0).to_string())2024-01-01 101.7 2024-01-02 113.4 2024-01-03 114.2 Freq: D Mean by weekday: 0 130.0 1 140.0 2 141.0 3 135.0 4 125.0 5 118.0 6 120.0
Summary
Trend, seasonality, and noise. Those are the ingredients of time. But how do we use our usual models, which expect a table of features? A very simple trick turns any time series into an ordinary table. Next, forecasting.
Key takeaways- Order matters, so never shuffle
- Trend, seasonality, noise
- Group by weekday and month