Neural networks for time-series forecasting Day 1

Time series data can be decomposed into four possible components:

1. the Trend component reflecting the long term progression of the series.
2. the Cyclical component that describes repeated but non-periodic fluctuations, possibly caused by the economic cycle.
3. the Seasonal component reflecting seasonality (Seasonal variation)
4. the Irregular component (or "noise") that describes random, irregular influences. Compared to the other components it represents the residuals of the time series.

Time series forecasting is the use of a model to forecast future events based on known past events. Meaning that, it is used to predict data points before they are measured.

Autocorrelation is the correlation of the value of a data item at a particular time with the values of previous data items.

The number of time steps back into the past that have data items with high autocorrelation to the future data item is called time lags.

Given the water demand data of the previous 12 months as input, a multi-layer perceptron model can be built to forecast the value of the random component of the water demand in the following month.

Spatio-temporal model not only incorporates time lags of the variable whose future values are of interest, but also other influential variables, and possibly their own lags. For example, for the water demand time series data, rainfall is seen as a lag indicator of water demand.

The output of the MLP is added to the extrapolations of the trend and seasonal components to get the forecast of water demand for the following month.

Perceptron for forecasting of linear time-series

1. Linear Auto-Regressive with  Exogenous (external)  input variables networks (ARx)

Neural networks for non-linear time-series forecasting

1. Time-lagged feedforward networks (sometimes are also called focused time-lagged networks)
2. Dynamically-driven recurrent (feedback) networks

A recurrent network feeds back the predicted output through feedback connection into the input layer with one unit time delay.

A Non-linear Auto-Regressive with Exogenous input variables (NARx) network model is a recurrent network that, on top of the variable being predicted, uses other external/exogenous variables as inputs.

MLP for forecasting Multi-step-ahead forecasting

1. A Non-linear Auto-Regressive with Exogenous (external)  input variables networks (NARx).

The nonlinear autoregressive network with exogenous inputs (NARX) is a recurrent dynamic network, with feedback connections enclosing several layers of the network.

Single step vs. Multi-step-ahead forecasting

Time-lagged and recurrent networks

Performance of ARx and NARx networks

In Spatio-Temporal Time-Lagged networks, the response of the time-series is affected by one or more time-series.

Dynamic networks can be divided into two categories: those that have only feedforward connections, and those that have feedback, or recurrent connections.

Recurrent-dynamic networks typically have a longer response than feedforward-dynamic networks.

For linear networks, feedforward-dynamic networks are called finite impulse response (FIR), because the response to an impulse input will become zero after a finite amount of time.

Linear recurrent-dynamic networks are called infinite impulse response (IIR), because the response to an impulse can decay to zero (for a stable network), but it will never become exactly equal to zero.

An impulse response for a nonlinear network cannot be defined, but the ideas of finite and infinite responses do carry over.

Dynamic networks are generally more powerful than static networks (although somewhat more difficult to train). Because dynamic networks have memory, they can be trained to learn sequential or time-varying patterns.

References

1. Grace Rumantir (2012). FIT5167 Lecture Notes. Monash University
2. MATLAB 2011b Help Documentation