Advanced Time Series Forecasting Methods

So far we have been talking about classical approaches when forecasting time series data. However, it is essential to explore alternative techniques that involve advanced methodologies such as machine learning and deep learning. There are mixed views regarding the accuracy of these last techniques. Some say that these advanced techniques underperform classical ones. However, a useful guideline is to consider classical techniques when dealing with data over a short time period and opt for machine learning or deep learning approaches when dealing with data over a long period.

Classical models

Before continuing let’s discuss what we exactly refer to by classical models. These models use traditional statistical methods and are based on the assumption that past values can be used to predict future values. These models are relatively simple to understand and implement, and they can be very effective for forecasting many types of time series, especially in situations in which data doesn’t exhibit complex relationships.

Stay up-to-date with our latest articles!
Subscribe to our free newsletter by entering your email address below.

Some models that we can find within this group are:

• AutoRegressive Integrated Moving Average (ARIMA) family of models: univariate models such as AR, MA, ARMA, ARIMA, SARIMA and SARIMAX.
• Exponential Smoothing: univariate models such as the simple, double and triple (Holt-Winters) exponential smoothing models.
• Vector AutoRegression (VAR): extension of AR models to handle multivariate data. Also includes further extensions like VARMA models.

Although they are generally computationally efficient, they have some potential disadvantages:

• Sensitive to the presence of outliers
• Sensitive to non-stationarity
• They may not be able to capture long-term trends in the data

Advanced models

Those are some reasons to consider more complex and advanced models based on Machine Learning and Deep Learning. The following diagram tries to break down the different types of models and their subtypes:

We refer to advanced models as those using Machine Learning algorithms to find patterns in data, enabling them to forecast future values. In contrast to the classical models, these are able to capture complex patterns, non-linear relationships, and long-term dependencies in time series data.

They can be broken down into those Machine Learning models that use Neural Networks or not:

• Machine Learning models like Linear Regression, XGBoost or Random Forest can be used to handle time series data that has been previously processed by reframing the problem as a supervised learning task.
• Deep Learning models are a subset of ML models that use artificial neural networks to extract complex patterns from time series data. Due to the sequential nature of the time series data, they need to be adapted to handle it. That’s the reason why they can be broken down into different types depending on the approach they follow:
  • Recurrent Neural Network (RNN)
  • Convolutional Neural Networks (CNN) or Temporal Convolutional Networks (TCNs)
  • Transformers

Recurrent Neural Network (RNN)

A Recurrent Neural Network is a type of artificial neural network designed to process sequential data, where the order of the data points is crucial. Unlike feedforward neural networks, RNNs have connections that form loops, allowing information to persist and be passed from one step to the next. This recurrent structure makes RNNs well-suited for tasks involving time series data. Following a diagram with the basic structure of an RNN:

There are two main types of RNNs depending on their structure:

Gated Recurrent Unit (GRU)

GRU is a variant of RNN that uses two gating mechanisms (reset gate and update gate) to control the flow of information making it more efficient in capturing long-term dependencies in time series data by mitigating the vanishing gradient problem. Below you can find a diagram of this architecture. We will go deeper in future articles.

Long Short-Term Memory (LSTM)

LSTM is another type of RNN that, like the GRU, also incorporates gating mechanisms to control the flow of information within the network. It is a more complex version of the RNN that includes three interacting gates (input gate, forget gate, and output gate) instead of two. This allows the LSTM to have more precise control over the flow of information within the cell. This fine-grained control enables LSTM to capture and maintain long-term dependencies in sequential data more effectively than GRU, making it well-suited for tasks that require modeling complex and distant relationships in the input data. As you can see in the diagram below, this architecture is more complex. We will go through it in a separate article.

Convolutional Neural Networks (CNN)

CNNs are commonly used for image processing but can be applied to time series forecasting by treating the time series as an image-like format. This type of neural network can capture local patterns and temporal dependencies in the data, effectively learning meaningful features for forecasting. Its advantage is the low number of parameters they require in comparison with the RNN.

Transformers

Transformers are a type of Deep Learning model that utilizes self-attention mechanisms to weigh the significance of different data points in the time series. They have achieved great success in various natural language processing tasks, and recently, they have been adapted for time series forecasting, demonstrating promising results.

Which one should I use?

As we mentioned in the introduction, some say that the classical methods are better, but for the long-term, Machine Learning and Deep Learning approaches can be more accurate.

Here you have some considerations when choosing your approach:

• The type of time series data: some types of time series data are more well-suited for certain techniques than others. For example, LSTMs are often a good choice for time series data that has long-term dependencies thanks to their gating mechanisms.
• The amount of data: if you have a lot of data, you may be able to use a more complex technique, such as an LSTM or transformer. However, if you have limited data, you may need to use a simpler technique, such as an ARIMA or Exponential Smoothing model.
• The complexity of the data: if you have time series data with non-linear relationships you may want to opt for an advanced methodology based on Machine Learning or Deep Learning to be able to capture them.
• The accuracy requirements: if you need very accurate forecastings, you may need to use a more complex technique. However, if you are only looking for rough estimates or for time series analysis, you may be able to use a simpler technique.