TimeXer for Time Series Forecasting with Exogenous Features

Now that we have a deep understanding of the inner workings of TimeXer, let’s apply it in our own small experiment using Python.

Forecasting with TimeXer

For this portion, we apply TimeXer on the popular EPF benchmark dataset. 

It contains information on the price of electricity in five different European markets. Most importantly, the dataset also comes with known exogenous features. The whole dataset is accessible on GitHub under the MIT License.

For this small experiment, we will compare the performance of TimeXer against NHITS and TSMixerx, as they are very robust models that also support exogenous features.

Here, we use the implementation available in neuralforecast, as I believe this is the easiest and fastest way to use deep learning models for time series forecasting. 

As always, the full source code for this experiment is available on GitHub.

Let’s get started!

Initial setup

First, let’s import the required packages for this experiment. We import the usual packages for data manipulation and visualization, as well as neuralforecast and utilsforecast for fitting the models and evaluating them.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

from neuralforecast.core import NeuralForecast
from neuralforecast.models import TimeXer, NHITS, TSMixerx

from utilsforecast.evaluation import evaluate
from utilsforecast.losses import mae, mse

Then, we read in the data and format it as expected by neuralforecast. Mainly, we create a unique_id column to identify the series, we name the timestamp column as ds and we name the target column as y.

We also remove the extra whitespace that precedes the name of the exogenous features.

BE_url = "https://raw.githubusercontent.com/thuml/TimeXer/refs/heads/main/dataset/EPF/BE.csv"
DE_url = "https://raw.githubusercontent.com/thuml/TimeXer/refs/heads/main/dataset/EPF/DE.csv"

BE_df = pd.read_csv(BE_url, parse_dates=["date"])
BE_df["unique_id"] = "BE"
BE_df = BE_df.rename(columns={
    "date": "ds",
    " Generation forecast": "Generation forecast",
    " System load forecast": "System load forecast",
    "OT": "y"
})
DE_df = pd.read_csv(DE_url, parse_dates=["date"])
DE_df["unique_id"] = "DE"
DE_df = DE_df.rename(columns={
    "date": "ds",
    " Wind power forecast": "Wind power forecast",
    " Ampirion zonal load forecast": "Ampirion zonal load forecast",
    "OT": "y"
})

Note that for simplicity, we forecast for two out of the five markets available in the dataset. Of course, feel free to forecast all of them, as the steps remain the same, just make sure to use the correct variable names.

Finally, let’s set some constants that will be reused throughout the experiment.

HORIZON = 24
INPUT_SIZE = 168
FREQ = "h"
BE_EXOG_LIST = ["Generation forecast", "System load forecast"]
DE_EXOG_LIST = ["Wind power forecast", "Ampirion zonal load forecast"]

Here, we use a horizon of 24 hours, and an input size of 168 hours (7 days). The data has an hourly frequency, and we define the name of the exogenous features.

At this point, we are ready to fit our models.

Training each model

In neuralforecast, we can initialize a list of models to be trained on a dataset. Here, we use TimeXer, NHITS and TSMixerx.

Below, we initialize the models for the Belgian market.

models = [
    TimeXer(
        h=HORIZON,
        input_size=INPUT_SIZE,
        n_series=1,
        futr_exog_list=BE_EXOG_LIST,
        patch_len=HORIZON,
        max_steps=1000
    ),
    NHITS(
        h=HORIZON,
        input_size=INPUT_SIZE,
        futr_exog_list=BE_EXOG_LIST,
        max_steps=1000
    ),
    TSMixerx(
        h=HORIZON,
        input_size=INPUT_SIZE,
        n_series=1,
        futr_exog_list=BE_EXOG_LIST,
        max_steps=1000
    )
]

In the code block above, we can specify the name of our exogenous features in the futr_exog_list parameter. This parameter expects a list of columns to be treated as external variables.

Also notice that for each model, we use the same input size, and maximum training steps. 

Finally, TSMixerx and TimeXer use the n_series parameter because they are multivariate models, meaning that they can learn the interdependency of multiple series. However, since we are modeling one market at a time, we set it to 1.

In the case of NHITS, it is a multivariate model, so the parameter is not used.

Then, all models can be trained on the dataset using cross-validation. That way, we get multiple windows of predictions that we can compare directly to the actual values.

In this case, we use ten non-overlapping cross-validation, as shown below.

nf = NeuralForecast(models=models, freq=FREQ)
BE_cv_preds = nf.cross_validation(BE_df, step_size=HORIZON, n_windows=10)
BE_cv_preds.head()

The entire process is done for the Belgian market only. Now, we can repeat the same steps for the Danish market.

models = [
    TimeXer(
        h=HORIZON,
        input_size=INPUT_SIZE,
        n_series=1,
        futr_exog_list=DE_EXOG_LIST,
        patch_len=HORIZON,
        max_steps=1000
    ),
    NHITS(
        h=HORIZON,
        input_size=INPUT_SIZE,
        futr_exog_list=DE_EXOG_LIST,
        max_steps=1000
    ),
    TSMixerx(
        h=HORIZON,
        input_size=INPUT_SIZE,
        n_series=1,
        futr_exog_list=DE_EXOG_LIST,
        max_steps=1000
    )
]

nf = NeuralForecast(models=models, freq=FREQ)
DE_cv_preds = nf.cross_validation(DE_df, step_size=HORIZON, n_windows=10)
DE_cv_preds.head()

Once the training process is done for both datasets, we can then evaluate the performance of our models.

Evaluation

Before calculating performance metrics, we can first visualize the predictions made for both series.

Below, we can see the predictions made for the Belgian market.

From the figure above, we can see that NHITS misses the granular variations in the data. However, TSMixerx and TimeXer seem to make better forecasts.

Similarly, we can then visualize the forecasts for the Danish market.

In the figure above, surprisingly, TimeXer now is the worst model, as it completely misses the early dip, while NHITS and TSMixerx generally follow the shape of the actual values.

This visual inspection is further confirmed by the table of performance metrics below.

Interestingly, TimeXer performs best on the BE dataset and NHITS performs worst there, but the situation is completely flipped for the DE dataset, where NHITS performs best, and TimeXer achieves the worst performance.

It is very hard for me to explain why that is the case. However, when testing TimeXer on other datasets, I did notice that it benefits from longer training compared to NHITS or TSMixer, so it might be that for the DE dataset, training for 1000 steps was simply not enough.

Of course, this experiment is not a thorough benchmark. The main objective is to show how you can implement TimeXer on your own datasets.

As such, keep in mind that:

• TimeXer is implemented as a multivariate model in neuralforecast, so it can model the relationship between multiple series
• TimeXer benefits from longer training, so set the training steps to a larger number than 1000
• the model supports exogenous features, which can be passed in the futr_exog_list argument. This means that the future values of the features are known and available at the time of forecasting.

Conclusion

TimeXer is a transformer-based model that combines a self-attention mechanism with a cross-attention mechanism.

The self-attention models temporal dependencies in the target series, while cross-attention captures relationships between the endogenous series and external variables. 

When working with TimeXer, it is recommended to train it for more than 1000 steps as it seems to benefit from longer training times.

In our small experiment, TimeXer achieved the best performance on one dataset, but the worst on another. Once again, keep in mind that this was not a complete benchmark. However, the performances are interesting enough to be worth trying in your own project.

As always, each problem requires its own solution, and now you can test if TimeXer is the best answer to your scenario.

Thanks for reading! I hope that you enjoyed it and that you learned something new!

Learn the latest time series forecasting techniques with my 👉 free time series cheat sheet 👈 in Python! Get the implementation of statistical and deep learning techniques, all in Python and TensorFlow!

Cheers 🍻

Support me

Enjoying my work? Show your support with Buy me a coffee, a simple way for you to encourage me, and I get to enjoy a cup of coffee! If you feel like it, just click the button below 👇

References

[1] Y. Wang et al., “TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables,” arXiv.org, 2024. https://arxiv.org/abs/2402.19072

[2] Official implementation of TimeXer — GitHub