Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Machine Learning approach

  • Time series prediction, or forecasting, can be very similar to modelling and prediction with tabular data.

    • A set of input variables, usually a single output variable.

    • Ordinary machine learning methods can be applied.

  • The inclusion of time can be done by adding one or more delayed variables, possibly including the response.

Validation

  • As soon as time is part of a model, extra care needs to be taken in validation.

  • Cross-validation*, training-validation-test splits are still relevant.

  • However, training, validation and test sets need to follow time chronologically and cannot overlap.

  • Instead of traditional cross-validation, one can perform backtesting with a sliding or expanding window:

Figures from Roy Yang’s bloggpost on uber.com

Shipping, oil, interest rates and exchange rates

  • These data are public data from the Norwegian Bank, SSB, Eurostat and U.S. Energy Information Administration for the period 2000-2014 (monthly).

  • The data are available at ResearchGate and were part of a Master thesis by Raju Rimal.

Inspect and visualize

Modelling without time

  • For starters, let us ignore time and build a simple prediction model for the exchange rate.

  • We will use scikit-learn’s Pipeline to combine standardisation (scaling) and linear regression and cross_val_predict to perform random K-fold cross-validation.

Backtesting

  • scikit-learn has a TimeSeriesSplit which creates segments for backtesting.

    • Expanding window is the default.

    • Sliding window can be applied by setting the right combination of parameters.

  • We will use scikit-learn’s cross_validate to perform the cross-validation based on the backtesting segments (cross_val_predict assumes that all observations will be test data at some point).

Question: Does the behaviour make sense with regard to what is included in and predicted from the model?

Question: Again; does the behaviour make sense with regard to what is included in and predicted from the model?

Exercise

  • Repeat the PerUSD predictions, but exchange LinearRegression with scikit-learns’s PLSRegression.

  • Check if the number of components in the PLS model has an effect on the explained variance (R2\text{R}^2), either manually or using a GridSearchCV.

Including the response variable in the predictors

  • As long as the training and test sets are not overlapping, we can include the response as a predictor.

  • Adding the response lagged can be done as a single variable or several variables (i.e., several different lags).

  • We will later look at ARIMA-type models where time lag is the main mechanism for modelling.

Five lags

Correlation between time series

  • To get an impression of the connection between different variables, one can compute correlations, e.g., in the form of a correlation matrix.

  • If one expects one variable to affect another variable at a later time, correlation with a lag can be computed.

  • The degree of connection between two time series may also be dependent on time.

    • A Sliding Window Correlation (SWC) shows local correlation in time windows.

    • The window size (and possible lag) can be tuned for series of quick or slow changes.

  • Note: Correlation does not equal causation.

    • There may not be a cause and effect, even though two phenomena show similar patterns. Beautifully illustrated by Tyler Vigen.

  • The concept of Autocorrelation will be covered later.

Pandas’ rolling() and shifts

  • When applying Pandas’ rolling() function, the index is used for matching the data points.

  • Therefore, we need to shift the index of the ExpNatGas to achieve a lag.

  • Because of the sliding window, the two series do not need to match in length.

Exercise

  • Combine lag and sliding window correlation.

  • Use ipywidgets to control:

    • window width

    • lag

    • selected variable to compare to PerUSD

    • bonus: visualize the sliding window like above