Streaming

• When data become available one and one or in a chunk-wise fashion, streaming models can be beneficial.

• A static model may be applied if we have enough training data and expect no changes to the underlying distributions.

• A dynamic model may be applied if we want to update based on what the model receives of new data.

  • Update model with some small weight on the new observation, either:

    • gradually aggregating to a model equivalent to training a model on all data, or

    • with a small memory loss on the existing model, leading to plasticity and capacity for learning (e.g., gradient descent methods).

  • Passive aggressive learning, where small deviations are disregarded while the model is updated when larger changes are detected.

  • If the models start from scratch, higher learning rates for the initial samples is more efficient.

• Streaming is relevant both for predictive models (tabular data) and forecasting models (autoregressive).