Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Synchronisation

  • When merging time-dependent data from different sources, matching them well is important, but also comes with some choices.

  • If time series are to be used in Machine Learning, exact synchronisation is usually needed, i.e., equal timestamps on each variable.

  • For plotting purposes different time resolution, e.g., weeks vs months, may not be a problem as long as the different cycles match up.

Accumulation

  • If a higher resolution time series is to be synchronized with a lower resolution series, some type of accumulation needs to be done.

  • Things to keep in mind:

    • Which type of accumulation should be done, e.g., sum for count data, mean for intensity data?

    • Should we use simple statistics, robust statistics or smoothed series data?

    • Are the low resolution recordings a series of snapshots (a) or an accumulation between (before (b)/after (d)) or around (c) timepoints (see illustration)?

      • This coresponds to the different uses of filters in the Noise reduction section.

      • ... which also means that it is possible to echange simple averages with other smoothers.

Question: Is the mean calculation above an accumulation of type a, b, c or d (as compared to the illustration)?

Interpolation

  • Timepoints may not match as easily as with days and hours above.

  • If one series is shifted slightly, the series are irregular in timesteps or have non-intuitive intervals, interpolation is an alternative.

  • When interpolating an irregular sequence in Pandas, one may need to resample to a higher frequency, fill the missing values and resample to the final frequency (see example below).

Time delays

  • Industrial processes often have a continuous or batchwise handling of raw materials into other materials or products.

    • When sensors record data along the production line, matching a piece of raw material to sensor readings can be done by adding a delay to the timestamp of the measurements early in the process or subtracting time from the later measurements.

  • For some processes, the delay is a known, fixed quantity.

    • For others the delay may be dependent on dynamic factors like raw material properties or process settings that add uncertainty to the time delay.

    • Synchronising such data, may require optimising correlations between sensors or using more advanced warping techniques.