A Compact and Efficient Neural Data Structure for Mutual Information Estimation in Large Timeseries

Fatemeh Farokhmanesh, Christoph Neuhauser, Rüdiger Westermann

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung

Abstract

Database systems face challenges when using mutual information (MI) for analyzing non-linear relationships between large timeseries, due to computational and memory requirements. Interactive workflows are especially hindered by long response times. To address these challenges, we present timeseries neural MI fields (TNMIFs), a compact data structure that has been trained to reconstruct MI efficiently across various time-windows and window positions in large timeseries. We demonstrate learning and reconstruction with a large timeseries dataset comprising 1420 timeseries, each storing data at 1639 timesteps. While the learned data structure consumes only 45 megabytes, it answers queries for the MI estimates between the windows in a selected timeseries and the corresponding windows in all other timeseries within 44 milliseconds. Given a measure of similarity between timeseries based on windowed MI estimates, even the matrix showing all mutual timeseries similarities can be computed in less than 32 seconds. To support measuring dependence between lagged timeseries, an extended data structure learns to reconstruct MI to positively (future) and negatively (past) lagged windows. Using a maximum lag of 64 in both directions decreases query times by about a factor of 10.

OriginalspracheEnglisch
TitelScientific and Statistical Database Management
Untertitel36th International Conference, SSDBM 2024 - Proceedings
Redakteure/-innenShadi Ibrahim, Suren Byna, Tristan Allard, Jay Lofstead, Amelie Chi Zhou, Tassadit Bouadi, Jalil Boukhobza, Diana Moise, Cedric Tedeschi, Jean Luca Bez
Herausgeber (Verlag)Association for Computing Machinery
ISBN (elektronisch)9798400710209
DOIs
PublikationsstatusVeröffentlicht - 10 Juli 2024
Veranstaltung36th International Conference on Scientific and Statistical Database Management, SSDBM 2024 - Rennes, Frankreich
Dauer: 10 Juli 202412 Juli 2024

Publikationsreihe

NameACM International Conference Proceeding Series

Konferenz

Konferenz36th International Conference on Scientific and Statistical Database Management, SSDBM 2024
Land/GebietFrankreich
OrtRennes
Zeitraum10/07/2412/07/24

Fingerprint

Untersuchen Sie die Forschungsthemen von „A Compact and Efficient Neural Data Structure for Mutual Information Estimation in Large Timeseries“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren