Skip to main navigation Skip to search Skip to main content

NApy: efficient statistics in Python for large-scale heterogeneous data with enhanced support for missing data

  • Friedrich Alexander Universität Erlangen-Nürnberg
  • Institute for Biomedicine (affiliated to the University of Lübeck)

Research output: Contribution to journalArticlepeer-review

Abstract

Background Existing Python libraries and tools lack the ability to efficiently compute statistical test results for large datasets in the presence of missing values. This presents an issue as soon as constraints on runtime and memory availability become essential considerations for a particular use case. Relevant research areas where such limitations arise include interactive tools and databases for exploratory analysis of biomedical data. Results To address this problem, we present the Python package NApy, which relies on a Numba and C++ backend with OpenMP parallelization to enable scalable statistical testing for mixed-type datasets in the presence of missing values. Conclusions Both with respect to runtime and memory consumption, NApy outperforms competitor tools and baseline implementations with naive Python-based parallelization by orders of magnitude, thereby enabling on-the-fly analyses in interactive applications.

Original languageEnglish
Article numbergiaf140
JournalGigaScience
Volume14
DOIs
StatePublished - 2025

Keywords

  • Python
  • efficient computing and parallelization
  • large-scale datasets
  • missing data
  • statistical software

Fingerprint

Dive into the research topics of 'NApy: efficient statistics in Python for large-scale heterogeneous data with enhanced support for missing data'. Together they form a unique fingerprint.

Cite this