ANTIC: Adaptive Neural Temporal In-situ Compressor
Sandeep S. Cranganore, Andrei Bodnar, Gianluca Galleti, Fabian Paischer, Johannes Brandstetter
TLDR
ANTIC is an in-situ neural compressor that adaptively selects and spatially compresses high-dimensional simulation data, achieving massive storage reductions.
Key contributions
- Introduces ANTIC, an end-to-end in-situ compression pipeline for large-scale simulations.
- Uses an adaptive temporal selector to filter informative snapshots during simulation time.
- Employs a spatial neural compression module learning residual updates between snapshots.
- Achieves storage reductions of several orders of magnitude while maintaining physics accuracy.
Why it matters
High-resolution simulations generate petabyte-scale data, posing a major bottleneck for HPC. ANTIC's in-situ approach drastically reduces storage needs by combining temporal and spatial compression, enabling more efficient scientific discovery without explicit on-disk storage.
Original Abstract
The persistent storage requirements for high-resolution, spatiotemporally evolving fields governed by large-scale and high-dimensional partial differential equations (PDEs) have reached the petabyte-to-exabyte scale. Transient simulations modeling Navier-Stokes equations, magnetohydrodynamics, plasma physics, or binary black hole mergers generate data volumes that are prohibitive for modern high-performance computing (HPC) infrastructures. To address this bottleneck, we introduce ANTIC (Adaptive Neural Temporal in situ Compressor), an end-to-end in situ compression pipeline. ANTIC consists of an adaptive temporal selector tailored to high-dimensional physics that identifies and filters informative snapshots at simulation time, combined with a spatial neural compression module based on continual fine-tuning that learns residual updates between adjacent snapshots using neural fields. By operating in a single streaming pass, ANTIC enables a combined compression of temporal and spatial components and effectively alleviates the need for explicit on-disk storage of entire time-evolved trajectories. Experimental results demonstrate how storage reductions of several orders of magnitude relate to physics accuracy.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.