Dataset Distillation by Automatic Training Trajectories

Dai Liu, Jindong Gu, Hu Cao, Carsten Trinitis, Martin Schulz

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung

Abstract

Dataset Distillation is used to create a concise, yet informative, synthetic dataset that can replace the original dataset for training purposes. Some leading methods in this domain prioritize long-range matching, involving the unrolling of training trajectories with a fixed number of steps (NS) on the synthetic dataset to align with various expert training trajectories. However, traditional long-range matching methods possess an overfitting-like problem, the fixed step size NS forces synthetic dataset to distortedly conform seen expert training trajectories, resulting in a loss of generality—especially to those from unencountered architecture. We refer to this as the Accumulated Mismatching Problem (AMP), and propose a new approach, Automatic Training Trajectories (ATT), which dynamically and adaptively adjusts trajectory length NS to address the AMP. Our method outperforms existing methods particularly in tests involving cross-architectures. Moreover, owing to its adaptive nature, it exhibits enhanced stability in the face of parameter variations. Our source code is publicly available at https://github.com/NiaLiu/ATT.

OriginalspracheEnglisch
TitelComputer Vision – ECCV 2024 - 18th European Conference, Proceedings
Redakteure/-innenAleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, Gül Varol
Herausgeber (Verlag)Springer Science and Business Media Deutschland GmbH
Seiten334-351
Seitenumfang18
ISBN (Print)9783031730207
DOIs
PublikationsstatusVeröffentlicht - 2025
Veranstaltung18th European Conference on Computer Vision, ECCV 2024 - Milan, Italien
Dauer: 29 Sept. 20244 Okt. 2024

Publikationsreihe

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Band15145 LNCS
ISSN (Print)0302-9743
ISSN (elektronisch)1611-3349

Konferenz

Konferenz18th European Conference on Computer Vision, ECCV 2024
Land/GebietItalien
OrtMilan
Zeitraum29/09/244/10/24

Fingerprint

Untersuchen Sie die Forschungsthemen von „Dataset Distillation by Automatic Training Trajectories“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren