Non-parametric Representation Learning with Kernels

Pascal Esser, Maximilian Fleissner, Debarghya Ghoshdastidar

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung

Abstract

Unsupervised and self-supervised representation learning has become popular in recent years for learning useful features from unlabelled data. Representation learning has been mostly developed in the neural network literature, and other models for representation learning are surprisingly unexplored. In this work, we introduce and analyze several kernel-based representation learning approaches: Firstly, we define two kernel Self-Supervised Learning (SSL) models using contrastive loss functions and secondly, a Kernel Autoencoder (AE) model based on the idea of embedding and reconstructing data. We argue that the classical representer theorems for supervised kernel machines are not always applicable for (self-supervised) representation learning, and present new representer theorems, which show that the representations learned by our kernel models can be expressed in terms of kernel matrices. We further derive generalisation error bounds for representation learning with kernel SSL and AE, and empirically evaluate the performance of these methods in both small data regimes as well as in comparison with neural network based models.

OriginalspracheEnglisch
TitelTechnical Tracks 14
Redakteure/-innenMichael Wooldridge, Jennifer Dy, Sriraam Natarajan
Herausgeber (Verlag)Association for the Advancement of Artificial Intelligence
Seiten11910-11918
Seitenumfang9
Auflage11
ISBN (elektronisch)1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 1577358872, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879, 9781577358879
DOIs
PublikationsstatusVeröffentlicht - 25 März 2024
Veranstaltung38th AAAI Conference on Artificial Intelligence, AAAI 2024 - Vancouver, Kanada
Dauer: 20 Feb. 202427 Feb. 2024

Publikationsreihe

NameProceedings of the AAAI Conference on Artificial Intelligence
Nummer11
Band38
ISSN (Print)2159-5399
ISSN (elektronisch)2374-3468

Konferenz

Konferenz38th AAAI Conference on Artificial Intelligence, AAAI 2024
Land/GebietKanada
OrtVancouver
Zeitraum20/02/2427/02/24

Fingerprint

Untersuchen Sie die Forschungsthemen von „Non-parametric Representation Learning with Kernels“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren