A Closer look at embedding propagation for manifold smoothing
Velazquez Dorta, Diego Alejandro (Centre de Visió per Computador (Bellaterra, Catalunya))
Rodríguez López, Pau (ServiceNow Research)
Gonfaus, Josep (Visual Tagging Services)
Roca i Marvà, Francesc Xavier (Universitat Autònoma de Barcelona)
Gonzàlez, Jordi (Centre de Visió per Computador (Bellaterra, Catalunya))

Data:	2022
Resum:	Supervised training of neural networks requires a large amount of manually annotated data and the resulting networks tend to be sensitive to out-of-distribution (OOD) data. Self- and semi-supervised training schemes reduce the amount of annotated data required during the training process. However, OOD generalization remains a major challenge for most methods. Strategies that promote smoother decision boundaries play an important role in out-of-distribution generalization. For example, embedding propagation (EP) for manifold smoothing has recently shown to considerably improve the OOD performance for few-shot classification. EP achieves smoother class manifolds by building a graph from sample embeddings and propagating information through the nodes in an unsupervised manner. In this work, we extend the original EP paper providing additional evidence and experiments showing that it attains smoother class embedding manifolds and improves results in settings beyond few-shot classification. Concretely, we show that EP improves the robustness of neural networks against multiple adversarial attacks as well as semi- and self-supervised learning performance.
Ajuts:	Agencia Estatal de Investigación PID2020-120311RB-I00
Nota:	Altres ajuts: this work was supported by the Generalitat de Catalunya under the Industrial Doctorate Program (grant number 2020DI62).
Drets:	Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial, la distribució, la comunicació pública de l'obra i la creació d'obres derivades, fins i tot amb finalitats comercials, sempre i quan es reconegui l'autoria de l'obra original.
Llengua:	Anglès
Document:	Article ; recerca ; Versió publicada
Matèria:	Adversarial robustness ; Few-shot classification ; Regularization ; Self-supervised learning ; Semi-supervised learning
Publicat a:	Journal of Machine Learning Research, Vol. 23, issue 1 (2022) , p. 11447-11473, ISSN 1533-7928

Adreça alternativa: https://jmlr.org/papers/volume23/21-0468/21-0468.pdf

27 p, 2.5 MB

El registre apareix a les col·leccions:
Articles > Articles de recerca
Articles > Articles publicats