Closing the gap in domain adaptation for semantic segmentation : a time-aware method
Serrat Gual, Joan (Universitat Autònoma de Barcelona. Departament de Ciències de la Computació)
Gomez Zurita, Jose Luis (Centre de Visió per Computador)
López Peña, Antonio M. (Universitat Autònoma de Barcelona. Departament de Ciències de la Computació)

Date:	2025
Abstract:	Semantic segmentation models need a large number of images to be effectively trained but manual annotation of such images has a high cost. Active domain adaptation addresses this problem by pretraining the model with a synthetically generated dataset and then fine-tuning it with a few selected label annotations (the "budget") on real images to account for the domain shift. Previous works annotate a percentage of either individual pixels or whole target images. We argue that the first is infeasible in practice, and the second spends part of the budget on classes that the pretrained model may have already learned well. We propose a method based on the annotation of regions computed by Segment Anything, a recently introduced foundation model for class-agnostic image segmentation. The key idea is to assign a ground truth label to each of a tiny subset of regions, those for which the model is more uncertain. In order to increase the number of annotated regions we propagate the ground truth labels to most similar regions according to a hierarchical clustering algorithm that uses the features learned by the pretrained model. Our method outperforms the state-of-the-art on the GTA5 to Cityscapes benchmark by using fewer annotations, almost closing the gap between the synthetically pre-trained model and that obtained with full supervision of the real images. Furthermore, we present competitive results for budgets less than 1% of samples and also for a larger and more challenging target dataset, Mapillary Vistas.
Grants:	Agencia Estatal de Investigación PID2020-115734RB-C21
Note:	Altres ajuts: acords transformatius de la UAB
Rights:	Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial, la distribució, la comunicació pública de l'obra i la creació d'obres derivades, fins i tot amb finalitats comercials, sempre i quan es reconegui l'autoria de l'obra original.
Language:	Anglès
Document:	Article ; recerca ; Versió publicada
Subject:	Active learning ; Domain adaptation ; Semantic segmentation ; Foundation model
Published in:	Machine vision and applications, Vol. 36 (2025) , art. 13, ISSN 1432-1769

DOI: 10.1007/s00138-024-01626-z

25 p, 4.1 MB

The record appears in these collections:
Articles > Research articles
Articles > Published articles