Learning effective RGB-D representations for scene recognition
Song, Xinhang (Chinese Academy of Sciences. Institute of Computing Technology)
Jiang, Shuqiang (Chinese Academy of Sciences. Institute of Computing Technology)
Herranz, Luis (Universitat Autònoma de Barcelona)
Chen, Chengpeng (Chinese Academy of Sciences. Institute of Computing Technology)

Data:	2019
Resum:	Deep convolutional networks can achieve impressive results on RGB scene recognition thanks to large data sets such as places. In contrast, RGB-D scene recognition is still underdeveloped in comparison, due to two limitations of RGB-D data we address in this paper. The first limitation is the lack of depth data for training deep learning models. Rather than fine tuning or transferring RGB-specific features, we address this limitation by proposing an architecture and a two-step training approach that directly learns effective depth-specific features using weak supervision via patches. The resulting RGB-D model also benefits from more complementary multimodal features. Another limitation is the short range of depth sensors (typically 0. 5 m to 5. 5 m), resulting in depth images not capturing distant objects in the scenes that RGB images can. We show that this limitation can be addressed by using RGB-D videos, where more comprehensive depth information is accumulated as the camera travels across the scenes. Focusing on this scenario, we introduce the ISIA RGB-D video data set to evaluate RGB-D scene recognition with videos. Our video recognition architecture combines convolutional and recurrent neural networks that are trained in three steps with increasingly complex data to learn effective features (i. e. , patches, frames, and sequences). Our approach obtains the state-of-the-art performances on RGB-D image (NYUD2 and SUN RGB-D) and video (ISIA RGB-D) scene recognition.
Ajuts:	European Commission 665919
Drets:	Aquest material està protegit per drets d'autor i/o drets afins. Podeu utilitzar aquest material en funció del que permet la legislació de drets d'autor i drets afins d'aplicació al vostre cas. Per a d'altres usos heu d'obtenir permís del(s) titular(s) de drets.
Llengua:	Anglès
Document:	Article ; recerca ; Versió acceptada per publicar
Matèria:	Scene recognition ; Deep learning ; Multimodal ; RGB-D ; Video ; CNN ; RNN
Publicat a:	IEEE transactions on image processing, Vol. 28, Issue 2 (February 2019) , p. 980-993, ISSN 1057-7149

DOI: 10.1109/TIP.2018.2872629

Postprint 13 p, 3.6 MB

El registre apareix a les col·leccions:
Articles > Articles de recerca
Articles > Articles publicats