A novel framework for retrieval and interactive visualization of multimodal data
Kalamaras, Ilias (Imperial College London. Department of Electrical and Electronic Engineering)
Mademlis, Athanasios (Centre for Research and Technology Hellas (Tessalònica, Grècia). Information Technologies Institute)
Malassiotis, Sotiris (Centre for Research and Technology Hellas (Tessalònica, Grècia). Information Technologies Institute)
Tzovaras, Dimitrios (Centre for Research and Technology Hellas (Tessalònica, Grècia). Information Technologies Institute)

Fecha:	2013
Resumen:	With the abundance of multimedia in web databases and the increasing user need for content of many modalities, such as images, sounds, etc. , new methods for retrieval and visualization of multimodal media are required. In this paper, novel techniques for retrieval and visualization of multimodal data, i. e. documents consisting of many modalities, are proposed. A novel cross-modal retrieval framework is presented, in which the results of several unimodal retrieval systems are fused into a single multimodal list by the introduction of a cross-modal distance. For the presentation of the retrieved results, a multimodal visualization framework is also proposed, which extends existing unimodal similarity-based visualization methods for multimodal data. The similarity measure between two multimodal objects is defined as the weighted sum of unimodal similarities, with the weights determined via an interactive user feedback scheme. Experimental results show that the cross-modal framework outperforms unimodal and other multimodal approaches while the visualization framework enhances existing visualization methods by efficiently exploiting multimodality and user feedback.
Derechos:	Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial i la comunicació pública de l'obra, sempre que no sigui amb finalitats comercials, i sempre que es reconegui l'autoria de l'obra original. No es permet la creació d'obres derivades.
Lengua:	Anglès
Documento:	Article ; recerca ; Versió publicada
Materia:	Cerca multimodal ; Visualització multimodal ; Interfície intel·ligents d'usuari ; Interacció Humà-Ordinador ; Multimodal Search ; Multimodal Visualization ; Intelligent User Interfaces ; Human-Computer Interaction
Publicado en:	ELCVIA : Electronic Letters on Computer Vision and Image Analysis, Vol. 12, Núm. 2 (2013) , p. 28-39, ISSN 1577-5097

Adreça original: https://elcvia.cvc.uab.es/article/view/v12-n2-kalamaras-mademlis-malassiotis-et-al
Adreça alternativa: https://raco.cat/index.php/ELCVIA/article/view/280906
DOI: 10.5565/rev/elcvia.518

12 p, 998.9 KB

El registro aparece en las colecciones:
Artículos > Artículos publicados > ELCVIA
Artículos > Artículos de investigación