MCLEMCD : multimodal collaborative learning encoder for enhanced music classification from dances
Gong, Wenjuan (China University of Petroleum)
Yu, Qingshuang (China University of Petroleum)
Sun, Haoran (China University of Petroleum)
Huang, Wendong (China University of Petroleum)
Cheng, Peng (A*STAR)
Gonzàlez, Jordi (Universitat Autònoma de Barcelona)

Fecha:	2024
Resumen:	Music classification is widely applied in the automatic organization of music archives and intelligent music interfaces. Music is frequently accompanied by other media, such as image sequences. Combining various types of media for various tasks is natural for humans but extremely difficult for machines. In this work, we propose a collaborative learning method to combine dancing motions and music cues for music classification and apply it to music recommendations from dancing motions. Dancing motions in the form of 3D joint positions contain cyclic motions synchronized with music beats, and a collaborative autoencoder is designed to fuse music cues into a dancing motion feature extraction module. The proposed method achieved 98. 07 % on the MusicToDance data set and 65. 29 % on the AIST++ data set. The code to run all experiments is available at https://github. com/wenjgong/musicmotion.
Ayudas:	Agencia Estatal de Investigación PID2020-120311RB-I00
Derechos:	Aquest material està protegit per drets d'autor i/o drets afins. Podeu utilitzar aquest material en funció del que permet la legislació de drets d'autor i drets afins d'aplicació al vostre cas. Per a d'altres usos heu d'obtenir permís del(s) titular(s) de drets.
Lengua:	Anglès
Documento:	Article ; recerca ; Versió acceptada per publicar
Materia:	Collaborative learning ; Multi-media processing ; Music recommendation
Publicado en:	Multimedia Systems, Vol. 30, issue 1 (February 2024) , art. 37, ISSN 1432-1882

DOI: 10.1007/s00530-023-01207-6

Postprint 23 p, 1.3 MB

El registro aparece en las colecciones:
Artículos > Artículos de investigación
Artículos > Artículos publicados