Presentation
JoruriPuppet: Learning Tempo-Changing Mechanisms Beyond the Beat for Music-to-Motion Generation with Expressive Metrics
DescriptionIn music-to-motion generation, the interplay between movements and music tempo variations significantly influences the emotional expressiveness and realism of performances. However, tempo-changing mechanisms remain underexplored in neural network-based music-to-motion tasks due to the scarcity of relevant datasets. Therefore, in this paper, we propose to use novel music features explicitly representing tempo variations, and introduce a dataset, JoruriPuppet, incorporating the Japanese traditional Jo-Ha-Kyu principle characterized by expressive tempo changes. Furthermore, we design three metrics to quantitatively evaluate the synchronization and expressiveness of generated motions. Experiments on our dataset highlight the limitations of SOTA methods in capturing fine-grained tempo changes. We demonstrate that integrating tempo-changing features into them improves neural network-based music-to-motion performance across existing datasets, validating the general effectiveness and applicability of our research.
Event Type
Technical Papers
TimeWednesday, 17 December 202511:34am - 11:45am HKT
LocationMeeting Room S426+S427, Level 4