Presentation
From Rigging to Waving: 3D-Guided Diffusion for Natural Animation of Hand-Drawn Characters
DescriptionHand-drawn character animation is a vibrant research area in computer graphics, presenting unique challenges in achieving geometric consistency while conveying expressive motion details. Traditional skeletal animation methods maintain geometric consistency but often struggle with complex non-rigid elements like flowing hair and skirts, resulting in unnatural deformation and missing secondary dynamics. In contrast, video diffusion models effectively synthesize physics-aware dynamics but suffer from stylized artifacts and geometric distortions when applied to stylized drawings due to domain gaps. In this work, we propose a novel hybrid animation system that integrates the strengths of skeletal animation and video diffusion priors. The core idea is to generate coarse images from characters retargeted with skeletal animations for geometric consistency guidance and to further enhance these images with video diffusion models in terms of texture details and secondary dynamics. We reformulate the enhancement of coarse images as an inpainting task and propose a domain-adapted diffusion model to refine regions requiring improvement, particularly those involving secondary dynamics, guided by user-provided masks. To further enhance motion realism, we propose a Secondary Dynamics Enhancement strategy during the denoising process that incorporates latent features from a pre-trained diffusion model enriched with human motion priors. Additionally, to address unnatural deformation resulting from hair sticking in skeletal animation, we introduce a hair layering modeling method that employs segmentation maps to separate hair from the body in the implicit fields, allowing our system to animate challenging hair-sticking characters more naturally. Through extensive experiments and a perceptual study, we demonstrate that our system generates high-fidelity animations with realistic dynamics and artistic integrity. The code and more animation results are available in the supplementary materials.
Event Type
Technical Papers
TimeWednesday, 17 December 20252:50pm - 3:00pm HKT
LocationMeeting Room S423+S424, Level 4