Presentation
All-at-Once: Modular 4D Gaussian Worldbuilding and RealityFX for Live Music Performance
SessionReal-Time Live!
DescriptionAll-at-Once transforms the emerging field of volumography—particularly Gaussian Splatting—into a real-time, modular worldbuilding instrument for live audiovisual performance. Instead of rendering static volumetric captures, we treat volumography as a performative medium that can be remixed, manipulated, and composed in real time.
Built upon a custom 4D Gaussian pipeline and interactive RealityFX system, the project enables performers to spatially “play” volumetric worlds as though they were musical instruments. Each musical gesture—beat, note, modulation—triggers a corresponding volumetric transformation: objects rotate, multiply, dissolve, and re-emerge in synchrony with sound. Music becomes an interface for world-making.
The result is a synesthetic experience where remixing music equals remixing reality. A simple bassline may ripple through volumetric space, while a chord shift transforms a photoreal room into a surreal dreamscape. By bridging Gaussian Splatting, procedural modulation, and spatial music mapping, All-at-Once proposes a new grammar for live volumetric performance—one that is poetic, participatory, and technically groundbreaking.
For the SIGGRAPH audience, this demonstration illustrates a conceptual and technical leap: from reconstructing the world to performing it. It showcases the first instance of RealityFX, a real-time volumetric effects layer for Gaussian media, and establishes a new paradigm where AI, graphics, and music converge into living, generative stages of performance.
In essence, All-at-Once invites us to imagine the next generation of concerts, theater, and XR experiences—where sound and vision truly merge into one expressive, real-time volumetric art form.
Built upon a custom 4D Gaussian pipeline and interactive RealityFX system, the project enables performers to spatially “play” volumetric worlds as though they were musical instruments. Each musical gesture—beat, note, modulation—triggers a corresponding volumetric transformation: objects rotate, multiply, dissolve, and re-emerge in synchrony with sound. Music becomes an interface for world-making.
The result is a synesthetic experience where remixing music equals remixing reality. A simple bassline may ripple through volumetric space, while a chord shift transforms a photoreal room into a surreal dreamscape. By bridging Gaussian Splatting, procedural modulation, and spatial music mapping, All-at-Once proposes a new grammar for live volumetric performance—one that is poetic, participatory, and technically groundbreaking.
For the SIGGRAPH audience, this demonstration illustrates a conceptual and technical leap: from reconstructing the world to performing it. It showcases the first instance of RealityFX, a real-time volumetric effects layer for Gaussian media, and establishes a new paradigm where AI, graphics, and music converge into living, generative stages of performance.
In essence, All-at-Once invites us to imagine the next generation of concerts, theater, and XR experiences—where sound and vision truly merge into one expressive, real-time volumetric art form.
Event Type
Key Event
Real-Time Live!
TimeThursday, 18 December 20255:00pm - 5:20pm HKT
LocationHall 3F, Level 3