BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:Asia/Hong_Kong
X-LIC-LOCATION:Asia/Hong_Kong
BEGIN:STANDARD
TZOFFSETFROM:+0800
TZOFFSETTO:+0800
TZNAME:HKT
DTSTART:19911015T033000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20251218T030653Z
LOCATION:Meeting Room S426+S427\, Level 4
DTSTART;TZID=Asia/Hong_Kong:20251217T095400
DTEND;TZID=Asia/Hong_Kong:20251217T100500
UID:siggraphasia_SIGGRAPH Asia 2025_sess134_papers_2005@linklings.com
SUMMARY:Single Image 3D Portrait Relighting with Generative Priors
DESCRIPTION:Pramod Rao (Max Planck Institute for Informatics; Saarbrücken 
 Research Center for Visual Computing, Interaction and Artificial  Intellig
 ence); Xilong Zhou (Max Planck Institute for Informatics); Abhimitra Meka 
 (Google Inc.); Gereon Fox and Mallikarjun B R (Max Planck Institute for In
 formatics); Fangneng Zhan (Harvard University); Tim Weyrich (Friedrich-Ale
 xander-Universität Erlangen-Nürnberg (FAU)); Bernd Bickel (ETH Zürich, IST
  Austria); Hanspeter Pfister (Harvard University); Wojciech Matusik (Compu
 ter Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts 
 Institute of Technology (MIT)); Thabo Beeler (Google Inc.); Mohamed Elghar
 ib (Max Planck Institute for Informatics); and Marc Habermann and Christia
 n Theobalt (Max Planck Institute for Informatics; Saarbrücken Research Cen
 ter for Visual Computing, Interaction and Artificial Intelligence)\n\nRend
 ering novel, relit views of a human head, given a monocular portrait image
  as input, is an inherently underconstrained problem. The traditional grap
 hics solution is to explicitly decompose the input image into geometry, ma
 terial and lighting via differentiable rendering; but this is constrained 
 by the multiple assumptions and approximations of the underlying models an
 d parameterizations of these scene components. We propose 3DPR, an image-b
 ased relighting model that leverages generative priors learnt from multi-v
 iew One-Light-at-A-Time (OLAT) images captured in a light stage. We introd
 uce a new diverse and large-scale multi-view 4K OLAT dataset, FaceOLAT, co
 nsisting of 139 subjects to learn a high-quality prior over the distributi
 on of high-frequency face reflectance. We leverage the latent space of a p
 re-trained generative head model that provides a rich prior over face geom
 etry learnt from in-the-wild image datasets. The input portrait is first e
 mbedded in the latent manifold of such a model through an encoder-based in
 version process. Then a novel triplane-based reflectance network trained o
 n our lightstage data is used to synthesize high-fidelity OLAT images to e
 nable image-based relighting. Our reflectance network operates in the late
 nt space of the generative head model, crucially enabling a relatively sma
 ll number of lightstage images to train the reflectance model. Combining t
 he generated OLATs according to a given HDRI environment maps yields physi
 cally accurate environmental relighting results. Through quantitative and 
 qualitative evaluations, we demonstrate that 3DPR outperforms previous met
 hods, particularly in preserving identity and in capturing lighting effect
 s such as specularities, self-shadows, and subsurface scattering.\n\nRegis
 tration Category: Full Access, Full Access Supporter\n\nSession Chair: Men
 glei Chai (Google)\n\n
END:VEVENT
END:VCALENDAR