Prof. WonSook Lee – School of EECS, University of Ottawa, Canada
Bio: Prof. WonSook Lee majored Mathematics for her BSc and MSc at the POSTECH in South Korea (one semester in the Birmingham University in UK) and then switched to Computer Science when she spent two years in the Institute of Systems Science in the National University of Singapore. She was involved in CieMed group, which is basically medical imaging research in collaboration with Johns Hopkins University. Then she moved to Geneva, Switzerland to do her PhD in MIRALab at the University of Geneva where her topic became Computer Graphics focusing on Virtual Human Modeling for Animation. Before coming to Canada in 2003 as a Professor, she also worked in several industries such as Korea Telecom and Samsung Advanced Institute of Technology in South Korea and Eyematic Interfaces, Inc in U.S.A. She is currently a Full Professor in the School of Electrical Engineering and Computer Science, Faculty of Engineering at the University of Ottawa and also the director of Lab of Images, Intelligence and Innovation (LIII). Her main research areas cover Computer Graphics, Computer Vision, Virtual/Augmented Reality, Machine/Deep Learning and Medical Imaging. She is the author or co-author about 140 peer-reviewed conference/journal publications. Through the years in the University of Ottawa, she has awarded several research grants such as NSERC DISCOVERY, NSERC RTI, CFI, ORF, ORNEC, CIHR/NSERC CHRP, NSERC Engage, SME4SME, NCE GRAND and Global Frontier R&D program by the National Research Foundation of Korea. Most of grants, she is the Principal Investigator.
THE PRESENTATION
Title: GAN and Diffusion based 3D Face Reconstruction from a Front View.
Abstract: In the domain of 3D face reconstruction from single images, the limitation of single front-view input data often hampers realism, particularly in capturing realistic profile views of the resulting 3D model. Recognizing the superior quality of 3D model reconstruction achieved through multiple-views, we capitalize on neural networks’ capacity to generate 2D images from extensive databases. Our method employs various deep-learning networks to generate realistic facial viewpoints, thereby providing additional input data that significantly enhances the quality of 3D face reconstruction. While traditional image-space editing faces constraints in content and style manipulation while preserving high quality, our approach leverages latent space editing, which offers enhanced capabilities for photo manipulation. By utilizing the GAN inversion method, we identify a latent vector corresponding to the input image and synthesize multiple pose images by exploring nearby latent vectors, thereby enhancing 3D face reconstruction. These synthesized images are then fed into Diffusion models, known for their superior representation of large-angle variations, to produce realistic profile views. Subsequently, all multi-view images are input into an Autoencoder for 3D face model prediction, refining texture and shape aspects to achieve greater realism. Experimental results validate the efficiency of our method in reconstructing highly accurate 3D models without reliance on a pre-existing 3D database.