Assistant Professor at UPenn. Research interests: Neural Scene Representation, Neural Rendering, Human Performance Modeling and Capture.
Philadelphia, PA
- Tweets 608
- Following 1,019
- Followers 4,567
- Likes 5,909
Thrilled to announce that I will be joining the University of Pennsylvania as an Assistant Professor in January 2023! I'm extremely grateful to all those who have supported me all the way and I look forward to working with students and colleagues at Penn! @PennEngineers @CIS_Penn
51
20
493
Check out our #ICCV2023 work "NeuS2" (vcai.mpi-inf.mpg.de/projects…), which reconstructs high-quality 3D geometry from multi-view images in <2 minutes. We derived a new analytical formula of the second-order derivatives for ReLU-based MLPs, leading to an efficient CUDA implementation.
3
54
297
43,642
I'm super excited to share our recent work: NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction. We propose to represent a surface as the zero-level set of an SDF and develop a new volume rendering method to train a neural SDF representation.
6
55
281
I’m recruiting multiple PhD students in my group at @CIS_Penn in the following areas:
- Neural Representations and rendering for 3D/4D Reconstruction
- 3D Generative Models
- Human Motion Generation
- LLM guided Graphics and Vision
- Neural Representations for Robotics
etc.
3
52
268
51,754
Check out our SIGGRAPH'23 work: "Drag Your GAN" (vcai.mpi-inf.mpg.de/projects……)! Experience the excitement of manipulating images by just "dragging" any points of the image. It's a fantastic hands-on interactive tool you don't want to miss! #SIGGRAPH2023
Have you thought about interactively 'dragging' objects in the image? Our #SIGGRAPH2023 work #DragGAN makes this come true!🥳
Paper: arxiv.org/abs/2305.10973
Project page: vcai.mpi-inf.mpg.de/projects…
5
15
133
22,237
Excited about my first visit to Hawaii to attend my first ML conference! Look forward to engaging with people on Neural 3D Representations and 3D Generative models at #ICML2023. Our paper NerfDiff will be presented at 11-13:30 Tuesday. Welcome to our poster at Exhibit Hall 1#231!
1
12
123
16,222
We released the code of NeuS: github.com/Totoro97/NeuS. Feel free to try it!😊
I'm super excited to share our recent work: NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction. We propose to represent a surface as the zero-level set of an SDF and develop a new volume rendering method to train a neural SDF representation.
15
104
I had to cancel my trip to CVPR due to a significant visa delay, which is truly disheartening. Many other people are also being affected by visa delays or denials. I sincerely hope that such an unfortunate situation never arises again for future conference participants.
3
5
103
39,957
Excited to be at @SIGGRAPHAsia! Our work, "DreamEditor: Text-Driven 3D Scene Editing with Neural Fields", led by @JjZhuang26958 @chenwangcw, will be presented at 14-15 today in C4.11. Come chat with us about Neural Representations and 3D Generative Models!
13
93
8,447
I am recruiting PhD students at @CIS_Penn this year! If you're interested in neural scene representation, neural rendering, human performance capture and 3D reconstruction, don't miss the opportunity to apply to my lab at: cis.upenn.edu/graduate/progr…. Deadline is Dec 15.
Prospective CS PhD Applicants: @CIS_Penn deadline is Dec 15. URL: applyweb.com/upenng/. Most importantly, you get to choose upto 4 faculty who you would like to review your application. Make best use of it. Learn more about our faculty at highlights.cis.upenn.edu/cis…. Good luck!
16
75
HDHumans won the Best Paper Award (honorable mention) at SCA @SympCompAnim! Congratulations to @marc_habermann and our team!
HDHumans got accepted at #SCA23. It can render neural avatars at very high resolution (up to 4K). How? We show that an explicit deformable mesh model can help to guide the NeRF. Importantly, we found that the NeRF can also guide the mesh deformation.
people.mpi-inf.mpg.de/~mhabe…
5
78
9,181
"NeuS" got accepted to NeurIPS 2021 as a Spotlight presentation! 😊 Congrats to the team! The code can be found here: github.com/Totoro97/NeuS
I'm super excited to share our recent work: NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction. We propose to represent a surface as the zero-level set of an SDF and develop a new volume rendering method to train a neural SDF representation.
10
75
Attending #ECCV2022 in Tel Aviv! First time in this lovely city! Happy to chat with you about research on Neural Scene Representation and Neural Rendering! I’m also looking for students to join my lab at UPenn in 2023! Send me an email if you want to chat!
2
4
67
Thrilled to introduce our SIGGRAPH 2022 work, "Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data", led by @RachelChuCat .This is the first attempt to incorporate the governing physics into a self-learning neural rendering framework for fluid reconstruction.
Thank @ErisZhang0326 for presenting our work at #SIGGRAPH2022!
This work reconstructs complex fluid scenes from RGB videos, allowing the existence of arbitrary obstacles and unknown lighting conditions for the first time :)
More information (code, videos) people.mpi-inf.mpg.de/~mchu/…
2
5
58
Want to know about the rapid advancement of Diffusion Models for Visual Computing? Check out our survey: arxiv.org/abs/2310.07204.
Diffusion Models offer transformative capabilities for visual computing.
In a new report, we overview the mathematical fundamentals and survey the quickly growing field of diffusion models for 2D, 3D, video, and motion generation and editing.
arxiv.org/abs/2310.07204
1
1
47
11,071
What I'm very surprised about is that it can also achieve high-fidelity thin structure reconstruction!! This is really thrilling to me as someone who has worked on thin structure reconstruction for a couple of years.
2
2
35
A very cool work led by Yuan Liu on a new generalizable neural representation, called Neural Ray, for novel view synthesis. It can generalize to unseen scenes with no or little training.
Happy to share our work "Neural Rays for Occlusion-aware Image-based rendering" for the novel-view-synthesis task.
Paper: arxiv.org/abs/2107.13421
Project page: liuyuan-pal.github.io/NeuRay…
(Results below are generated without per-scene training.)
2
6
34
The SIGGRAPH 2022 Technical Papers Call for Submissions is now available! This year, there are two ways to submit groundbreaking research via the Journal Papers or Conference Papers tracks! Here is more info about these tracks: s2022.siggraph.org/program/t…
It's submission season at SIGGRAPH! 🤩 📝 💻 ✨ Make sure you bookmark our "Submit to SIGGRAPH" page to stay up-to-date on #SIGGRAPH2022 program submission openings and deadlines. Technical Papers is open for submissions with more programs opening soon! bit.ly/3o2mfHH
1
5
34
My lab will conduct research at the intersection of Computer Graphics, Computer Vision and AI, with a focus on neural scene representations, neural rendering, human performance modeling and capture, and 3D reconstruction. (2/n)
1
1
26
I'm looking for motivated students to join my lab in 2023. You can find information about my research and more at lingjie0206.github.io/. (3/n)
1
5
23
The code and data of our NeurIPS 2020 (spotlight): Neural Sparse Voxel Fields are available! Check out: github.com/facebookresearch/…
(1/3) Super excited to present our recent work: Neural Sparse Voxel Fields (NSVF): a hybrid neural scene representation for fast and high-quality free-viewpoint rendering.
Joint work with @LingjieLiu1 (MPI), Zaw Lin (NUS), Tat-Seng Chua (NUS) and Christian Theobalt (MPI).
1
3
25
Jiatao @thoma_gu will present our #ICCV2023 work, SSD-NeRF, this morning (Wed, Oct 4) at 10:30-12:30. If you're interested in our work and want to talk with him about 3D Generation and Diffusion Models, welcome to our poster 031 in Room "Foyer Sud"!
Our paper 'Single-Stage Diffusion NeRF' will be presented at #ICCV2023. We merge 3D diffusion with NeRF into a holistic model, providing priors for both 3D generation and reconstruction (from an arbitrary number of views). Check it out here: lakonik.github.io/ssdnerf/ #NeRF #AI
19
4,226
Welcome to our poster at #NeurIPS2023 for the amazing work led by @youngjoongkwon !
#NeurIPS2023
We propose DELIFFAS, a deformable two-surface representation parameterizing a surface light field, which allows real-time and photorealistic avatar synthesis.
Great Hall & Hall B1+B2 (level 1) #210
Thursday (Dec 14) 10:45 am - 12:45 pm CST
vcai.mpi-inf.mpg.de/projects…
15
2,647
If you're interested in neural scene representations and neural rendering, feel free to join us in 40mins on the Q&A live session of our NeurIPS Spotlight paper: Neural Sparse Voxel Fields:
Q&A session at Dec 8th, 2020 @ 17:30 CET (8:30 AM PST)
The code and data of our NeurIPS 2020 (spotlight): Neural Sparse Voxel Fields are available! Check out: github.com/facebookresearch/…
2
14
We released the code at github.com/19reborn/NeuS2. Joint work with Yiming, Qin, @marc_habermann , @KostasPenn and Christian.
2
12
1,154
Check out more details in our paper: arxiv.org/abs/2106.10689 and at our project page: lingjie0206.github.io/papers…
2
12
Exciting news! Want to generate stunning images from texts in real time? Check out our recent work: BOOT (jiataogu.me/boot/). Our key idea? Distilling a pretrained diffusion model into a single-step model, with no need for training data!
🪘🪘New pre-print!! I’m delighted to share our latest work @Apple MLR
“BOOT👢: Data-free Distillation of Denoising Diffusion Models with Bootstrapping.”
We explore a novel method that can distill your favorite diffusion models into ONE STEP without using training data!🔆 (1/6)
1
11
2,972
Different from existing neural surface reconstruction methods (like DVR and IDR), NeuS does not require foreground masks as supervision to converge to a valid surface. Unlike NeRF, NeuS can extract high-quality surfaces.
1
9
Stunning work! 3D generation moves so fast!
DreamFusion: Text-to-3D using 2D Diffusion
paper: openreview.net/pdf?id=FjNys5…
abs: openreview.net/forum?id=FjNy…
project page: dreamfusionpaper.github.io/
DeepDream on a pretrained 2D diffusion model enables text-to-3D synthesis
8
Don’t miss the valuable opportunity to work with @XingangP , the leading author of DragGAN!!
(1/2) We are actively seeking PhD candidates from various countries to foster diversity in our research group at Nanyang Technological University. Know someone interested in a PhD with us? Please refer them to our team. Thanks for supporting diversity in academia! 🌍🎓
1
10
2,478
A nice survey! Cannot agree more about “I can easily see the field moving back to SDF-style implicit representations or even voxels, at least at inference time.”
2020 was the year in which *neural volume rendering* exploded onto the scene, triggered by the impressive NeRF paper by Mildenhall et al. I wrote a post as a way of getting up to speed in a fascinating and very young field and share my journey with you: dellaert.github.io/NeRF/
1
10
Amazing work!!
As opposed to NeRF methods that render conventional RGB images, this work renders raw lidar scans from novel views using a prototype single-photon lidar. Better few-view reconstruction than using point clouds and a step towards more general neural rendering of light transport!
9
2,380
Come join us!
Virtual coffee break at #CVPR2023!
Coffee break is an excellent time to chat and connect! Grab a coffee and chat with us 10:00 - 10:30 AM PDT.
Lingjie @LingjieLiu1 (UPenn) and I will be there!
Link: calendly.com/jbhuang0604/cvp…
1
1
6
2,649
Awesome work by @marc_habermann et al. Creating controllable realistic characters with high-fidelity geometry details and dynamic textures becomes possible!
Check out our new work which synthesizes photo-realistic renderings of humans just by providing a driving motion. Thanks to the team (@xuweipeng000 @MZollhoefer @GerardPonsMoll1 @LingjieLiu1 and Christian Theobalt)
Project page: people.mpi-inf.mpg.de/~mhabe…
7
You may apply through cis.upenn.edu/graduate/progr… (DDL: Dec 15) and list me as your preferred supervisor.
6
4,061
Wow, nice work!
Excited to share "Volume Rendering of Neural Implicit Surfaces" (VolSDF): a volume rendering framework for implicit neural surfaces, allowing to learn high fidelity geometry from a sparse set of input images.
with @thoma_gu @yoni_kasten @lipmanya
arxiv.org/abs/2106.12052
(1/8)
1
6
Furthermore, we proposed a progressive training strategy for stable hash encoding training, and an efficient training method for further accelerating the learning of dynamic scenes to within ~20 seconds per frame.
1
1
6
1,455
Disappointed with your CVPR reviews? Wanna give it another try to a top conference? You may consider the upcoming SIGGRAPH 2022, especially the new conference track with submissions in 7 pages! The paper registration deadline is Jan 26, and the submission deadline is Jan 27!
Submit your papers to SIGGRAPH next week! If your CVPR submission on image synthesis/graphics/animation doesn't get the reviews it deserves on Monday, then withdraw and submit to the new SIGGRAPH technical papers conference track instead!
2
5
This method allows us to reconstruct physically-plausible and high-fidelity fluids from sparse multiview RGB videos. Check out the code and more video results under people.mpi-inf.mpg.de/~mchu/….
4
The key idea is to represent a 3D scene by considering visibility probabilities defined on camera rays emitted from input images.
4
Thanks for your interest! There are some points we need to clarify:
1. As mentioned by Sarlin, our method is not strictly better than SuperGlue. The results in the paper all use SIFT as descriptors while SuperGlue is optimized on SIFT. our method outperforms it under this setting
2
2
Super excited about our TVCG work! We propose a simple but efficient non-data-driven method for high-quality 3D shape segmentation. Code and data are available: github.com/clinplayer/SEG-MA…
2
Good question. In our setting, we must calculate the second-order derivatives for the parameters associated with the point normal (the first-order derivative of point location).
1
2
223
Checkout our TVCG work about geometry-based 3D shape segmentation: github.com/clinplayer/SEG-MA…. We propose a simple but efficient non-data-driven method for high-quality segmentation, which can also facilitate data annotation process for deep learning. Code and data are available!
We propose a geometry-driven method for 3D shape segmentation, in the age of deep learning XD! Code available: github.com/clinplayer/SEG-MA…
2
2. Our method is not designed to replace SuperGlue. SuperGlue is a great matcher while our method is used to prune correspondences produced by matchers. We showed in our experiments that we can achieve improvements by combining SuperGlue with LMCNet.
1
Hi, thank you for your interest. I haven't read DualField. Our method differs from UNISURF in that UNISURF represents the surface by occupancy values and gradually reduces the sample regions at some predefined steps to make the occupancy value converge to the surface,
1
1
Ya, I saw that. I was trying to say that superglue works better with superpoints rather than SIFT.
1
A fantastic collaboration with our incredible team @Po_lhr @toomanyyifans @VGolyanik @AbermanKfir @jon_barron @bermano_h @ericryanchan @talidekel @holynski_ @akanazawa , K. Liu, @BenMildenhall @MattNiessner, B. Ommer, C. Theobalt, @peter_wonka @GordonWetzstein
1
423
Hi, if rendering an image with volume rendering at inference time, it would be two times slower than NeRF since it needs one more pass for normal calculation. However, since we can have a good surface after training, sphere tracing would accelerate the rendering process
1
1
while we represent the scene by an SDF and define a new weight function based on SDF in the volume rendering equation, therefore, the zero set of the network-encoded SDF is expected to represent an accurately reconstructed surface upon successful minimization of a loss function.
1