Director, Research Scientist @AIatMeta Digital Human/Vision/Graphics デジタルヒューマンの研究してます。 #pifuhd #sapiens All opinions are my own.
Pittsburgh
- Tweets 951
- Following 521
- Followers 4,518
- Likes 2,402
Pinned Tweet
Relightable Codec Avatars is now extended to full-body! At #SIGGRAPH2025, we will present Relightable Full-body Gaussian Codec Avatars. Key contributions include learnable Zonal Harmonics and deferred learnable radiance transfer for specular! Check it out!
neuralbodies.github.io/RFGCA…
2
30
177
13,715
#CVPR2020 にオーラルで採択された、1枚画像から人の高解像フル3D化を実現する研究「PIFuHD」のコードがgithubで公開されました！！！！
GPUが無くても自分で画像をアップロードして3Dモデルが作れるようにGoogle Colabのデモも公開中です！#お家でデジタルヒューマン
github.com/facebookresearch/…
15
1,396
4,050
OpenAIのDALL-E （GPT3のテキスト→画像版）が凄まじい。複雑な文章入力に対しての高品質でそれっぽい生成画像は、もはやクリエイティブとしか言いようがない。（画像は”犬を散歩させるチュチュを履いた大根”の入力に対する生成結果）
openai.com/blog/dall-e/
2
412
862
自分達の#SIGGRAPH論文がITMediaで特集されました🎉🎉
携帯での簡単なスキャンで表情もコントロールできる本人そっくりのアバターが作成できるようになります！
iPhoneの自撮りで本物そっくりな動く3Dリアルアバター 米Metaが技術開発 itmedia.co.jp/news/articles/… スマホで撮影した短時間のスキャン画像から頭部アバターを生成。シワや歯など細部も正確に再現され,動く表情も自然に表現される。視点が変わっても頭部の角度に応じた光の当たり具合を表現。
2
202
675
📢 Check out 𝗥𝗲𝗹𝗶𝗴𝗵𝘁𝗮𝗯𝗹𝗲 𝗚𝗮𝘂𝘀𝘀𝗶𝗮𝗻 𝗖𝗼𝗱𝗲𝗰 𝗔𝘃𝗮𝘁𝗮𝗿𝘀!
Our latest codec avatars using 3D Gaussians generalize to novel lighting (OLAT, envmap) with *all-frequency* reflection (see video for hair and eye reflection) in real-time! shunsukesaito.github.io/rgca…
8
72
416
45,941
動きキレッキレのアバターを全自動で３Dスキャン群から作り出す技術『SCANimate(=SCAN+Animate)』がCVPR2021にオーラル採択＋公開されました🎉
リグと服のシワなどの変形(PSD)を３Dスキャンから直接学習することによりリアルなアバターアニメーションが手作業なしで作れます→scanimate.is.tue.mpg.de/
1
85
405
今年のSIGGRAPH共著論文が公開されました！デジタルヒューマン・アバターの照明、表情、顔向きを”リアルタイム”かつ”実写レベル”で自由にコントロールできます！（動画は全てCGです）facebookに移っての初仕事（？）です。
ビデオ👉piped.video/5YigyNvt4GE
Webサイト👉sai-bi.github.io/project/sig…
106
404
🎉 We just released code, data and model for our #SIGGRAPH2023 work "CT2Hair"!! CT2Hair achieves fully automatic reconstruction of production-ready hair strands from computed tomography (CT) scans!!
Code&Data: github.com/facebookresearch/…
Project: yuefanshen.net/CTHair
6
87
394
56,125
Brown/MIT/トロント大/Google/NVIDIA/TUMの研究者とのサーベイ論文及び論文検索サービスが公開されました🎉🎉
NeRFやDeepSDFなど、この1~2年で爆発的に研究が進んでいる“MLPを用いた時空間場を表現する技術＝Neural Fields”の基礎から応用例まで幅広くカバーしています！
brownvc.github.io/neural-fie…
1
67
391
🎉 Excited to share our #3DV2024 paper: "Diffusion Shape Prior for Wrinkle-Accurate Cloth Registration".
We show how to incorporate a powerful diffusion prior into cloth 3D registration, and achieve wrinkle-accurate alignment!
Project: www-users.cse.umn.edu/~guo00…
3
60
350
45,506
Wow, both Relightable Gaussian Codec Avatars and URHand are selected as Oral at #CVPR2024 !!
RGCA: shunsukesaito.github.io/rgca…
URHand: frozenburning.github.io/proj…
3
50
304
22,834
📢 Looking for high-res and generalizable NeRF rendering for face and body without requiring 3DMM or SMPL?
Check out our #ECCV2022 paper, KeypointNeRF!! The code is just released!!
Project: markomih.github.io/KeypointN…
Code: github.com/facebookresearch/… (1/4)
5
67
307
#ECCV2024 にオーラル採択された人物特化型基盤モデル「Sapiens」のコードおよび学習済み重みが公開されました！！
史上最高精度の4タスク(姿勢推定、領域分割、深度・法線推定）モデルに加え、3億枚人物画像による事前学習モデルも公開中です。#おうちでデジタルヒューマン
github.com/facebookresearch/…
2
53
308
28,377
📢 Does your avatar lack history-dependent secondary motions such as soft-tissue dynamics?
Try our #ECCV2022 paper, AutoAvatar!! Now you can learn secondary-motion from raw scans *without* mesh registration.
Project: zqbai-jeremy.github.io/autoa…
Code: github.com/facebookresearch/…
(1/N)
2
45
282
🚨 Need super photorealistic hair model? Check out our new #ECCV2022 paper, NeuralStrands🧠💇♀️!!
Website: radualexandru.github.io/neur…
w/ awesome intern @radu_alex_rosu, @Ziyan_Wang1994, Chenglei, @SvenBehnke, and Giljoo!!
1
44
253
Who at #CVPR2024 wants to see Relightable Gaussian Codec Avatars live in VR? Come join Codec Avatar workshop on June 18th (Tue)!! We offer VR demo during the break! There's some exciting announcement (and also my talk) there as well!
4
35
240
31,032
Need volumetric capture at home? Check out our PIFuHD (#CVPR2020 Oral) on github, enabling high-res 3D human capture from a single picture!! We also provide Google Colab demo, where you can upload your own photo and play with PIFuHD on the cloud!! #pifuhd
github.com/facebookresearch/…
4
67
237
📢Looking for real-time drivable volumetric avatars? Check out out #SIGGRAPH 2022 paper, DVA!!
We extend Mixture of Volumetric Primitives [Lombardi+ 2021] to articulated human bodies to enable real-time/high-fidelity rendering of clothed human!
PDF: arxiv.org/abs/2207.09774
4
42
228
🚨 Excited to introduce our #CVPR2023 paper, MEGANE🤓!!
MEGANE is an interaction-aware compositional 3D morphable eyeglasses and head model, supporting photorealistic rendering & relighting. (1/7)
👉Project: junxuan-li.github.io/megane/
4
35
195
37,033
Check out our upcoming #CVPR2021 Oral paper: SCANimate! Now you can build a pose-driven avatar with naturally deforming clothing from incomplete raw scans *WITHOUT* surface registration or clothing template! Key idea in the following threads (1/N)
Project: scanimate.is.tue.mpg.de/
4
32
188
📢 Check out 𝐒𝐚𝐩𝐢𝐞𝐧𝐬 (#ECCV2024, Oral)!
Largest human-centric foundational models that natively supports 1K high-res inputs, achieving SOTA on 2D kpts, seg, depth and normal!
Code and models (each task + pretraining) are ready to use TODAY! github.com/facebookresearch/… (1/6)
5
34
183
21,521
#ECCV2020 に採択されたリアルタイムで動くPIFuの進化版「Monoport」がGithubで公開されました！！
Monoportを使ってウェブカメラ一台で仮想空間から３Dプレゼンができるシステムも#SIGGRAPH RTLで明日発表されます！
project page: project-splinter.github.io
github: github.com/Project-Splinter/…
2
53
174
#SIGGRAPH2023 に採択された、CTスキャンから高精度の3D髪モデルを復元する「CT2Hair」のコードがGithubで公開されました！！！
CTデータと生成結果もダウンロードできるのでぜひ遊んでみてください！
github.com/facebookresearch/…
2
28
166
21,227
#ICCV2021 New paper is out!!
ARCH++: Animation-Ready Clothed Human Reconstruction Revisited
pdf: arxiv.org/pdf/2108.07845.pdf
abs: arxiv.org/abs/2108.07845
ARCH++ produces results which have high-level
fidelity and are animation-ready for many AR/VR applications
3
26
156
💡Check out our #SIGGRAPHASIA 2024 paper, URAvatar: Universal Relightable Gaussian Codec Avatars. Now high-quality relightable avatars can be created by 𝐞𝐯𝐞𝐫𝐲𝐛𝐨𝐝𝐲!
We learn a universal relightble prior and show how to use it for quick adaptation from a phone scan.
6
26
164
18,826
📢Join me tomorrow at our #CVPR tutorial on Neural Fields (Neural Implicits/Neural Scene Representation)! You will learn foundation and latest advances of this exciting "field"! I will talk about Digital Human applications and its open questions! neuralfields.cs.brown.edu/cv…
3
26
149
おお、動いた！Sapiens、HuggingFaceで誰でも試せるようになってるので、是非！#sapiens
huggingface.co/spaces/facebo…
2
20
155
22,102
Hey I'm in Sydney! We have two papers to present @SIGGRAPHAsia
1. Human-SGD (human-sgd.github.io/) 12/14/2023 (Thu), 11:50am-12:00pm
High-quality 360° textured human from a single image. No supervision by 3D scans required. Key idea: Use shape-guided diffusion to inpaint!
2
28
140
16,568
実写クオリティの髪の3次元復元・レンダリングを自動で実現する「Neural Strands（ニューラルネットによる毛束表現）」が #ECCV2022 に採択されました！動画は全て実写ではなく提案手法による合成結果になります。
🚨 Need super photorealistic hair model? Check out our new #ECCV2022 paper, NeuralStrands🧠💇♀️!!
Website: radualexandru.github.io/neur…
w/ awesome intern @radu_alex_rosu, @Ziyan_Wang1994, Chenglei, @SvenBehnke, and Giljoo!!
40
136
CVPRで大好評だったニューラルフィールドのチュートリアルの録画及びスライドが公開されました！是非是非
Neural fields are emerging as useful signal representations in computer vision & beyond. Our full-day introductory @CVPR tutorial on the topic is now public.
Video: piped.video/PeRRp1cFuH4
Slides: drive.google.com/drive/folde…
Web: neuralfields.cs.brown.edu/cv…
28
130
🚨 #ECCV2022 に採択された「KeypointNeRF」のコードが公開されました！
PIFuやPIFuHDと同様に個別学習一切なしで数枚の全身や顔画像から下記のような結果を自動生成できます！
コードはこちら👉github.com/facebookresearch/…
📢 Looking for high-res and generalizable NeRF rendering for face and body without requiring 3DMM or SMPL?
Check out our #ECCV2022 paper, KeypointNeRF!! The code is just released!!
Project: markomih.github.io/KeypointN…
Code: github.com/facebookresearch/… (1/4)
40
128
自分達のチームが開発したコーデックアバターを使った弊社CEOとLex Fridmanの対談が公開されたみたいです。実写と区別がつかないクオリティのアバターを使ったメタバースでの対談は世界初では？
piped.video/MVYrJJNdrEg?si=l3gT…
15
118
17,184
📢 Check out 𝗨𝗥𝗛𝗮𝗻𝗱: 𝗨𝗻𝗶𝘃𝗲𝗿𝘀𝗮𝗹 𝗥𝗲𝗹𝗶𝗴𝗵𝘁𝗮𝗯𝗹𝗲 𝗛𝗮𝗻𝗱𝘀!
URHand is a universal relightable hand prior! You can turn URHand into relightable "Your Hand" from a quick phone scan! Our *linear* model is the key for scalable training! frozenburning.github.io/proj…
1
13
111
13,851
Facebook AIのブログでPIFuHDめっちゃフィーチャーされてる！！
At #CVPR2020, Facebook AI is pushing the state of the art forward in many important areas of CV, including core segmentation tasks, architecture search, transfer learning, multi-modal learning, 3D reconstruction, & more. Learn more here: ai.facebook.com/blog/faceboo…
18
102
The code of SCANimate (#CVPR2021 Oral, Best Paper Nominee) is finally released on Github🚀🚀 github.com/shunsukesaito/SCA…
Also CVPR Q&A session will be held in Session 3 (6-8am in ET on June 22). See you soon!!
w/ Jinlong, @qianli_m, @Michael_J_Black
7
13
102
OMG!! SCANimate is nominated for best paper @CVPR!!! 🚀
SCANimate (scanimate.is.tue.mpg.de/) and TUCH (tuch.is.tue.mpg.de/) were both nominated for best paper @CVPR. Congratulation to @psyth91, @qianli_m, Jinlong Yang, @LeaMue27, @ahmedaaosman, @SiyuTang3, and Chun-Hao Huang. Well done!
5
10
97
📢📢 Check out our new #CVPR2023 paper on real-time renderable & relightable 3D hands with global illumination effects (e.g., self-shadowing, SSS etc)!! It even supports two-hands relighting!! Kudos to our awesome intern @s1wase!!
I'm thrilled to announce that our paper on the neural relightable hand model has been accepted for CVPR2023! I owe a big thanks to my mentors @psyth91, Tomas Simon, and collaborators during my internship at Meta! Check out more details on our website! sh8.io/#/relightable_hands
1
9
95
19,600
データセットの公開を約束したにも関わらず反故にした論文は強制的にリジェクトするようにCVPR, ICCVを含むCVFの論文のルールが更新されたとのこと。
Motion: All CVF conferences will have a dataset review process for papers promising a dataset. Acceptance will be conditional on the dataset being ready and reviewed by the camera-ready deadline. Papers that do not comply will be rejected. Comments please: tinyurl.com/wro6ljl
43
92
[PIFu, Colab対応しました!! /PIFu supports Google Colab!! ]
ローカルで環境構築できない人でもDemoコード回せます。/ Give it a try even if you don't have your own GPU!!
github.com/shunsukesaito/PIF…
@r_natsume @akanazawa @HaoLi81
5
26
84
SIGGRAPH Trailer is out!! Our volumetric avatar modeling from a phone scan is also featured!!
自分達の論文も特集されてます〜
piped.video/YGmWqQ6oz3U
2
10
78
Super excited to share our work Human-SDG @SIGGRAPHAsia !!
We show how to create high-quality 360° textures on any monocular human reconstruction (e.g., #pifuhd, #ICON, #ECON) with no 3D scans!
Check out @jbhuang0604's amazing video! 👉 piped.video/watch?v=IfHiGSsm…
Super excited to share our work @SIGGRAPHAsia:
3D Human Digitization from a *Single Image*
💡 Core idea: Synthesize consistent and detailed appearances using a 3D shape-guided diffusion model!
2
14
75
19,137
Check out our latest #SIGGRAPH2021 paper on relightable + animatable avatar from Facebook Reality Labs!! Our work enables for the first time *real-time relighting* of photorealistic avatars!!
Full Video👉 piped.video/5YigyNvt4GE
Project 👉sai-bi.github.io/project/sig…
3
12
76
Check out our #CVPR2022 paper, COAP! Our compositional neural implicit body model is highly generalizable to unseen poses and identity!!
COAP also offers efficient and robust self-intersection and collision handling, which is critical for human-scene interaction!
📢 Our generalizable neural implicit body leverages a localized encoder-decoder to model volumetric humans #CVPR2022
COAP is useful for resolving self-intersections and collisions with other objects neuralbodies.github.io/COAP
w/ @psyth91 @aayushbansal @MZollhoefer @SiyuTang3
1
14
75
Colabでいろんな画像アップロードして#pifuhd タグをつけて結果をTweetしてもらえると、今後の改善につながるのでご協力お願いします〜
colab.research.google.com/dr…
4
24
64
Wow super nice!! PIFu is now on @huggingface!
Here is a quick @huggingface Spaces demo for the original PIFu project. From a single image, it can generate a 3D model with colors!! While it's an old method (2 years old 😂) it's still very very impressive
github.com/shunsukesaito/PIF…
huggingface.co/spaces/radame…
2
9
65
Very cool benchmark/dataset for 3D neural avatars!! And super happy to see our KeypointNeRF (ECCV 2023, with @marko_mih) outperforms other baseline methods across all categories in SSIM and PSNR!!
dna-rendering.github.io/inne…
Excited to share our new dataset work DNA-Rendering for high-fidelity 3D neural avatars 💃🕺, which is accepted by #ICCV2023. It provides 1500 ID*outfits, 1187 motions, 60 views and 6.75M frames up to 4096 resolutions!🔥🔥🔥
Project page: dna-rendering.github.io/
1
13
57
11,284
Check out our #ECCV2024 work on expressive whole body 3D Gaussian Avatars (incl. face and hand control) from a monocular video!! Led by super Gyeongsik!
Attention! Meta dropped a new highlight based on monocular videos that will definitely be under-hyped.
[ECCV '24] Expressive Whole-Body 3D Gaussian Avatar
arxiv.org/abs/2407.21686
Project: mks0601.github.io/ExAvatar/
YouTube: piped.video/GzXlAK-sBKY?si=N4j1…
1 / n
2
5
61
5,665
Wow! Sapiens is nominated for ECCV best paper!!
Presentation info:
Oral 1C, Wed, 2nd Oct, 1:30 p.m. CEST, Silver Room.
Poster 113.
GitHub: github.com/facebookresearch/…
3
5
60
5,890
Presentation today @3DVconf ! Jingfan and I will be in poster session. Please drop by!
🎉 Excited to share our #3DV2024 paper: "Diffusion Shape Prior for Wrinkle-Accurate Cloth Registration".
We show how to incorporate a powerful diffusion prior into cloth 3D registration, and achieve wrinkle-accurate alignment!
Project: www-users.cse.umn.edu/~guo00…
4
59
4,557
CMUのKeenanによるDifferential GeometryレクチャーがYouTubeに上がり始めた！ビジュアリゼーションもすごい綺麗。要チェック
This lecture is part of a course on #DiscreteDifferentialGeometry at @CarnegieMellon @SCSatCMU.
Full playlist: piped.video/watch?v=FRvhgkGK…
Course webpage: geometry.cs.cmu.edu/ddg
Course notes: cs.cmu.edu/~kmcrane/Projects…
1
8
56
足が映っていなく画像で画像の切り出しがうまくいってないケースが散見されたので、アルゴリズムを改善して足が見切れている場合でもうまくいくように改善しました
Google Colab Link: bit.ly/37sfogZ #pifuhd #
2
10
53
有志によるテック企業AI研究者に特化した給与提示額(bade, RSU, etc)の統計がまとまってるウェブサイト。世界でAI研究者が大体どれくらい貰ってるかがよくわかる。給与交渉の際には参考にしたい。
Introducing AI Paygrades (aipaygrad.es)! Statistics of industry offers for AI jobs. The goal is to reduce information assymetry so candidates can make informed decisions and negotiate better. Submit your information and spread the word! With @abhshkdz.
6
53
PIFuをリアルタイムにする研究がECCV通りました〜 Stay Tuned!!
Oh congratulations @ruilong_li , @yuliangxiu, @psyth91 Zeng Huang, Kyle Olszewski for their ECCV2020 paper: Monoport: Monocular Real-Time Volumetric Teleportation. Our SIGGRAPH 2020 RTL! demo is based on this paper, which demonstrates how to make #PiFU real-time! Stay Tuned!
6
50
SCANimate (scanimate.is.tue.mpg.de)がCVPRのベストペーパーにノミネートされました！2年ぶりのノミネート嬉しすぎる！
The #CVPR2021 candidates for Best Paper can be found here: cvpr2021.thecvf.com/node/290 Congratulations to all of the authors whose papers are in contention!
1
2
46
Wow! Clothed human reconstruction at scale!!
Another narrowing domain: 3D Body Reconstruction
PIFuHD + fashion dataset + Blender
Processed ~300 full model images (few seconds x img). No clean-up.
Fast camera to hide hacky UV projection
#b3d #3dml #3dbodyreconstruction
6
46
🐱Check out our latest work on building behavioral simulators of animals and humans from casual video captures!!
Sharing my recent project, agent-to-sim:
From monocular videos taken over a long time horizon (e.g., 1 month), we learn an interactive behavior model of an agent (e.g., a 🐱) grounded in 3D.
gengshan-y.github.io/agent2s…
1
6
46
5,922
Wow!! Two Minute Papers @twominutepapers features PIFuHD! Thank you!!
3
3
45
📢 Come to see our #CVPR23 poster (#40) of MEGANE this afternoon if you are interested in interaction-aware generative modeling, relighting, and digital humans!!
🚨 Excited to introduce our #CVPR2023 paper, MEGANE🤓!!
MEGANE is an interaction-aware compositional 3D morphable eyeglasses and head model, supporting photorealistic rendering & relighting. (1/7)
👉Project: junxuan-li.github.io/megane/
1
9
42
8,659
NeRFでも使われているPositional Encodingがなぜ高解像の復元に適しているのか理論的に分析した論文が公開されたっぽい。下についてるAuthorのSIRENとの違いに関するコメントも興味深い。
people.eecs.berkeley.edu/~bm… We figured out why the "positional encoding" used in NeRF works so well! NTK theory says that using an MLP with a Fourier basis yields a composed kernel that is good at interpolating (as per basic signal processing). Excited to see what people do with this
6
43
コンピュータビジョンのトップラボの一つ、MPI-ISのMichael Black（前所属先のボス）による、31年間のICCVの変遷と自身の経験、そしてCVの未来についてまとめたブログがめちゃくちゃ面白い。
perceiving-systems.blog/en/n…
3
43
My oral presentation session is happening NOW!
📢 Check out 𝗥𝗲𝗹𝗶𝗴𝗵𝘁𝗮𝗯𝗹𝗲 𝗚𝗮𝘂𝘀𝘀𝗶𝗮𝗻 𝗖𝗼𝗱𝗲𝗰 𝗔𝘃𝗮𝘁𝗮𝗿𝘀!
Our latest codec avatars using 3D Gaussians generalize to novel lighting (OLAT, envmap) with *all-frequency* reflection (see video for hair and eye reflection) in real-time! shunsukesaito.github.io/rgca…
5
42
3,075
Appleが学習データ外のカテゴリーにも一般化されるような高精度な3次元の復元の論文出してる。Multi-view integrationなどPIFuと近いアイデアが使われてるみたいだけどより洗練されている印象。
1/n Check out our @Apple research paper "On the generalization of learning-based 3D reconstruction" (or 3D43D) arxiv.org/abs/2006.15427
4
39
"Digital Human meets Neural Fields" happening now!
📢Join me tomorrow at our #CVPR tutorial on Neural Fields (Neural Implicits/Neural Scene Representation)! You will learn foundation and latest advances of this exciting "field"! I will talk about Digital Human applications and its open questions! neuralfields.cs.brown.edu/cv…
1
1
34
CVの教科書に研究が載る実績を解除した
Szeliski先生のComputer Vision: Algorithms and Applicationsの二版、八月にドラフト完成してた
szeliski.org/Book/
PBRTと合わせて読めばレンダリングとCVの知識一通り得られて「勝ち」では？合わせて2500ページくらいありそうだけど。
1
36
去年に引き続き自分たちの論文がSIGGRAPHトレーラーに載りました！
piped.video/VBZ2sDxvZQE
1
8
37
6,050
Super excited to give a talk in #CVPR23 workshop on high-fidelity neural actors tomorrow!
Attending #CVPR2023 soon? Check out our workshop with great line-up of speakers. @synthesiaIO research is looking forward to meeting you in person!
Speakers: @K_S_Schwarz, @psyth91, @dancasas, @GerardPonsMoll1
Workshop: hfna-workshop.github.io/
ActorsHQ: actors-hq.com/
2
36
5,503
arXivの論文リストのトップに欧米の有名ラボばかりよく見かける理由、知ってますか？知る人ぞ知るarXivの論文リストの順位の上げ方に関する興味深いスレッド。（引用数とリスト順位が有意に相関があるのは知らなかった。。。）
Have you ever wondered why papers from top universities/research labs often appear in the top few positions in the daily email and web announcements from arXiv?
Why is that the case? Why should I care?
5
35
Amazing to see how far pixel-aligned implicit functions have come within 3 years!! Very nice results, congrats @thiemoall !!
Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing
abs: arxiv.org/abs/2204.08906
project page: phorhum.github.io/
1
2
36
If you are at #ICCV2023, must check out NCHO (Neural Composition of Humans and Objects) led by Taeksoo @taeksu98 (he was an undergrad when starting collaboration!) We enable, for the first time, learning "compositional" 3D generative avatars from raw scans. (Oct 5th, 2:30-4:30pm)
Super excited to share our work NCHO at #ICCV23!!
NCHO is a first ‘compositional’ 3D avatar generative model learned from real-world 3D scans, that enables separate controls over humans and objects. (1/8)
with @psyth91 and @jhugestar!
Project page: taeksuu.github.io/ncho/
5
35
5,189
Super happy to be chosen as outstanding reviewers at #ICCV2023!! Also congrats to my intern @Frozen_Burning !!
1
2
33
6,611
指導教官、大学やめるってよ
Big News! After 7 wonderful and exciting years as professor, I'm happy to announce that I'm leaving @USC and the @USC_ICT , and I will fully commit to @PinscreenInc as CEO in order to take it to the next level!
1
30
🤲Interested in real-time relighting, 3D hands, and digital humans? Come to see our #CVPR2023 poster (#18) of RelightableHands this morning!
I'm thrilled to announce that our paper on the neural relightable hand model has been accepted for CVPR2023! I owe a big thanks to my mentors @psyth91, Tomas Simon, and collaborators during my internship at Meta! Check out more details on our website! sh8.io/#/relightable_hands
8
29
4,498
Deep Learningに使われる回転(SO(3))の表現方法で6次元表現がいいって言われてたけど、SVDによるさらに良い方法が提案されたらしい？
I'm excited to share this new work. Take a look if you care about representing 3D rotations in deep networks.
"An Analysis of SVD for Deep Rotation Estimation," with @jakelevMtl, @_machc, @KefanChen7, @Jimantha, @akanazawa, and @rostamiz.
Preprint: arxiv.org/abs/2006.14616
8
30
🚨Check out @jiyuval_'s amazing #CVPR2024 work, InterHandGen!
We build a *unified* framework to build strong prior for interacting two hands w/ or w/o an object. We significantly boost the generation quality via Cascaded Reverse Diffusion! Code 👉 github.com/jyunlee/InterHand…
🚨 InterHandGen (#CVPR2024) is one of the first works on the generative modeling of 🤲interacting two hands (with or without an additional object). This diffusion-based prior be incorporated into any existing method via SDS-like loss to boost the estimation accuracy!
(1/🧵)
28
7,208
OMG! Check out our work SCALE (#CVPR2021)!! Using SCALE, you can model clothed avatars (incl. skirts!) with *EXPLICIT* shape modeling but *NO* clothing template or registration!
Wanna know how? 👉qianlim.github.io/SCALE
Colab. w/ @qianli_m, Jinlong, @SiyuTang3, @Michael_J_Black
7
26
🎉 Are photo-real relightable avatars only useful for creating pretty pictures? No! Re:InterHand (#NeurIPS2023) shows significant boost in monocular two-hand estimation with synthesized data!
Download data for boosting and benchmarking your method here👉mks0601.github.io/ReInterHan…
2
27
3,813
2. Diffusion Posterior Illumination for Ambiguity-aware Inverse Rendering (vcai.mpi-inf.mpg.de/projects…)
12/14/2023 (Thu), 11:10pm-11:25pm
Diffusion-prior meets inverse rendering for high-resolution illumination recovery!!
1
4
28
3,171
Wow! Very nice work! Amazing that you guys made such an awesome dataset AIST++ @ruilong_li @akanazawa ! Also this became possible with the dedicated work to build the original AIST dataset by @shuhei2306 et al.!
google.github.io/aichoreogra…
1
4
25