Assistant Professor in Comp Neuro and ML at @epfl_en @CampusBiotech | #deeplabcut co-developer | past: @LMU_Muenchen, @uni_tue and @Harvard
Geneva, CH
- Tweets 1,303
- Following 742
- Followers 3,789
- Likes 2,559
Pinned Tweet
This summer I spent time in Alaska doing field work with bears. It was a wonderful experience to learn more about bears and meet our collaborators!!
6
1
79
8,039
Excited to share #DeepLabCut, which also contains a first glimpse into my odor guided navigation work with @UtopianCynic and @MatthiasBethge arxiv.org/abs/1804.03142v1 …
6
75
215
Thrilled to announce that I will start as an assistant professor at @EPFL_en in early 2020! Thanks to my mentors: Andreas Herz, @MatthiasBethge and @UtopianCynic. I’m looking forward to move with @TrackingActions to Switzerland!
15
5
151
What is the integrative logic of proprioception? Sensory information is collected in muscle spindles distributed in the body, but how is the information integrated. We took a computational approach to this problem... cell.com/cell/fulltext/S0092…
4
21
127
16,625
.@marius10p thanks for the idea - seems like nothing is camouflaged to #DeepLabCut - Trained with !! 14 frames !! for 8min on GPU (6,000 iterations). Frame-by-frame prediction. Paper: goo.gl/g7UVXN @sofiabiologista
4
32
106
Thanks to @snsf_ch for funding my research project titled: "A theory-driven approach to understanding the neural circuits of proprioception".
I'm looking for a postdoc, PhD student and research engineer. Please RT.
2
31
80
Do you want to learn how to track poses, analyze what animals are doing, identify your favorite bear or link behavioral to neural data?
Join the hands-on, 5 day Cajal course from 6 to 10 November! tinyurl.com/yckpd5ek @Cajal_Training
6
20
61
29,629
Thank you for this recognition. Receiving this award with Mackenzie means a lot to me. We are also grateful to all past and present collaborators as well as our group members for making this possible. Thanks for the fantastic journey to @NeuroVenki, @MatthiasBethge, @DulacLab, ..
🍾Deeply humbled, honored 🙏🏼, and so, so happy to share this with my best friend & life partner @TrackingPlumes 🥰.
Also a huge thanks to our mentors and labs - it’s been a privilege to work with you🙏🏼
2
2
50
5,019
One major idea we are putting forward in this work is that proprioception can be understood normatively as having to solve action-recognition from receptor inputs! These "actions" can then be used for perception, balance, motor control and learning...
✨#tweetprint: Introducing our ANN model(s) of the proprioceptive system!
@OpenSimSU+spindles+ANNs+action-recognition task.
We find task-training really matters for ANNs + trained units resemble S1 neurons
preprint: biorxiv.org/content/10.1101/… project page: mathislab.org/deepdraw
1
5
50
Are you interested in motor skills, musculoskeletal control and reinforcement learning?
Check out our manuscript: biorxiv.org/content/10.1101/…
2
18
52
18,016
Very excited to introduce the beta version of the @DeepLabCut Model Zoo! We envision that over time, for common experiments training will no longer necessary. For now, during stay at home orders dog and cat models might be particularly useful.
Did someone say dogs? 🐶
✨ Introducing the DeepLabCut Model Zoo ✨
A growing collection of trained models for you to immediately use - no training required, no installation required (see below)! 🐕🐵🐈🐕🧍🐭 #openscience
mousemotorlab.org/dlc-modelz…
2
10
47
I’m looking for students to work on ‘deep phenotying’ and theories of sensorimotor learning! Apply!!!!
1
30
37
Implicit shape representations, such as Signed Distance Fields (SDF) are great for 3D reconstruction. But do they help 3D pose estimation? I'm sharing our recent work HOISDF, which has just been accepted by #CVPR2024 work led by PhD student Haozhe Qi!! arxiv.org/abs/2402.17062
1
6
41
11,860
Mice pulling manipulandums? Voila... @TrackingActions
cell.com/neuron/fulltext/S08…
1
15
36
The Schools of Computer and Communication Sciences (IC) and Life Sciences (SV) at EPFL invite applications for a faculty position at the intersection of computational and life sciences. @epflSV @ICepfl Please RT. epfl.ch/about/working/joint-…
2
17
30
I had a great time speaking at @SonyCSL in Tokyo. Thanks to @neuromusiker for being a wonderful host! #deeplabcut
1
7
29
My lab will be working on quantifying behavior and computational models of motor control! I’ll be recruiting soon... please reach out to me if you are interested!
1
3
31
We had great fun participating & writing the paper with all the winning teams and organizers-- check it out! I'm quite excited what one can learn about biological motor control based on the new simulators! Our team: @chiappa_alberto @pablo_tano8 @nisheet0 @pouget_alex
MyoChallenge 22 paper is out 🔥
Have a look at the smart solutions 🧠 of last year's winners
Lots of good ideas in it for this year's manipulation challenge😈
➡️ proceedings.mlr.press/v220/c…
1
3
28
4,821
Greatly enjoyed contributing a tiny part to this exciting cancer study of Profs. @EPFL_altug_lab and Camilla Jandus. I joined the project after discussions at the @epflSV retreat in Jan 2020 and learned a lot!
video clip of the multi-cell tracking w/#maDeepLabCut from: advances.sciencemag.org/cont…
1
4
28
We are looking for a few more teaching assistants --> so please apply!
So far, we have local hubs in Athens, Bochum, Buenos Aires, Geneva, Kigali, London, Munich, Okinawa, and Oxford... you can pick any of those or a virtual hub, when you apply as a student!
Do you want to learn how to track poses, analyze what animals are doing, identify your favorite bear or link behavioral to neural data?
Join the hands-on, 5 day Cajal course from 6 to 10 November! tinyurl.com/yckpd5ek @Cajal_Training
2
1
23
9,505
Now out in the International Journal of Computer Vision! link.springer.com/article/10…
🐻 WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models biorxiv.org/content/10.1101/…
from @amathislab 🎊
1
4
25
8,389
The official job ad with application details is online: recruiting.epfl.ch/Vacancies…
👋 @amathislab & @mwmathislab are looking to jointly hire a research software engineer at EPFL - one of the nicest places to work, very multicultural, super team oriented, work hard/play hard with Mount Blanc & Lake Léman in your backyard type of vibes 🏔️🧗♀️🎿⛵️Ad coming soon! 👀
11
22
9,805
Second day on the job & excited to participate in a symposium on “Emerging technologies for chemosensory research” symposium at #ISOT2020 with @DulacLab, @ShohamLab & Florin Albeanu (live from my chalet) achems.org/virtual/index.php…
1
2
20
Nice preprint on modeling grid cell firing patterns in arbitrary enclosures... I really hope to see recordings in some of these "open fields" biorxiv.org/content/10.1101/…
1
18
Trails of hominid footprints, “motion” captured by Pliocene deposits at Laetoli that date to 3.66 million years ago, firmly established that early hominoids achieved an upright, bipedal, and free-striding gait (Leakey & Hay, 1979). Beyond fossilized locomotion, behavior can [...]
🥳#teamDLC is delighted to present our Primer on Motion Capture with Deep Learning in @NeuroCellPress!
📑 want to know more about neural networks, optimization, data augmentation?
📈 pitfalls to avoid?
Paper: cell.com/neuron/fulltext/S08…
Complete w/code: github.com/DeepLabCut/Primer…
2
17
Check out this exciting work by PhD student @GabeffValentin co-advised with @devistuia! This is the full version of the article that was selected as an oral at CV4animals #cvpr2023.
🐻 WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models biorxiv.org/content/10.1101/…
from @amathislab 🎊
3
20
2,367
Check out the nice spotlight by @HansScherberger in Neuron: sciencedirect.com/science/ar…
What is the integrative logic of proprioception? Sensory information is collected in muscle spindles distributed in the body, but how is the information integrated. We took a computational approach to this problem... cell.com/cell/fulltext/S0092…
1
1
17
4,160
We are looking for a postdoc/graduate student/research intern to work on Brown Bear Identification and Behavior Analysis!
mathislab.org/currentopening…
1
6
15
Having fun seeing dancing sharks, listening to student talks and giving a #deeplabcut demo @NSB_MBL at @MBLScience Woods Hole!
4
18
Join us at this exciting summer school with @dyamins, @MatthiasBethge, @BrechtLab, @TrackingActions, @eywalker, @AToliasLab, Josh McDermott, Maneesh Sahani, and many more!!
🤩 Exceptional registration opened to attend the #FENS #SummerSchool 2022 on "Artificial and natural computations for sensory perception: what is the link?", taking place on 22-28 May in Bertinoro, Italy. Apply before 18 March. loom.ly/GorZLVk
@BathellierL @AToliasLab
1
6
16
Neuro jobs at @epfl_en -- The Ecole polytechnique fédérale de Lausanne (EPFL) invites applications for two faculty positions at the Tenure Track Assistant Professor level in the Brain Mind Institute of the School of Life Sciences. epfl.ch/about/working/facult…
9
15
This was a great collaboration with Rick, who has been one of the earliest adapters and contributors to #DeepLabCut!
"On the inference speed and video-compression robustness of #DeepLabCut" -up to 1,000 FPS, testing different GPUs, and showing that DLC is to robust video-compression! - save time & data space. New work from @TrackingPlumes & Rick Warren. @biorxivpreprint biorxiv.org/content/early/20…
4
15
Really love the technical solution behind this GUI, which is actually more of an interactive plot -- pure @matplotlib mastery by @jessy_lauer! So happy to have him on the team. Thanks @cziscience for the support!
#maDeepLabCut: multi-animal support + high perf. pose estimation. We will highlight updates in the coming days, but here is 1:
This NEW tracking GUI built w/@matplotlib allows editing errors or swaps, & deal with occlusions:
📹 piped.video/bEuBKB7eqmk
📹 piped.video/YSRQT8N2vFE
1
4
16
Join the Cajal hands-on neuroscience course Quantitative Approaches to Behaviour, in which I will be taking part with @DeepLabCut and friends!
19 July - 8 August 2020 at the
Champalimaud.
Apply here: fens.org/Training/CAJAL-prog…
5
12
Congrats to the team! So proud of PhD students @chiappa_alberto, @pablo_tano8 and @nisheet0 and co-Alex: @pouget_alex
Team stiff_finger (@pablo_tano8 @chiappa_alberto
@nisheet0) receiving their award plaque and part of the organization team (@CaggianoVitt @GDurandau @Vikashplus ) during #NeurIPS2022
1
1
13
Too bad I’m not going. Have fun everybody!
14
Excited to be part of this amazing symposium #LocationLocationLocationandTime (I like the nod to Minkowski space that embeds us all)!
1
2
11
Super proud of my first two PhD students @LStoffl, who finished his first study on using transformers for pose estimation and @a_marinvargas, who wrote a review on ML for motor neuro!!! arxiv.org/abs/2103.12115 and arxiv.org/abs/2103.11775
We developed an end-to-end trainable multi-instance pose estimation model with transformers -arxiv.org/abs/2103.12115 - great work by PhD student @LStoffl and Master's student @vmaxmc2!!!
1
1
12
You can attend the course online, or join at one of several satellites across the globe (this map is likely not final...). #GreenDistributedDLC
Applications extended till 24 Oct for our new #NeuroKit on Modern Approaches to #Behavioural #Analysis. Learn the philosophical and observational skills required to understand #behaviour 👉 loom.ly/3uL0Vvw
📆 21-25 Nov 22
🌏 Online
👥 @TrackingPlumes @amathislab @taunbot
1
7
12
Thanks to my wonderful team, collaborators and the DLC community! 2022 was great fun!! Happy holidays!!! --> Next, I'll be off the grid for a while! #ski
🎁 The #DeepLabCut 2022 Year in Review 🗓
- what we achieved, what you loved, where we are going in 2023!
- THANK YOU to the amazing community
Read on our blog now⬇️
(and a new DLC release will be out before we head off for a holiday break 2.3💜✅🙏)
link.medium.com/m6A6cUpSZvb
2
12
4,558
New work with awesome summer student @mertyuksekgonul, @Perf_Genetics, @bethgelab and @TrackingActions
Thus, transfer learning - by using pretrained-ImageNet networks - boosts robustness & performance for pose estimation, and is really important for lab-sized datasets to achieve robust results! This work was led by: @TrackingPlumes Preprint: arxiv.org/abs/1909.11229 5/
4
12
Great work and quite happy about the name choice ... Rock me Amadeus!
Of course, this is NOT a🪄 solution, it's a proof of concept & we, led by 🧙@shaokaiyeah, are actively improving AmadeusGPT!
We greatly thank @kinematikAI for hosting this demo in 🤝@mwmathislab & the @TheValleeFdn for funding!
piped.video/watch?v=cVikZ8Oe…
1
12
1,892
Thanks to @BohacekLab for your exciting talk. On a personal note, it was my first talk in Austria, since I finished high school!
A real treat to present together with the @DeepLabCut developers @TrackingPlumes and @TrackingActions in a symposium at the International Winter Neuroscience Conference #IWNC in Sölden, Austria. For both Alexander and me our first science presentation in our motherland!! 😯😊🥳🇦🇹
13
#CompNeuro issue @CurrentOpinioninNeurobiology tinyurl.com/y98lclw8 covering #RNNs, #sleep, #learning, #BMIs, #gridcells, #maxent, etc.
9
10
DLC 2.1 includes dynamic auto-cropping. This dramatically improves the inference speed for open field setups (like for the rat below)where the animal is much smaller than the frame and no prior tracking is necessary. #DeepLabCut
3
2
10
Great Q&A with Andreas Herz (my wonderful PhD supervisor!) cell.com/current-biology/ful…
2
10
Fantastic work led by AI4Science fellow Stephane d'Ascoli & Sören Becker!! You have data and want a symbolic ODE -- ask #odeformer! #ai4science
🚨 ODEFormer is on Arxiv! arxiv.org/pdf/2310.05573.pdf
We show that Transformers can recover the differential equations governing dynamical systems from noisy & irregularly sampled trajectories.
Very fun collaboration with @SorenBecker, @TrackingPlumes, @pschwllr & @k__niki!
🧵⤵️
3
11
3,230
We had a great time meeting many colleagues in Kobe while visiting Prof Atsushi Iriki at @riken_en! Next stop Hokkaido for a conference! @TrackingActions @neuromusiker @phyloinfokobe
1
9
Arm movements induced by noninvasive optogenetic stimulation of the motor cortex in the common marmoset pnas.org/content/early/2019/…
1
9
Great for capturing behavior ;)
9
Sliman was a wonderful character & great scientist. I enjoyed every moment with this force of nature. I'll miss him💔
Horrible news but a beautiful tribute. Sliman will be missed.
Sliman Bensmaia, leading expert on the neuroscience of touch, 1973-2023
news.uchicago.edu/story/slim…
9
2,103
Really excited to host @slimanjbensmaia next Wednesday at EPFL. memento.epfl.ch/event/cnp-se… @CNP_EPFL @NeuroLeman
1
10
@valedisanto, @WilliamMenegas, @pranavm42, @meet10may, @UnitAfrican, @amathislab, @gkneuro, @TrackingNoise, @akaijsa, @jessy_lauer, @Nyktofob, @naoshigeuchida, @stes_io, @georgelauder, @zhoumu53, @LStoffl, @glopesdev, @json_dirs, @Perf_Genetics, and many others!! 🙏
3
9
621
Faculty search at the interface of life and computer science!! @ICepfl @epflSV -- This broad search also integrates Comp. Neuro, NeuroAI, etc. epfl.ch/about/working/joint-…
1
2
9
That cake!
It's been a pretty intense tracking week! Amazing course and better people. Can't wait to put all this knowledge into action! #deeplabcut #londonhub @DeepLabCut @Cajal_Training @taunbot @SofiMinano
1
9
Yesterday, warming up for @ScienceMarchTUE at the botanical garden @tueboga w @TrackingActions Thanks to @PlantEvolution for the shirts!!!
7
This is the second time we organize this course. Last year we had 93 students from across the globe!
Here are the logos of the hubs we had then, from London, Nairobi, Munich, Buenos Aires and Warsaw. This year we hope to cast an even wider net!! Join us.
1
2
8
1,567
Check out the cool work of the DLC AI residents!!
🎉 #DLCAIResidency - @Nirelkadzo & #teamMegaDLC
🔥 The team hooked up 💚MegaDetector+DeepLabCut 💜 and deployed it on 🤗!
📚 Read about it in the #cookbook: deeplabcut.github.io/DeepLab…
𐂐 And hey, it's live, go test it out! @huggingface App 🤗: huggingface.co/spaces/DeepLa…
9
This was a great collaborative project!! Check out DLC-Live!
🥳𝗗𝗟𝗖-𝗟𝗶𝘃𝗲! fast, low-latency posture-based feedback! new python pkgs PLUS integration into
- @bonsai_rx
- Autopilot
- a new DLC-Live! GUI
Kane et al @gkneuro, @glopesdev, @json_dirs @TrackingPlumes @TrackingActions
📜biorxiv.org/content/10.1101/…
📹
piped.video/KDSgddOqHtM
4
Amazing paper from amazing people!!! Congrats.
📣Part 1 of our quest to better understand the brain was @DeepLabCut.
🔥🦓Now Part 2: Introducing #CEBRA to jointly model neural dynamics & behavior with self-supervised learning. Hypothesis- or data-driven, highly consistent, decodable neural latents
arxiv.org/abs/2204.00673
🧵👇
7
Special thanks to @slimanjbensmaia with whom we had deep discussions during his visit to EPFL in 2021 and of course @TrackingActions for discussions throughout.
2
8
802
@SofiMinano @andrada_marica @Nirelkadzo @scientistjonas @Zanemit @AcaGavric @NasFilippova @huang_shuhong @shura_gav
The course is co-directed by @taunbot
1
1
8
962
Really like how this plot highlights the stereotypical movements!
3D arm movements were analyzed using multiple cameras and the awesome @DeepLabCut
Monkeys optimized their behavior (although differently) across trials despite not being restrained. I.e. we can do conventional analyses as in “constrained” experiments but within a large env 7/n
1
6
We show that HOISDF achieves state-of-the-art results on the widely used hand-object pose estimation benchmarks (DexYCB and HO3Dv2). Check out our project page for more details amathislab.github.io/HOISDF - This work was done with Chen Zhao and Mathieu Salzmann (EPFL).
1
7
678
Very grateful for the funding & thrilled to start the AI Residency Program next summer!
The @DeepLabCut developed by the Mackenzie Mathis Laboratory and the Alexander Mathis Group, receives an EOSS award and launch an AI Residency Program! Congratulations!
More info: bit.ly/3zzWylt
@cziscience @epflSV @TrackingActions @TrackingPlumes @EPFL_ELLIS_unit
2
7
The full version is now out: elifesciences.org/articles/8…
🥳 @akaijsa & @pranavm42 et al introduce task-driven modelling of the proprioceptive system, now out in @eLife!
Our work combines deep learning & biomechanics to test theories of💪sensory representations
elifesciences.org/articles/8…
1
6
1,821
Standard CPU also work for testing (just much slower). Depending on image size GPUs with as "little" as 6GB suffice...
6
The final version just appeared today in @NeuroCellPress authors.elsevier.com/sd/arti…
6
That's a beautiful paper!
The grids are a torus, and continuous attractors are real!! Neuropixels makes a difference, as do outstanding postdocs and collaborators! #Gridcell #KiloNeurons @KISNeuro #CAN biorxiv.org/content/10.1101/…
7
Organized by @DimaRinberg! I'll be talking about @DeepLabCut, the model zoo, and odor-guided navigation (work with @UtopianCynic, @siddjakes29, @TrackingActions, @jessy_lauer, ...)
6
Awesome to be there!!!
#EOSSmtg @cziscience kick off day 1! Beyond excited to see all the incredible tools, open source enthusiasm, and focus on inclusion and community (And so fun to have @DeepLabCut w/@matplotlib on the meeting cover)
4
Apparently ... I joined Twitter 7 years ago?!? Fortunately, I only spent a few minutes of those 7 years on here ...
5
Why do I bring (c) up: Whatever the motivations for "citing/not citing" are: forgetting, purposeful omitting, not being able to cite something because it didn't yet exist... IMHO authors should give other authors the benefit of a doubt... 4/
1
5
Dynamic cropping is run just like the standard analyze_videos but with setting the dynamic flag: deeplabcut.analyze_videos(path_config_file,[video], dynamic=(True,.1,30)). See: github.com/AlexEMG/DeepLabCu…
1
5
Thanks to all the co-authors (@pranavm42, @kaijsandbrink, @clmich, @TrackingActions, @MatthiasBethge) that have made this work a great journey!
5
Prior workshop materials are available here github.com/AlexEMG/DeepLabCu…
We will post the 2020 slides soon!
1
5
If you want to be a (paid) teaching assistant, to either teach online or organize a local hub please apply here form.jotform.com/23197423297… or reach out! You can also chat with the amazing previous TAs such as
@facuemina @ChuckleScience @Sabrineiitor @SofiMinano @NicoleVissers1
1
2
6
833
Of course, big thanks also to all the co-authors on the two papers @SebHausmann, @TrackingActions and @vmaxmc2
5
“Great ad” after watching this.... @BohacekLab
Very happy to share our latest work! Frustrated with expensive and inflexible rodent behavior testing systems, we compared @DeepLabCut head-to-head with 2 leading commercial systems. But first, please turn up the volume to 11 for this trailer 🙂 #Rammstein #MachineLearning
1
6
A great post on the power of *appropriate* data augmentation for making more robust pose estimation models by @scientistjonas !!
🎉 #DLCAIResidencyWeek - @scientistjonas🦇
Jonas has been hard at work extending the @napari_imaging-DLC GUI to include multi-cam labeling, & testing how to make robust models with unbalanced data (🦇 flying left->right).
📚how he solves it: #DLCBlog 🔥deeplabcut.medium.com/balanc…
5
Character recognition form muscle spindle input is just one example for this objective, but I think it has many advantages and as we show is sufficient for the emergence of "interesting" coding properties!
1
5
Great title!
Self-attention mechanism can be viewed as the update rule of a Hopfield network with continuous states.
Deep learning models can take advantage of Hopfield networks as a powerful concept comprising pooling, memory, and attention.
arxiv.org/abs/2008.02217
github.com/ml-jku/hopfield-l…
3
Congrats & great paper!
It's out! The lab's first paper: Mixture Coding and Segmentation in the Anterior Piriform Cortex. Work lead by twitter-less Sapir Penker frontiersin.org/articles/10.…
1
6
It was a pleasure to listen to @EdvardMoser sharing the fascinating discovery of a toroidal neural code during navigation & sleep! biorxiv.org/content/10.1101/…
#EBBS2021 fully hybrid format seems to be working!
We have @EdvardMoser, Nobel Prize winner presenting his work and an engaging discussion animated by Pierre Lavenex and @TrackingPlumes. @EBBS_Science Meeting taking place right now at Forum Rolex @EPFL_en!
Thanks everyoneü
4
Amazing work!
Very proud to share our last @Nature paper. We shown that agents trained to localise in an environment, spontaneously develop grid-like representations. This shows that grid cells support vector-based navigation (e.g. shortcut). deepmind.com/blog/grid-cells…
2
Sadly still timely & worth listening to: openvault.wgbh.org/catalog/A…
4
Check out paperswithcode.com/paper/hoi…
Implicit shape representations, such as Signed Distance Fields (SDF) are great for 3D reconstruction. But do they help 3D pose estimation? I'm sharing our recent work HOISDF, which has just been accepted by #CVPR2024 work led by PhD student Haozhe Qi!! arxiv.org/abs/2402.17062
4
1,004
Version 1.0 of pypi.org/project/deeptinberg… will hopefully be my tenure package... prospective #students and #postdocs join and help me fill it with code!
2
1
5