design * prod * eng * technical staff @lumalabsai ・ latent space explorer * generator ・ spatial design eng ex @leapmotion @nasaJPL ・ av ml artist
San Francisco, CA
- Tweets 3,472
- Following 1,937
- Followers 7,531
- Likes 22,643
Pinned Tweet
latent space shines when it’s interactive in realtime, we love dynamic feedback
We’re all looking at the latent space, but @graycrawford is inside of it.
1
1
66
10,423
I realized chess pieces can be redesigned to be purely geometric attack directions
306
1,687
19,058
1,597,276
four months of conversation
ALT Five months of iMessage conversation bubbles filling up the entire phone screen
26
127
2,548
I’d love to share my VR Master’s Thesis, completed May ’19
✨Developing Embodied Familiarity with Hyperphysical Phenomena✨graycrawford.com/thesis
a culmination of VR prototypes + philosophical essays on
🌌spatial intuition with novel physics
✍️bodily ownership+representation
39
333
1,700
Working on some VR visual programming UI with the #VisualEffectGraph
Interested in how particle-based wires can be "displays" of the data they carry, or reactive to the hand as input
28
322
1,535
Currently diving into
guiding #VisualEffectGraph particles
with my @LeapMotion hands
using @aman_gif's MeshToSDF converter
Already seeing *so many* places where this can go; stoked to explore further
28
220
1,332
loved sharing these ‘wide view’ explorations of iMessage history! 💬
hints at how spatial OSs give so much room to spread out
there’s so much personal meaning embedded across innumerable conversations
I’m stoked how XR can let us recall their breadth
capitalized time
29
90
1,203
188,283
I'm continually astounded by Unity's #VisualEffectGraph
This feels like legit magic 💫
12
132
776
Friends!
@aman_gif and I built a system for exploring neural-network-generated images, maneuvered by your hands in real-time
Traversing bigGAN's high-dimensional space of pseudo-real images is enthralling, like divination via movement, so we called it XOROMANCY
13
180
704
I’m working on using my Apple Watch for haptics in VR + @LeapMotion
There’s potent sensory flexibility here, seeing the collision at the fingertip but feeling it at the wrist — repetition likely fusing them more
I fully expect Apple to use wrist haptics in their headset AR push
10
121
597
Growing some UI for spatially navigating the branching structure of reply threads
Interested in directly swiping through parallel piles of nested replies
(instead of the current "💬3" that can hide entire massive branches)
Made on-device, pushing @createwithplay to its limits
26
50
536
Excited to share my fave project while at @Ultraleap,
Waft Locomotion! 🍃
Wafting with your hands allows nimble, fluid movement in VR —
Moving through the world is deeply human and feels good!
Any truly spatial computer must uphold our natural mobility.
→ 🧵
21
67
475
77,657
I’m struck by how Virtual Virtual Reality’s leash communicates the mass of rendered objects. By making the intermediating structure "elastic", its mechanical connection maps the mass of the grabbed object to the visual displacement of the leash.
8
60
376
These past few months I and some friends (@supercgeek @lucas0choa @lumar_isa @GautamBose18 @YujinAriza) built six (!) @LeapMotion North Star AR headsets with help from the STUDIO for @CreativeInquiry and @cmuetc.
Shout-out to @noazark's clutch component sourcing!
11
63
310
fully generative XR OSs have SO much potential!
entire environments, our approach to space, material, scale, context…
I’ve been loving using image prompts in Midjourney v4 — critical to specify what words alone cannot (and the text prompt couldn’t even fit in here)
🧵:
13
50
305
Working on cloth physics in Unity, seeing what parameters I might map to sonic qualities.
I might try to measure the change in surface area during displacement...
6
39
282
Thrilled to announce that I’ll soon be releasing my debut album, FILAMENT
Accompanying FILAMENT is a selection of tracks shaped into interactive, embodied VR instruments / environments
I’m sharing an early peek today @ OC6 @ the Indie Dev Lounge — would love to see you there!
11
37
232
tbh cool to see an entire conversation reply tree
it branches out so widely —
sucks how phones only show one tiny stem at a time
ALT the twitter conversation branches out, with replies streaming down vertically in a normal scroll. any time a message has multiple replies, the replies stack up as parallel columns, and they can seed downstream replies that continue branching
what a neural net is doing internally is not an effective procedure or an algorithm. you know that bc it can give a response in constant logical time, independent of the complexity of the question. an algorithm simulates the net, but the weights don't implement an algorithm
14
16
246
32,757
inching closer to fully generated OSs and environments
imagine interfaces "prompted" in realtime by our actions
(using #cogvideo)
70mm f/2.3 photograph of a vast 3D floating document room of a misty submerged hypermodern macOS desktop file system UI, photograph of 3D iOS room
5
32
219
I've been working on this prototype for Vision Pro —
spreading + branching Twitter threads along multiple reply chains
there's so much space available!
why confine everything into a clipped window frame?
(also a great stress test for nested scroll views)
Growing some UI for spatially navigating the branching structure of reply threads
Interested in directly swiping through parallel piles of nested replies
(instead of the current "💬3" that can hide entire massive branches)
Made on-device, pushing @createwithplay to its limits
9
30
208
46,841
Hi, I'm looking for full-time work! I’m a VR interaction designer+developer skilled in Unity. I’ve just graduated from CMU’s Interaction Design Master’s program, & I’m esp. interested in VR prototyping, hand UI, & authoring environments. Ready to move! graycrawford.com
27
43
183
I'm thrilled to share that this week I joined @ultraleap / @LeapMotion as an interaction design engineer!
Stoked to work in the SF office amidst the tracking + interaction R+D group!
22
2
166
Twitter deserves a far richer interface!
We naturally create branching conversations and must be able to see and search them fluidly
@realGeorgeHotz
Growing some UI for spatially navigating the branching structure of reply threads
Interested in directly swiping through parallel piles of nested replies
(instead of the current "💬3" that can hide entire massive branches)
Made on-device, pushing @createwithplay to its limits
22
13
151
scrubbing through a song’s spectrogram
ALT screenshot of iOS Music app with a spectrogram in place of the linear playhead timeline
5
7
136
After playing with the Quest’s hand tracking at #OC6:
I’d put it at ~60% Leap Motion’s fidelity
~200ms latency was most distracting
Finger occlusion was comparable
Pinch interactions very solid
Any fast movements immediately freeze at the hand model’s last known location
5
17
131
Beautiful hand tool details in @normalvr’s Half + Half
like this gimballed teleportation slingshot for hide+seek
love these wonky mechanicals!
@maxweisel
7
20
127
we can't really force the knight to only move along a single L, it has 8 spots it can jump to, check the line of sight to those squares
17
4
120
135,888
I reprogrammed myself quite quickly by injecting the .pngs into chess.com using the Stylish Chrome extension (forgive the older piece design)
as the board becomes more sparse,
it becomes even easier to parse
2
1
119
38,777
Picsew stitches together a png from a screen recording — but downsamples when it caps at ~65k px tall
1
1
105
real-time bigGAN traversal with @LeapMotion hand tracking feels incredible
[w @aman_gif]
What does it feel like to move your body through 14 dimensions simultaneously (xyz + quaternion wxyz per hand)
4
16
100
I’ve been working on this spectrographic music library! 🌊
It’s lovely how indicative the visual texture is — genres are immediately distinctive, song structure+moments stand out on sight
so sad how normal audio scrubbers hide everything; so self-similar, zero detail
7
11
107
6,289
I’m fascinated by how each song’s spectrogram is so distinctive
after a couple listens, I recognize the song *on sight*
music libraries can be much more than mere lists of metadata, music deserves better representation
(this lovely album is Medieval Femme by @FatimaAlQadiri)
8
11
102
16,710
I prefer to see my entire day’s scroll at once
![<https://s.mj.run/C_LsOLb9KBE> <https://s.mj.run/7A9PwMp1kW8> <https://s.mj.run/cR3OfeZQzQY> <https://s.mj.run/YI-xysW4kIQ> <https://s.mj.run/Wey_B74Xfl4> <https://s.mj.run/o7Ij82XvQMg> <https://s.mj.run/MIKIVsN6dvY> 70mm wide view top-down still of the macOS "neighborhood" calendar messages studio arena, multiple projects hovering in the passages between rooms forming a massive chronological environment (like a branching transit map in 3D), morning light --style raw --ar 2:3 [great-grandchild variation]](/pic/orig/media%2FF5OsgTubMAALJXj.jpg)
ALT <https://s.mj.run/C_LsOLb9KBE> <https://s.mj.run/7A9PwMp1kW8> <https://s.mj.run/cR3OfeZQzQY> <https://s.mj.run/YI-xysW4kIQ> <https://s.mj.run/Wey_B74Xfl4> <https://s.mj.run/o7Ij82XvQMg> <https://s.mj.run/MIKIVsN6dvY> 70mm wide view top-down still of the macOS "neighborhood" calendar messages studio arena, multiple projects hovering in the passages between rooms forming a massive chronological environment (like a branching transit map in 3D), morning light --style raw --ar 2:3 [great-grandchild variation]
2
7
95
7,407
branching conversation threads can form such a range of unique structures —
almost “displays” of their own convo patterns
reveals the potent messages thickened by sparking many replies, or where the extended continuities have like a laminar flow, or where it is literally shallow
4
12
87
exploring graphs using 3D hyperbolic geometry:
“the amount of magnification varies across the display, enabling examination of a small area’s fine details whilst maintaining a view of the whole graph as a frame of reference”
does spatial memory scale to hyperbolic arrangements?
ah man, you sent me down a deep rabbit hole the first time you posted that. refreshing to see old ideas that weren't limited by convention. Since then i bcame obsessed with Walrus, idk if youve seen it.
caida.org/tools/visualizatio…
5
12
93
text2video is starting to show causal relationships too
soon we'll reach with our generated hands
[two hands manipulating multiple handles on a complicated tool emitting an AR hologram interface within a misty spatial desktop file system UI]
3
8
77
so tragic that Likes are now private!
I loved browsing exclusively via my friends' Likes arranged multicolumn in TweetDeck... RIP
great for avoiding the For You algo... alas
1
6
81
5,673
140mm f/2.3 photograph of two hands manipulating a complicated tool emitting an AR hologram interface within a misty spatial desktop file system UI with VR buttons and dials, dappled golden hour lighting with mammatus clouds emerging from the mist, parhelion atmospheric halos
3
9
77
thought I’d share a thread of my explorations with @midjourney ⛵️
in my years of text-to-image generation I’ve never wielded such rapid clarity
developing intuitions for prompts is truly an odyssey through infinite novelty
this for music + video could make artists of us all
5
8
77
music streaming apps should show full spectrograms!
you can instantly recognize genres,
remember specific melodic moments
return to a favorite artist and you know exactly where to skip to
albums can be represented so much more truthfully than mere lists of track names
I love how spectrograms reveal @samvoltek’s album Utility
imagine if your entire library was spectrographic!
instead we’re just scrolling through track metadata
ALT Spectrograms for each song from the Barker album Utility. X axis is time. Y axis is frequency. Brightness is amplitude at that frequency at that moment. Super interesting to see which songs are more melodic and ambient, vs percussive. Notice the bass lines, and which songs are super active in the high end. You can see the moments where a break happens.
2
4
77
9,711
that’s tight, I’ve been extruding these up in Blender for some test 3D prints; I’ll try that too
4
71
21,090
@aman_gif and I are honored by the inclusion of XOROMANCY, our gestural neural-network-imagery explorer
in the STUDIO for @CreativeInquiry’s 30th anniversary retrospective exhibition INTERSECTIONS
showing at @MillerICACMU 12-6pm daily until November 17
4
21
74
Excited to share my fave project while at @Ultraleap,
Waft Locomotion! 🍃
Wafting with your hands allows nimble, fluid movement in VR —
Moving through the world is deeply human and feels good!
Any truly spatial computer must uphold our natural mobility.
→ 🧵
3
3
74
6,436
I loved sharing streamdiffusion at SXSW!
So cool seeing ppl explore realtime image gen for the first time ~
We co-discovered so many lovely pockets, tons of embodied play while I livetuned parameters
Fascinating how ppl responded to more-humanoid forms vs overall composition
5
4
70
4,523
Using realtime physics to control sound parameters in Dreams —
grab the container and manually jostle the ambient cones around, add some kinetic energy into the system
Great to have an authoring environment that's so deeply performative
indreams.me/element/oapYoWnB… #MadeInDreams
3
14
61
Making my hand a nebula + granular synth! ✨
All #VisualEffectGraph + @LeapMotion + MeshToSDF
Music+interactions for my incoming @optia_ release
Wild how the particles still feel like part of my body even though they diverge so much from my base anatomy
2
14
64
My first entirely 2D, non-VR piece made with Unity's VFX Graph is
Arc, a gravitational particle sim, made in the STUDIO for @creativeinquiry
Feel free to try out the Mac build — I keep surprising myself by discovering new particle behaviors: graycrawford.itch.io/arc
5
5
54
I'm convinced that fluent latent-space exploration is going to be huge for creative output and inspiration.
Through GANbreeder I've discovered forms I previously never could have conceived, and am developing intuitions for how bigGAN structures adjacent textures across images.
3
10
50
I love the patches of long messages, or rapidfire linksharing thumbnails
like the idiosyncratic landmarks that help orient+navigate within sketchbooks
51
Today, XOROMANCY is premiering publicly at NY Live Arts' Live Ideas 2019! newyorklivearts.org/event/op… It's up the whole month of May.
Explore in realtime a wild space of neural-network-generated images!
@aman_gif and I worked hard on this, and can't wait to see what you discover!
1
7
50
excited to finally install songs in my house
been thinking a lot about music representations + forms…
sad that most often “songs” are represented/recognized by their title text,
versus some more direct rendering of their sonics
3
9
51
eager to run diffusion on FaceTime so I can prompt myself cloudy if I’m not feeling 100
6
4
50
9,933
Loving Dreams' logic system, especially how responsive and surveyable the parameters and gizmos are in play mode
Here I'm mapping camera focus distance to R2, and object spin to controller angle
So great how the nodes, wires, & gizmos all display data in realtime
#MadeInDreams
4
50
STUNNED by what we can make with #Ray2 ✨
I’m so glad to have contributed to this model!
Excited to continue exploring its latent space, there is SO MUCH to discover 🌌
Introducing Ray2, a new frontier in video generative models. Scaled to 10x compute, #Ray2 creates realistic videos with natural and coherent motion, unlocking new freedoms of creative expression and visual storytelling. Available now. Learn more lumalabs.ai/ray.
2
4
55
3,965
I love how the blur isn’t purely optical depth of field,
it’s like the material itself is dissolving, diffusing
ALT <https://s.mj.run/OOkHXfXPvOU> <https://s.mj.run/QRZpnatzN6o> <https://s.mj.run/DMpMfvfXLOs> <https://s.mj.run/-Oh8m9QC0uY> <https://s.mj.run/o7Ij82XvQMg> --ar 2:3
4
3
48
6,628
spectrograms!
I love @djHIMERA's album Sharing Secrets — the arpeggiation and ambient sections shine through so clearly here, each song has such distinct visual character
3
4
49
3,362
physical modeling synthesis for all UI sounds!
end the tyranny of baked audio file assets!
imagine a god having to attach a unique audio file to each type of object collision 😵💫
i've improved my string simulator!
a string pluck sounds different depending on where on the string you pluck from. in this simulation both the sound & visual vibrations of these different plucks are synthesized using some basic Fourier math
3
2
44
🗺
feel like it’s harder to build mental maps of our computing environs when we’re only ever shuffling objects into and out of view 📑
everything so tucked away 🗃 manual excavation
usually we move our bodies around territory & build maps innately 🧚
new OS needs “new feet” 🕸
7
3
47
I hope that spatial UI for long-term use is designed/placed to encourage good posture
perhaps moving higher or fading out as slouching increases
Could users set the UI to not just encourage but enforce good posture? Is this sort of self-conditioning analogous to a shock collar?
5
1
41
This is some fascinating work on external embodied representation in @holoception
Seems like we can project and unify our proprioception with external, scaled representations
1
2
41
My first VR piece is STRATA, a musical exploration of dynamic spatial interfaces, mapping cloth dynamics to sonic characteristics [ @optia_ ]
If you have a @LeapMotion + Vive, I'd love for you to play it! github.com/graycrawford/STRA…
4
12
45
my Apple Music data has every scrub/pause/skip event 2015-2024, down to the ms
which sections of each song I rinse most,
which drops I ⏩ to,
which verses I skip
I hate ❤️ or 💔 whole tracks, such gross actions
I instead want tracks highlighted like the yt relevance histogram
2
46
2,505
(sidenote for spatial UI sketchers)
I found it super useful to have a gray background, using black lines for matter and white lines for rendered objects
makes clear exactly what's UI and what's not
2
6
45
