I hand-wrote a 500-LoC RL stack to make hacking on RL research much easier.
Most RL stacks are either massive and unhackable, or duct-taped research scripts. I am open-sourcing Mithrl, a modular RLVR stack.
Next items on my checklist: adding more complex environment examples, supporting multi-gpu + async RL, and QoL fixes.
I might scrap external runtime dependencies (Huggingface PEFT + vLLM) and write purpose-built, simpler versions from scratch if I feel the need.
If you want to experiment with RL and are looking to own sovereign tools, I’d love to get on call, understand your requirements and help integrate for free.
20
19
176
17,314
I built software to cut docker image sizes by more than 80%. It uses dynamic analysis and and has a LOT more room for improvement.
The demo here shows an ETL workflow docker image, cut down from 1.28GB to 193MB (-85%)!
If large docker image sizes are becoming a production problem, I'd be happy to explore how we can help.
74
233
2,908
353,171
Introducing BeyondPDF, a smart, free, fully-local (wifi-less) PDF search app for MacOS. "Free" has always connoted to "It's free but we will steal your data"... not anymore.
Whats wrong with MacOS's Preview? It's great... but I NEED smart/semantic search. Searching for words is tedious but searching for ideas is wayy easier.
I converted a light embedding model from @huggingface to CoreML and used zero libraries (vectordb + tokenizer is all raw code)... the installer itself is 42mb lol incl. the ML model, so its light and has a reasonable footprint.
It's 99% perfect. My goal is to improve perf/UX with every release as more feedback rolls in. I urgently needed this myself and now, you can enjoy it too.
Available only on the app store (recommended, linked below) or as a .dmg from my website (also linked).
76
105
1,314
193,616
GPU Poor? **Friends** is all you need.
i built a distributed training module from scratch that performs model training over a cluster of M-series macs connected over the same network.
this was inspired by @anyscalecompute's Ray, @hnasr's work, @ShopifyEng's merlin article and @maximelabonne's spirit.
drop a comment if i should drop a tutorial, the more people wokring on MLE / distributed systems, the merrier!
76
75
916
100,846
gpu poor? I wrote a simple distributed training module to train a vision model on three M1 macbooks. it's open sourced!
i wanted to build a distributed training tool for M-series macs but I faced two hurdles: network latency is extremely high and my friends weren't happy with lending out their macbooks during exam season LOL.
i pivoted it to a federated learning module (where network costs are acceptable). it still needs more tweaks like removing the centralized datastore and compressing payloads but its 90% there :) so contributors are welcome
federated learning is a hard problem to productionize... but its not that hard to understand. thanks to @jafermarq from @flwrlabs (hiring!) for giving me tips on optimizations.
got questions? shoot away :)
35
55
771
104,271
I rejected job offers and took a gap term to work on my own idea for my last @UWaterloo coop. this is my plan
25
11
390
24,740
AI chatbot startups have raised $100M+ right now but have it all wrong.
Your chatbots point users to a 100-page zendesk doc on "how to ***" but why not just complete the task?
I built ActBot: set up your own agentic chatbot on any website!
The demo below shows ActBot provisioning a GPU on @runpod with my requirements.
30
16
368
86,258
Introducing Agenata: a free, open-source web annotation tool. The demo below shows a live annotation of adding an item to cart on uber eats.
Currently, SOTA web agents underperform because:
1) Fewer web trajectory datasets lead to poor reasoning and planning over different domains
2) Websites have a ton of edge cases, making it hard for SOTA llms to reason over complex DOMs
If this is something that interests you, please reach out!
21
21
287
44,559
Introducing Spotty, an open-source self-hosted orchestrator (like Kubernetes) for spot instances.
Spot inst. are 60-90% cheaper and great for dev clusters and non-critical infra.
There are other challenges I am working on adjacently:
> spotty container monitoring to debug system level issues
> reduce deployment times to improve iteration speed
> integrated file system for built-in persistence
I dogfood-ed Spotty with a few projects and I love it (there is still a long way to go before other devs see the value).
I am working on making infrastructure cheaper, deployments easier and time-to-prod faster. If this interests you, reach out!
15
13
246
23,925
I wrote a short article about LFM-2's (by @liquidai ) hybrid architecture w/ illustration + simple pytorch impl.
13
18
193
16,449
I built a tool to cut edge AI docker images by 75%. The demo here trims a Flask inference server running @moondream's VLM (vision model).
we trim a VLM inference server from ~400MB to ~100MB (75% reduction *excluding model weights*) while preserving performance. This also works for containerized data jobs, ML tooling, workflows, etc.
we have limited seats on our waitlist. If large container images are a headache, I'd love to chat.
6
7
173
11,287
great video, if ppl are interested in this, simon's post is awesome too - siboehm.com/articles/22/CUDA…
1
14
151
22,374
Docker trimmer cuts a batched ML job image by 50%!
we are getting some serious interest and are still pretty early, I am opening up more spots on our private alpha, sign up on the waitlist link below.
the job pulls an sklearn ML model + specified data file from S3, runs ML inference on the data batch and batch writes it to @MongoDB.
would love thoughts :)
6
9
107
5,587
weekend ship: I made a minimal python version of NGINX called minginx-py (find repo below). videos explains it
8
3
100
7,904
i suspect adam (just quit as ceo of aws) and ilya (just quit as CS at openai) are building a gpu startup tg
3
1
84
8,973
i love living in downtown toronto!!!!! everyone i am grabbing dinner with is in a 15-min walk radius
there have been times where i took 2 hour trains from waterloo to have dinner w people
6
2
84
5,172
Self-building, on-the-fly APIs with agents. Demo ⬇️
The world needs devs to spend more time architecting developer-friendly APIs not code monkeying the same boilerplate.
I time-boxed myself to build a simple self-building, and soon self-healing, API that can build and fix itself by figuring out your requirements on the fly.
2
2
76
8,265
I built an AI agent that can perform end-to-end tests on your webapp with natural language test definitions.
traditional heuristic evaluations cost $10,000+ dollars, an llm-agent-based heuristic eval can cost < $1,000, even for large projects!
shoutout to @ivan_yevenko from @globe_engineer and @natfriedman whose code I forked and built upon.
If this is of interest to you, DM me and let's talk on how we can bring this into your company.
12
4
74
6,142
thats the biggest problem with ML dependencies, they are extensible and bloated + dependencies creates a hoard of unused package. static analysis can only get you so far. would love to chat more if this is a pressing problem. Here is a quick explainer - theblueprint.dev/p/cut-docke…
2
62
12,146
after a lot of interest, finally wrote about "distributed training from scratch" - link in replies
target audience is newbies and the aim is to build intuition, I welcome comments and suggestions!
4
1
62
3,593
i came to sf a month ago not in the best mental shape, with a fleeting sense of purpose. over a month of patient counsel from friends i hold dear, i rediscovered my “why”. today i’m leaving sf aligned and energetic.
4
63
2,882
Today, I'm thrilled to announce that I've coined a new term in computer architecture: Coreplay. Think of it as foreplay for CPUs. Warm-ups, prefetching, and all the things that get your processor in the mood to perform. Excited to continue making meaningful contributions to this field.
7
61
3,396
today was the first session of Toronto Systems Group (@basilyusuf1709 and me). we want a space for systems enthusiasts (fpgas, compilers, k8s, ml sys, etc). today we covered NGINX and tried to implement a barebones version
8
1
60
2,925
life update:
- took coop term off to work on my own idea
- moved to mumbai for 4 months
- distributed training course is cooking on the side
dropping something soon :)
6
1
55
6,011
I built a patent search engine to look through billions of patents (my demo only does millions for now) through semantic search.
The infra for this is very hard and required some work on the embeddings / vectordb front.
To scale up your embedding search, you need:
- small embedding models finetuned to your needs
- quantization and a vectordb that supports your chosen method
Have questions on Agents / Sem. Search / Applied AI for your org? Shoot me a DM :)
Shoutout to @metaphorsystems for doing this by default!
6
54
9,989
app store: apps.apple.com/app/beyondpdf…
website (for .dmg): omkaark.github.io/BeyondPDFI…
4
55
8,139
for most people, symposium was a fun and inspiring event. for a lot of us on the org / volunteer side, it was that + a half-marathon + 3x upper body days
5
49
2,630
got out-shipped but this is sick!!!!
One more Apple announcement this week: you can now run your personal AI cluster using Apple devices @exolabs_
h/t @awnihannun
4
1
46
5,877
yo brian, thank you for helping me build the thermoclinic neucliophillic ai agent reactor that, since the study, has helped cure cancer (currently in stealth mode)
45
6,866
exactly, i hate companies charging for this... i do like obsessing on the ux, thank you for noticing
1
45
4,364
Hey @metaphorsystems (@jeffzwang @SarahChieng)! I use google and metaphor frequently, and recently, I have been thinking about how Query Biased Snippets on Metaphor's search results could really enhance user experience.
Here's a quick video/demo of metaphor with QBS.
Things to note:
- This could be a game-changer for AI agents and LLM tools. It gives them a richer context about links, rather than just relying on cosine similarity rankings.
- At the 1:06 mark, my localhost shows you a sneak peek of how the UI looks with QBS integrated.
- My demo is not prod-worthy, do not use beautiful soup scraping, it is a recipe for tanking throughputs
I'm really excited about this and would love to chat more if you're interested.
8
2
42
10,161
Hey Ronald this is awesome, I built a repo that does that can run training over a cluster of macs - github.com/omkaark/simple-fe…
4
1
43
2,396
I never truly connected with any profs at my time in waterloo because I didn't go to any class
funnily today was my last day in uni and thankfully, I connected with one prof properly
4
43
2,139
we are apparently charting the app store in utilities!!!!!! can we run this up?
Introducing BeyondPDF, a smart, free, fully-local (wifi-less) PDF search app for MacOS. "Free" has always connoted to "It's free but we will steal your data"... not anymore.
Whats wrong with MacOS's Preview? It's great... but I NEED smart/semantic search. Searching for words is tedious but searching for ideas is wayy easier.
I converted a light embedding model from @huggingface to CoreML and used zero libraries (vectordb + tokenizer is all raw code)... the installer itself is 42mb lol incl. the ML model, so its light and has a reasonable footprint.
It's 99% perfect. My goal is to improve perf/UX with every release as more feedback rolls in. I urgently needed this myself and now, you can enjoy it too.
Available only on the app store (recommended, linked below) or as a .dmg from my website (also linked).
3
3
41
3,783
wow the gap term is over and it went by like a breeze
heading back to waterloo tomorrow for my unofficial gap term 🫡
3
40
2,328
hot take: i don't get the openai hate + i'm glad they exist
you hate em cuz you ain't em
4
2
38
2,731
shoutout to @globe_engineer, i took their DOM optimizations
shoutout to @tryramp and @karimatiyeh for beating me to build this
this is super crude but works well - github.com/omkaark/actbot
2
2
37
4,169
Everyone's talking about the Nike CEO who went from intern -> CEO.
Canada also has an example - @david4mckay of RBC. He went to the University of Waterloo!
3
3
37
3,285
i know a waterloo first year still looking for a summer internship. good full-stack, AR and embedded experience but most importantly, done to learn. anyone interested in a chat?
3
1
32
8,661
i think this form of rollout is so powerful. build good technology with no particular use case in mind -> put it out there -> watch what people do with it -> double down by RL'ing on p90 use cases
gradient descent into people building awesome technology for the love of the craft
We’ve developed Claude for Chrome, where Claude works directly in your browser and takes actions on your behalf.
We’re releasing it at first as a research preview to 1,000 users, so we can gather real-world insights on how it’s used.
2
31
2,234
prolly the next @socraticainfo symposium will look like this... except a canadian flag
33
1,782
tobias is smart, super ambitious and will grind no doubt, any company that hires him is lucky!
1
1
31
4,702
my favourite part about meeting billionaires is the reminder how similar we are and that maslows hierarchy of needs apply universally
5
1
29
1,793
unsubscribing from claude felt like letting an employee go, but 1000x less painful obv. didn't even feel anything when I unsubbed from openai. there is something to this
2
28
1,440
🚢 quick ship 🚢
I wrote a simple, open-source container monitoring tool called Spotty Monitoring. It's used as a sidecar container and exposes container metrics via api.
Scheduling containers on existing instances based on resource usage is the next step. Currently, on Spotty, scaling workflows can only be done by provisioning new instances.
Use it with your containers through `docker pull omkaark/spotty-monitoring:latest`.
3
3
29
2,670
as a first time cto, you build internal tools from scratch
as a second time cto... you buy the tool
4
26
2,847
an employee's worth is the domain knowledge they possess which is not in GPT's pretraining data
3
3
28
1,953
im SO bullish on the mlx. when apple builds, they build seamless integration within their ecosystem. Federated Learning is great across mac mps backends, but.... I have some thoughts on this, but I will let you extrapolate :) maybe I'll even build and OS it
2
28
2,017
my intention is to get this to be readable, DX-oriented and open source it so people can learn that federated learning is actually not hard (on a dev scale)
2
27
3,730
It's a lot more work than people think to open-source a repo. When my repo is private, I get away writing malformed code to iterate on ideas fast.
My repos are a big part of people's first impressions (and my brand), so I spend a good few good hours refactoring before I OS it
2
27
1,617
GitHub searches through 300,000,000 repositories instantly.
Learn the nitty gritties of their search from a 5 min article.
Read here: theblueprint.dev/p/how-githu…
4
25
6,653
this might not mean much to many, iykyk... almost a month of research amounted to this finally working
4
25
2,332
If a service is free, you are the product. gpt was never meant to have just a consumer biz model, it does not scale as well... now enterprises (like apple) will fill oai's pockets for us to use it... parallels with how Google provides search for free. giving it for free leads to model getting used by 100x more users which means openai will see an explosion of data. high diversity of input + actually good model responses are collected. this results in exponentially increasing tokens + higher diversity of training data which means openai has the data needed to train gpt-5 (assuming a 100x model and scaling laws apply). whoever was behind this is a mastermind. now only compute is the blocker. inference compute will be solved but training compute is the bottleneck here.
4
1
23
2,364
m1 Mac performance on 512x512 fp32 matmul:
PyTorch (cpu + gpu) = 0.5ms
naive cpu kernel = 500ms
my cpu kernel = ??
video/report coming soon, these benchmarks are important
1
24
1,169
seeing a friend organically opening BeyondPDF, making a search and deriving value is magical, 1.0.2 is coming
3
1
24
2,208
sales is deeply entrenched in everyday life. there was a time when I HAD to haggle on sublet rent. I was competing with 3 other potentials for a premium apt.
the biggest problems renters have is coming back to a dirty place and unreliable payers. I showed pics of me cleaning my apt before moving out + payed 2 months upfront and boom, got the place for a $250/mo discount.
i've landed other awesome deals w similar approaches
24
920
today and tommorow is open doors in toronto, lots of science, arts and arch buildings are open for visits like KITE, AGO, MARS, check them out!
23
1,110
i used to love art as a kid but the hustle and bustle of my teenage years inhibited me. seems like i have picked it back up and added my own flavour
3
22
1,140