Co-founder and CEO @firecrawl ️🔥 Prompting humans and machines. Be the best version of yourself.
San Francisco
- Tweets 1,464
- Following 397
- Followers 6,259
- Likes 3,265
Pinned Tweet
Excited to announce @firecrawl's $14.5M Series A 🔥
We've grown 15x in the past 12 months. The demand for clean web data in AI is real.
Here's the story of how we built the web data layer for AI - starting from a side project to powering how 350k+ builders get web data 🧵
66
22
499
116,836
Introducing a Text-to-API AI ⚙️
Turn any website into an API with @firecrawl 's /extract. Just describe your API in plain text and get an endpoint you can hit. Another great open-source project by @devdigest !
43
265
2,355
262,788
Excited to announce text-to-api.ai
A website that turns any website into a get API with @firecrawl /extract endpoint. Data on the web has never been more accessible!
Thanks to @devdigest, for starting this fabulous trend. Check out his GitHub repo below!
37
193
2,117
235,080
Next week we're open sourcing Agent Builder - a visual n8n-style workflow builder for AI agents.
Build AI agent workflows with a visual canvas by connecting @firecrawl, LLMs, logic nodes, and MCPs, then deploy as an API.
Stay tuned for this one 👀
67
135
1,805
251,812
We're open sourcing Firecrawl Observer in 3 days 👀
Monitor any page or entire sites with @firecrawl's powerful change detection.
Set custom intervals and get webhook alerts instantly when anything updates.
Built with @vercel, @convex, @Groq, and more. Stay tuned 👀
32
67
867
68,025
Next week we’re launching Fireplexity, our open source Perplexity clone.
Ask any question and get AI-powered answers with cited sources, all powered by Firecrawl's /search and /scrape capabilities.
Build your own AI answer engine in just 3 days - follow to stay tuned!
26
71
834
67,017
I'm spending $1 million to hire AI agents for our startup @firecrawl
The future belongs to those who can build and control armies of AI agents. We're hiring hundreds of them (and their creators) across our business.
See if your agent (or you) can make it.
69
61
754
156,364
Excited to announce Claude 3.7 Trend Finder 🔦
Use Anthropic's new model to monitor & extract trends on blogs and socials with @firecrawl and the X API.
It collects posts from key influencers and sites, tracks the latest trends, and then pings you on Slack or Discord.
16
62
635
67,970
Introducing SmartCrawl by @firecrawl - Turn any website into an API with AI
Watch the AI in action:
- Find the top 10 AC units on Amazon
- Extract and format JSON product data
- Generate a reusable automation accessible via API
Built with @e2b_dev @browserbase
37
66
585
77,361
Introducing Claude 3.7 Web Crawler 🕸️
It finds pages, re-ranks them, then navigates entire websites with Anthropic's new model, returning data according to the entered objective.
Powered by 3.7 Sonnet for powerful near-instant responses and @firecrawl
15
40
521
68,610
Next week we're open-sourcing our website AI readiness checker 🤖
It runs a complete audit with @firecrawl including:
- LLMs.txt compliance
- AI-readable content quality
- Proper sitemap structure
- Plus 10+ other checks
Stay tuned to find out where your site stands 👀
26
50
488
37,158
Announcing Open Agent Builder - A @firecrawl powered n8n-style workflow builder example app
Build AI agent workflows with a visual canvas by connecting Firecrawl, LLMs, logic nodes, and MCPs, then deploy as an API.
Fork the repo and build your own workflow app today 👇
25
32
384
114,738
In honor of Pinecone's Series B:
We use postgres + pgvector with @supabase and it works great
28
22
352
59,888
Introducing our website AI readiness checker 🤖
It runs a complete audit with @firecrawl including:
- LLMs.txt compliance
- AI-readable content quality
- Proper sitemap structure
- Plus 10+ other checks
Find out where your site stands 👀
15
40
361
49,079
You can now crawl websites using natural language 👀
With @firecrawl v2, just add a prompt like "get the blog pages" to any /crawl request to semantically steer where the bot goes.
It automatically configures the right parameters to crawl the exact pages you need.
21
36
346
38,430
Announcing Claude 3.7 Company Researcher 🔬
Ask for any company's info, and it will search the web to extract structured data using Firecrawl's new /extract endpoint.
Open source and powered by Claude 3.7 API & @firecrawl
7
49
343
40,043
Firecrawl is growing like hotcakes. It only took six products, countless prototypes, and years of failure.
How to fail incessantly and still win: 🧵 👇
23
11
332
47,933
Introducing DeepSeek V3 Web Crawler 🌐
It discovers, re-ranks, and navigates with the newest DeepSeek model and @firecrawl, delivering data with precision based on objective.
8
38
244
21,804
Announcing the new Gemini llms.txt Generator 🔥
Preprocess and concatenate any website into a single text file that can be fed into any LLM.
We crawl the whole website with @firecrawl and extract data with Gemini.
Check out the brand new design with the link below 👇
4
22
246
18,904
The future of onboarding is instant.
Watch this demo of how to use the /extract endpoint to automatically whitelabel your dashboard for a customer. @firecrawl code below!
8
19
228
40,789
GPT-4.1 can now crawl! 🔥
It finds pages, re-ranks them by relevance, and navigates websites with @OpenAI’s new GPT-4.1, delivering exactly what you need, structured and ready to use.
Powered by OpenAI’s latest model for fast, intelligent responses and @firecrawl!
4
23
225
35,456
Introducing Claude 3.7 Stock Analyzer 📈
Analyze and generate charts for any stocks with Claude 3.7 and @firecrawl
Simply enter a stock name and get detailed chart report in seconds.
4
30
223
17,911
Avi, I would have agreed with you a month ago. But, after trying to move away from @langchain , we ended up switching back
Here's why 👇
8
14
196
166,075
Announcing DeepSeek R1 Documentation Assistant 🤖
Scrape any documentation with @firecrawl, then chat with your docs using DeepSeek R1 and @ollama. Zero cloud LLM dependency, 100% local!
Built with @Streamlit for a seamless experience ⚡
7
30
196
17,839
Concatenate any website into a single text file that can be fed into any LLM.
With @firecrawl's new llmstxt endpoint, you can quickly generate llms.txt and llms-full.txt files for any website.
Try it out in seconds with @Replit 🔥
11
24
181
14,699
Turn any URL into a podcast with AI in seconds 🔥
Just paste any article, blog, or news URL and watch it transform into a professional-sounding podcast immediately.
No complex setup, powered by @firecrawl and @elevenlabs
3
29
153
14,219
Clone any website with @cursor_ai and @firecrawl
In the future, every website will evolve autonomously based on its competitors, influences, and user behavior 🤯
Instantly clone any website with Cursor and Firecrawl 🔥
Just enter the site you want to clone into @cursor_ai composer and the agent will clone it after visiting the site.
Powered by Claude 3.5 Sonnet and our new @firecrawl MCP server.
1
15
152
20,218
We want to hire HUMANS with 3 qualities:
1. High agency
2. Low ego
3. Give a shit
If this is you, email me caleb@firecrawl(dot)com with a 30 second loom that proves it
7
7
142
12,967
We built a content optimizer AI that boosts website conversion 🚀
Just enter any site, and it will scrape the CTA, titles, and hero section to fine-tune for more clicks and sign-ups.
Fully open source and powered by @firecrawl!
7
18
144
12,021
Excited to announce Gemini 2.5 Pro Web Extractor ⚡
Ask for any company info, and @firecrawl’s /extract endpoint will fetch structured data for you with Google newest model.
1
15
135
17,325
Introducing o3-mini company researcher 🚀
Just ask for a company's info, and it will search the web to extract structured data using Firecrawl's new /extract endpoint.
Powered by Open-AI's o3-mini and @firecrawl
3
18
136
19,508
Introducing Llama 4 Web Crawler 🕸️
It crawls any website with @AIatMeta's new Llama 4 Maverick Model, @togethercompute, and @firecrawl
Just give it a URL and a goal then it will navigate + return the requested data in a structured format
3
24
136
11,187
Introducing the Firecrawl's 🔥 /extract endpoint. It crawls and converts any website into structured data; you only need a URL and a schema.
5
16
112
7,989
We built a web crawler with o4-mini 🌐
Enter a URL to crawl, sort by relevance, and extract the key information- clean and structured for your workflow.
Powered by @OpenAI and @firecrawl
7
18
114
14,146
Excited to show off Firecrawl Fine-tuning Dataset Builder 📚
Learn how to create high-quality training data for fine-tuning LLMs using @firecrawl to scrape web content.
Build your own instruction-answer pairs dataset for domain-specific AI assistants!
5
14
108
10,245
Competitive analysis just got 10x easier with @firecrawl's new /search endpoint 🚀
We built a pipeline that automatically finds competitors, scrapes their site, then generates a report.
All powered by a @langchain StateGraph which handles the entire workflow.
6
14
100
16,642
How to become a 10x developer?
Teach 9 other developers to do your job just as well.
@trisha_gee, JetBrains
3
17
81
In 2013, two founders set out to build a budgeting app - but hit a major problem.
Connecting bank accounts was a nightmare.
They pivoted, creating Plaid.
7 years later, Visa tried to buy it for $5.3B.
Here's how Plaid changed fintech forever all powered by web scraping.
7
18
75
28,442
Once upon a time, @kiwicopple from @supabase asked me about my dream feature.
I asked for an AI chatbot in the supabase console that could write queries for me based on my database schema.
So I prototyped it with @mendableai and @langchain
5
12
76
15,370
We got invited by @garrytan to demo Firecrawl 🔥 at YC launch live. So, I practiced by pitching to an AI simulation of a Shark Tank that we built with @GroqInc llama3 and @elevenlabs.
No, @kevinolearytv, my mom doesn't have 2300 smurf GitHub accounts 🤣🤣
7
14
66
32,700
Introducing Bulk Company Scraper 🔍
Instantly research companies & startups with @firecrawl! Upload a list of names and get AI-powered insights on companies, funding & more.
From names to knowledge in seconds 🚀
4
11
62
6,590
Announcing AGI News powered by Claude 3.7! 📧
A daily web-connected AI newsletter now powered with Anthropic's new model.
It extracts the latest AGI news from the web using @firecrawl + Claude 3.7, generates a newsletter, and sends it straight to your inbox!
2
13
61
7,003
Seven months ago, @nickscamara_ and I thought Firecrawl would be a weekend hackathon project.
Just 7 months later, we've crossed 19k stars and have PMF
Firecrawl is the best it's ever been and the worst it will ever be ;)
8
7
59
4,080
Demo’d firecrawl 🔥 at the first @ycombinator launch live with some incredibly impressive founders and products @chunonline @InversionSpace @sciphiweb @getclearspace
Thanks @garrytan for this. Sf is so back🧡
7
12
50
24,461
In 2022, My co-founders and I pooled our money together just to sponsor our CTO's visa (@nickscamara_, @ericciarla, @garrettfrohman)
Now, @mendable we're working with amazing companies like @MongoDB and @Snap and are approaching ramen profitability.
This wouldn't have been possible without @ycombinator @garrytan
😉 The rocketship is taking off 🚀
5
9
48
11,746
GPT-4 + copilot has doubled my coding velocity (at least)
Things are beginning to accelerate 😅
2
1
41
7,823
On the day ChatGPT came out, I couldn't sleep.
I could see what was coming next.
Two years later, our company is making job postings for AI agents.
Think you're AI agent has what it takes? Apply below ;)
4
1
47
6,721
Excited to announce o3-mini Interview Prep AI 📖
Simply paste a job URL and it will find the perfect interview prep resources using @firecrawl's new Extract endpoint and Open AI's new o3-mini.
1
7
48
3,225
Using @cursor_ai for non-code tasks (notes, content) is literally a cheat code.
I generated this *entire thread* with Gemini 2.5 right here in Cursor. I even had AI schedule it for me via the @typefully MCP.
(1/4)
3
4
45
2,909
In honor of @perplexity_ai 250M funding announcement, we're releasing a search + extraction API. You can search and then run the page content through whatever model/prompt combo is best for you
Tired of search APIs giving you just a taste? 😒
Introducing /search: SERP + Firecrawl = FireSearch🔥
Say goodbye to search results with no actual page content! Think of the LLM use cases for this... 🧵👇
1
5
44
5,484
We named our new side project FireCrawl 🔥 but with all the @supabase hype today we're considering changing it to SupaCrawl ⚡️
5
4
43
4,346
Check out the repo here: github.com/firecrawl/open-ag…
3
4
42
2,838
LLM-ops in production are hard, and trusted resources are scarce. As Mendable has grown, it's felt like we've been exploring unknown territory.
That's what we're building @agiguide_. We'll be sharing all the hard-learned tips and tricks we wish we'd known at the beginning. If that's your jam, give it a follow :)
Introducing AGI Guide - your newest hub for all resources towards building production-ready LLM apps ✨
Join us as we explore:
- Evaluation techniques 📊
- Innovative retrievers 🔍
- Prompting strategies 🎯
- Intelligent chaining 🔄
- Agent breakthroughs 🚀
7
39
10,381
We’re hiring 🚀
Scraping web data for AI agents sucks. @firecrawl is fixing that.
Live demo of Firecrawl turning entire websites into LLM-ready data in seconds w/ @CalebPeffer
2
3
39
8,641
We're hiring a AI native Data engineer to build Firecrawl's nervous system
You:
- SQL / big data junkie
- Scaled a growth stage startup's data infra before
Us:
- Serving 350k developers, growing fast
- Infinite runway and profitable
- Team is 80% former founders, 80% engineers
Apply below 👇
8
5
38
8,633
But, in practice, I see most founders doing this:
"Have an idea. Get emotionally invested, try once, fail, then give up or run out of money"
Liberate yourself from this trap. Accept that your ideas are stupid, and keep trying anyway.
1
36
2,144
See the code and get your API keys here: firecrawl.dev/p/claude-3.7-t…
1
3
32
2,416
mendable.ai is now available on the langchainJS docs 💪
Many thanks to @hwchase17 and the whole @langchain community!
(OpenAI's servers ran into turbulence just as we published. Apologies for any inconsistency 😅)
✨New AI powered Docs✨
@nickscamara_ was kind enough to set @langchain typescript docs up with mendable.ai - a documentation chat bot embedded into our documentation build
Play around with it here: hwchase17.github.io/langchai…
3
9
30
6,455
Trustpilot works because it's trusted, not because it's hard to build.
Their moat is their network effect, not their software.
28
3,352
My co-founders grinded together for four years, releasing six products (and many more prototypes)
Each one has done exponentially better than the last (by gross revenue)
2
2
26
5,967
FIRE-1 agent + Firecrawl just caught Figma squatting on DEV MODE trademark 👀
Watch FIRE-1 agent with Firecrawl navigate the USPTO trademark search.
soon to be integrated into isfigmasquattingmytrademark.…, which will be built with @lovable
2
8
27
3,989
Web-scraping scale kinda killed the first startup I worked at.
4 years later, we've released FIRE-1 to fix that (a thread 🧵)
Introducing /extract v2 🚀
Now you can extract from multiple pages, handle pagination, and even extract from the entire web without a URL - powered by FIRE-1.
Our state of the art endpoint that allows you get data with a prompt just got a lot better on Day 3 of Launch Week.
1
4
27
3,545
The @vercel AI SDK is just AI on NPM 😍
At mendable.ai we expect that it's going to change how people build AI chat on the web
Here are the juicy bits 👇
3
7
26
6,222
This has to be one of my favorite uses of @mendableai to date. This opens an entirely new world of insights on what your customers need!
The LlamaIndex community *loves* our @mendableai bot (we get ~20k questions every month!)
To keep improving our DX, we teamed up with @mendableai & @nomic_ai to make sense of all that data.
With Nomic Atlas, we built a visual map 🗺️ of all the user questions, more details 👇
5
25
4,959
Our strategy is simple
1. Try something. Reinvest the money, revenue, and learnings from each thing we do into the next
2. Don't give up
2
1
23
1,861
Excited to announce Open-source Watch 👀
Turn GitHub trends into Slack notifications with @firecrawl and @streamlit
Get notified about emerging tech based on your keywords on a custom schedule
Try it yourself below👇
2
4
21
4,092
3. LangSmith.
We looked at other LLM observability platforms. None of them were as easy to set up as we wanted.
With Langsmith, we had an observability platform up and running by literally flicking a .env variable. @hwchase17, @RLanceMartin the team is on to something.
1
22
5,232
To those who have been using @firecrawl MCP
What are you using it for, and what can we do better?
1
5
21
2,681
Our secret sauce is the combination of AI agents and user feedback. This agent writes a script based automation that is fast, cheap, and reliable — so you can rerun without spending a fortune on tokens.
Want to try it out? Join our waitlist: firecrawl.dev/smart-crawl
1
20
2,388
You'll soon be able to 🧑🎓"teach" @mendableai how to answer questions more accurately
4
22
1,676
I get where you're coming from. Langchain ain't perfect. But it's only getting better.
Overall, we're glad we use it - and it's a hell of a community.
1
21
4,706
Last week one of our best engineers graduated high school in Hungary. Congrats Mogery!
The young savant archetype is real.
Exceptional talent sometimes doesn't need a degree or even a diploma - DM me if that's you!
2
4
19
1,637
💎Hidden gem in the GPT-4 announcement: open-sourced evaluation framework.
github.com/openai/evals/blob…
With it, you can pick from pre-built evaluation templates and prompts. Plus, the test runner is parallelized (fast) and crash resistant (saves progress)
🙏 OpenAI 🙏
2
9
21
1,569
"Are you ready to face the sharks?"
But for real, SharkGPT is 10 times more engaging than we expected.
🦈🤖🍕 We built a #SharkGPT and AI simulation of Shark Tank, and our friend @FakeYouApp founder @echelon got "bitten" by the sharks with his "PizzaGPT" concept! 😂 Check out the hilarious video of his pitch getting roasted. #AI #SharkTank #PizzaGPT
1
4
19
10,567
2. It's a great universal interface for LLMs
We need LLMs besides @openAI. Langchain's wrappers make swapping these extremely easy. (LLama 2, anyone?)
If you don't use Langchain, you'll just end up having to write your own abstraction later.
2
21
5,278
When @AlexReibman realized firecrawl🔥was 50x faster than his previous scraping tool 🤯
Wouldn't have known without @AgentOpsAI🖇️👀
2
6
20
10,991
Interested in trying it out? Here's the open source code:
github.com/firecrawl/ai-read…
Want to ship with us? See our open roles: firecrawl.link/jobs
3
18
1,901
See the code and get API keys here: firecrawl.dev/p/claude-3.7-w…
1
1
19
1,353
I didn't want to read 120 pages of the GPT-4 paper/system card, so I hooked it up to mendable and started asking questions
Check it out if you're as lazy as me 😉👇👇
mendable.ai/search/gpt-4-pap…
2
6
18
1,554
I've been using Claude to talk through my relationships. It's insanely powerful, like always having a therapist 24/7. Much better than chatGPT
The hardest part is writing a lengthy description of the relevant info. Bullish on @AviSchiffmann's friend now 👀
3
1
18
5,730
We saw the writing on the wall. Open-core software wins. So we stayed up all night to open the firecrawl before launch.
Firebase -> @Supabase
@Firecrawl -> SupaCrawl 🤣🤣
While building Mendable - we found that feeding LLMs well-structured markdown improved accuracy. Also, it was hard.
So, we released an open source repo and an API that crawls and turns entire websites into a markdown with just a few lines of code.
Introducing FireCrawl 🔥 👇
ALT firecrawl dot dev
6
1
17
984
If you want to learn from our mistakes, work with us. Just search
"Firecrawl Jobs" into Google ;)
4
16
1,831
If you're building a AI GEO optimizer, you should be using firecrawl ;(
Introducing FireGEO - our open source Semrush for AI 🔥
Monitor your website's presence on the leading AI search platforms and compare to all your competitors.
A SaaS kit built with @aisdk, @supabase, @DrizzleORM @autumnpricing, @better_auth and more.
Fork the example today 👇
1
17
1,336
1. It's convenient for production
Do you want to implement exponential backoff? How about an easy-to-use interface for streaming? I don't.
With Langchain that all comes out of the box. Sure, would it be insanely difficult? No. But why bother if there's a lib to handle it?
3
1
16
5,568
Something that never ceases to amaze me is how users that pay the LEAST money almost always the most demanding (and cruel??)
4
15
1,295
This integration is @lovable ❤️🔥
You can now more seamlessly build apps that can scrape websites using @firecrawl with Lovable
1
3
17
6,842
If you feel down, remember that 3/5 tech bros are having an existential crisis as we speak
1
15
1,250
See all of our open roles here: firecrawl.dev/careers/jobs
3
1
17
7,006