Call every LLM API like it's OpenAI 👉 github.com/BerriAI/litellm
try it today ➜
- Tweets 1,007
- Following 182
- Followers 5,264
- Likes 1,058
Posting 500 flyers around SF today.
Hitting up: fidi, marina, Hayes, South Park 💪💪
Check out ReliableGPT today!! github.com/BerriAI/reliableG…
8
1
28
2,930
Tomorrow, 10AM Start: Come build your GPT-3 apps with us tomorrow: lu.ma/qs9ujmmm?tk=n5aWcb
🍇 BerriAl Swag for All
🎁 Special Prizes for the Top 3 Winners
🤖 Build your LLM apps in minutes
Tomorrow, 10AM Start: Attend the @Berri_AI (YC W23) Hackathon at our HQ in SF!
🍇 BerriAl Swag for All
🎁 Special Prizes for Top 3 Winners
🤖 Build chatGPT Apps in minutes w/ berri.ai
Join Us: lu.ma/qs9ujmmm?tk=n5aWcb
3
16
26,864
Berri's (berri.ai/) first hackathon is in the books!!
Thank you to everyone that showed up, and all the awesome projects that got built. We'll be showcasing some of these, over the next few days!!
2
2
13
1,361
BERRI ON THE STREETS!! Celebrating our Launch producthunt.com/posts/berria…
@ishaan_jaff will be distributing stickers and berries around cafes in SF 🍇🍇
2
3
13
2,547
😱 BerriAI on the Times Square Billboard!!
👉 Join us tmrw for the world's largest chatGPT Hackathon: lu.ma/b9bcda9t
📣 s/o to @brexHQ #GrowWithBrex for makingt his happen
12
467
🪄✨ Introducing Pipelines.
🙇🏻♂️ Ever felt stuck trying to figure out the right chunking/vectorizing/search strategy for your LLM App?
🚀 Pipelines let you spin up an LLM App in 2-lines of code.
Want help? Join us today and launch in 30mins 👉 lu.ma/s2nhhru9
2
11
3,461
📣Announcing #HackAI's 1st speaker - @RazRazcle
👀 @humanloop does some incredible work in enabling co's to bring LLM's to production. Catch his talk @ #HackAI this weekend.
👉 Register for the event: lu.ma/b9bcda9t
6
11
2,211
📣 Want to use #chatgpt to answer COMPLEX customer / internal questions?
Need a way to:
1. Reason through several different information sources?
2. Provide References of what it looked at
🥁 Announcing ComplexInformationQA
Try it today 👉 lnkd.in/gNNV9Zvq
2
1
8
892
BerriAI just launched on @ycombinator's Launch YC!
🚅 LiteLLM - Call all LLM APIs using the OpenAI format [Llama2, Anthropic, Huggingface, etc.]. Check them out: ycombinator.com/launches/JKe…
2
9
515
New release updates:
🦙 added support for @gpt_index - thanks @ravithejads
👻 added support for wrapper functions - thanks @HingumTringum
🐛 fixed a bug which wasn't parsing 'with' statements
try berri today 👉 github.com/ClerkieAI/berri_a…
1
2
8
806
🤔 Using chatGPT for answering customer questions?
👀 Want to give it the conversation history?
😱 Just pass the history to your Berri api endpoint, and we'll manage the rest
Learn more 👉docs.berri.ai/api-reference/…
2
9
823
📣 Announcing #HackAI's 2nd speaker - Dhen Padilla
👀 Dhen built LLM Infra @ streamlineclimate.com - and is talk about the challenges of building PRODUCTION-GRADE LLM Apps @ #HackAI
👉 Register for the event: lu.ma/b9bcda9t
3
7
862
add it today -
docs.litellm.ai/docs/observa…
New integration: @LiteLLM
You can log requests using Supabase across all LLM Providers @OpenAI, @Azure, @huggingface, and more!
Learn more: supabase.com/partners/integr…
ALT integrations LiteLLM
1
8
1,417
Berri builds in public 😤
🤫P.S.: @krrish_dh and @ishaan_jaff have something special planned for our first 1000 deploys 👀
1
8
1,795
🔥 We just launched models.litellm.ai (made by
@yujonglee) View 200+ LLMs supported by @LiteLLM , and view their context window and pricing
🔥 We just launched models.litellm.ai (made by 👑 @yujonglee)
👉 Look up 200+ LLMs supported by @LiteLLM , and view their context window and pricing
🛠️ Fix PR for allow using custom header for litellm api key (h/t Andres Aldana)
💡 [Proxy-Fix + Test] - POST /batches and GET /batch endpoints
📖 Docs - using /batches endpoints with LiteLLM Proxy docs.litellm.ai/docs/batches
⚡️ [Proxy-Fix] - raise more descriptive errors when crossing tpm / rpm limits on keys, user, global limits
7
395
😅 Bringing #chatGPT to your users **requires** continuous testing
😱 Small changes (e.g. to the prompt, etc.), can have massive impact
🚀 Every instance on Berri now gets automatic testing + evaluation scores!
👉 Get started: play.berri.ai
1
2
7
620
First stop - Another Cafe.
If you’d like to get an in-person demo, drop a comment here / dm me and I’ll make my way over.
📣Going to every office/coffee shop/chipotle joint I know in #SF to get ppl to spin up a #chatGPT App through @Berri_AI
🫡Goal for today: hit 100+ deploys
🙌Taking over @Berri_AI Twitter for live-tweeting the journey. Come through.
1
7
586
ANNOUNCING T5 🔥🔥
Need an alternative to GPT? Want it to be deployed on your infra?
PRESENTING T-5!!
Live on the playground today 👉 play.berri.ai/
Live on the API 👉 docs.berri.ai/api-reference/…
1
6
686
🚀 5 updates to LiteLLM 🚅
👋 @helicone_ai + @supabase integrations for auto-logging LLM responses
👀 @infisical integration for reading in your API token
🥳 exposing `api_version` and `org id`
🛠️ fixes to exception-mapping
✅ Support for async calls
github.com/BerriAI/litellm
1
7
1,067
EOD - had 130 ppl try and deploy w/ 59 successful. Lots to fix in the pipeline. Tomorrow we go for the remaining 138 deploys.
3
7
928
Fyi @teknium, @Shagn1000 we launched a new Load balancer on LiteLLM it can 20x your throughput on Ollama, vLLM, any Litellm supported model. Here's how to use it docs.litellm.ai/docs/routing
6
375
👀QA on your codebase.
✈️we're running pilots..
👇drop a link to a github you'd like this for
3
6
1,921
@Berri_AI hackathon is live !!! Some amazing apps coming out by 6pm Thank you to @fondocom for giving us this space 🫐🎁
2
6
1,034
🤔 Do you run a 2-sided marketplace?
😅 Are your support teams overwhelmed with questions specific to each listing?
🚀 Learn how to 10x your customer support in 10 minutes w/ #chatGPT next SATURDAY!
👉 sign up here: lu.ma/na4e119c
2
6
1,195
huge thanks to @sebderhy for the help 😃
LiteLLM v0.13.0 ⚡️ Support for @anyscalecompute endpoints with streaming live now. Try it here: docs.litellm.ai/docs/provide… s/o to @sebderhy for the help
🔎 Use LiteLLM to easily benchmark Hugging Face LLMs docs.litellm.ai/docs/tutoria…
5
453
Use @huggingface TGI LLMs with ALL OpenAI compatible benchmarking tools
💥 Trying to evaluate if the latest LLM is good for production ? LiteLLM proxy allows you to run @huggingface TGI LLMs (100+LLMs) on LM Eval Harness
docs.litellm.ai/docs/tutoria…
🪨 Supper for Bedrock amazon.titan-embed-text-v1 LIVE docs.litellm.ai/docs/embeddi…
docs.litellm.ai/docs/tutoria…
1
4
482
😱 You can now create a LLM App to QA your docs straight from our website (berri.ai/)
🚀 If it takes you more than 15 minutes to go from idea to shareable prototype, we'll buy you coffee.
5
388
Berri Lightning Hacks:
You can now query an API url straight from berri.ai/.
Seen below: an api call to semantic scholar, getting all relevant papers and turning it into a chat interface [IN UNDER 2 minutes]
4
739
📣 Trying to integrate #chatGPT into your product?
😱 Privacy, PII, Finetuning, etc. are just some of the concerns you're trying to navigate?
🚀 We'll take you from #Pinecone Prototype to Production-LLM App - in JUST 2 WEEKS!
👉Apply: lnkd.in/gVexdwaK
2
1
5
414
New @cohere embedding model on LiteLLM
💥 Thrilled to announce - DAY1 support for Cohere embed v3 models on LiteLLM (it promises better performance than OpenAI embeddings) Try it here: docs.litellm.ai/docs/embeddi…
⚡️ Use LiteLLM Proxy to compare 100+LLMs on LM Eval Harness benchmarks: docs.litellm.ai/docs/tutoria…
4
395
❤️ We just launched support for Llama-2-7b-32K-Instruct on liteLLM: github.com/BerriAI/litellm/
Love the awesome work done by @togethercompute
Introducing our newest long-context model:
💬 Llama-2-7B-32K-Instruct
Fine-tuned using Together API, the model is now available to use with our APIs & Playground: api.together.ai
Try it out and send us feedback!
together.ai/blog/llama-2-7b-…
1
5
565
We reduced failed customer requests in prod by 80% using caching.
Open-sourcing the code to do this yourself.
github.com/BerriAI/reliableG…
1
5
334
Thank you for the shoutout @Txtetxu1
If you haven't checked out his newsletter yet - would highly recommend it:
txetxu.substack.com/p/conneu…
1
4
342
BerriAI <3 Replit
come hack with us this Saturday (03/11) in SF: lu.ma/i27b3fyn
Learn how Y Combinator-backed startup, @Berri_AI, is using Replit to let users build ChatGPT bots for any document, website, or knowledge base.
blog.replit.com/berriai
2
4
1,150
THE BEST DEAL ON THE INTERNET is now here!!
Announcing: THE BEST DEAL ON THE INTERNET!!! Try now: berri.ai/
Unlimited #chatGPT instances, 10,000 queries/day, index finetuning, etc.
s/o: @TechCrunch for telling us that our pricing was too expensive - techcrunch.com/2023/04/04/th…
4
631
4
485
Just stopped by LinkedIn office to teach developers how to use Berri producthunt.com/posts/berria…
4
686
Afraid #ChatGPT is going to steal your data? 😱
Feeding in sensitive data into LLM applications is (rightfully) worrying! With solutions like #PrivateGPT rising in popularity, how can you harness LLM superpowers while protecting your company's data. 🧵 👇 (1/4)
1
1
4
1,023
💥 New demo video on using LiteLLM: loom.com/share/bd4e9029a1da4…
3
291
legendary ⚡️
Finally we did it with @LiteLLM and @spheron!
"Load Balancing Multiple LLM Endpoints"
Here’s how it works:
1️⃣ Deployed multiple LLM models on Spheron’s decentralized GPU network.
2️⃣ These endpoints automatically connect to LiteLLM without any manual setup.
3️⃣ LiteLLM handles the magic—load balancing API calls across all models to keep things fast and efficient.
The result?
✔️ Scalable LLM deployments made simple.
✔️ Lower costs thanks to Spheron’s infrastructure.
✔️ More time for devs to build awesome stuff instead of worrying about infra.
This is the future of deploying and managing AI models. What do you think? Let’s talk!
1
416
Signup for #BerriAI's exclusive 2 week cohort where we'll help 10 companies build on-prem chatGPT apps that:
- Have 99% uptime and can scale 📈
- Achieve ⚡ fast response times of <1 second.
- Ensure exceptional coverage for user questions (>90%) 💡
👉 shorturl.at/uKM03
1
3
1,154
huge thanks to @phduc06 for the awesome contribution !!!
LiteLLM v14.1 ⚡️ evaluate @OpenAI @cohere , @AnthropicAI, @huggingface LLMs on 100+ benchmarks
Try here: docs.litellm.ai/docs/simple_…
🍰 Major improvements to token trimmer - thanks to @phduc06 for the awesome work on this docs.litellm.ai/docs/complet…
github.com/BerriAI/litellm
3
539
Build your app with custom data + custom prompts today berri.ai/
🚨 Build your own chatGPT app for your data (📄) + add custom prompts at berri.ai/
In this demo I build an app analyzing Apple's 10k filing with a custom prompt instructing the LLM it is a financial analyst
🚀 Build with @Berri_AI and get a web app + API endpoint!
1
3
602
💥 LiteLLM v1.7.21 Proxy - Handle 1000+ requests/second with @Azure, @perplexity_ai online models, @OpenAI 💥 Try it here: docs.litellm.ai/docs/simple_…
⚡️LiteLLM Router now load balances based on each deployment RPM / TPM h/t @mikkolehtimaki for the load test docs.litellm.ai/docs/routing
🔎 Support for @perplexity_ai pplx-online LLMs - pplx-7b-online, pplx-70b-online docs.litellm.ai/docs/provide…
⚡️ OpenAI, Azure Embeddings now use Async Clients docs.litellm.ai/docs/embeddi…
📖 Proxy Request Logs now hidden behind verbose prints
2
429
🤯 Ever wished you could build your own #Notion chatbot?
🙇🏻♂️ Use @gpt_index's index + @langchain's agent and spin up #chatGPT in <1min.
☕️ if it takes you any longer.. coffee's on us (Venmo: @Krrish-Dholakia)
berri-ai.ghost.io/creating-c…
2
158
Proud to announce LiteLLM is now live on @CodiumAI's pr-agent repo.
1 line change and they now support 5 new LLM providers
check out the PR here - github.com/Codium-ai/pr-agen…
1
187
I just received a coffee from someone on @buymeacoffee!buymeacoffee.com/clerkieai/c…
2
298
@ramsri_goutham are you attending #HackAI? Let's build our own chatGPT Plugin!
Link to event - lu.ma/b9bcda9t
2
63
I'm going to be launching on Product Hunt tonight ! Join my discord to stay tuned: discord.com/invite/KvG3azf39…
🚀 @Berri_AI is launching on @ProductHunt tonight 12:01am. Join our Discord community to stay up-to-date with our launch: discord.com/invite/KvG3azf39….
Here is everything we built in the last 5 days leading to the launch:
1
1
901
AAAND WE’RE DONE!! ✅✅
~500 posters put up 🆙 in <10 hrs!
Thank you for following along ❤️
Check out ReliableGPT: github.com/BerriAI/reliableG…
2
347
check out @so_claros today!!
Added reliableGPT to pacman-prompts, and started using it in prod at @so_claros
It's already helped us out a couple of times, especially when the OpenAI API is throwing a tantrum
Thanks @ishaan_jaff and @krrish_dh!
1
2
348
What are wrapper functions?
For more complex use-cases. If you're taking a user query and doing multiple things (LLM calls, api calls, etc.) with it, you can put them in a wrapper function and pass the wrapper function to Berri.
2
146
Thanks @SethDeGroot!
@WillManidis if we can't get you to prod (live with this for all your users) in a week, we'll give you $100 in OpenAI credit.
calendly.com/kdholakia/30min
2
35
Starting today, you can use the documentQAPipeline.
We're going to work to optimize these flows increasingly over time - making them faster, more reliable, etc.
Want to use/contribute a pipeline for your scenario? DM us to get started!
1
2
287
💥 Excited to launch Day 1 support for @AnthropicAI Claude-3, use 100+ LLMs in one format. 👉 Start here: docs.litellm.ai/docs/provide…
2
256
Btw we'd love to chat @Txtetxu1
calendly.com/d/xz2-fqd-gqz/b…
2
163
We would love to chat!
calendly.com/clerkie/30min
1
2
332
Found + fixed a ui bug on berri. Had to buy Kyle a coffee for it - but it was worth it. Should be easier to put url's on the site now. #1kIn5days
2
408
I just received 3 coffees from someone on @buymeacoffee!buymeacoffee.com/clerkieai/c…
1
365
Switching it up. Berri is going to streets. On market street. Come thru if you’re looking for an in-person demo.
246