Version and test your agents. The right way 🍰
NYC
- Tweets 616
- Following 270
- Followers 5,789
- Likes 799
Pinned Tweet
Announcing our seed fundraise!
And that we are hiring in NYC!
Don't learn to code, learn to prompt engineer.
Excited to announce our $4.8M raise led by @neversupervised at ScOp, @pboyce at Stellation Capital, along with @joshuabrowder, @gokulr, @benln, @romainhuet, @OfficialLoganK, @ByrneHobart, @akilian,@GabeStengel and operators from OpenAI, Anthropic, Microsoft, and Google.
@Jonpon101 and I built PromptLayer for domain experts like doctors, lawyers, and therapists— not for ML engineers. Today, hundreds of the top AI teams use our prompt CMS for prompt management, evals, and observability.
techcrunch.com/2025/02/07/pr…
Come join us @promptlayer - we are hiring in NYC 🗽
2
2
22
12,639
Great article by @beam_cloud on how to use @langchain and @promptlayer with their platform!
docs.beam.cloud/getting-star…
6
43
8,906
🪩 New Analytics Page 🪩
Now you can track and visualize:
1. Cost 💰
2. Latency 🏎️
3. Model Usage 🤖
4. Prompts 📃
Great for teams, we all know that one person who will live on this page! 🍰🍰🍰🍰
1
3
35
21,660
PromptLayer is hosting the 1st ever Prompt Engineering Tournament 🏆
Come for pizza, beer, and good company.
Compete to win a @rabbit_hmi r1 and the admiration of your peers 🐰
See you at the beautiful @Company amphitheater on March 12th. lu.ma/cs8vn7es
2
7
30
35,832
Big news... PromptLayer now fully supports @MistralAI models.
Open-source language models are seriously good. Try them out today in through the Playground, evals, or logs.
open-mistral-7b
open-mixtral-8x7b
mistral-small-latest
mistral-medium-latest
mistral-large-latest
2
6
27
6,390
If you didn't hear, the new @OpenAI models let you define functions in your prompt and reliably get JSON responses to call said functions.
And best of all... the new function types are fully supported in PromptLayer 🍰
See more at openai.com/blog/function-cal…
2
20
4,293
🏃♂️Search and Speed updates 🏃♂️
Our dashboard loads 10x faster and search is now optimized for big accounts.
3
19
5,881
📣 PromptLayer now supports Claude-100k 📣
One line of code to start tracking & monitoring your @AnthropicAI app
❌ client = anthropic.Client
✅ client = promptlayer.anthropic_client
Set it up in <5 min ✨
3
3
17
7,009
Speak is building the best way to learn a new language, powered by LLMs.
As they expanded to over 10 new markets this last year, PromptLayer helped them generate content, refine prompts, and scale content workflows. Launching in a new language became quicker, and tasks that would have normally taken months took a single week.
PromptLayer enabled non-technical teams at Speak, particularly in content and product, to iterate on AI features without requiring constant engineering support.
Read the full case study: blog.promptlayer.com/how-spe…
2
5
17
1,703
A lot of customers have told us they are working on migrating their prompts from @OpenAI to open source models like @MistralAI
So we wrote up a guide on how to do it. Check it out 🫡
blog.promptlayer.com/migrati…
4
17
4,729
We are cohosting a NYC event with @arizeai in a few weeks!
@Jonpon101 will be talking about building LLM systems with one of our favorite customers @RogoData 🍰
Don't miss it if you want to learn how to build LLM applications (in real world businesses not just academic papers)
3
2
17
4,575
Very cool new release by @OpenAI. Structured outputs for the new gpt-4o model.
This means you can now force JSON response in strict mode AND provide a schema for it to adhere to.
No more "mis-using" tool calls to force schema.
openai.com/index/introducing…
3
17
3,812
Everyone is talking about tool calling and structured outputs these days.
It's the most important thing for a prompt engineer to know.. but every model does it a little differently.
We put together a chart to compare the slight differences between models ✍️
1
2
11
1,008
Fine tuning might just be the future. (now that it's available for @OpenAI gpt-3.5)
It's easier than ever to use PromptLayer to prepare some fine tuning training data. Just click the export button in the search bar!
Filter export json by tags, query, or metadata ✨🕺
1
4
14
1,733
Bunch of new features coming out this week around prompt templates. Systematic prompt engineering.
🍰 Prompt update commit messages
🍰 Prompt release labels
More info in thread below 🧵
4
4
14
1,495
🪩 Feature Friday 🪩 You can now programmatically access older versions of prompts through our Python library and our API
1
14
2,215
Our long awaited Javascript library is now out!!
`npm install promptlayer`
Learn more on the docs 🫡
2
3
13
1,565
One of the most commonly asked for features!
Great for PMs, SREs, and generally understanding user & agent behavior
The next phase in @promptlayer’s master plan..
🔭 User event & agent analytics 🔭
Every request can now be tagged with user behavior & agent metadata.
Stop jerry-rigging Mixpanel & Segment in your LLM stack. Commonly requested feature from the PMs and SREs using PromptLayer.
1
12
1,158
🚀 New Blog Post: The Prompt Engineering Triangle – the Future of GenAI
At PromptLayer, we've been thinking deeply about where prompt engineering is headed. So we decided to write a blog post on the topic!
Key insights from the post:
1️⃣ Prompt Engineering Is Business Logic
Prompt engineering isn't just technical—it's how businesses encode their unique requirements into AI systems. As long as companies have distinct needs, prompt engineering will remain essential.
2️⃣ Business Logic Belongs Outside Code
Critical AI decisions shouldn't be buried in Python files. Business stakeholders need a platform where they can directly view and adjust how their AI systems behave.
3️⃣ Beyond Traditional Prompting
The future might not look like writing prompts at all! Whether it's crafting datasets, designing evaluations, or new approaches we haven't discovered yet—what matters is having a platform that adapts to how businesses shape their AI systems.
We explore these ideas through what we call the "Prompt Engineering Triangle" and examine how different paths between prompts, datasets, and evaluations could unlock new possibilities.
Check it out 👇
blog.promptlayer.com/the-pro…
3
13
2,687
It is super easy to test out the new @AnthropicAI Claude 3 models on PromptLayer! 🍰
All you need is to take your prompt template to the evals page and select either of the new models! You can then compare the results with any other model on PromptLayer.
1
2
11
1,697
New bio...
Credit to a customer discovery call today where our product was described as "data-driven prompt engineering". Love it 🙏
1
10
1,239
PromptLayer now has a "Forgot Password"
Too much fun building cool features for you but figured it's about time for this one 😜
3
1
10
1,449
Available now in the PromptLayer playground, registry, and evals. Happy prompting 🫡
New @Google developer launch today:
- Gemini 1.5 Pro is now available in 180+ countries via the Gemini API in public preview
- Supports audio (speech) understanding capability, and a new File API to make it easy to handle files
- New embedding model!
developers.googleblog.com/20…
1
2
11
2,852
📣We have added @OpenAI Functions to our Prompt Registry! You can now create prompt templates that have functions in them! 🎉
Read more in our docs docs.promptlayer.com/feature…
1
10
874
Thanks for the shoutout on Forbes, Michelle!
There is a lot of complexity and work to be done to stand-out in the AI world to come. Prompt engineering is the process 🍰
@Forbes @ForbesTech @CatalyzingInnov
forbes.com/sites/michellegre…
4
2
10
3,007
Great happy hour last night at MLOps World
PromptLayer x @modal_labs x @FeatureformML
What a powerful room 🍰 💚 🎟️
1
9
1,754
Just dropped a new example tutorial. Search recipes!
Build a recipe recommendation bot using:
✨ @OpenAI for models
🌲 @Pinecone for vector storage
🍰 @PromptLayer for prompt engineering
Many of our users have RAG workflows, so this should help!
docs.promptlayer.com/recipe_…
5
10
1,198
UPDATE: We just made it even simpler to use @AnthropicAI.
Make sure to use the one line code swap below:
❌ import anthropic
✅ anthropic = promptlayer.anthropic
📣 PromptLayer now supports Claude-100k 📣
One line of code to start tracking & monitoring your @AnthropicAI app
❌ client = anthropic.Client
✅ client = promptlayer.anthropic_client
Set it up in <5 min ✨
1
2
9
1,349
Check out our favorite prompts from the tournament..
We combed 🪮 through every prompt submitted (list in the appendix) and found the prompt engineering strategies that were most effective.
Thanks to @pranav_kanchi for the writeup.
blog.promptlayer.com/our-fav…
1
10
2,825
🧵 On Prompt Routers
The naive approach to building a chatbot: big monolith prompt
The better approach: prompt router + modular task-specific prompts
1
9
1,340
We just added support for Llama3 on PromptLayer. @AIatMeta 📈📈📈
You can run use the playground, run evals, and compare it against GPT-4.
The top 20 open source LLMs are now on PromptLayer 🍰
Thanks to @CloudflareDev for making this so easy!
3
9
3,567
Ellipsis is building the ultimate coding agent and developer productivity tool. We are excited to share how the team is using PromptLayer to build their AI code review agent!
As the @ellipsis_dev team rapidly grew to handle 500K+ daily requests, they used PromptLayer to reduce debugging time by 90% and avoid spending hundreds of hours building an in-house solution.
PromptLayer allows engineers to pinpoint errors in three clicks, and we give non-technical stakeholders insights and tools to help participate in this process. For an agentic product like Ellipsis, faster iterations and more collaboration translates to a seriously better product.
Read the full case study here: blog.promptlayer.com/how-ell…
1
2
8
1,765
"You're not going to build defensibility through basic machine learning for most companies. You're going to win by working with the domain experts. The best prompt engineers treat it as a black box. LLMs are getting more complicated not less."
2
8
783
Join us next week for this event! Our very own co-founder @Jonpon101 will be hosting a panel with Tumas Rackaitis of @RogoData
Join us in NYC Sept 7 for Building LLM-Powered Systems that Work in the Real World 🙌
Speakers from @TCVTech @promptlayer @bentomlai @pinecone @llama_index @Bazaarvoice @RogoData
Panels, demonstrations, and networking opportunities!
SAVE YOUR SEAT: arize.com/nyc-llm-toolchain-…
1
7
1,642
Our team is at MLOps World in Austin this week.
Come to our happy hour on Thursday! We'll be cohosting it with @modal_labs and @FeatureformML
Sign up at lu.ma/pu3ivvzr
3
8
3,147
Good prompt engineering is about one simple thing:
Shorten the feedback loop. Iterate as quickly as possible.
2
7
638
Prompt engineering basics tip: JINJA2
Extremely powerful. You might remember it from the old web days...
Use it to swap out complicated Python code for simple jinja2 prompts →
{% for item in user_info %}
- {{ item.key }}: {{ item.value }}
{% endfor %}
blog.promptlayer.com/prompt-…
2
5
601
ParentLab is revolutionizing parenting support through the product Era's AI-powered conversations that provide timely, personalized guidance to parents.
PromptLayer's visual prompt CMS allows @ParentLab's educators to rapidly iterate on the AI's voice, no technical background required. The results: 10x faster prompt deployment, 400+ engineering hours saved, and 700 prompt revisions in just 6 months.
PromptLayer also provides critical analytics to track edge cases, usage, and costs as ParentLab scales.
Read the full case-study here: blog.promptlayer.com/how-pro…
1
7
927
🚀 New Blog Post: Prompt Engineering Guide to Summarization 📚
The second installment in our prompt engineering tutorial series is now available!
Key insights from the post:
1️⃣ Extractive vs. Abstractive Summarization
Extractive summarization selects and uses exact sentences from the original text, ideal for preserving wording in legal or technical documents. Abstractive summarization rewrites content in new words, improving readability for blogs or opinion pieces.
2️⃣ Handling Large Documents and Iterative Refinement
A basic chunking technique breaks text into 500-token segments, summarizes each chunk, then combines these summaries. This process is refined through multi-turn prompts, focusing on main arguments and discarding minor details for a more coherent and accurate final output.
3️⃣ Domain-Specific Summarization
Fine-tuning models on specialized datasets enhances understanding of domain-specific terminology and concepts, crucial for fields like medical research.
Check it out 👇
blog.promptlayer.com/prompt-…
2
4
699
We'll be releasing some guides on the subject soon. In the meantime, you know where to find us.
PromptLayer seamlessly allows test-driven prompt engineering. Or should we say eval-driven...
1
3
7
901
Try out Claude 2 with PromptLayer!
Introducing Claude 2! Our latest model has improved performance in coding, math and reasoning. It can produce longer responses, and is available in a new public-facing beta website at claude.ai in the US and UK.
ALT Claude 2's performance on the GRE, USMLE, and Multistate Bar Exam. Claude 2 on GRE: Verbal: 165; Analytical Writing: 5.0; Quant Reasoning: 154. Claude 2 on USMLE: Step 1 (5-shot) 68.9; Step 2: 63.3; Step 3: 67.2. Claude 2 on Multistate Bar Exam (5-shot): 76.5
1
7
1,212
AI's robotic tone problem: it's not just annoying, it's harmful.
blog.promptlayer.com/unlocki…
"GPT-speak" stems from formal training data and overcautious responses, creating unnatural interactions.
Strategic prompt engineering and model selection can transform AI communication into genuine, empathetic dialogue without fine-tuning complexity.
2
7
764
We just pushed a huge docs overhaul 📣📣📣
New getting started tutorial & tons more code snippets
docs.promptlayer.com/
2
3
7
2,966
Yes.... we know creating @OpenAI function schemas is annoying. Use our new function dialogue to build the schema interactively 🦾
1
2
7
1,053
Using PromptLayer for batch jobs is easy
Thanks @thenewboston_og for making a video on it
piped.video/watch?v=8hW-Ojwp…
1
2
6
782
General purpose AI is hard to do.
OpenAI and Anthropic have some of the worst prompts I've ever seen.
1
6
823
Quick update! You can now attach model parameters to prompt template versions 🙏
docs.promptlayer.com/feature…
1
5
818
🎉🎉🎉
Big news!
You can now use @promptlayer with #CodeGPT and start logging all interactions with @OpenAI models
1
5
815
NY tech week off to a great start! Gave a demo along with @GocharlieAI and @CamcorderAI for some marketing tech folks 🍰
2
5
645
Well the time has come... @OpenAI is no longer the only game in town. Lots of new models to try.
@LiteLLM has been doing a great job of letting you swap in and out different models (litellm.ai)
Excited about our new integration.
Read more docs.promptlayer.com/languag…
1
4
5
1,085
LLaMA2 is live 👀👀👀
✨NEW LAUNCH! LLaMA2 chat API & open-source playground💫:
We're releasing tools that make it easy to test @meta's latest LLM & add it to your own app with @replicatehq.
Playground: llama2.ai
Live chat API here: replicate.com/a16z-infra/lla…
Repos & instructions below:
1
2
6
1,617
Can't wait for @arizeai Observe 2023 summit! We'll be there.
Signups are open arize.com/observe-2023/
2
6
1,159
Our talk with @RogoData on how they use PromptLayer to build a RAG application at scale 🍰
Some amazing insights by their CTO on prompt engineering & prod language models.
piped.video/watch?v=0s6jESIZ…
3
6
942
Subject Matter Experts, not ML Engineers, will drive the future of AI.
The next wave of AI builders won't need coding skills - they'll need domain expertise. Here's why prompt engineering is democratizing AI development.
Traditional tech roles are being flipped: lawyers, doctors, and teachers who understand their field will create better AI solutions than pure engineers.
Watch a podcast with our co-founder @imjaredz on the subject: piped.video/watch?v=1SnLCWQ8…
The future belongs to domain experts who can harness AI tools, just like Excel empowered non-technical knowledge workers to manage data.
1
6
493
Some new updates to our Prompt Registry page....
Pagination & more
Also our newest engineer Nahuel's first commits 🙏🎉
2
6
628
It's not a hackathon without PromptLayer cakes
Team working towards sponsors' outcomes
🤝
Sponsors bringing their A-Game to the event.
Thanks @AnthropicAI @modal_labs @praxisnation @AntimetalCloud @huggingface @JoinPasses @Meta @promptlayer @LastMile @AntimetalCloud @nomic_ai @HF0Residency @EverywhereVC
1
5
575
Monolithic prompts are simple to build. But are not a good idea if you expect long conversations.
Every new user message will be appended. Every new AI response will be appended.
Longer prompt ➡️
Longer context window ➡️
Slower & more expensive responses ⌚️ 💰
1
2
5
1,005
🍰🚨🍰
🚨 @promptlayer update: We added versioning to our PromptRegistry 🍰!
Now you can see your old prompts and track how they improve 🕺
1
1
5
877
As a reminder... using PromptLayer with @OpenAI also takes one line of code.
❌ import openai
✅ openai = promptlayer.openai
2
5
456
Cool article on the AI Stack by @MongoDB and @richmondalake
Love the PromptLayer shout out!
AI is collaborative. Couldn't agree more.
mongodb.com/resources/basics…
2
5
492
Make sure to update promptlayer through pip before trying this 🫡
🍰⛓ Big news. You can now use @promptlayer with @langchain
Langchain speeds up development by 10x.
Promptlayer speeds up debugging by 10x.
Start using them together with one line.
1
5
497
Check out this great blog post by @emeka_boris how he created a Twitter bot using Magniv 🎉🥳
python.plainenglish.io/build…
1
5
Everything is prompt engineering
Everything is prompt engineering
Everything is prompt engineering
4
499
Part 2 of the tutorial is out!
Adding world information to your chatbot 🫡
piped.video/watch?v=C2_nVCUe…
🚨 Building your own ChatGPT 🚨
Amazing new tutorial on @thenewboston_og. Takes you from a blank slate to building your own personal ChatGPT.
Emphasizes all the best practices of prompt engineering. Great watch 🍰
piped.video/watch?v=Reujw-Hc…
1
5
545
🚀 New Blog Alert! 🚨
Check out this fantastic intro to prompt engineering by on the PromptLayer blog:
blog.promptlayer.com/underst…
🔑 Key highlights:
- What prompts are and why they matter
- The stateless nature of LLMs
- Anatomy of an effective prompt
- Tips for crafting masterful prompts
This article is just the beginning! 📚 It's the first in a series that will dive deeper into specific applications and advanced strategies for prompt engineering.
3
5
534
📢 PromptLayer is now on Discord 📢
Join our discord community to ask questions, learn from peers, and hear about our product roadmap!
discord.gg/VMZVbyeb
1
5
1,551
Today, we are entering a new era.
The emergence of GPT-3 and Deep Learning has broken the traditional approach to Artificial Intelligence.
Innovation comes from hackers, no longer from researchers.
1
5
295
Best practices in prompt management and collaboration. Read more below.
❌ Prompts in code
✅ Prompts in CMS
❌ Monolith system messages
✅ Modular prompt snippets
blog.promptlayer.com/scalabl…
2
5
452
🍰 x ⛓️🦜
🚨 PR to add @promptlayer support for `ChatOpenAI` on @langchain has been merged! Should be on the @langchain python library soon!
5
790
You can now request more resources through our task decorator.
It is as simple as adding a resource dictionary with your desired cpu and memory needs.
2
4
We now support function calling from @AnthropicAI
Was so easy to implement because of our model-agnostic prompt schema refactor 🤓
5
369
🚨 Building your own ChatGPT 🚨
Amazing new tutorial on @thenewboston_og. Takes you from a blank slate to building your own personal ChatGPT.
Emphasizes all the best practices of prompt engineering. Great watch 🍰
piped.video/watch?v=Reujw-Hc…
2
5
873
Prompt engineering is a delicate balance between automation and human labor. When done right, the right tweak of a prompt can be more important than months of regular coding.
1
1
4
291
🚨 New feature alert 🚨
Chain Magniv tasks together and trigger a new task on successful completion.
Learn more at docs.magniv.io/documentation…
1
4
Prompt inheritance. This is great way to modularize your master prompt into a bunch of re-usable snippets ✂️
1
1
5
1,512
“The first reaction with prompts is that really relying on them in your codebase as much as you do source code will be kind of hacky and fragile”
We agree. Prompts belong in the Registry.
The (now routine!) presence of a prompt in the middle of a pipeline that achieves state-of-the-art (as in the study below) is what “prompted” this post.
The first reaction with prompts is that really relying on them in your codebase as much as you do source code will be kind of hacky and fragile…but it’s fine *if* the output of that part of the code can tolerate statistical noise.
To reduce the noise a bit, we probably want something like pip’s version freezing for prompts, where the exact model and version number is always included alongside the prompt in some form.
(Again, all this is likely obvious to anyone logging a lot of time in the space, just some musings prompted by looking at an interesting paper.)
2
4
571
