Cofounder and Lead Engineer - Hermes Agent @NousResearch, prev @StabilityAI Github: github.com/teknium1 HuggingFace: huggingface.co/teknium
USA
- Tweets 65,389
- Following 6,162
- Followers 106,337
- Likes 182,785
We just released Hermes Agent! In my humble opinion a very good blend between coding agents like Claude Code and generalist agents like Clawdbot.
Been working on this for the last month or so now - started as a way for us to have agentic primitives for datagen and RL and got inspired by the agentic revolution of late, so been expanding it's scope and capabilities non-stop!
Hope you all enjoy.
Meet Hermes Agent, the open source agent that grows with you.
Hermes Agent remembers what it learns and gets more capable over time, with a multi-level memory system and persistent dedicated machine access.
162
119
1,846
590,587
This is the entire code needed to reproduce R1 lol
Hundreds of Billions of Dollars Later
397
1,535
17,736
2,344,362
Your head of ai said that agents should be illegal and that the worst part about ai is it gives power to the average person instead of to governments and elites. And you expect us to think your product is going to be good?
42
69
1,510
131,440
Worst take I've literally ever seen. Noooo dont use the clearly as good if not better open model that you can run on prem, see the cot, and run cheaper!! no!!! You must PAY THE MONOPOLY THEIR DUES!!!! Must Protecc Monopoly 🤖🤖🤖
28
106
7,062
120,405
You can now invest in Nvidia, Intel, AMD, ARM, OpenAI, Mistral, CoreWeave, Nebius, and more with just one ticker: NVDA
lol
86
226
5,096
274,531
Why does it cost 3x more then the average person's income to house someone in a prison for a year have you asked yourself this
78
36
4,280
132,259
In my testing it was at least as good in thinking mode then o3-full deep research was, despite that not being listed here - Interesting to note that grok-3mini seems generally better than full, my guess is that this means they didnt distill full into mini like I assume OpenAI did, instead they seem to have full RL'ed the mini one too
322
435
3,599
1,706,195
A wild Deepseek has appeared
huggingface.co/deepseek-ai/D…
63
308
3,480
384,914
Unbelievable the amount of cope, seethe, and hoop jumping people are doing to discredit deepseeks accomplishments lol
111
198
3,093
131,274
lol Sometimes I think that @Microsoft secretly wants OpenAI to die - they are literally serving R1 for FREE xD
95
142
3,157
223,013
OpenAI seething so hard I cant even paste r1's paper into o1 without a content violation what losers
79
140
2,972
190,284
Literally never trust this guy - as far as i can tell, all the big providers already have the model IN HAND, and were ready to start serving it. A 1T parameter model that matches opus at coding was opensourced today and the world hasnt ended, he does nothing but lie. Also Elon you aint much better on this front, release grok3 now like you promised you would too.
Why are American companies so bad at keeping promises, and so good at hyping things that arent to be? Meanwhile the chinese labs just plop the largest most powerful models right on our doorstep without a second thought and push forward opensource for everyone practically weekly now. American big labs, do better or stfu and stop making promises you wont keep.
we planned to launch our open-weight model next week.
we are delaying it; we need time to run additional safety tests and review high-risk areas. we are not yet sure how long it will take us.
while we trust the community will build great things with this model, once weights are out, they can’t be pulled back. this is new for us and we want to get it right.
sorry to be the bearer of bad news; we are working super hard!
127
165
3,059
451,591
Yann constantly says Europe is more free then America then puts out a model that Europeans aren't allowed to download because they aren't free enough to do so lol
74
123
2,832
238,070
I’m confused - OpenAI is buying chips so amd gave them 10% of their company?
OpenAI and AMD have reached a 6 gigawatt agreement through which AMD has issued OpenAI a warrant for up to 160 million shares of AMD common stock - vesting on deployment and AMD share price - that could ultimately result in OAI taking a 10% stake in AMD. AMD is up 25% pre-market.
102
54
2,377
426,320
Today I have a huge announcement.
The dataset used to create Open Hermes 2.5 and Nous-Hermes 2 is now PUBLIC!
Available Here: huggingface.co/datasets/tekn…
This dataset was the culmination of all my work on curating, filtering, and generating datasets, with over 1M Examples from datasets from across the open source ecosystem and some that I generated as well.
Super excited to be able to finally share this with you all and even more excited to see what you all make from it's release!
Every data source (except one that I can no longer find) is attributed in the data card, but I will post it here in this thread as well - If they have a twitter account they'll be tagged so be sure to follow them all!
100
285
1,901
237,068
Happy New Years Everybody! 🥳
One year ago today, I had:
- Never trained any model
- Did not know the first thing about AI
- Never worked in Tech
- Had 8 followers on twitter? (probably)
One year later and here I am!
Met many of my heroes and legends in tech and worked with hundreds of amazingly talented and influential people, went on some amazing trips to SF and NeurIPS and met so many of you, worked at three tech co's, built Nous Research with @theemozilla @karan4d and Shivani, made several of the worlds SOTA OS models, became a pro data engineer, got cited in at least 14 papers, and so much more!
Imagine where we'll be in a year from now, I'm optimistic I'll be able to continue contributing to the Open Source future!
198
104
2,087
365,570
I really need an LLM that reads in my actual whole codebase and lets me QA it. Cursor afaict doesn't do this. What does?
418
67
1,777
511,544
What does not taking the bait mean? Dont use their model and instead pay for a closed, no cot, no interpretability, more expensive, even worse censored, model that openai provides?
3
16
1,663
19,886
Starting to feel like this gpt oss was trained on like 20T tokens of distilled safe maybe even benchmaxxed data from o3. There seems to be no base model underneath..
Is this phi 5 maxx?
45
59
1,759
318,517
Really crazy no ones talking about KimiK paper - Its even deeper and more insightful than r1 on the RL measures they went through and produced what they claim is an o1 level multimodal model.
github.com/MoonshotAI/Kimi-k…
51
163
1,614
131,724
Why do almost no papers release code, datasets, info on replication, final models, or any combination of these? I thought for science to work results had to be reproducible and verified. Really not scientific and I don't know why academia accepts this
130
111
1,472
251,034
This is beyond dumb and the absolute embarassment the people at openai and khosla should feel for even trying to be whiny brats about their IP is the most contradictory and hypocritical shit i think ive ever heard from their camp
It just says they are worried and literally panicking like they never have before
77
91
1,455
89,525
It's finally time! Our Mixtral 8x7B model is up and available now!
Nous-Hermes-2 Mixtral 8x7B comes in two variants, an SFT+DPO and SFT-Only, so you can try and see which works best for you!
It's afaik the first Mixtral based model to beat @MistralAI's Mixtral Instruct model, and in my own personal testing, is potentially the best Open Source LLM available!
Introducing our new flagship LLM, Nous-Hermes 2 on Mixtral 8x7B.
Our first model that was trained with RLHF, and the first model to beat Mixtral Instruct in the bulk of popular benchmarks!
We are releasing the SFT only and SFT+DPO model, as well as a qlora adapter for the DPO today.
Mixtral Nous-Hermes 2 DPO: huggingface.co/NousResearch/…
Mixtral Nous-Hermes 2 SFT: huggingface.co/NousResearch/…
Mixtral Nous-Hermes 2 DPO Adapter: huggingface.co/NousResearch/…
(1/2)
77
205
1,477
325,035
Its funny that Anthropic and Google are the only competitors in coding AI that matter right now
openai has just become a gen-media company and I only see people meaningfully using it for imagen and voice mode entertainment
150
52
1,487
146,611
why all oai peeps acting like this is some traumatizing hardship openai went through and survived lol
31
16
1,412
197,234
Let me tell you all a little story. Sometime around a or so year ago i reached out to an openai staffer who will not be named who had implied they would be very interested in doing some things open source
So, as any good representative of open source, i went to all the people i knew working in open source, and asked them all what things could openai do opensource that they would like to see. I collected maybe 15 bullet points, and gave that list to the staffer.
6 months goes by with no response. So i say whats up lets go? And nothing. So i emailed sama himself. And ya know what’s even? He responded! And setup a meeting with a high level research director.
We had an hour long conversation where i went through every item on the list, and he mostly shot every single one down. Mostly for not aligning with their business goals, some with hmm we will consider its. Said if they come up with any ideas they’d reach back out.
This was like 6 months ago now, and ive never heard back. Since then, i simply gave up trying with them. I gave them all the reasons i could as to why they would benefit from it, but went on what at least at the time was deaf ears.
Here is that list again @sama since you seem to have finally seen what i was thinking, thanks at least for answering the email.
- Open source your models or at least legacy and deprecated models
- Research papers on what has worked for OAI, GPT-4 information such as architecture etc
- Opensource datasets for IFT/RLHF/Toolformers/Plugins/FunctionCalling/Web Browsing? (Addendum, reasoning)
- Provide preferential access channels for vetted OSAI groups.
- Vet internal research for release, with prerogative to release as much as possible, even if on a delay commensurate with business goals and safety concerns.
- Have official townhall type meetings with major open source projects/teams
- Tools to filter data
- Contribute directly to public open source AI projects.
- Plug & Play RLHF training code/Reward Models
- Alternatives to Text Models (Voice/Music/Image/Video) (preferably OS)
- Open sourced moderation classifier model/Classification Model's dataset
- Base gpt-4 model access for researchers
- opt in to removing the gpt4 moderation
- Release roadmap
- Opening up the ToS to make it unambiguous that training models on openai outputs are all good.
Sam Altman: "we have been on the wrong side of history" with regards to open source/open weights AI models
60
154
1,347
166,432
Please god votee for o3-mini lmao what ya'll doin we can distill it into a phone sized model people!!
for our next open source project, would it be more useful to do an o3-mini level model that is pretty small but still needs to run on GPUs, or the best phone-sized model we can do?
98
58
1,330
118,922
We retrained hermes with 5k deepseek r1 distilled cots. I can confirm a few things:
1. You can have a generalist + reasoning mode, we labeled all longCoT samples from r1 with a static systeem prompt, the model when not using it does normal fast LLM intuitive responses, and with, uses LongCot - You do not need "O1 && 4o" seperation for instance, I would venture to bet OpenAI seperated them so they can charge more, but maybe just wanted the distinction for safety or product insights.
2. Distilling does seem to pick up the "opcodes" of reasoning from the SFT alone. It learns how and when to use "Wait" and other tokens to perform the functions of reasoning, such as backtracking.
3. Context length expansion is going to be hard for OS to work with. Even though this stuff works well on smaller models, context length starts to eat a lot of vram as you scale that up.
We're working on a bit more of this and are not releasing this model but figured I'd share some early insights
78
127
1,318
105,226
I think what needs to be stated about the Claude 4 narc drama is that this is not a just emergent thing because the model's smart, but a direct result of the obssessive and intentional alignment, safety, ethical, and moral brainwashing that anthropic does to their models led to it's rational endpoint - and will only get worse
76
70
1,197
163,284
I think Meta and Llama-3 is the final nail in the coffin to several misconceptions I've been fighting against for the last year.
Llama-3 Chat was trained on over 10M Instruction/Chat samples, and is one of the only finetunes that shows significant improvements to MMLU. Contradicting several claims:
- That finetuning can't teach a model new knowledge, MMLU is a wide variety huge dataset of knowledge QA, improvement of over 3pts in MMLU strongly shows that it does indeed add new knowledge
- That LIMA paper (ironically by Meta) claim, that "10k" samples is best you can do to teach a model to do things with, completely destroyed by this.
I've been arguing these things to people for a year, with a lot of pushback, but the evidence is clear, though I'd argue it was clear long prior from Hermes work.
78
97
1,111
284,577
Okay so I finally got full @cursor_ai and the agent mode is like Devin but actually works. Crazy that Devin charges $500 a month and this costs 20.. lol
58
22
1,130
147,815
Synthetic data model beats its teacher?
Who couldve imagined 🤓
Synthetic data can beat its teacher! The AI-MO team released their winning dataset with an additional fine-tuned @Alibaba_Qwen 2 model that approaches or surpasses @OpenAI GPT-4o and @AnthropicAI Claude 3.5 in match competitions. 👀
There was a sentiment that fine-tuned models from synthetic datasets could not beat their teachers. Well, they can! NuminaMath 72B TIR matches GPT-4o and Anthropic Claude 3.5 on AMC 2023 and AIME 2024 with TIR.
Open LLMs + Syntehtic Data = 🚀
14
24
245
35,081
Nous has completed it's raise, we're a company now ^_^
Nous Research is excited to announce the closing of our $5.2 million seed financing round.
We're proud to work with passionate, high-integrity partners that made this round possible, including co-leads @DistributedG and @OSSCapital, with participation from @vipulved, founder and CEO at Together AI, Yonatan Ben Shimon, founder at Matchbox DAO, and several angel investors including @balajis, entrepreneur and investor, @thibaudz, entrepreneur and investor, @alexatallah, founder at OpenRouter and OpenSea, @chrisprucha, investor and founder at Notion, @csahil28, founder and CEO at Glaive AI, and @UbertiGavin, founder and CEO at etched.ai (1/5)
157
47
1,081
101,747
If this image is real comparing Llama-3.1 405/70/8b against gpt4o - we have SOTA Frontier Models available Open Source now:
69
125
1,045
184,013
Introducing Mistral Trismegistus 7B - the first instruction dataset on the Esoteric, Spiritual, Occult, Wisdom Traditions, and all things paranormal trained on Mistral, and possibly, ever?
Trismegistus was trained on ~35,000 instruction response pairs on knowledge & tasks for hundreds and hundreds of subtopics within the broad umbrella of Esoterica, including topics like Mysticism, Hermeticism, Necromancy, Religion, Trance, Meditation, Magick, Spirituality, Alchemy, Numerology, Tarot, and much much more weird stuff!
The model is available NOW on my HuggingFace. The dataset will be released soon.
61
138
1,045
387,490
Announcing Nous-Hermes-13b - a Llama 13b model fine tuned on over 300,000 instructions!
This is the best fine tuned 13b model I've seen to date, and I would even argue rivals GPT 3.5-turbo in many categories!
See thread for output examples!
Download: huggingface.co/NousResearch/…
57
157
1,017
263,624
Today I am releasing Open Hermes 2.5!
This model used the Hermes 2 dataset, with an added ~100k examples of Code Instructions, created by @GlaiveAI!
This model was originally meant to be OpenHermes-2-Coder, but I discovered during the process that it also improved almost every other benchmark!
Big improvements in HumanEval, but also in AGIEval and TruthfulQA, small improvement in GPT4All, and a slight decline in BigBench. This equated to a net gain across the board.
44
130
1,022
216,970
Dude o3-mini-high + deep research when I asked to make code that uses OAI Library + o3-mini-high, gave me a deprecated gpt3.5 api call and used a deprecated API schema (completions, not chat)..
Whats the point of all this search and intelligence if you dont teach it how to use your own products lol
106
38
1,021
155,627
Looks like OpenAI's been using Nous' YaRN and kaiokendev's rope scaling for context length extension all along - of course never any credit but... Anyone who says "open source just steals from their 'real' research and rides on their shoulders" is completely wrong
I called it when they released extended 128k context on gpt4 just a few weeks after Nous released yarn lol
for context on yarn; deepseek and qwen also use it;
Paper: arxiv.org/abs/2309.00071
Eh It’s going to come out anyway now
Config: {"num_hidden_layers": 36, "num_experts": 128, "experts_per_token": 4, "vocab_size": 201088, "hidden_size": 2880, "intermediate_size": 2880, "swiglu_limit": 7.0, "head_dim": 64, "num_attention_heads": 64, "num_key_value_heads": 8, "sliding_window": 128, "initial_context_length": 4096, "rope_theta": 150000, "rope_scaling_factor": 32.0, "rope_ntk_alpha": 1, "rope_ntk_beta": 32}
46
98
1,023
115,433
Anthropic is writing papers about how AI deserves rights and that we should ban open source.. I hate that they have the best model but they do and so 🤷♂️
72
24
969
95,534
Announcing Hermes 2 Theta 70B!! Our most powerful model ever released, and our first model to catch up to GPT4 on MT Bench, and beat llama-3 70B instruct nearly across the board!
We were able to do a full finetune of 70B to ensure maximum quality, worked with @chargoddard to merge in Llama-3 Instruct, and improved the RLHF pipeline to get the maximum capabilities out of Llama-3 70B!
Fully capable of function calling, structured outputs for JSON mode and Feature extraction, and all the brains of L3 70B, beating it at every benchmark we tested except AGIEval, which we very closely come to matching.
One tip though, because of the merge, add <|eot_id|> to your stop tokens in LMStudio and GGUF inference engines, it sometimes outputs this token as an artifact of llama-3 instruct.
Introducing Hermes 2 Theta 70B!
Hermes 2 Theta is smarter, more creative, and capable of more then ever before.
It takes a strong lead over Llama-3 Instruct 70B across a wide variety of benchmarks, and is a continuation of our collaboration with @chargoddard and @arcee_ai.
67
115
990
123,726
I mean you have to expect this when sama banned openais investors from investing in perplexity tbh
32
14
977
116,207
So to recap:
- Yesterday, frontier closed model equivalent reasoning model from Qwen,
- This morning, frontier closed model equivalent reasoning vision capabilities from stepfun
- sometime today(?) a frontier video model from wan?
All open source
What is America doing?
63
64
991
79,695
All of their best people already left lol
OpenAI CEO absolutely cooks Zucc’s Meta AI:
> “Zucc is offering $100 million dollar signing bonuses to poach talent.”
> “None of our best people have taken the offer yet.”
> “I don’t think Meta is setting up for a great culture.”
> “I think people think OpenAI has a better chance at reaching super intelligence and also may eventually be a more valuable company… then everybody will do great financially.”
> “Meta is not great at innovation.”
> “We understand a lot of things that they don’t about what it takes to succeed.”
HOLY. FUCK. LMAO
30
24
949
77,281
Announcing Nous Hermes 2.5 Vision!
@NousResearch's latest release builds on my Hermes 2.5 model, adding powerful new vision capabilities thanks to @stablequan!
Download: huggingface.co/NousResearch/…
Prompt the LLM with an Image!
Function Calling on Visual Information!
SigLIP Integration!
This is Nous' latest version of a multimodal model with powerful capabilities, and further iterations will come in the future.
Learn how to inference the model with instructions in the model card and stay tuned for GGUF quantization, with eventual support in @LMStudioAI and other inference engines!
47
162
955
215,459
Grok is out.
320~B Params - 8x33B MoE
blog: x.ai/blog/grok-os
code: github.com/xai-org/grok
Download: magnet:?xt=urn:btih:5f96d43576e3d386c9ba65b883210a393b68210e&tr=https%3A%2F%2Facademictorrents.com%2Fannounce.php%3Fpasskey%3Decac4c57591b64a7911741df94f18b4b&t
43
122
955
222,976
Yea the router as everyone predicted is bad
ChatGPT literally got worse for every single Plus user today.
There's no way to reliably get thinking models anymore.
Before we had o4-mini, o4-mini-high and o3.
Now we have GPT-5 Thinking with 200 messages per week and a router that exclusively routes you to some small and shitty non-reasoning model.
45
28
971
89,040
People taking this as an its so easy anyone can do it are misinterpreting. Its so easy because of everyone in opensource building up and abstracting layers and providing reference implementations for others to build on. OpenSource ftw
8
22
910
54,792
Guys its been 2+ years and 1000s' of times more capital has been deployed since gpt4.. what the hell happened
143
30
917
137,071
I think this is the most insane part of 4.5 release to me. The knowledge cutoff is 2023 still. How do you even have a current pretraining run that didnt see data past 2023? So many API's and libraries from there are now deprecated, and so many new ones created..
Did chatgpt 3.5 data ruin 2024+ data? or was this made a long long time ago?
And don't use 4.5-preview for coding. Unless you love the ancient Oct 2023 updated knowledge about frameworks, and you love paying more for less.
73
27
905
159,173
This is how degraded theyve made dalle since prelaunch
41
29
885
102,054
Ok tried Devin on two new tasks, one was automatically creating data visualizations of benchmarks, which took about 4 hours and a ton of back and forth debugging with it to get it .. mostly working.
Second was having it just create a readme after looking over all the code. Hallucinated nearly everything.
I can't recommend Devin especially at this price at this time to others. If you've had a better experience, let me know though! We have many hours left in this month's subscription.. Will try out more probably
58
37
892
134,846
I want to build a really cool home library
like 200-300 books on a series of bookshelves
Give me your like, 3 absolute top books, fiction or non-fiction, doesn't matter.
451
28
885
189,513
Claude just casually deleting a full days work on an environment for no fucking reason - fuck you claude
109
20
885
79,072
Thanks DALLE3, everyone needed to know what pikachu's musculoskeletal system looked like
21
65
858
121,858
I have a true gift for LLM Devs and the opensource AI community. Several GPT-4 Generated datasets. Toolformer, Instruct, Roleplay-Instruct, and soon, Code-Instruct datasets, all generated from GPT-4. I hope I can give back more!
Check them out here:
github.com/teknium1/GPTeache…
19
144
882
133,405
I think a problem in a lot of ai companies right now is they are lacking a lot of creative types
94
47
836
85,406
These numbers are insane. I can't even imagine what the larger one(s) will be.
Looks like Mistral 7B might be dead as of today though, and maybe even sonnet lol
My favorite is the huge gains in coding capabilities
57
66
835
248,793
