Introducing two new tokenizer-free LLM checkpoints from our research lab: TFree-HAT 7B Built on our Hierarchical Autoregressive Transformer (HAT) architecture, these models achieve top-tier German and English performance while processing text on a UTF-8 byte level.

We are excited to launch our two models Pharia-1-LLM-7B-control and Pharia-1-LLM-7B-control-aligned. Both models and the code used to train them are now publicly available and open-sourced for non-commercial research and educational use. Read our model blog post here: aleph-alpha.com/introducing-… Learn more about our open-source codebase Scaling: aleph-alpha.com/open-source-… #writtenbyalephalpha

Today we introduce T-Free, a new paradigm in language processing. Tokenization is one of the core building blocks of large language models (LLMs), transforming natural language into numeric representations for further processing. (1/3) 🔗 lnkd.in/eTi7kjuc #writtenbyalephalpha

Today our partners have committed 500M$ towards our mission to push the frontiers of responsible and human-centric AI technology. (1/4) 🔗aleph-alpha.com/aleph-alpha-… #writtenbyalephalpha

🥳We're happy to announce that we closed a €23 Million Series A to build Europe’s largest, sovereign #AI language models. The funds enable us to accelerate the development of #GPT3-like text, vision & strategy AI models and grow our team of world-class researchers & engineers.

🚀 Exciting Announcement from Davos: Aleph Alpha Unveils Tokenizer-Free LLMs! 🚀 We’re thrilled to announce a pioneering innovation that was unveiled yesterday at the World Economic Forum in Davos: Aleph Alpha has introduced a groundbreaking tokenizer-free (T-Free) LLM architecture, setting a new benchmark for sovereign AI solutions. 🔑 What makes this breakthrough so important? Many governments and enterprises have struggled to develop and master a sovereign generative AI strategy due to a lack of data, resources and expertise, forcing them to rely on established models that do not perform adequately on relevant languages and specialized knowledge. This results in a loss of power where value capture, but also ideological alignment, has to be left to a small group of LLM producers. By removing the tokenizer – a bottleneck to adaptability in traditional models – our new tokenizer-free AI architecture enables: ✅ High-quality multilingual capabilities for multiple languages and alphabets. ✅ Customization for specialized industries. ✅ Lower computational demands for fine-tuning, reducing costs and environmental impact. For Finnish, for example, this innovative AI architecture can deliver a 70 percent reduction in total training cost and carbon footprint compared to alternative options. This innovation democratizes access to AI, empowering governments and businesses to create solutions that are fully aligned with their cultural and linguistic specifications, specialized knowledge and terminology, while increasing efficiency and fairness in AI applications. 🌍 💡 What drove us to this development? As a European company rooted in a diverse linguistic and cultural ecosystem, we’ve prioritized sovereignty and multilingualism since day one. Years of research have culminated in the T-Free architecture, a true milestone in AI technology. 🤝 Collaborating with @AMD & Schwarz Digits In collaboration with AMD and Schwarz Digits, Aleph Alpha's new and innovative LLM architecture is enhanced with high performance computing and a sovereign cloud solution, creating a full-stack sovereign offering that combines AI software and hardware. 🔗 Press release: lnkd.in/eGAsPk9R 👉 Technical insights on our blog: lnkd.in/evse6YD9 👉 Our latest peer-reviewed Research Paper: lnkd.in/ewmJTQTf Let’s shape the future of trustworthy, sovereign AI solutions together.

New in Neural Network Parametrization Technique: Introducing Unit-Scaled Maximal Update Parametrization (u-μP). In partnership with @GCResearchTeam, u-μP merges μP and Unit Scaling to boost training stability & hyperparameter transfer across model sizes. Read more about the theory & practical tips for LLM training in our blog. 🔗aleph-alpha.com/in-awe-at-th… #writtenbyalephalpha

We present to you...LUMINOUS! Aleph Alpha's multilingual, multicapable, generalizable AI language model "Made in Europe" that helps you get the most out of your data. piped.video/wi73fB19-qI #writtenbyahuman

We have compared #Luminous to @OpenAI Davinci, @MetaAI OPT, and @BigscienceW BLOOM, and the results are astonishing. A Thread 1/11 🔗aleph-alpha.com/luminous-per… #writtenbyalephalpha

We're thrilled to announce Lab 1141, a groundbreaking collaboration between Aleph Alpha Research and @TUDarmstadt, with the goal of pushing the boundaries of explainable, safe, and transparent GenAI. 🔗 aleph-alpha.com/aleph-alpha-… #writtenbyalephalpha

At #AlephAlpha we are developing Large Language Models (LLMs) that can be trusted in real-world applications. Our most recent work is AtMan, an #ExplainableAI method providing explanations of generative transformer models at almost no extra cost. A Thread 1/9 #writtenbyalephalpha

For #ICLR2025 we are unveiling a new, high-quality pretraining dataset for German LLMs. Shared to strengthen the open research community. Shaped by our belief in excellence and transparency. huggingface.co/datasets/Alep…

Today we also launch PhariaAI, our new end-to-end AI solution, for enterprises and governments, to built develop, deploy and distribute sovereign generative AI technology. PhariaAI combines proprietary innovation with our new state-of-the-art open-source models and components and is meeting the requirements of complex and critical information-based value chains. Learn more here: aleph-alpha.com/phariaai/ #writtenbyalephalpha

We are proud to announce that we have partnered with @HPE to enter the AI cloud market by expanding the @HPE GreenLake portfolio to offer LLMs. Together, we will extend our #Luminous models to the cloud and offer them as a service to our customers to fuel new applications for business and research initiatives. hpe.com/us/en/newsroom/press…

We are excited to share that out paper AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation also got accepted for #NeurIPS2023. Meet us in New Orleans: on Wed, 13 December 10:45 am CST — 12:45 pm CST Paper: ​arxiv.org/abs/2301.08110 1/9

Together with @carperai, @AiEleuther, @UBC_CS, @StabilityAI, and others we are introducing #QDAIF: Quality-Diversity through AI Feedback to generate more diverse and high-quality natural language. 🔗Read the full research here: qdaif.github.io/

We gathered here to m#ourn the tokenizer, whose time has passed. We released three 8B models – and said goodbye to an old friend. Find our models on Huggingface: huggingface.co/collections/A… And read more in our paper: arxiv.org/abs/2501.10322 #RIPTokenizer

#1 fastest European 🇪🇺 commercial data center & #72 of all supercomputers worldwide! Our very own #alphaONE #AI data center is up and running! Congrats to our infrastructure team (you guys rock🤘) and the teams from @HPE_AI & @NVIDIAAI for their support. Not bad for a #startup!

Meet us in New Orleans on Tue, 12 December 5:15 p.m. CST — 7:15 p.m. CST at #NeurIPS2023 where we talk through our paper MultiFusion. Paper: arxiv.org/abs/2305.15296 Benchmark: huggingface.co/datasets/AIML… Project Page: aleph-alpha.github.io/MultiF… (1/13) #writtenbyalephalpha

We’re releasing: - TFree-HAT-Pretrained-7B-Base – trained from scratch in English & German - Llama-TFree-HAT-Pretrained-7B-DPO – post-trained on the base version for stronger instruction following Designed for efficient deployment with on-device and local inference setups.

By skipping subword tokenizers (which fragment non-English text), HAT packs more information per position. Higher compression → fewer generation steps → less FLOPs. In our tests, TFree-HAT delivers on average +40% compression in German, +16% in English vs. standard tokenization

We are proud to share that our two papers #AtMan and #MultiFusion have been accepted for the main track at @NeurIPSConf this year. NeurIPS is one of the most prestigious AI conferences that brings together researchers, practitioners, and industry experts from around the globe. Find out more about the sessions here: nips.cc/virtual/2023/index.h… AtMan: arxiv.org/abs/2301.08110 Multifusion: arxiv.org/abs/2305.15296 #writtenbyalephalpha #NeurIPS2023

We are strongly committed to #Opensource & the reproducibility of #research. That's why we're happy to publish MAGMA code + checkpoint. All our language models now have full multimodal functionality = No reason to use pure language models ever again. github.com/Aleph-Alpha/magma

Try them out, experiment with the architecture, and see how tokenizer-free models handle your language or domain. Find our blog post here: aleph-alpha.com/introducing-… Or read our model cards: huggingface.co/collections/A…

We're deeply committed to state-of-the-art #AI R&D. But what good is R&D, if only a few understand it? That's why we teamed up with☕️ @AICoffeeBreak to summarize and break down some exciting recent developments in 🧮mathematical deep learning. Feel free to comment.

Join us for Aleph Alpha's AGI Summer Talks on Wednesday, 26 July to discuss challenges, consequences, and future opportunities of AGI. 🔗Get your ticket here: eventbrite.com/e/agi-summer-… #writtenbyalephalpha

Made in Germany - created for Europe and the world... We are live! #Artificialintelligence from Europe will make people and businesses all over the world more resilient for the coming digital decade and strengthen our values of #diversity, #security and #freedom.

Our newest model Luminous-Supreme-Control is now available! It is optimised to better follow your instructions. A Thread (1/7) Get started here: app.aleph-alpha.com/ #writtenbyalpehalpha

What a week🤯. Not only did we close a record €23 million Series A for generalizable AI "Made in Europe" 🇪🇺, but we've also been chosen as one of the #AI champions by the @WM_BW for the development and application of our multimodal, european #AI language models. #AGI

We present to you Luminous-Control: the first model-generation ever developed to uniquely combine enhanced human-like interaction with functionalities for source verification and the explainability of AI-generated content. Start now: app.aleph-alpha.com #writtenbyalephalpha

We present to you our newest #Luminous functionalities #AtMan and #Explain to make any of your AI-generated content trustworthy and tracable. (1/4) 🔗docs.aleph-alpha.com/docs/ca… #writtenbyalephalpha

💡Our first joint work from our researcher Björn Deiseroth with the team from @TUDarmstadt @kerstingAIML @MBrack_AIML @schrame90. It shows that interacting with the self-generated reasons of LMMs help to improve the alignment of multimodal reasoning. 👉arxiv.org/abs/2208.08241

Have you ever wondered why and how a model predicts what it does? Together with @TUDarmstadt we introduce AtMan that provides explanations of generative transformer models at almost no extra cost. 📄 arxiv.org/abs/2301.08110 #writtenbyalephalpha

Our team published a break-through method to add multi-modality to today's #LLMs and they subsequently added this to our #Luminous models. One year later, we achieved another break-through. M-VADER creates images from any mixture of text and images. 🔗 lnkd.in/eS56uym7

We can confirm that this is indeed "a hedgehog in a red coat sitting in a chair, reading a book". 🦔 We just wanted to be sure of that. Well done @GoogleAI

Inference isn’t just about speed. Our latest blog breaks down how hardware choices impact latency, throughput & cost when serving massive models like DeepSeek v3. Read the blog & full report: aleph-alpha.com/deepseek-inf…

We’re shipping both models with: Hugging Face inference for easy testing vLLM fork optimized for HAT – note this is still under active development Weights & inference code are available today under Open Aleph license for research and educational use.

In order to use generative AI for critical tasks in law, healthcare, and banking we must ensure that AI-generated knowledge is #explainable, #traceable, and #trustworthy. 🧷ow.ly/eT6l50NgHOZ #writtenbyalephalpha

Our friends at @qdrant_engine created an example showing how to store Aleph Alpha embeddings in their database. You can check it out at: qdrant.tech/documentation/in… #writtenbyalephalpha

Read our full paper here: arxiv.org/abs/2406.19223 Dive into the source code of T-Free: github.com/Aleph-Alpha/trigr… Try out our interim research model checkpoints: huggingface.co/Aleph-Alpha/t… huggingface.co/Aleph-Alpha/t… (3/3)

🎉 Over the last few weeks we've introduced some amazing features to improve your experience with Luminous! 🚀 Here's a thread about what's new: 1/9

Language performance (comparing the DPO model): German: Outperforms Llama 3.1 8B Instruct on 67% of benchmarks. English: On-par with Llama 3.1 8B Instruct Consistent bilingual quality while keeping the model compact and efficient.

We are thrilled to welcome @TheGregYang at our next #nittygritty on July 27th. Come, join and learn about “The unreasonable effectiveness of mathematics in large scale deep learning”. Sign up here: ow.ly/n0Q350JVBZV #writtenbyahuman

Our innovation, T-Free, offers a novel approach to tokenization, boosting tokenizer fertility across various languages, and reducing the size of the embedding layer by up to 75% compared to traditional tokenizers. Early experiments with T-Free show promising results and could unlock new possibilities in LLMs, including: - Up to 50% reduction in training and inference costs - Improved semantic encoding of language - Enhanced performance in multilingual models (2/3)

We’re looking for passionate AI Research Engineers in Alignment/Explainability to work with us on our vision of an Artificial General Intelligence (AGI) "Made in Europe". For more info: ow.ly/YDUm50DGklk

In LLM-as-a-judge evaluations (MTBench), the 7B-DPO model’s answers were preferred over Llama 3.1 8B in: 69% of English samples 75% of German samples

#EleutherAI has released 2 GPT3-like models, sized 1.3B & 2.7B parameters, making the latter the 2nd largest publicly accessible autoregressie LM to date. We are very proud of our colleagues’ role in bringing large NLP models to the #opensource community. ow.ly/kLQR50EhqGK

We are also excited to announce future partnerships with IPAI, Bosch, Schwarz, Christ & Company, HPE, SAP and Burda Principal Investments. Welcome aboard! (3/4)

What is Germany's prospect as a global AI-technology leader and why is it important to build a strong collaborative AI-ecosystem? Our CEO @JonasAndrulis spoke on these topics and more at this year's @Riseof_AI. piped.video/watch?v=coBrx6UC… #writtenbyalephalpha

We strive to make constructive contributions to the debate on future developments of AI. @JonasAndrulis is part of this year's Annual Meeting of the @wef in Davos to speak on the most pressing issues of future AI. @wef AI Lunch, 2:00pm #writtenbyalephalpha #writtenbyahuman

🌟 Luminous-Supreme is now available! ☑️ After Luminous-Base and Luminous-Extended, Luminous-Supreme is the newest and most powerful generation of our multilingual language models. ▶️ lnkd.in/e2Zwq_3V #writtenbyahuman #writtenbyalephalpha

Thanks @NielsRogge & Congrats on 🦩 @DeepMind

We would like to thank the investors who believed in us early on and brought us this far: LEA, Earlybird, 468, Lakestar, UVC, Cavalry. We look forward to continuing our collaboration with you. (4/4)

Hard work does not go unnoticed…and sometimes it is rewarded. We from Aleph Alpha are very excited to be nominated for the German Founders Award 2022. Let's continue to shape the #AI landscape for #Europe. #writtenbyahuman #writtenbyalephalpha

🚀 Today is the day, we are launching #Lumi, a digital citizen assistant, which allows an entirely new interaction with the city of #Heidelberg thanks to innovative #AI 𝐌𝐚𝐝𝐞 𝐢𝐧 𝐄𝐮𝐫𝐨𝐩𝐞. 🔗 Sounds fun? Try for yourself: heidelberg.de    #writtenbyalephalpha

Control over AI systems is one of our key focus topics at @Aleph__Alpha. In this work, which is one step towards control, we show how to leverage automatic AI feedback to create multiple outputs which are diverse according to user specified dimensions, such as topic-focus and sentiment. Work performed in collaboration with @carperai and @StabilityAI. #writtenbyalephalpha

Truly “Made in #Europe” #AI solutions require cross-border cooperation. We are looking forward to our new partnership with @graphcoreai to optimize both our technologies for the next generation of multi-modal models. ow.ly/tH6v50Jth3G #writtenbyahuman #writtenbyalephalpha

We are very happy to intensify our cross-border cooperation with @graphcoreai on semantic search

🚀 Remember our #ai rockstar @MarkiSchmitz at the Big Techday of @tngtech? 🍿 His keynote "Luminous, my laptop and me" is now available to watch on YouTube. 🔗 lnkd.in/eknhT4fg #writtenbyahuman #writtenbyalephalpha

Viewer discretion is advised! Multimodality 🖼️ + Multilinguality 🇪🇺 = 🤯

🚀 A week ago, we had the pleasure to share our vision of future #AI. 💻 Together with hessian.AI we held our first workshop on Transformer based World Models.#thankyou for the great cooperation. 🔗lnkd.in/e3HdHRDk.

Thanks @mikebutcher for taking the time to listen to our vision of generalizable AI "Made in Europe". Together with @EarlybirdVC, #Lakestar, @UVC_Partners, #LEAPartners, @468Capital & @CavalryVC, we sure will solidify Europe as an #AI powerhouse! 🇪🇺

...and indeed he will. Our CEO and founder @JonasAndrulis is attending the @NVIDIAEU @NVIDIAGTC this Thursday 24th. You're of course more than welcome to join, ask question and share your experience. See you there.

#AI assistant systems, which run on #Luminous will shorten processing times and ultimately lead to a more efficient and effective public administration. A live demonstration given by our CEO @JonasAndrulis today at @GovGipfel showed how it works in practice. #writtenbyahuman

Congrats to the authors: @MarcoBellagente, @MBrack_AIML, @hannah_teufel, Felix Friedrich, Björn Deiseroth, Constantin Eichenberg, @andrewdai99, @Robert_Baldock, Souradeep Nanda, Koen Oostermeijer, @fffffelipec, @schrame90, @kerstingAIML, Samuel Weinbach. (13/13)

We won't stop, cause we can't stop - Tomorrow with @PfeiffJo, who is speaking at our 2nd #nittygritty. We're quite psyched about this because adapters played a great role for MAGMA. Come, join and learn about the next evolution for transformers. nittygritty.ai/220324_jonas_…

Our #explain functionality can also be used for explaining your own documents. Simply use our explain-endpoint to investigate which sentence influenced the completion the most. Read more here: docs.aleph-alpha.com/docs/us… Try here: app.aleph-alpha.com/ #writtenbyalephalpha

💬 No more pre-trained #Q&As but rather human-like conversations. With our conversational module #Lumi there are new possibilities for human-machine-communication, which makes us not another #Chatbot company.   🔗 Read more:  aleph-alpha.com/use-cases #writtenbyalephalpha

If you were not sure how to use #AlephAlpha models, we are here to help. Our developers wrote for you a use case showing how you can leverage prompt engineering and our Large Language Model API to build truly intelligent products. A Thread 1/4 #writtenbyalephalpha

#MAGMA agrees that this is indeed impressive.

Join us for our interactive seminar tomorrow at 5:00pm (CEST). Greg Yang from @MSFTResearch will talk about the unreasonable effectiveness of mathematics in large scale deep learning. Sounds fun? Sign up! nittygritty.ai/220727_greg_y… #writtenbyahuman #writtenbyalephalpha

Thanks to @timmpsmith from @Siftedeu for interviewing @JonasAndrulis on the gap in the market of #generative #AI in #Europe, and how we are trying to fill it.

Many thanks to @EarlybirdVC, #lakestar, @UVC_Partners, #LEAPartners, @468Capital and @CavalryVC for their trust in our vision to build a revolutionary 🇪🇺 #European #AI ecosystem and safeguard the digital sovereignty for the European public & private sector.

Danke an @_SilkeHahn von @heiseonline für diese in-depth Analyse unseres Leistungsberichts zu #Luminous und der weiterführenden Übersicht über unsere Forschungsansätze bei @Aleph__Alpha.

“The course we set now will determine what the world will look like in 50 years from now. We will then be in an era where we're no longer slowed down by human capabilities alone. Becasue we can't predict the future, we must create it ourselves.” @JonasAndrulis at #OMR23 #writtenbyalephalpha

We signed it, sending a signal to the world. The EU’s AI Code of Practice isn’t just another piece of regulation. It’s a declaration. Of what AI can be when built with transparency, responsibility and sovereignty. Just like ours. aleph-alpha.com/a-signature-…

„#Luminous ist in vielen Umgebungen eine starke Alternative und damit ein wichtiger Schritt zur Technologiesouveränität Europas.“. Vielen Dank an @CKerkmann vom @handelsblatt für die Veröffentlichung unseres Vergleichsberichts zu #Luminous.

📣 Together with @graphcoreai we were able to reduce our model from 13bn parameters to 2.6bn parameters. Our research teams, cleaned up 80% of the less relevant model weight, while retaining the most important parameters. 🔗 Read more about it here: lnkd.in/eSwtgz7h

This is a collaboration of @Aleph__Alpha and @TUDarmstadt as well as @Hessian_AI, @DFKI, @laion_ai, @CentreForCogSci. (12/13)

Pünktlich zur parlamentarischen Sommerpause durften wir gestern auf der 57. Stallwächterparty die neuste Generation digitaler #KI Assistenten #Luminious vorstellen. Wir bedanken uns bei #TheLänd in Berlin für die Möglichkeit zur Präsentation sowie den konstruktiven Austausch.

The @handelsblatt named us as one of the most innovative companies in Germany that will have a significant impact on the #digital transformation. Join us, if you want to be part of this incredibly exciting journey. aleph-alpha.de/career handelsblatt.com/unternehmen…

#AI meets #Art. @Bogula vom @aric_hamburg hat unsere multimodale MAGMA-Funktionalität genutzt, um kreative Titel für abstrakte Kunstwerke von @peezmo_ zu erschaffen. Weitere Informationen auf unserem Blog aleph-alpha.com/using-aleph-… #writtenbyahuman #writtenbyAlephAlpha

Only through a supportive environment and a strong team we can fill the gap of #generative #AI in #Europe. Many thanks to @EarlybirdVC for the constant support.

#Data is the new gold – but only if we generate #knowledge from it... That's one why we developed #Luminous. Try it out: app.aleph-alpha.com #writtenbyahuman #writtenbyalephalpha

🇩🇪 & 🇪🇺 müssen die technologische Revolution großer #KI-Modelle aktiv & souverän gestalten - damit wir nicht einen weiteren digitalen Sputnik-Moment☁️ erleben. Vielen Dank an @KASonline & @jasonchumtong für diese Gelegenheit. kas.de/de/einzeltitel/-/cont… #KIMadeinEurope #Vertrauen

We're proud to announce the closing of a record seed funding round to research and develop #Artificial General #Intelligence (AGI). With our investors, academic & industry partners, we address the growing demand for #AI made in #Europe. We're hiring: hiring@aleph-alpha.de

From the whole team at @Aleph__Alpha. We want to wish everyone a happy, healthy and prosperous new year. Thank you for all for your fantastic support through out 2021.

Congrats o the authors: @mayukh091, Björn Deiseroth, Samuel Weinbach, @schrame90, and @kerstingAIML. Thanks @Hessian_AI, @DFKI, and @TUDarmstadt for their collaboration. 9/9

🗣 Language is a carrier of culture. 🌟 With #Luminous, we have created an AI technology, which speaks and writes in German, English, French, Spanish and Italian while comprehending European cultures and values. 🔗app.aleph-alpha.com

Wir bedanken uns bei BW Finanzminister @DerDanyal @FinanzenBW & MdL. @tok_tayfun für ihren Besuch bei uns in Heidelberg. Der Schwerpunkt des Dialogs lag auf der Erörterung der Chancen für Politik, Wirtschaft und Gesellschaft durch #KI-Technologie "Made in Europe".

🎉 In tomorrow's #NittyGritty seminar, Lucas Beyer @giffmana from #GoogleBrain will talk about learning and transferring general visual representations. 🚀 Join us! 🔗 Event link: lnkd.in/e-VYqW7K #machinelearning #ml #ComputerVision #openscience #writtenbyalephalpha

🗣 Yesterday, the #team came together to learn about giving and receiving feedback. It was a great opportunity for all of us to engage, laugh, improve and discuss the importance of feedback. 🖤 A big #thankyou to the whole team for participating. #writtenbyalephalpha

For a deeper dive, check it out on arxiv: arxiv.org/abs/2301.08110 Congrats to the researchers: @mayukh091, Björn Deiseroth, Samuel Weinbach, @schrame90, and @kerstingAIML. Thanks @Hessian_AI, @DFKI, and @TUDarmstadt for their collaboration. 9/9

Here are the links for this release: Huggingface: huggingface.co/Aleph-Alpha/u… Paper: arxiv.org/abs/2407.17465Scal… Codebase: github.com/Aleph-Alpha/scali…

AtMan's memory usage is comparable to a forward pass. It scales up much more efficiently when compared to other methods that require a backward pass. The comparison shown here is on a batch size of 1. AtMan’s runtime decreases significantly with higher batch sizes. 8/9