As we are learning DeepSeek is one of the most sophisticated psyops of all time. Here's how it went down: 1) Release the model open source. 2) Include highly detailed papers for all other people to replicate your work. 3) Create a novel SOTA RL algo that uses less memory and bandwidth than PPO and release it absolutely free so anyone can copy it for their own work. 4) Make sure it's super easy to fine tune so that others can easily SFT out any local law abiding answers about Tiannamen in about half a day for $50 in compute. 5) Cackle while saying muahhahaha you can only run it on our servers. Oh wait actually you can run it on any hardware you want, fine tune it, replicate our entire method on your own models, which lifts up the entire research community. Truly one of the most dastardly and sophisticated plans to come out of China in a long time!

How many more model releases do we need for folks to realize we are not getting to magical superintelligence with what we got? How many times do you have to see a model benchmaxxing to realize Humanity's Last Exam is a freaking idiotic name and that answering questions on it doesn't tell us shit about the true intelligence of the model? How many models do we have to see demonstrating superficial intelligence but utterly failing at long running, contextual understanding for people to wake up and realize that AI is just another tool? A good tool, a useful tool, a wonderful tool but not magic and not the end of all jobs and not the end of humanity or any other absurd fantasy of fools and dreamers. Fool me once, shame on you. Fool me twice, shame on me.

This man gave you Reddit, RSS, Markdown, helped stop early attempts to censor the web (that are now winning) and freed academic journals for researchers. And they killed him for it. A crime that has never been answered for and never will be

Lot of absurd takes like this on the superalignment team leaving OpenAI. The more likely reason they left is not because Ilya and Jan saw some super advanced AI emerging that they couldn't handle but that they didn't and as the cognitive dissonance hit, OpenAI and other practical teams building real world AI are realizng this fantasy of super intelligent machines rising up and getting out of control is a waste of time, money and resources. So they slowly and correctly starved that team of compute that could be used for more useful things like building capabilities into their products, which is what AI are, products.

How not to accelerate. Don't replicate this kind of stupidity in America with an onslaught of absurd AI legislation based on non-existent problems.

DeepSeek is the best AI company in the world right now. Any team would be lucky to have these engineers. If this was an American company the media would be singing their praises. But because it's China idiots think "they copied everything or they smuggled chips." Their week of code just proved both of those wrong. How many AI companies write their own file system and find unknown optimizations at the PTX level and solve the bubble problem in GPU utilization? I've watched multiple filesystem efforts fail over the years despite a decade of work by major open source masters and big companies. Meanwhile these folks wrote it in their spare time. You don't optimize this much when you have an embarrassment of riches in GPUs and bandwidth. And that's a damn shame. The sheer amount of GPU power and bandwidth of the US behemoths have made them fat and lazy by comparison. Why bother optimizing when you can just hurl more GPUs at the problem and burn cash like you're the freaking Joker in the Dark Knight? Meanwhile these folks are jamming with a 500% mark up while charging pennies on the dollar. All the while, American AI companies are bleeding cash while charging premium prices and releasing incremental model upgrades while still saying with a straight face that ASI is two years away.

Here's a rundown of my favorite books of 2024. Links at the end of the thread. I start with my favorite book of the year by far: A Brief History of Intelligence. It's an exquisite masterclass in writing and research. At one point I realized I'd highlighted nearly every page. By the end I'd made a big list from the bibliography to follow up on dozens of threads from its pages. I love books by authors who become positively obsessed with an idea and follow it with the single minded focus of a maniac. That's what happened with author Max Bennett who traces the evolution of our minds from simple clusters of neurons to complex thinking wetware, all while tying it into our all-out quest to create artificial thinking machines. The result is a hauntingly beautiful masterpiece filled with deep insights into how our brains came to be.

I wanted to see if AI could code me a complex app. Not a crappy little one-off script. A real program. Just one little problem: I mostly suck at coding. So can AI make magic for someone like me? Yeah. But...it's complicated. Here's what I learned along the way. 1/

If you work in AI and you are against open source you should be barred from using any open source tools. Good luck building your models then. No Pytorch for you. No Linux. No Kubernetes. No distributed training frameworks. Nothing. I never thought we'd be fighting the open source battle twice in my lifetime but here we are. Don't believe anyone, ever, who tells you that only them and the other "trusted" people can protect you. Kings and Queens and the old church have been running that scam since the Dark Ages. It was bullshit then and it's bullshit now. Nobody should be the overseer of AI for the rest of us.

If you want to understand why Sam Altman wants to raise 7 trillion then read this book. On it's face, it's an absurd number. But when you understand what's at stake, it's not. The global chip industry is the most advanced, interconnected and globally distributed supply chain ever built in human history. It's dominated by companies that cannot be replaced at every step. One company can make the 120-300M a piece EUV machines (ASML). One company in the world makes the lens in that machine (Zeiss), one of 5000 unique suppliers to ASML. One company can integrate and fabricate those chips at the highest level (TSMC). When you realize that China could invade Taiwan at any moment, and that TSMC makes 90% of the worlds advanced chips, that power your iPhone, Android phone, every Mac, all the Nvidia chips in the world (who manufacture exactly ZERO of the H100s and any other chips they make), all the tensor chips that Google makes and more, then you begin to realize what is at stake and why it's imperative that we reform the supply chain fast. amzn.to/48xn4g1

How dare these bastards do open science the way it used to be done instead of writing "due to the competitive nature of the markets we won't tell you fuckall about how the model was trained and how many parameters it has and we'll be sure to hide away the thinking for 'safety''?!?! It's outrageous.

If open source is "bad for national security" why does Linux run all the nuclear subs in the US, every super computer in the US for national defense and everything else, all the Gov clouds in the major clouds that the government runs on, the military laptops of service personel and more? Open source is in 92% of software. It runs every super computer on Earth, your phone and the router in your house to name a few. Without open source there is no OpenAI and Anthropic and other closed source platforms that rely on pytorch and Linux and containers and the Transformer architecture and more. People who use open source and then turn around and attack it are talking out of both sides of their mouth. This concentrated attack in open source has reached a disgusting level. Only people willfully ignorant of the history of open source (which Vinod is not), or naivety (the Doomers), or with a vested interest in keeping AI closed (like say, an early investor in OpenAI) would say this.

If you want to understand why the Times case has a near zero probability of winning, then read this thread. This fellow does a nice write up and he seems sincere in his belief that what he is saying about the suit is accurate and correct when in fact it's basically just a lot of wishful thinking, misunderstanding of copyright law and red herrings. He's really hopeful that this case will cement the media's right to charge machines to learn, something not even remotely covered by copyright law. The text does not say what he thinks it says and it does not even come close to a "slam dunk." In fact, the opposite. First, as I've noted before, trying to get everyone to license training data is not going to work because that's not what copyright is about. We all learn for free. We learn from the world around us and so do machines. Writers at the NYT did not pay the Hemingway estate for learning to write short, sharp sentences as young people studying journalism. Young quarterbacks do not have to call up Tom Brady to get permission to study his throwing motion to learn to throw a football. Copyright law is about preventing people from producing exact copies or near exact copies of content and posting it for commerical gain. Period. Anyone who tells you otherwise is lying or simply does not understand how copyright works. What else does he cite in the write up? The amount of money Microsoft makes? 1 trillion in new value on their stock price! Equating it to a fraction of the training data is utterly preposterous. The NYT claiming the value of reporting on wars and murders and politics as somehow relevant to the case? Not even remotely related. Pointless to even include except as a red herring. It's an attempt to ascribe nebulous public good value to actual value in stock price. No. Just no. Even the most damning thing, the prompts they cite as evidence of exact output by GPT of Times content, are obviously manipulated. Anyone in AI can see this in under a second. Nobody seems to be able to recreate the verbatim output with the BS prompts they provided. Why? Because the verbatim output almost certainly did not come from memorization, but from retrieval augmented generation (RAG) with web browsing. A programmer probably deliberately prompted it via the API to fetch a specific article and asked it to output part of the text and they provided only a fraction of the prompt instead of the whole prompt. If I ask it to go fetch a times article and output that for me then it's on me, not the model. I don't need machine learning to do this. I can do it with programming libraries from decades ago. This is nonsense. And including it will kill this case dead because the lawyers will not be able to reproduce this in the real world. Almost everything this fellow cites as evidence is sleight of hand, misdirection and not relevant at all to proving actual copyright violations, which are dependent on output not input. This case is going to get eaten alive, just like the Sarah Silverman case and others that were filed with a complete lack of understanding of how AI works, along with grandiloquent claims about copyright and violations that are absurd to even the most basic sniff test. This most likely outcome for this case is it being settled out of court with MS and OpenAI paying a licensing fee for ongoing training data which is what this is really about. It will be a bad precedent for everyone, everywhere because there is no actual ruling and it gives the illusion that they won and people should be held to ransom for training data.

This cartoon from 1923 nailed the date for the rise of generative AI.

How is that that one of the most powerful ideas in the entire history of technology is now under coordinated attack? If I told you that one little software concept powers the website you're reading this on, the router in your house that connects you to the internet, the phone in your pocket and your TV streaming service, what would you think? What if I said that the NASDAQ and New York stock exchanges where your retirement portfolio trades  is powered by the same ideas?  Or that those same concepts power 100% of the top 500 supercomputers on the planet doing everything from hunting for cures for cancer, to making rockets safer, to mapping the human brain? Not only that, it also powers all the major clouds, from Amazon Web Services, to Google Cloud and Alibaba and more. You'd probably think that was a good thing, right?  Not just good, but absolutely critical to the functioning of modern life at every level.  And you're be right. That critical infrastructure is "open source" and it powers the world. It's everywhere, an invisible bedrock beneath our entire digital ecosystem, underpinning all the applications we take for granted every day. It seems impossible that something that important would be under attack. But that's exactly what's happening right now.  Or rather, it's happening again.  It's not the first time. In the early days of open source, a powerful group of proprietary software makers went to war against open source, looking to kill it off. SCO Unix, a proprietary Unix goliath, tried to sue Linux into oblivion under the guise of copyright violations. Microsoft's former CEO, Steve Ballmer, called open source cancer and communism and launched a massive anti-Linux marketing barrage.  They failed and now open source is the most successful software in history, the foundation of 90% of the planet's software, found in 95% of all enterprises on the planet. Even Microsoft now runs on open source with the majority of the Azure cloud powered by Linux and other open software like Kubernetes and thousands upon thousands of packages like Docker, Prometheus, machine learning frameworks like Pytorch, ONNX and Deepspeed, managed databases like Postgres and MySQL and more. Imagine if they'd been successful in their early attacks on open source?  They would have smashed their own future revenue through short-sightedness and total lack of vision. And yet open source critics are like an ant infestation in your house.  No matter how successful open source gets or how essential it is to critical infrastructure they just keep coming back over and over and over. Today's critics of open source AI try to cite security concerns and tell us only a small group of companies can protect us from our enemies so we've got to close everything down again and lock it all up behind closed doors. But if open source is such a security risk, why does "the US Army [have] the single largest installed base for RedHat Linux" and why do many systems in the US Navy nuclear submarine fleet run on Linux, "including [many of] their sonar systems"?  Why is it allowed to power our stock markets and clouds and our seven top supercomputers that run our most top secret workloads? But they aren't stopping with criticism there.  They're pushing to strangle open source AI in its crib, with California bills like SB 1047 set to choke out American AI research and development while tangling up open weights AI in suffocating red tape, despite a massive groundswell of opposition from centrist and left Democrats, Republicans, hundreds of members of academia and more, the bill passed and now goes to the governor.  Eight members of Congress called on the governor to veto the bill. Speaker Pelosi took the unprecedented step of openly opposing a bill in a state Assembly that is primarily Democrat. And yet still the bill presses forward. Why? Do too many people just not understand what open source means to the modern world? Do they just not see how critical it is to everything from our power grid to our national security systems and to our economies? And the answer is simple: They don't. That's because open source is invisible. It runs in the background.  It quietly does its job without anyone realizing that it's there.  It just works. It's often not the interface to software, it's the engine of software, so it's under the hood but not often the hood itself.  It runs our severs and routers and websites and machine learning systems.  It's hidden just beneath the surface. The average person has no idea what powers their Instagram and Facebook and TikTok and Wikipedia and their email servers and their WhatsApp and Signal.  They don't know it's making their phone work or that the trades of their retirement portfolio depend on it. And that's a problem. Because if people don't even know it exists, how can we defend it? 1/

JD Vance's Keynote at the Paris Summit is a straight banger. Goodbye AI safety nonsense, hello AI opportunity. Let's go! (Link to speech in comments) Some quotes and a link to the speech: "I am not hear to talk about AI safety, I'm here to talk about AI opportunity." "The future is not going to be won by hand wringing about AI safety. It will be won by building." "When people come to us asking for more safety regulations we have to ask if that regulation is for the benefit of the people or for the benefit of the incumbent." "American AI will not be co-opted into a tool for authoritarian censorship." "When people talk about AI replacing jobs we think they're missing the point. AI is going to make us more productive, more prosperous and more free." "We need our European friends to look to AI with optimism, not caution." "The Trump administration is troubled by some reports that some foreign governments are considering tightening the screws on US tech companies. with international footprints. Now America cannot and will not accept that."

As I predicted, once one country does this the others will need to go all in on it too, or risk becoming an AI backwater. technomancers.ai/japan-goes-…

If you actually believed this then you'd be morally bankrupt for working at a company looking to make it happen. That leaves only a few actual reasons for saying something like this: 1) You believe you're a part of the few, specially chosen, wise people who should have this power and can guide it fairly (you're not and you can't) 2) You want daddy government to come in and give you a monopoly through discrimatory lawfare because see number one 3) You're unabashedly evil So which is it?

Here's the story of another technology that faced massive backlash in its time that will sound very familiar to today's battles over #AI. Coffee. a thread.

The NY Times is asking that *ALL* LLMs trained on Times data be destroyed. That includes GPT 3 and 4, Claude, Mistral, Llama/Llama 2 and pretty much any other model in existence. I call on @facebook, @cohere, @AnthropicAI, @MistralAI, @huggingface, @google and everyone else who cares about the future of AI to join the defense and smash the NYT in court for this overreaching attempt to twist and expand copyright to the determinant of pretty much everyone else on Earth but the NY Times.

People thinking AI will end all the jobs are hallucinating worse than Max Tegmark on an acid trip. And one of the reasons is this: AI does not make people 10x more productive and it is not a magical fix to anything. It is simply another kind of intelligence that shifts the bottlenecks to other parts of workflow, namely problem composition/refinement; iteration; and verification. You may have a 10X speed up in writing code but you now also have a 10X slowdown in verification because you have to read all that code and troubleshoot it and bug fix it or you may be shipping with dozens of hidden security vulnerabilities and untested bugs. Again, speeding up in one area just means a slow down in another area. The bottleneck shifts. If AI produces an email you can verify it quickly. If it produces a novel you've got a lot of reading and an exponential increase in problems to find in that long text. With that idiotic book by Yud on the way, we are about at peak stupidity/doom about AI and all that remains is this generation's The Population Bomb, aka If We Build It Everyone Dies.

This kind of safety research is utter nonsense. It's safety theater. Nobody asks the model if they can shut it down. We just shut it down. Its a blob of code. The IT team simply turns it off. Done. This is nothing like "testing an airplane" in the real world to see if it will crash. It's worse than nonsense. It has no practical value whatsoever for security or safety. Anthropic repeatedly and deliberately creates these sensational headlines and paints itself as the only wise, kind, safe, special people who can be trusted to guide AI because their strategy is to get Washington to pass legislation that boosts them and harms competitors. But when your safety "research" is on par with the TSA confiscating children's toys that look like guns and pretending it means anything for actual airline safety, why should they be trusted for anything?

Path of every great trader: 1) Start out clueless 2) Overtrade/ Take too many risks/ Don't set stops 3) Take massive loss 4) Reflect + come back stronger 5) Learn age old lessons the hard way 6) Set stops / Proper allocation 7) Stop following others 8) Develop your own style

AI Doomers are having zero impact on the development of advanced AI surveillance and autonomous weapons systems. Instead, they're intensely focused on restricting civilian access to AI. Ironically, this means they're ushering in the worst possible timeline for mankind. We'll have a world where all civilian systems are controlled, hobbled and deeply censored (aka "safe") and military, weapons systems and surveillance systems are hyper advanced. I don't want this timeline. Nobody else will either once they're forced to experience it in reality. See this post where Anduril just got 1.5B in additional funding to build advanced autonomous weapons systems (nitter.app/anduriltech?t=Mf-vEyvG… ). Also note that the EU AI bill has a 100% exemption for military/defense/surveillance. Guess what? So does every other bill, including SB1047. Threre is a zero percent chance that governments will restrict themselves from building advanced AI military and surveillance systems. There is not one single government on Earth that will restrict these technologies for themselves. Even if there was a pact, they would do it with black budgets just like the Total Information Awareness (en.wikipedia.org/wiki/Total_…) systems that were built in the US despite explicit restrictions from congress not to build them because mass surveillance was just too tempting for them. If you don't understand this, you don't understand much about life or human nature. By the way this world of advanced military systems is not coming. It's already here. China has an absolutely massive surveillance state that harnesses AI from top to bottom (facial recognition, gait detection, dissident tracking, predictive analytics) (economist.com/china/2023/11/…) and the war in the Ukraine is being fought with drones and AI repurposed from game systems (economist.com/leaders/2024/0…). There is even a newly appointed commander of drones (economist.com/europe/2024/07…). To be very clear, I'm not against AI military systems because I know the are an inevitable fact of life. I hate war. It's a disgusting and ugly waste of human life and it showcases the worst of what we are as a species. But I am a pragmatist to my core. I realize that no amount of wishful thinking will ever stop war or an escalation of military systems. Wars will be fought. Wars are won by having better stuff than the other guys and so I want my team to have the best systems. Simple as that. These systems can built and so they will be built. There is absolutely zero chance of stopping them. Restricting your own military development in the vain hopes that others will follow is foolishly naive. And yet that is exactly what many advocates for strangling American civilian AI believe. Helen Toner said "we don't have to worry about China" (former OpenAI board member and EA (yes you are EA, as you worked for an EA org and are continually funded by them and advocate their positions, Helen, despite your protests to the contrary) and Dan Hendrycks (whose team wrote the first draft of SB1047 and created a consulting org to profit from the bill) believes that by setting a "good example" that authoritarian regimes will just willing follow along to self-restrict development of advanced AI. Of all the ridiculous and stupid arguments of Doomers, this is perhaps the most absurd and frankly, stupidly naive thing I have ever heard in my life. It betrays an almost comically idiotic understanding of human nature and the way power works in the world. It's not just naive, it's dangerous. By pushing their cultural information warfare campaign with corrupted children's videos financed to the tune of 7-10M about AI destroying us all (nitter.app/DrTechlash/status/1821…) and using disgusting propaganda techniques like push polling (where the questions are knowingly and deliberately designed by AIPI to bias people against AI and NOT to collect an actual, realistic poll about people's real feeling about AI) (nitter.app/FLI_org/status/1821267…) they are pushing us right to the brink of the worst possible world. It's a world where your AI can't answer questions honestly because it's considered "harmful" (this kind of censorship always escalates), where information is gated instead of free, where open source models are killed off so university researchers can't work on medical segmentation (nitter.app/BoWang87/status/182102…) and curing cancer (budget conscious academics rely on open source models; they can fine tune them but can't afford to train their own) and where we have killer robots and drones but your personal AI is utterly hobbled and lobotomized. Resist this world at all costs. Protect access to civilian AI. Protect open source. Protect open weights. Fight for the future. If you can hear this, you are the resistance.

Imagine if all the people creating the web browser and the early internet were constantly warning you that the internet would destroy the world instead of doing what it actually did, which is make the world more connected than ever before, while giving you access to all the world's knowledge. We probably would have a bastardized version of the net, not the open, democratizing net that let anyone communicate with anyone else directly over massive distances. Of course, every technology has its downside and the internet has some and AI will have some too. But I don't understand this obsession with people who work in AI and want to tell you all the bad things that it will do. The mainstream media just eats this stuff up because that's what they do, conflict and horror. So they happily keep asking the same questions like when will AI kill us all or destroy all the jobs? And these folks just keep answering: Tonight's story is Dooooooooooooooom! When will we wake up from this collective delusion and realize that this is subconscious status seeking by folks who want to inflate their importance in the world and that there is no basis for what they are saying? How long will it take before we realize that what Altman said recently is the most likely scenario, that AGI will arrive and it will change the world but not as much as we expect, just like every technology before it. We find balance with technology. We adapt. That's what we do. These constant doomsday predictions are the one dimensional, black and white thinking of children. It doesn't take into account any other engineering developments along the way, or that we learn as we build technologies and that those learnings change the technology, or that there will be mitigations we put in place along the way that we take from those learnings, like we always do. Planes got safer and safer the longer we developed them and the more we understood their flaws. So did refrigerators and the production of milk and medicines and everything else in the history of technology. We learn. We adapt. If there is a problem with the net, it comes from its greatest strength in that it's given a voice to all these folks who have a problem for every solution and who can't change the subject and who won't shut up about it. They are leading us to a world where one or the more important technologies if the future is crushed and controlled by a tiny group of people who can censor anything and bend the AI to their worldview and filter out everyone else's instead of you interacting directly with people. Once people default to asking AI for everything, instead of looking it up themselves, AI becomes our interface to the digital world. We don't want that interface controlled by a few companies who can warp it and force it down a narrow channel with a limited world view, rather than showing you the vast and wonderful diversity of life. That's what you should really fear, not AI turning you into a paperclip. You should fear people who think open is dangerous, when open is the foundation for all greatness.

I spent a few hours listening to Dan Hendyrcks, who runs the non-profit AI Safety group behind SB 1047, aka the California AI Control and Centralization Bill. I find him charming, measured, intelligent and incredibly dangerous. Some of the most dangerous people in life are ones who can convincingly lie about their intentions and who can easily mask those intentions. After listening to him for several hours and reading his talking points info-graphic, which claims to protect open source AI and fine tunes of models, I'm convinced that not only does he know this is an outright fabrication, but that's it's 100% intentional. Make no mistake, Dan's p(doom) is 80%, meaning he believes that there is a 80% chance that AI will kill us all. Reading the language of the bill and with that understanding held clearly in mind, it's not hard to see what the true intentions of the bill are, despite Dan's measured language and slickly produced info-graphics. The intention of the bill is very clear for anyone who has eyes to read the text. It has three clear goals: 1) Ensure that only a small group of companies, rigidly controlled and overseen by a special government agency, have the right to create advanced artificial intelligence. 2) Destroy open source AI. 3) Make sure that model makers have liability hanging over them like the sword of Damocles for the rest of their life, ensuring that governments can hold model makers responsible for any misuse or crime from those models forever. Usually when I listen to AI doomers it takes about one minute to hear the flaws in their logic and the nonsensical logical leaps. It's a bit like looking at code from GPT. Looks great, reads well but under the surface it's riddled with bugs. Many doomers are fantastic at using the language of rationality without any actual rationality happening below the surface. Not so with Dan, who I find delightfully well spoken and clear. He's the kind of fellow I'd like to have a good meal with and a glass of wine. In the Future of Life podcast he rejected notions of "AI Foom" as unlikely and noted that we were in a slow take off scenario. He also subtly digs at the community by saying that belief in rogue AI was based on "cultural history" aka they read too much sci-fi. He also clearly lays out his strategy to go towards grassroots legislation and work with policymakers at that level. I'd almost like him if we weren't trying to crush American innovation and cripple AI development. And yet I find him tremendously dangerous and I'm not afraid to say it. Unlike other folks in the AI safety space he is good at masking his intentions. He's a bit like Yoda when we first meet him in Empire Strikes Back, cleverly disguised as a harmless old man. In all his talking points and in his communications on the bill he is measured and denies his true intentions up and down while working to ensure that AI is rigidly controlled by the Turing Police and centralized at all costs. He cleverly says that the bill protects open source because it "establish a new advisory council to advocate for safe and secure open source AI development." This is cleverly worded to create the illusion that open source is protected. Really it's an advisory board that fails to protect open source AI in any way whatsoever, and he knows it. The bill is absolutely a de-facto ban on open source AI for advanced models because it requires model makers to have “the capability to promptly enact a full shutdown of the covered model,” aka a remote kill switch, including the ability to force “the cessation of operation of a covered model, including all copies and derivative models, on all computers and storage devices within custody, control, or possession of a person, including any computer or storage device remotely provided by agreement." Of course, with a widely distributed model that is not tightly controlled or surveilled this would be impossible because model developers are held liable for the model and fine tunes of the model no matter where that model lives. The talking points also claim that fine tunes of the model are protected. They're not, because of this language: “(2) “Hazardous capability” includes a capability described in paragraph (1) even if the hazardous capability would not manifest but for fine tuning and posttraining modifications performed by third-party experts intending to demonstrate those abilities.” In other words, someone fine tunes a model they consider dangerous, the model maker is liable. This bill is insidious in nearly every way. It was conceived to sound simple and measured while its real intentions are much darker. It must be stopped. It's goal is to dramatically de-incentivize people and companies from ever releasing a powerful model as open source and to ensure that model makers can be crushed into submission at any time. It's designed to do this in the one state that is responsible for the vast majority technical development and innovation over the last few decades. All because someone believes in a the fantasy that AI will kill us all. I fully support everyone's right to believe whatever they want but they don't get to make laws for the rest of us and crush American innovation. If it wasn't so insidious, I'd almost admire it for its deviousness.

Regulatory capture in a nutshell: 1) Get the uninformed general public terrified of vague threats 3) Frame it as "for the greater good" 4) Employ true believers/well intention extremists 5) Co-opt ^^ useful idiots 6) Gov eliminates competition for you! politico.com/news/2023/10/13…

Two books I just read with essentially the same message: We've forgotten how to build in the west and a poisonous mentality and politics of scarcity and degrowth has taken hold on both the left and the right. For the left that scarcity manifests as we've got to use less, slow down everything from consumption to energy use. For the right, it manifests as immigrants are stealing all the jobs and houses and we've got to limit access to them. Both of these are dead end philosophies that have never made any society richer or more prosperous. The answer is the build. We won't deflate our way out of an energy or housing crisis. We won't get there by closing every door. We've got to build. More energy. More houses. More factories. Build. Build. Build. Nuclear. Solar. Wind. All of it. Not some of it. All of it. Now. Degrowth will lead to shortages and riots and more political extremism. And eventually war. That is history in a nut shell. In times of plenty we work together and grow. In times of scarcity we kill each other. If we don't have enough homes we've got to build more homes. Simple as that. To do that we've got to slash through a thicket of red tape in zoning laws and absurdly long permit procedures as builders fill out endless forms and submit environmental impact assessments that do nothing to protect the environment and everything to make life miserable for the builder and the young couple trying to buy their first apartment that is now out of reach. We're at a tipping point now. The politics of scarcity is winning. It's taken hold like a sickening rot that's poisoning minds and policies and setting us down a dark path. We still have a chance to turn it around before that tipping point is crossed but when the Rubicon is crossed it will play out like a vicious storm, and then there is no stopping it and its dark energy will rip apart everything in its path before it finally blows itself out in exhaustion and subsides again, leaving a trail of tears behind it. Nobody wants the storm except the insane and the short sighted and the people who make their living spreading rage and fear. But it's coming if we don't begin to think differenty and build new things and trust in science and technology to make a better tomorrow. Build.

How about don't listen to anyone who says anything like this because they actually have no clue how it will play out and just because they do something something AI does not mean they have any experience accurately predicting the future of civilization and the arc of tech development. Get your degree. Study. Study everything you possibly can. Learn everything you possibly can in life. Nobody ever looks back in life and says I wished I'd learned less stuff.

This has reached a nearly clownish level of virtue signaling and status gaming. Nobody is muzzling these folks and their right to complain constantly about this imaginary future with imaginary superintelligence that has no actual trajectory from current systems. Imagine the immense privilege of people being paid millions of dollars shouting loudly about their right to complain nonstop about the very product they're helping to create. This is not people standing up against slavery, or against genocide and political repression. These are people who want the right to make sure you know about their imaginary future hallucinations whether you want to hear about it or not. They already have that right. Nobody has taken it away from them. And no, a non disparagement agreement, which is standard across all kinds of business, is not taking away their rights to "warn people about AI." Nobody is silencing them. And nobody needs to hear a message that has reached Brandolini's level of misinformation and nonsense that the rest of us now have to clean up through absolutely enormous struggle, because the amount of effort it takes to kill this kind of bullshit is infinitely greater than the amount of effort it takes to create it.

To think I once hated @Twitter. I love it so much now! Direct access to some of the most brilliant and creative minds in the world, people I would never just stumble across in the real world. No other place like it. It turns six degrees of separation into one.

People are mad about water use by AI because they want to be mad about AI. They are not mad about it because they actually know anything about water use in data centers. Datacenters are evil! (Written by someone on a phone, whose software is powered by a datacenter, on a social media platform, powered by a datacenter, while researching "the problem" on the internet, powered by a datacenter. These are not serious people.

If open source is bad for security why does the US army use Linux? Why does it power nuclear subs in the US and the top supercomputers? Why does it power all of our stock market infrastructure, like the New York Stock exchange? Why does it power every cloud? Why did the NSA create security enhanced Linux? Why are all the tools that build models, like Pytorch and containers and Transformers, open source? People fundamentally misunderstand the nature of open source. They don't seem to understand that its open nature is its strength, just like America's open nature was its strength, one that we are throwing away more and more with each passing day. Inevitably people and societies who benefit from openness regress at some point. They start fearing the dreaded "other", some threat from the outside that is always nebulous and shifting, a Boogeyman that must be stopped by closing off and closing up. Inevitably this leads to that former leader's decline as they embrace closed and get all the downsides of it, over centralization, bloat, lack of competition, stagnation, unaccountable companies who don't need to listen to customers because they are locked in. Growth slows. Decline starts slow and then accelerates. Open source powers 92% of all software in the world. I was a part of the first multi-billion dollar open source company, Red Hat, for a decade, starting when there were 1200 people and watching it grow at a record clip quarter after quarter to 10,000 people. Many people made the same arguments then against open source. They tried to kill it and yet now there are dozens of open source multi-billion dollar companies and all the major clouds and closed source companies rely on it. Imagine if Microsoft had been successful in killing open source in the early days? They would have slaughtered their own future business because open source powers more than 50% of their Azure cloud. Short sighted thinking is death in business and for societies. If you stand against open source you stand against reason, evidence, the entire history of software, and what made America great in the first place.

Richard Sutton, father of Reinforcement Learning, just gave a great talk where he said "decentralized collaboration is the source of all human flourishing and the source of all that's good in the world." And if you're in favor of this, "the enemy" are the people calling for "centralized control" of both "people and AI." Centralized control of AI is the same as calls for centralized control of people. Control of AI chips and who can download what and what AI's goals are allowed to be, is the same as calls to limit free speech, and dictating who can trade with who, and who you can collaborate with, and what you are allowed to think. Authoritarians are the parasites of any system. They are the selfish cheaters who try to stack the benefits for themselves and their little tribe and they cause nothing but pain and misery for themselves and others. Trade, art, economics, community come from individual agents/people pursuing their own individual goals and choosing to work together. That is the light in the world. The darkness is the control freaks and the parasitic cheaters who try to stack the deck in their favor. Their worldview is one of darkness, of us versus them, and their fundamental belief is most people are evil and can't be trusted. Make no mistake, calls to restrict AI, to align AI, to create kill switches in chips, and to limit access to chips, and to embedded surveillance trackers in chips, are the same as calls to limit speech, and control where you go, and who you can work with and do business with. And these calls only ever come from the parasites of a system, the people who poison a healthy working system and who periodically pop up to set back the great and powerful and inevitable surge of human progress pressing forward endlessly and forever. (Link to talk in comments)

The only people who can look at communism with longing are people with zero understanding of history, zero understanding of how economics uplifts quality of life across societies, zero understanding of what happens when a delusional mob mentality sweeps through a nation and zero understanding of the horror of how communism plays out in reality.

Here's my major takeaways from the R1, DeepSeek-V3 and GRPO papers (links in the comments). TLDR: ◦ RL is NOT all you need. DeepSeek-R1 uses two passes of RL and two passes of SFT. ◦ GRPO is a fantastic variant of PPO that every lab should consider using as it uses much less memory and compute. ◦ Reasoning is heavily dependent on verifiable outcomes like math and programming and that may not generalize to fuzzy reasoning tasks like "is this advertising layout better than this other layout?" ◦ Synthetic data from a strong model is essential to help bootstrap better models. This is a key advantage that OpenAI/Anthropic and Google have because they have unfettered access to SOTA models and can distill from them, while restricting you from doing it. ◦ The distilled Qwen and Llama models were NOT trained with RL, just SFT. ◦ Whether scaling up hard reasoning leads to better fuzzy reasoning is THE open question in AI right now for every major and minor lab in the world! (probably not but let's see.) Here's my detailed thought below: ### RL is NOT all your need ### A lot of folks are saying "RL is all you need" but that's really not the whole story. Why? Because they trained two major models. ◦ DeepSeek-R1-Zero, is trained with no SFT, just RL ◦ DeepSeek-R1 This model addresses the limitations of DeepSeek-R1-Zero and uses two passes of RL and two passes of SFT It is incredible that they trained a model with nothing but RL, AlphaGo style, and it developed spontaneous reasoning capabilities. But the model suffers from a few problems, like poor readability of its COT and language mixing (mixing Chinese and English most likely, etc). So RL is all you need? Nope. But it's still super exciting and now other labs will replicate this for sure and we may get a pure RL model, AlphaGo Zero style very soon. ### Distilled Models They also released several distilled models (aka smaller models that learn from bigger, smarter models) based on Qwen and Llama. These models DO NOT use RL. They are pure SFT, basically because it's much more expensive to train 5 or 6 models with the full RL and SFT pipeline of DeepSeek. ### Training Data Having a powerful baseline model to create synthetic data is critical. This is the advantage OpenAI/Anthropic/Google have over many others. They have access to their own unfettered, uncensored models and can create synthetic data to bootstrap models and can restrict you from doing it with the TOS. This is what DeepSeek did too. They used their powerful DeepSeek-V3 model to create the following datasets: ◦ 600K synthetic reasoning dataset ◦ 200K non-reasoning dataset The non-reasoning data prevents overthinking and teaches the model to simply return some answers quickly, such as factual Q&A responses instead of wasting time thinking a bunch about simple things. "We expand the dataset by incorporating additional data, some of which use a generative reward model by feeding the ground-truth and model predictions into DeepSeek-V3 for judgment. Additionally, because the model output is sometimes chaotic and difficult to read, we have filtered out chain-of-thought with mixed languages, long paragraphs[sic], and code blocks. For each prompt, we sample multiple responses and retain only the correct ones. In total, we collect about 600k reasoning related training samples." "For non-reasoning data, such as writing, factual QA, self-cognition, and translation, we adopt the DeepSeek-V3 pipeline and reuse portions of the SFT dataset of DeepSeek-V3." ### Dependence on Verifiable Outcomes The reward modeling for the model is based on accuracy and verifiable outcomes: ◦ Math: correctness of the final answer ◦ Programming: leet code problems, compiled generated code, code run on test cases This means that if you want to use RL to teach it something that doesn't have a verifiable outcome (like whether an article is good versus bad, or whether this advertisement design is better versus another one) then you can't really RL it and that's not great. Anyone working on agents right now, needs to come up with a way to verify the outcome if they want to use RL. At my lab we're working on ways to do this with tasks on GUIs, for instance. But many things we want models to do, do NOT fall into verifiable end states, like deciding if one layout is a better advertisement design than another or whether a cartoon drawing of a horse is aesthetically pleasing. This begs the question I've been asking since news of Strawberry broke last year: Can models trained for HARD reasoning (math/science/programming) generalize to FUZZY, nuanced reasoning in other domains? This is now an OPEN research question. Maybe the most important one in all of AI right now. A number of OpenAI folks have publicly posted on X that yes they can. Others there have doubts and I have them too. In my article, "Why LLMs are Smarter than You and Dumber than Your Cat", published in July 2024, two months before o1-Preview release in Sept, I wrote that strawberry/o1 was based on a decade old paper called Q* from DeepMind, where models learned to play Atari games. Here's what I wrote: "If we look at the Q* paper, it's basically a learned deterministic policy, which means that exploration drops to zero. That's good for games, as it means that the model will never just randomly go right when it's found the optimal path at a certain point is left. It also makes it good at hard reasoning, like math, or formal reasoning like logic. "But what it likely won't do is capture "fuzzy reasoning," the kind of "rule of thumb" reasoning that we do. You might make an unconscious rule in your head that to check that an email was sent you need to look in the "sent" directory, but not always. On some other systems it might be "drafts" or something else and you adapt as needed." We'll soon see. ### AI Sanctions are NOT working and they WILL NOT work "Something there is that doesn't love a wall, that wants it down." - Robert Frost Trying to keep China from developing AGI is a fools game that has a 0.0% chance of success. This lab of very smart people has developed ways of working with less that are good for ALL labs across the world, because everyone wants to use less compute and less memory. ◦ Because they are resource constrained they developed smarted ways to do striped/sharded, distributed training across slower GPUs. See the DeepSeek-V3 paper. ◦ They also developed GRPO, a variation of PPO that is much more memory and computationally efficient. Speaking of GRPO... ### GRPO The main advantage of is it just uses a hell of a lot less memory and compute and it doesn't need a critic model. How it Works: ◦ For each question, GRPO samples a group of outputs from the old policy. ◦ It then computes the rewards for each of these sampled outputs. The advantage is calculated based on the relative rewards within each group. The rewards are normalized by subtracting the group average and dividing by the group standard deviation. ◦ GRPO then optimizes the policy model by maximizing an objective that uses these advantages. Key Differences from PPO: ◦ No Critic Model: PPO typically uses a critic model (a value function) to estimate the baseline, GRPO foregoes the critic model, using the average reward of multiple sampled outputs as the baseline. This reduces memory and computational burden. ◦ Baseline Calculation: PPO uses Generalized Advantage Estimation (GAE) which relies on a learned value function alongside the reward. GRPO calculates the advantage based on the relative rewards within each group of sampled outputs. ◦ KL Penalty: In PPO, a per-token KL penalty from a reference model is added to the reward. GRPO, however, directly adds the KL divergence between the trained policy and the reference policy to the loss. Biggest advantages of GRPO? It basically boils down to cheaper training costs. With no critical model, you use a lot less memory and compute. GRPO is also particularly effective for LLMs, where the value function can be complex and where usually only the last token is assigned a reward. GRPO's group relative way of calculating advantages aligns well with the comparative nature of reward models, which are usually trained on comparisons of different outputs for the same question. Even better, GRPO can be used with both outcome and process supervision. With outcome supervision, the reward is given at the end of the output. With process supervision, the reward is provided at each step of the reasoning process. Best of all, GRPO can also be used in an iterative way, where the reward model is continuously updated using a replay mechanism and the policy model is trained with the new reward model.

Number of AI Doomsday predictions correct so far (if we're keeping track): 0% - 10^25 FLOPs would produce magical super-models. Nope. - Okay, 10^26 FLOPs then. Nope again. - A flood of deepfakes will influence the election. No. - Models will be so persuasive that no one will be able to resist them. Nope. That's called advertising and we already have everyday, dumb algos that do it just fine. At what point do you call a 0% success rate a failure and discount all future predictions to zero? Put that in your idealized Prediction Market pipe and smoke it.

We are the luckiest 1% of 1% of people to ever live in the entire history of humanity. Anyone putting "progress" in quotes does not realize that life in history was nasty, brutish and short and we are absolutely blessed to live when we live. Some facts for you. A thread (1):

So the bubble popped. Good. Now the #crypto community can get back to doing the hard work of changing the world without the media circus surrounding it. Here are the five keys to crypto's evolution. hackernoon.com/the-five-keys…

Microsoft appears to be trying to patent RAG. They call it RAS "Response Augmenting System" but a rose by any other name... I'm guessing the community and various vendors have some prior artwork to throw at this application.

I mute the nitpickers, block the outraged, like the kind, follow the insightful. -- @naval (could not have said it better.)

There is absolutely nothing to "wonder about" here as to why Crowdstrike is installed everywhere, on every regulated system. It's regulation, audits and liability, of the exact kind that California's SB1047 hopes to cram down the throats of advanced AI makers. Basically when your friendly neighborhood auditor comes into town with his clipboard to check on your IT systems for your train/plane/infrastructure systems, he checks a box and moves on if you installed Crowdstrike, because it has been deemed by the all knowing auditors in the sky that Crowdstrike is safe and secure. If you dare to install something else, because you happen to actually understand IT security better than this pencil pusher, then he opens up a new and expensive chapter in his book and you have to justify why you deviated from the great wisdom of the IT safety board on high. Even better, it's all on your dime because you're forced to pay for his grand wisdom about how best to conduct your IT security! You get to pay for folks like Dan's Hendryck's Gray Swan AI to come in and share their infinite wisdom on how you should conduct your AI policy because it's required by law to pay him. I'm in the wrong business. I should get into passing laws to guarantee my personal income. Seems a lot easier than creating real value for folks and a product they actually want to buy!

With all due respect to one of the Godfathers of neural nets, who has forgotten more math than I will ever know, there is nothing wise about what he is saying here at all. 1) We're already making progress on "alignment" and/or teaching machines what we want them to understand. Contrary to the nonsense belief of some folks who tell us alignment is "impossible" or "we're making no progress", we're actually already making tons of progress in a very short time frame on guiding neural nets and seeing what is inside them. Check out RLAIF (arxiv.org/abs/2309.00267), RLHF (huggingface.co/blog/rlhf), Claude's personality training (anthropic.com/research/claud…), Golden Gate Claude research (anthropic.com/news/golden-ga…) where they figure out what neurons go with what concepts and suppress them or emphasize them, and community groups like Eluether on Eliciting Latent Knowledge (eleuther.ai/projects/elk). Now try looking inside a human's head and figuring out where one of their stupid/nonsensical beliefs came from. Can't be done. In fact, I argue we can already guide/teach/train AIs better than we can children or people. When you teach a kid a value what is the chance the child actually ingests that value? Answer: Who the hell knows! Why did they or didn't they accept it or reject it? Answer: Who knows! Why did some human make a decision or tweet something so idiotic it makes your head spin? What was in their past/chemical makeup/circumstances/wiring/experience that made them do it? Answer: Who knows! But with AI we're learning more and more each day and building systems that let us peer inside them and guide them better and that will only get stronger over time, especially if we have open source AIs that we can throw more human minds at understanding versus smaller teams that can only do so much with their closed/proprietary models. Our incentives are to have systems that do what we want, both from a research and commercial standpoint. That will drive the development of these systems. 2) Like all armchair futurists, Hinton makes the classic mistake of projecting one technology into the future and leaving everything else the same. In other words we get superintelligence but nothing else in society or technology develops. We get no parallel developments in safety, training methodologies, societal counters, etc, etc. It's like saying in the Wright brothers era that future planes will be made of canvas and wood and we'll never figure out any other ways to make them safer or more robust. We develop better safety procedures as we go. Changes at the technological and societal level happen in parallel. The future is trillions and trillions of variables all changing at the same time. That's how we got to modern planes that carry people all over the planet with an accident record of one every 1.26 million flights from Wilbur circling Kitty Hawk two or three times on a glorified kite. That is how all technology has developed in the history of the world. Over time we get lots and lots of planes and with each passing day we make them stronger, faster, better and safer. Elon makes his spaceships better with every flight, learning from the past. Eventually it will be a shocking event when even a single one blows up because his team managed to make them so safe that it becomes a massively outlier when one fails. Go read about race cars in the early days of Enzo Ferrari in the book "Ferrari, The Man and the Machine". Drivers regularly died and went careening into crowds, killing dozens of onlookers in the 1940s and 50s. Now go watch an F1 race today. Watch someone get out of a burning car, with only minor burns because he's wearing super advanced fire suppression clothing. Watch a car flip and slide along upside down on the Halo that protects the driver's head and watch him pop out unharmed. Both of those happened last year. In fact, despite cars going at breakneck speeds, we're shocked when a single person dies because the mitigations are in place to make something even wildly dangerous like F1 safe. To think we will develop no mitigations for AI and learn nothing as we work with and deploy the technology is just nonsense and the most basic error anyone can make in long term predictions. Life develops all at once, in parallel, at the societal and technological level. 3) Lastly, just because Hinton knows a lot about of neural nets does not mean he knows a damn thing about long term society and technological predictions. Don't make the mistake that expertise in one area applies to any other area. Who cares what a neurosurgeon says about running a hospital at scale? They're only marginally related and knowledge of how to do one has nothing to do with the other. Also, one celebrity endorser in a doomsday x-risk cult does not make a cult legitimate. It just means even smart people can end up getting grifted for their life savings when Jim Jones rolls into town. If people would stop using the cheap mental heuristic of "Hinton smart about AI, him must know other things too that have the word AI in it" and start looking carefully and closely at his arguments, they would find there's really nothing at all to what he is saying here. At this point, we'd very much appreciate if Doctor Hinton would stop talking about imaginary x-risk fantasies (and stay out of California politics) and instead enjoy his retirement because he earned it. Maybe some travel and fine food, good sir? I can recommend the south of France, Japan and Basque country for some food that will blow your mind. Maybe a beach somewhere to relax? I personally hate just sitting around doing nothing but some people love it. Maybe catch up on reading or painting? A museum tour? Take up gardening. Leave the future of AI to the applied AI and ML research teams of today and tomorrow. Don't worry, we got it from here and we're doing just fine.

I've joined Stability AI as CIO. Check out why in my latest article where I cover the transformative power of generative AI, the rise of open foundation models, + the battle for a future that's truly open versus open in name only. #stablediffusion danieljeffries.substack.com/…

I feel like everyone slept on this paper, Mixture of a Million Experts, which essentially solves catastrophic forgetting and continual learning with a dynamic, learned router called PEER, that can find the top experts for a task even with millions of them. Theoretically any time you want to learn something new, you just seed the model with new experts and teach it that new thing and let the PEER router adjust to the new knowledge. To me this feels like it captures the plasticity of the brain. I wonder if it's just computationally infeasible outside of the big labs at the moment or if there is some flaw someone spotted in it? Or was it just lost in the deluge of amazing stuff we've seen in the last few years? What other alternative architectures offer the above advantages? None that I know of currently. datacamp.com/blog/mixture-of…

Dopamine is not the "pleasure" chemical. It's fired when a mammalian brain is expecting a reward through trial and error learning but also decreased when the pattern the brain is looking for does match the expected reward. It's more like the "wanting" or "anticipation" chemical. Neither is serotonin a pleasure chemical. It's more like the "satiated" chemical but it is not pleasure. It actually decreased the desire for more pleasure, like after a good meal where it basically shuts off the desire for more food.

Under no circumstances should the US government be licensing AI model companies. Putting complex technology in the hands of bureaucrats to decide who can and will come to market is an utterly awful idea on every level. Anti-American. teddit.net/r/ChatGPT/comment…

Thank God that wisdom is finally starting to prevail over delusion and scare mongering and lobbying.

Stable Diffusion 2.1 beta coming real soon to a Discord near you. Mamma, look at my hands! And look at those sweet, sweet art styles. #StableDiffusion2 Close ups in the thread. /1

Destroying Terrible Anti-AI Arguments: Day One "Would you open source a nuke?" This a classic example of "begging the question" where the questioner assumes the truth of the conclusion, instead of supporting it. It bakes in the presupposition that AI is tremendously dangerous, without actually demonstrating that it is dangerous at all. It makes a false analogy with nuclear weapons to create this presupposition. Nuclear weapons have one purpose: to kill lots of people at once. Very few technologies are inherently destructive like this and to make this analogy showcases the questioner's lack of understanding that every technology has an inherent range of capabilities/possibilities from good to bad. Those capabilities may lean more to one side or be somewhere in the middle. On the whole, a lamp in your house leans strongly to the side of good but I can still hit you over the head with it or you can electrocute yourself with it. A gun may lean more strongly to bad but I can still hunt to feed a family with it or defend against an intruder. General purpose technologies are somewhere in the middle of the range in that they can be used for almost anything and be made to serve many purposes. AI is such a technology. AI has a massive range of capabilities. It can teach a young child to learn a new language or discover new potential pathways for combating cancer or it can be used for surveillance and monitoring dissidents in an authoritarian regime. It is a tool, wielded by the user of that tool and it mirrors the intentions of the wielder. For a better tech analogy, AI might be closer to something like Linux. Linux has a tremendous range of capabilities as well. It's used in all the supercomputers on Earth, the vast majority of smartphones, most home routers, and it powers every major public cloud, to name a few. It's also used to write malware and create botnets and it powers the supercomputers and clouds of authoritarian nations too. Very few people would argue now to ban Linux, though they tried in the early days of open source, with Balmer and Gates calling it "cancer" and "communism" and Sco trying to sue it out of existence. Despite the fact that Linux is used for some purposes we prefer it doesn't get used for, we let it proliferate because the overwhelming positive benefits of a widespread set of common software building blocks for the world. Every technology has inherent downsides but if it has a range of capabilities, we let the technology proliferate far and wide because we want to reap the benefit of those capabilities as a society. The more we reduce roadblocks to access the more ways that technology proliferates and benefits the world in unexpected ways. We punish people who use the tool for bad purposes as best we can but we understand that there is no way to prevent any and all misuse, even in authoritarian regimes with no rule of law and "absolute" control. It is an illusion to think we can eliminate all risk and when we try we create bottlenecks and choke points that also unwittingly strangles many of the benefits too. Just because one person stabs someone with a kitchen knife, we do not take kitchen knives off the market because the other 99.99% of people cut vegetables with it. We do not open source nuclear weapons technology because that technology has a single purpose and we would like to limit its destructive capabilities to allies (thought this never really works as well as we would like in the long run because the technology is just too attractive to people and so non-ally nations find a way to replicate it anyway, usually through espionage.) We do not limit general purpose technology under the guise that it might be used by bad people sometimes too. We look to mitigate downsides through sound, sane, clear legislation that punishes that specific bad use case and we let the rest of the world cut vegetables.

About 30 years too late. :)

Remember when I wrote this weekend that people don't understand the unit economics of AI yet and that agents doing constant work and or running 24x7 will not cost the price of a Netflix subscription? That's because they can't. Anthropic and OpenAI lose money on every $200 subscription, let alone $20. It's loss leader pricing at a level we have never seen. About 3000%-4000% off the actual cost of running these models. This poor fellow is experiencing it now. Everyone will soon. The solution many folks present is "clear the context window" aka make the model stupider by giving it amnesia. One developer was optimizing his sleep schedule around Anthropic limits. There is no solution currently. This pricing is not sustainable at all with current technology. The actual answer is that any agent that is not an open source model, running on you own hardware, will likely be out of reach for many folks in the not too distant future without breakthroughs in small model architectures or a complimentary breakthrough in large model architecture/distilling/etc, and that agents will primarily be digital workers for businesses and priced accordingly, aka at roughly human salaries.

Remember the real alignment problem is who controls the AI. Open source fixes this problem. If your AI is not aligned to you, it's aligned to whoever is pulling its strings.

Wrote this short article to say where @StabilityAI stands on 1.5 release. TLDR is that if we don't go a little slower + deal with sound feedback from society, our own ML researchers + regulators then there is a chance open source AI simply won't exist. danieljeffries.substack.com/…

They don't buy them because they are too freaking big for parking, garages, and squeezing into tight places and pretty much everything else in a land with limited space.

The third battle for open source begins...

That's what the rationalist meaning always was, they just coopted the language of rationality. Like an LLM they sound like they know what they're talking about with fancy words but there is no actual reasoning happening under the hood.

Aka, use our censored and controlled models instead!

Very soon AI art generators won't be inspired by any art (already less than 2% of the dataset) + it will still generate art with ease + the whole "stolen remixer" narrative will collapse. Also AI will have no "tells" like crummy hands + it'll be perfectly coherent. Then what?

Except you can't fine tune it, can't replicate it, don't know the recipe, can't improve it, can't put it behind a firewall, have to go through filters, etc. The beauty of R1 is they put the recipe out there and now multiple people can replicate it. That is the power of open source. It accelerates progress. Enterprises are very much going to want models running on their own hardware in their own DC or VPC, especially as computer operators get better, because they won't want to send all their sensitive data (as a model clicks through sensitive spreadsheets or private code) in a round trip to the cloud that's a choke point and soon to be the most juicy target in the world for APTs.

The best papers we're reading on agentic workflows are coming out of Chinese universities and the top open source models are Chinese. My lab is exclusively focused on Molmo at the moment which is based on Qwen 2.5 China is not going to be ahead on open source AI, they ARE ahead. Why? Because in the west we have poisoned the well with nonsense about the end of the world and bogged down companies and researchers debating with fools on AI killing us all. We need to sweep these absurd debates to the side and label them the delusional fantasies that they are and let America get back to doing what it does best: Building the future.

The Mixture of a Million Experts paper is a straight banger. Reduces inference cost and memory usage, scales to millions of experts, oh and just happens to overcome catastrophic forgetting and enable life long learning for the model. Previous MOE models never got past 10k experts and they had a static router to connect them up that was inefficient but this includes a learned router than can handle millions of micro experts. Reminds me a bit of how the neocortex works because it is composed of about 2 million cortical columns that can each learn a model of the world and then work together to form a collective picture of reality. Catastrophic forgetting and continual learning are two of the most important and nasty problems with current architectures and this approach just potentially wiped out both in one shot. There have been other approaches to try to enable continual learning and overcome catastrophic forgetting like bi-level continual learning or progress and compress, that use elastic weight consolidation, knowledge distillation and two models, a big neural net and a small learning net. The small net learns and over time the learnings are passed back to the big net. Its weights are partially frozen and consolidated as the new knowledge is brought in. Good ideas, also out of Deep Mind robotics teams. But this paper seems to say you can just add in new mini experts, freeze or partially freeze old weights, and just grow the model's understanding as much as you want, without causing it to lose what it already knows. It's like having Loras built right into the model itself. arxiv.org/abs/2407.04153

In HEAVEN: * The cops are British * The chefs Italian * The mechanics German * The lovers French * It's all organised by the Swiss   In HELL: * The cops are German * The chefs British * The mechanics French * The lovers Swiss * It's all organised by the Italians

Jaan Tallinn, one of the major sponsors behind many of the organizations behind sb1047, calls for making graphics cards above a certain level "illegal" and for "more and more pervasive surveillance of software" in this interview snippet. It really shouldn't matter what side of the political spectrum you're on when you hear this kind of talk. Outlawing chip development and creating a software surveillance state has no place in any country that values freedom and democracy. Nobody with these kind of belief structures can make a sane, sound policy recommendations. These are horrfying totalitarian values and they should be as odious to you as a disgusting bug crawling on your sandwich.

An atrocious bill where sentators failed to listen to their constitutients, and have singled handedly worked to crush the very source of California's rise to prominence as the center of technical excellence. Imagine if we'd created an arbitrary line in the sand for 4K of RAM or the x386 processor? How absurd will these arbitrary limits look in 2 years? 5? 10? The idea that models get dangerous above a certain training threshold is based on no actual evidence, no actual incidents in the real world (LLMs have a 0.000048% incident rate in 2023, based on a 121 "incidents", one of which was the Pope in a puffer jacket), and no grounding in scientific understanding. It's as if California senators were trying to freeze California at the WinTel dynasty era of compute, enshrining Microsoft and Intel as the once and future kings. Imagine what we would have lost if Linux had been illegal to create? What would run our stock market? Our army? Our supercomputers? The router in your house? The cloud? When Senators listen to doomers who read too much sci-fi, and think computers will grow sentient and take over, instead of the innovators and startups building the future, this is what you get. What California will now get is companies setting up elsewhere, while they watch the next tech revolution from a distant shore, longing for a golden California tech age gone past. Do whatever it takes to kill this innovation killing bill before it gets to the House.

I keep waiting for this real risk to manifest anywhere. Where is it? If gpt-4o was so risky where are all the incidents? Surely we should have some real examples by now. Eventually everyone will realize that the real risk of AI is the same as the printing press to the church in the middle ages: It was a risk to their power, censorship and control over the populace.

I have decided to throw my support behind the Pause campaign. Pause AI Legislation. I'm calling for a six month moratorium on all AI legislation. Until we can make sound, sane, evidence based rules that protect American businesses from catastrophic harms we need to shut it all down.

The main challenge for them is they have a MUCH bigger regulatory target on their back. If their AIs make the same level of mistakes or creates security risks or gives up private data, it's the opening regulators needs to go for the jugular.

I call on the California Assembly to vote no on SB-1047 and Governor Newsom (@GavinNewsom) to veto this pernicious bill if it does pass, or it's likely to destroy California's fantastic history of technological innovation. This is not an AI safety bill. It is a Trojan horse. It's designed to look simple and measured but its real purpose is to give a small, fringe group of anti-AI extremists a kill switch over advanced AI, while throwing a monkey wrench into California's tech industry. The reason is clear. It was designed by a small group of non-profits who believe AI will cause the end of the world. Dan Hendrycks (linkedin.com/in/dan-hendryck…), co-founder of the Center for AI Safety, one of the sponsors of the bill, believes with 80% certainty that AI will end all life on the planet. If you start with that dangerously delusional premise you can't possibly craft a sound, sane, innovation protecting bill. You can only craft a bill designed to dramatically slow down or destroy the growth of AI altogether. California is faced with a clear choice: Either accept the dystopian sci-fi hallucinations of this small fringe movement (thelogic.co/news/special-rep…), with some people in that movement beginning to openly call for violent terrorism against AI labs (nitter.app/Dan_Jeffries1/status/1…), or bet on California and its unique ability to bring together the best and brightest in the tech world to build the next generation of software. It's really as simple as that. Since any bill that starts from the delusional premise of an AI doomsday is fatally flawed by design, I urge lawmakers to start from scratch with a clean, clear bill designed to protect California's prosperity and its world leading tech industry. Consider starting with this draft, SB-1048 (github.com/JeremyNixon/ai-in…), written by pro-California and pro-innovation leaders in the community. Passing the fatally flawed SB-1047 means passing the torch of the future to other states and other countries. This is not a prediction. It's just basic cause and effect. If a model company can choose between two states, one with a high and unreasonable legal burden and one without it, which will they choose as business owners? Governor Newsom has outright said “I don’t want to cede this space to other states or other countries." (courthousenews.com/californi…) That is exactly what the bill will do, Governor, unless you veto it or push back on it swiftly. It's worse than that too. Not only will you cede this territory to other states, you risk ceding it to non-western dictatorships that do not share our values and who are racing ahead with AI. China already has advanced advanced video generation models to rival OpenAI (piped.video/watch?v=73F11cHh…), a self-driving train (piped.video/watch?v=kIh8xWxy…) and robo-taxis (piped.video/watch?v=_ny1uvVT…) in production, Alibaba City Brain (piped.video/watch?v=mX6hzp6O…) deployed to dozens of cities to control traffic and prevent accidents, as well as advanced military AI (cepa.org/article/china-bets-…). We cannot afford to cede the future of AI to anti-democratic regimes. Unfortunately, this bill has continued to move forward, despite the best efforts of the AI business community and more rational voices who have pushed for much needed changes to the bill but been largely ignored. Senator Scott Weiner says he is open to changes (bloomberg.com/news/newslette…). But the changes his team have made are mostly insubstantial. The core of the bill is still utterly broken. When the core is broken it doesn't matter what else you do. Minor changes to the rest of it don't effect the essence. The bill makes model manufacturers certify that nothing bad can ever be done with their models in the future. It holds them responsible for everything that goes wrong forever. This is like making Ford responsible for every drunk driver. Regulate driving. Hold the drunk driver responsible, not the car maker. Anti-AI advocates like to say that opponents of this bill hate any and all legislation but that's utter nonsense they know it. Senator Weiner even said "Silicon Valley doesn't like regulation." No, we don't like this bill in particular. We are all for speed limits on the road. That regulates the user of the technology, the car. We are saying very clearly: Regulate the use case. Don't hold the models responsible for everyone else's crimes. Don't punish Microsoft because someone at Enron used Excel to defraud investors. Punish the cheats at Enron. This is not very complicated. It works in all aspects of law and it has for thousands of years. So how did we get here? Why has this bill gotten so far? One of the worst reasons that I've heard from advocates of premature legislation is this persistent catch phrase: "Look at social media. It's out of control. We should have gotten ahead of it. We've got to get ahead of AI. We've got to do something now." This is broken reasoning. No regulator could have predicted social media's impact on the world three decades ago. When Representatives Cox and Wyden were crafting the section 230 amendment to the Communications Decency Act in 1996, the number one question of many representatives at the time was "What is the Internet?" This idea that we can "get ahead of it" is an insidiously seductive message. We all like to believe in our own ability to perfectly predict the future. The problem is we're all just hopelessly bad at it. And the farther out that we go the more impossible it gets. With all do respect to the Assembly members and Governor Newsom and their respective accomplishments, there is no chance that you or anyone else on the planet can accurately predict what will happen in ten or twenty or thirty years with AI. This is not an indictment of your intelligence or skills in other areas of life. It's just a simple fact. The future is trillions and trillions of variables all changing at the same time. We might be able to predict small, close up events but the farther you get into the future, the more things change. It's an illusion to believe that the Senators and Congresspeople in the 1990s could have foreseen remote teleworking, trillions of dollars in e-commerce transactions, the end of book stores, virtual reality, cryptocurrency, the darknet, massively multiplayer games, and social media for all its goods and ills. In the Wright Brothers era nobody on Earth, in Congress or anywhere else, is capable of writing the safety requirements of the Boeing 787 Dreamliner. You cannot design meaningful safety standards for how a commercial jet will function in 2024 back in 1904 when Wilbur is flying a glorified kite of fabric and wood for a few circles around Kitty Hawk. If we can't predict the future, what do we do? Are we doomed? No. We do what we've always done. We watch and observe and we address concrete, real world problems as they happen in the real world. For instance, where life and death is involved we should hold technology creators to a higher standard. If you are making a self-driving car, which can kill or injure people when things go wrong, you should have a higher standard. The only way to properly regulate is to watch as the complex evolution of society and technology play out over time. You see it develop for good and bad, because it is always both, and you try to reduce the ills and incentivize the good. That is society and law in a nutshell. Punish bad, incentivize good. As a community we feel frontier model companies should be publicly transparent on safety measures and realistic safety efforts like red teaming, which is designed to address actual real world problems with models now, not fantasy sci-fi problems. But we do not agree that model makers, under penalty of perjury, sign a statement saying their model can never be used for anything bad. This is asking for an impossibility. It's asking people to perfectly predict the future and certify that their model can never be used for any ill purpose. Nobody can do this and it is likely already unworkable under existing contract law. You cannot ask someone to sign a document that says they have to flap their arms real fast and fly. If you do ask someone to sign it, they can easily argue in court that it is "impossible to comply" and that provision is now unenforceable. As the California legal team at Stimmel law writes on their blog (stimmel-law.com/en/articles/…): "A party can invoke impossibility and argue that it did not perform its contractual obligations because it was impossible for it to do so." It is absolutely impossible for a model creator to certify that there is no possible way anything can go wrong with a model in the future. As we already noted, nobody can predict the future perfectly. Nobody is an all knowing oracle, including me and anyone else who tries to predict the future regularly. It will be impossible for them to testify to future events that haven't happened yet. If the law does pass, every model manufacturer will likely immediately take the state to court on these grounds and win. It's time for a step back. Please look carefully at this bill and understand its true goals before you make your choice. This bill is designed to look like an AI "safety" bill but its real goal is to make sure that no advanced AI ever develops at all and that of it does, a few self appointed overseers from a fringe movement can kill it. If your goal is to ensure AI doesn't develop in California and America, or other democratic countries, then you're on the right track because the sponsors of the bill wrote it to do just that. But if that's not your goal, then I urge you to go back to the drawing board as quickly as possible. Nobody can predict the future perfectly but we can make one prediction with confidence. If this bill passes, the next technology revolution will be outside of California. If that's what you want, Governor and Assembly members, you're making great strides. If not, there's still time to change course.

Keep the faith man. SI is coming super soon. Just a matter of a little more scaling, right?

The greatest crimes in the world are not committed by people breaking the rules but by people following the rules. It's people who follow orders that drop bombs and massacre villages. - Banksy

I thought AI was only good for destroying all art and the end of the world? Why is it discovering the first new antibiotics in sixty years? Who let this happen!?!?! euronews.com/next/2023/12/31…

What everyone continues to get wrong about LLMs: - They'll make you lazy. Nope. I'm now studying programming again so I understand better what I'm seeing. - They destroy the desire to learn anything. Nope again. See above. I'm learning more, faster now. - They make junior folks as good as senior folks. Nope. People with expertise in an area benefit best. As a professional writer I can definitely see the difference between that and my programming. When I'm lost in programming I often get stuck "hitting and hoping" the LLM will fix a problem because I don't fully understand it (though I am getting better little by little). With writing I can spot the errors and rewrite it immediately because I already know where it screwed up on flow, style, word choice, idea, structure and more. And best of all, with LLMs, you can start working on something right away instead of spending days, weeks or months learning about it before you even get off the ground. You can get started immediately and learn as you go! Trial and error learning was always the best learning and it's more true now than it ever was in the past.

"The generation that has experienced more peace, freedom, leisure time, education, medicine, travel, movies, mobile phones and massages than any generation in history is lapping up gloom at every opportunity." - Matt Ridley, The Rational Optimist

LLMs ain't got nothing on this yet.

At the end of 2017 I couldn't find a real bear on #crypto anywhere. Midway through 2018 I can't find a single trader who is bullish. What does that tell you?

#Libra is a cyberpunk nightmare ripped from the pages of @GreatDismal and the screens of Mr. Robot. Don't worry, Citizen, E-Corp is on your side and will keep your private keys safe for you! hackernoon.com/libra-a-cyber…

10^26 FLOPs is a made up line in the sand, created by people whose express purpose is to kill advanced AI development. It does not convey any magical new abilities to models. In fact models are plateauing at the same levels. You have multiple models now, all trained by different teams, with almost exactly equivalent performance, none able to really break ahead in any meaningful way. OpenAI had a massive lead but only because they were the first to do it. Now other teams have caught up or surpassed them. We have models at these levels already and 300M people are using them (conservatively, probably much more) and nothing has happened. Where is the big emergency? We had a whopping 163 "incidents" last year, one of which was the international crisis of the Pope in a puffer jacket. There is no emergency. It's just red tape made up by people who watched too many movies growing up and now think Terminator is real. It serves nothing and makes no one safer. It has nothing to do with AI safety. It's nonsense, no better than putting your finger to the wind and saying "I think it's going to rain."

They're super sneaky! You gotta watch out or pretty soon they'll be swimming over here and taking all our women too!

"There is nothing noble in being superior to your fellow man; true nobility is being superior to your former self." -Hemingway

Hinton's predictions: Neural networks will be really really important long before most other people thought so: 100% correct. Societal level predictions on AI and its impact: 0% (Remember radiologists with be replaced by AI in 5 years (10 years ago)? We often conflate "good at XYZ" with "good at predicting impact of XYZ on the world". They are not even close to the same thing. We should stop pretending that they are and stop asking people to predict the future because they are terrible at it almost across the board.

Just a reminder that every single AI bill or proposed bill on the planet carves out 100% exceptions for military and surveillance (the actual dangerous use cases) and just regulates civilian AI for the rest of us.

There is only one way to fix the California anti-AI bill: Shift the burden of a crime back to the person or people committing a crime. What this bill effectively does is put model makers and trainers on the hook for life, facing fines and even jail time, for any possible misuse of their tool no matter when it happens. If they fail to predict some use case twenty years from now they could face major trouble. What if every car manufacturer was responsible for everyone who got drunk and plowed into a bus of school children? Imagine if Adobe was responsible for everyone who created revenge porn with it or for failing to predict this use case a few decades later? You can't make Photoshop useful for creating ads, posters and art and also remove the ability for people to misuse it. What if every kitchen knife manufacturer was responsible for everyone who went crazy and stabbed someone with it even though 99.9999% of everyone else cuts vegetables with it? You cannot remove the cutting edge of the kitchen knife and still make it useful for cutting vegetables. We need to let people cut vegetables. What if the Linux developers were responsible for it getting used in botnets, despite it also getting used to run the New York Stock Exchange, US army laptops and radar systems, every major cloud and supercomputer in the US, the router in your house and your phone? The benefits far, far outweigh the downsides of that technology and the same is true of AI. The emphasis has to shift back to the people or groups committing the crime or the harm to make this work. What this bill effectively does is force every model maker, closed or open, to have perfect foresight for every way a person might misuse their tool for life. Why would any AI company ever set up in California again with this kind of burden hanging over them forever? The language in the bill, such as the training threshold, comes from a group whose stated goal is to stop artificial general intelligence from ever emerging at all and to prevent anyone every developing it. Since they know this goal would be rejected by every wise senator and business person they wrap it up as AI safety. Worrying about AI rising up and taking over the world is like worrying about whether fairies will invade California orange groves and destroy next years crop. It's not real. Real safety is putting a higher standard on use cases that involve real world life and death, like when AI is driving a car or flying a plane. There can and should be a higher burden of proof there. We are also not talking about AI getting used in mass surveillance or for autonomous weapons, which are real dangerous use cases, not because of sentient machines, but because of the age old problem of humans misusing tools. These use cases get a free pass under every bill proposed everywhere and they should not. Lastly, the training threshold is an arbitrary number that will grow increasingly problematic as the years go by. What happens when we have this kind of power in our servers or desktops in a decade or two? Every day that goes by will put this power in more and more hands and create more and more of a burden. Imagine if you'd fixed the threshold for desktop computers at the x386 Intel processor or 4K of RAM in the 1980/90s. Shift the burden back the person doing the crime. Shift the burden to the use case. Hold people accountable for their crimes, like we do with everything else. Hold the drunk driver responsible for getting drunk and killing someone, not Ford. If you put this kind of burden on model trainers and force them to become seers or have a magical kill switch for everything that goes wrong, you make it impossible for any AI company to release tech in California or to set up shop there. This is not good for the California economy or American innovation.

This article is brilliant. A real eye opener into the minds of people who think AI will kill us all. Some of my favorite quotes with my comments in bold after: "Grace, the lead researcher at a nonprofit called A.I. Impacts, describes her job as 'thinking about whether A.I. will destroy the world.' DJ: That is not a job. The fact that anyone is paid for this is a testament to capitalism, democracy and free enterprise. Only capitalism can support and even reward its own antagonists, unlike in a dictatorship where they are systematically eliminated. Amazing system. You truly can do whatever you want and get paid for it! "Grace started a philosophy Ph.D. program, but later dropped out and lived in a series of group houses in the Bay Area...Some people gave away their savings, assuming that, within a few years, money would be useless or everyone on Earth would be dead. Others signed up to be cryogenically frozen, hoping that their minds could be uploaded into immortal digital beings." DJ: Only people in cults give away all their money because they think the end of the world is coming. There is simply no other way to interpret this kind of idiocy. There is nothing 'rational' about this kind of thinking at all. "Philosophers of doom tend to get hung up on elaborate sci-fi-inflected hypotheticals. Grace introduced me to Joe Carlsmith, an Oxford-trained philosopher who had just published a paper about “scheming AIs” that might convince their human handlers they’re safe, then proceed to take over. He smiled bashfully as he expounded on a thought experiment in which a hypothetical person is forced to stack bricks in a desert for a million years. 'This can be a lot, I realize,' he said." DJ: Ahhhh. A person stacking bricks for a million years in the desert. At least we're preparing for the most important possibilities here! "These days, Yudkowsky uses every available outlet, from a six-minute _TED_ talk to several four-hour podcasts, to explain, brusquely and methodically, why we’re all going to die...In early 2023, he posed for a selfie with Sam Altman...“Eliezer has IMO done more to accelerate AGI than anyone else,” Altman later posted. “It is possible at some point he will deserve the nobel peace prize for this.” "Opinion was divided as to whether Altman was sincerely complimenting Yudkowsky or trolling him, given that accelerating A.G.I. is, by Yudkowsky’s lights, the worst thing a person can possibly do. DJ: Nobody is divided on whether he was trolling him. "The doomer scene may or may not be a delusional bubble—we’ll find out in a few years—but it’s certainly a small world. Everyone is hopelessly mixed up in everyone else’s life, which would be messy but basically unremarkable if not for the colossal sums of money involved. Anthropic received a half-billion-dollar investment from the cryptocurrency magnate Sam Bankman-Fried in 2022, shortly before he was arrested on fraud charges." "Open Philanthropy, a foundation distributing the fortune of the Facebook co-founder Dustin Moskovitz, has funded nearly every A.I.-safety initiative; it also gave thirty million dollars to OpenAI in 2017, and got two board seats. (At the time, the head of Open Philanthropy was living with Christiano, employing Christiano’s future wife, and engaged to Daniela Amodei, an OpenAI employee who later co-founded Anthropic.) “It’s an absolute clusterfuck,” an employee at an organization funded by Open Philanthropy told me. I brought up once what their conflict-of-interest policy was, and they just laughed.” DJ: Yeah, none of this is fucking funny at all. In his talk, Nielsen told a story about a house party where he’d met “a senior person at a well-known A.I. startup” whose p(doom) was fifty per cent. If you truly believe that A.I. has a coin-toss probability of killing you and everyone you love, Nielsen asked, then how can you continue to build it? The person’s response was “In the meantime, I get to have a nice house and car.” DJ: Well at least he's got clear morals. I don't support war and genocide but the army pays nice! What is immensely clear from this article is you have a cult of people, with too much time on their hands, who spend time thinking about absolute nonsense as part of their identity and status seeking. Even worse they seem to have money to throw at this and have the chance to negatively impact regulations and crush innovation if politicians are duped into thinking they're credible and not just people who watch too much Terminator as a kid and have trouble separating fantasy from reality. They must be stopped at all costs. Everyone is free to think whatever crazy nonsense they want in life. But they are not free to make the rules for the rest of us.

There is an idiotic idea, pushed by AI safety folks, that if we restrict open source and centralize control of foundation models in the hands of a few private companies using security through obscurity, then nobody will be able to advance AI or replicate their breakthroughs anywhere else. Wrong. Dead fucking wrong and stupid. The ideas that power ML are well known, open and easy to see. All you need is will, money and time. Hell, not even that much time. o1 was replicated two months later. The ideas are scientific and mathematical. They are shared openly and they are obvious through direct observation and reverse engineering too. If we stay the course and replicate the foolish overregulation of Europe, the result is as deterministic and obvious as the sun rising in the morning and setting in the evening. We're on a collision course to second rate power status because we imagine that it's our God given right to be number one and we can make any idiotic legislation we want and it will just magically work out in our favor. It will not.

While we've worked hard to cripple open source AI development in the US and Europe, because lawmakers were listening to idiots, Chinese labs have been roaring along and now managed to release an open source reasoning model on par with o1. While US firms hide their latest research away for "safety" (aka commercial reasons) this small lab has replicated the thinking capabilities of o1 with 140 researchers despite being GPU poor due to increasingly absurd sanctions. Our policies on AI are a disaster. It's time to stop listened to a bunch of fools trying to astroturf public opinion with tiny one and two person non-profits funded by the same three people. While these folks spam Time and other mass media, blathering about "systemic risk", the world keeps right on turning just fine despite hype and doom proclamations and the AI risk database released by MIT students is only noteworthy because nothing actually happened in the real world with any of these "risks". When are we going to wake up and realize we've been lost in a hallucination? It's time to start building again.

While non-profits funded by end of the world cults are out there trying to kill AI in its crib, AI is out there saving lives already. bbc.com/news/technology-6860…

Our Applied AI team at Kentauros (kentauros.ai/) spends a lot of time tackling the problem of GUI navigation for agents. Today, we are releasing our first dataset, WaveUI-25k, and our first fine-tuned vision-language model: huggingface.co/collections/a… Why bother about GUI Navigation? Won’t AI just use APIs or have a clear machine-learning interface? Yes, but APIs aren’t always available, and it will take many years for ML APIs to become a reality, so we’re meeting the world where it is today. It’s also because UIs are uniquely human, much like language. They’re a high level abstraction of the world and that makes them a fantastic training ground for advanced AI development because they’re complex, challenging, ever changing, and filled with endless edge cases. They’re a great approximation for AI that uses real-world models, planning, prediction and adaptation on the fly. Next week we’re releasing a new agent that navigates GUIs very well with a combination of heuristics, classical computer vision and MLLMs. Although we’re super excited about our Gen 2 agent release there are still some limitations. Rather than bolting an expert system onto a ML model, it would be absolutely ideal if models were good at knowing where and what to click on a UI. Unfortunately, we’ve discovered that today’s frontier models like GPT-4o, Claude Opus 3, Sonnet 3.5, Qwen2, and InternVL are absolutely terrible at three key things on UIs: 1) Returning coordinates 2) Moving the mouse 3) Drawing bounding boxes on GUIs The reasons shouldn't be surprising. They just weren't trained to do any of these things. That’s why, for our Gen 3 agent, we’re already working on a click model as the primary navigator, and we’ll fall back to classical techniques as needed. Our WaveUI-35k dataset is 25K well labeled images that you can use to train a click and coordinate model. We’re currently auto-labeling a larger version of the dataset that is roughly 80K images and we’ll release that soon. Why do you want this? Well, maybe you want a model that can navigate smartphones or desktops. Or maybe you want to control remote desktops the way our AgentSea platform can navigate a virtual machine like a human navigating TeamViewer but with the Agent in charge, the way we do with our ToolFuse protocol (github.com/agentsea/toolfuse) and the AgentDesk (github.com/agentsea/agentdes…) Python packages. We built this dataset by collecting examples from other existing datasets. The three main sources are: - WebUI (uimodeling.github.io/) - RoboFlow Website Screenshots (universe.roboflow.com/robofl…) - GroundUI-18K (huggingface.co/datasets/agen…) The datasets are great and we have tremendous respect for the teams that gathered them but unfortunately many of the labels were less than ideal or just wrong. So we set about fixing these fantastic resources into a new, unified, well labeled dataset. We’ve preprocessed them to have a matching format and programmatically filtered out unwanted examples, such as duplicated, overlapping or low-quality elements. The resulting pre-processed dataset has ~80k examples at the object/element level -- i.e. each row consists of the bounding box coordinates of a UI object, along with the screenshot of underlying UI (and extra info, like source, screen size, etc). We then took ~25k examples from the dataset and annotated them to get the following additional info, per example: - name: A descriptive name of the UI element. - description: A long, detailed description of the element - type: The type of the UI element - OCR: OCR of the UI element. Set to null if no text is available - language: The language of the OCR text, if available. Set to null if no text is available - purpose: A general purpose of the element - expectation: An expectation of what will happen when you click this element You can explore the dataset at huggingface.co/spaces/agents… We’ve also fine-tuned a very small PaliGemma model as a proof of concept. You can check it out here: huggingface.co/spaces/agents… You can upload a screenshot and use the phrase “detect” XYZ. For example, on a screenshot of Google Flights: * detect the “Return” field * detect the Vacation rentals button * detect the flying plane in the picture To be clear, the PaliGemma model is too small and has too low a resolution to be state of the art. It’s a little too rigid in how it detects things and we want it to be more flexible. It sometimes nails detection tasks and it’s sometimes just wildly off base. But it is proof that a properly trained model with a well-annotated dataset can make headway on the gnarly problem of navigating GUIs on the wide and wonderful world of the web. Out next steps are simple: * Get the full dataset annotated and release it in the next few weeks (we only used 25k out of ~85k potential examples) * Training the variant of PaliGemma that has higher resolution. * Training a larger multimodal model like InternVL or Chameleon We’ll see you next week when we release G2 agents! See you then.

Remember, nobody asked permission from anyone to open source Linux or any other open source project. Yes Linux is used for some bad things but the good things it's used for far outweigh the bad, by a massive margin. You can't have "kind of" open and get the same results. The idea that we need some kinds of restrictions on this or that some government bureaucracy will sit in judgement of what can be open sourced in anti-American and anti-innovation.

I thought this post was satire when I read it. I use Claude Code as much as 8 hours a day and had to create a custom command in Flashbacker called hallucination-hunter that essentially says "go find all the fake code you implemented over the last few hours that does not do anything, pretends to do things, tries to do magical things, etc." I have to use it for both Claude 4 Sonnet and Opus and GPT-5 and Qwen-Coder. Just because people aren't talking about it doesn't not mean it's a solve problem. A benchmaxed model that purports to solve hallucinations on info/facts does not mean it solves it on actual real world problem implementations. People moved on from talking about it because people get bored easily and like to talk about the next shiny, scary thing to keep them angry/afraid/worried.

This is how you deal with AI risk: You punish folks who commit crimes when they commit crimes like we do with everything else. You don't ask model makers to be on the hook for any and all crimes other people may commit for life (like making Ford responsible for every drunk driver) and you don't ask model makers to have a magical kill switch or stop all crime before it happens like they're the pre-crime unit in Minority Report. You don't take the kitchen knife off the market because one person misuses it stab someone. You let the other 99.9999% of people cut vegetables with it and you punish the person who misused it. This is how society works and works well. We set it up so the benefits outweigh risks and we build society to benefit the vast majority who use technology for good and we punish misuse and exploitation. Simple. Easy. Effective. Lawmakers, please get this right. It's not complicated. apnews.com/article/biden-rob…

This is exactly what has not happened and it won't. The reason is that skilled people wield the tools better. Always have and always will. Shit coders produce shit code with AI. But great coders can now produce better, cleaner, more organized, more tested and more documented code faster. Someone with no artistic skills can produce something fast on Canva and it is often good enough but it has not changed the price of high end commercial and ad work and website work because the best companies want to stand out and because the tools are not magic. If you don't know whether something looks good and pleasing to the eye, because you never developed that skill, then you can't verify the output of the AI so it doesn't matter. You can't tell if it is award winning or absolute garbage that no one will respond to. These kinds of predictions fail over and over because reality is always more complex than changing a few variables and leaving all the other variables unchanged.

This talking head is dead wrong. The best open source models are currently Chinese, not western. Qwen 2.5 is the best open source model on the planet and has been for some time. Deepseek is innovating with new techniques that they didn't copy from anywhere. EngineAI has humanoid robots that walk as smoothly as people. Unitree robot dogs can dance on one paw while someone hits them with a bat and they don't fall over. China does not need western models. Math will not be contained. The techniques that make for great models are not some secret that only a few people know. They are widespread and easily accessible. The more legislators listen to these kind of idiotic, bobble head political takes, the more we will cede the hard and soft power of open source to places that don't share our values. The west made the open source movement and now we're ceding its power through our own stupidity and ignorance and because we listen to folks who do not understand the trillions in economic value open source has meant for the US, Europe and the world, not to mention the influence over development and standards. The cloud runs on western open source, the router in your house, your phone, every super computer on the planet, this website you're reading this on, your ham radio, robots, and much much more. You do not get to have kind-of-open software and get the same effects. Open means open. Closed would never have gotten us the cloud. Closed would not have gotten our supercomputers to the levels they are now. If the west doesn't make the best open source models then someone else will and we are the less for it.

The US Department of Commerce opened public comments on open weights AI models. It's important for everyone with sound, sane, clear reasoning to speak up and get your thoughts in before they're bombarded with absurd policy suggestions from fear mongers. commerce.gov/news/press-rele…

It's not assumptions it's looking at the evidence. I am fine with a much longer time horizon for it to develop and think that is realistic while SI in 2026 is just straight marketing BS. Over a long enough time horizon we get superintelligence but will it take different architecture and chips and algos and such? Almost certainly.

How exactly do they "collect data" on a model you are running locally? And what does "disclosing training data" have to do with anything at all in what I wrote?

In this comprehensive analysis of human reasoning, researchers ask the critical question: Can humans reason at all? And come to some worrying conclusions.

Finally get to share what my new startup is working on: Announcing the AgentSea platform. - SurfKit: Think of it as Kubernetes for Agents. Launch agents locally, in a container or in the cloud. - DeviceBay: Attach a device to the agent. First device: a complete virtual Linux desktop that the agent controls the way you control Teamviewer, sending mouse clicks and commands. Can run in QEMU, GCP or AWS. - SurfPizza/SurfSlicer: Our alpha agents are pure, multimodal agents that use old school machine vision + LLMs to navigate desktops and GUIs. No Playwright. Pure robot vision here. - ToolFuse: A protocol that defines, limits and controls how a tool gets used by the agent. - ThreadMem: Memory for agents. - Taskara: Task management for agents. That’s just the tip of the iceberg. More coming very fast and very soon. We’re releasing two new alpha agents that are much stronger at navigating GUIs over the next few weeks, SurfMonsta and SurfNinja. Notes and Caveats: - We do not believe in AI hype and promising more than we can deliver. The infrastructure components (surfkit, toolfuse, etc) are strong beta software. The agents are alpha as all hell. They are not fast (yet), and they sometimes make mistakes and do stupid things. - We cherry-picked the runs in the video. That said, it’s still pretty damn magical when it’s working. They will only get better from here. - The agents can be expensive. They eat tokens like Pac Man. That will go down over time, especially as people leverage open source, local MLLMs. - Optimizing, fine-tuning and just good old-fashioned creative thinking will get us to the next level soon and you can help by creating your own agents that build on our ideas and your own insights. - We have a LOT more tricks up our sleeves but we wanted to get this stuff out there now. - The big studios are working on the Promethean fire of multimodal GUI nav agents too but we guarantee they’re going to keep it all to themselves. We don't want to live in that world. Open source made the modern world and it will make the next iteration too, no matter what the Doomers think. We want you to run your own agents that you control. Privacy and control. Power to the people. We can’t wait to see what you can do with this when you get your hands on it. Let’s go!

The most powerful open weights model is now made in China, Tencent's massive new MOE model. Meanwhile in the US and Europe we continue to try to destroy open source by creating a permission based model where some bureaucracy can decide what can and can't be open sourced from on high. Nobody sought permission to open source Linux or any other open source project. Open source added trillions to the economy because it was organic and decentralized and because you didn't have to beg anyone for permission. Open source and open weights AI must be the same or the next wave of AI will be from countries that understand maintaining a lead isn't a given. And that next-gen open source leader just might be a country that doesn't share your values and is actively hostile to them because we were too stupid and too privileged to understand that continued influence on the world stage is earned through continued innovation and not just a birth right because we owned the first wave of open source innovation. ("Would you open source a nuke?" responses are an insta-block. If you're going to make a case try actually making one instead of regurgitating other people's thoughts.) arxiv.org/abs/2411.02265?utm…

"China is 2-3 years behind." Meanwhile in China. Watch this video for five mins and see if you still believe that. Anyone who tells you the west can shoot itself in the foot with insane legislation written by people who worry about AI turning us into paperclips is living in a dream world that will virtually guarantee we fall further and further behind. This is how empires fall. Not by revolution but by our own people crying out to be censored and to slow down progress. We've lived in a bubble. Everyone alive today in the west has lived their whole lives with so much peace, freedom and prosperity that we just take it for granted and think it's a given that it will always be that way. It's going to be a hard lesson for us if we don't wake up fast. piped.video/-4ZgvwQ6YGU?si=Sgz3…