A post on my decade long journey with Infrastructure-as-Code, why I built my own minimal library in pure TypeScript and why I think you should at least entertain the idea of "un-bundling IaC" and returning to simplicity. I built alchemy after years of working with every other option, from CloudFormation, CDK, to Pulumi, Terraform and Kubernetes. IaC is non-negotiable in my opinion, and is one of my favorite technologies as a developer. I started with CloudFormation since I worked at Amazon and really hated that. JSON is just not expressive enough for me and debugging broken stacks was was always something I dreaded. I'm pretty sure the CFN website still squashes the whole error message into a 10 pixel wide box 😅. The CDK is an upgrade on that (because yay for TypeScript) but it is still always going to be limited by the CloudFormation service, which is very slow to change and use. Being developed by corporate giant Amazon and with the founder moved on to new ideas, you just have to accept that things will move more slowly and you have no say or control over that. It's how things are in a big centralized company. One mega wart with the CDK technically is its awful coupling to synchronous I/O, making it damn near impossible to do anything async. Every user inevitably runs into this, googles it and is met with "fuck off" in the GitHub issues. Then we're forced to hack around it, which is not pretty. Not allowing is async is basically not allowing JavaScript. I doubt it'll ever be fixed. They're also stuck on CJS. Taking a dependency on the CDK and using it anywhere other than locally will 10x the size of your app and destroy your DX. It's a tangled mess of of complicated, legacy TypeScript code generating complicated, proprietary JSON files uploaded to the complicated, opaque, proprietary CloudFormation service 🫠 Why? How does this serve the user? Later, I moved on to Pulumi, which was a nice change of pace since it runs locally and is much faster. You can cancel a deployment by hitting Ctrl+C (hallelujah!) and it also supports more than just AWS, which is great because everything should be managed sensibly in code. However, Pulumi is largely a wrapper around Terraform. If you thought you were going to escape the layers, you were sorely mistaken. That JavaScript is just lipstick on the pig that are clunky "providers" implemented with Go, running in a separate process. You can't just run Pulumi everywhere (like the browser) and it even struggles to run in AWS Lambda if you're using ESM. Pulumi Custom Resources are possible but more of an afterthought and a PITA to implement. They have many sharp edges and gotchas, like the resource code having to be serialized and their coupling to CJS - so if you import the wrong thing with ESM, it just breaks. I couldn't even use AWS SDK v3 in a simple custom resource because of how leaky these hacks are. If Pulumi bricks itself by getting into some weird state, good luck fixing it. Their state files are complicated and their providers are opaque, so you're really just stuck running `pulumi refresh` and praying, or surgically removing and restoring resources one by one. It's not transparent. I've used Terraform when I've been forced into it. It does exactly what it claims to do, but I don't love it because it's a custom DSL and a heavy toolchain. And for what? Calling a few CRUD APIs? Way overkill. If it doesn't already exist in Terraform, then just forget it. Every time I think about implementing a custom resource for Terraform, I just can't bring myself to do it. Let me just write a function in my language, please! Lately, I've been using SST because I've been doing a ton of web development. At first, I really liked SST because of its local development experience. `sst dev` gives you live deploy, a TUI multiplexer and a proxy from the cloud to your local code. This is great for building web apps in AWS. But, SST is a wrapper around Pulumi!! (which is a wrapper around terraform (which is a wrapper around the underlying SDK ( ... ( ... ( ... )))) ... Layers. Layers. Layers. When will it ever end? As my app grew, SST's bugs and opinions ate away at me. I got blocked by broken resources that have race conditions and it was impossible to work around (still is, by the way). I also wanted to deploy a nested app using another `sst.config.ts` but it conflicted with their assumptions around generated sst-env.d.ts files. Again, I was let down by the complexity and opinions(!) of my chosen IaC framework. And for what? To help me call a few CRUD APIs and track my Resources? Honestly, it just seems insane how far we've drifted away from simplicity in this area. This became even more apparent as I started using Cursor more and more to write code. You see, I've found myself doing more frontend than I've ever done before, and I'm not a good frontend developer. So, I relied on Cursor to write most of the code for me and (as many others have experienced) it totally blew my mind 🤯 Cursor is just really, really, really good at TypeScript, React and Tailwind. I've built a functioning and (if i don't say so myself) good looking SPA. It's been a blast. As a "backend guy", i feel my world has opened up. But, this got me thinking ... you know what else Cursor is really great at? Perhaps even better at? 👉 Interacting with CRUD APIs. While there's a ton of frontend training data for LLMs, there's just as much (if not more) training data for CRUD lifecycle operations. They've also ingested all of Kubernetes, Terraform, Pulumi, SST and the AWS CDK's training data. Modern LLMs know it very well. Long story short, I discovered that Cursor can pretty much one-shot the implementation of Resources. All of the resources in Alchemy are entirely generated on-demand (5 minute time investment, tops). When I run into a bug, I just explain it to Cursor who fixes it immediately. Working this way may seem like more work at first glance, but in practice I think it's not. I will never get blocked by working this way. I will never have to wait for another person to prioritize my problem, merge a fix and release the change. I will never be confused about what's actually happening under the covers. This is a game changer for me. It means we don't need tools like Terraform to build layers of "provider" suites for us. We just need the engine - the bit that tracks state and decides what to create/update/delete. The rest can be generated at near zero cost. I suspect a lot of people gripe with me on this claim of "zero cost" because they have not yet embraced the AI-generation workflow. Their instinct is right, but outdated. Before LLMs, I would agree that this was infeasible, but I think we've finally crossed the threshold where LLMs can do this work trivially and respond in real-time to your requirements. If this is not obvious to you, then I suggest forcing yourself to practice with AI code generation. When it fucks up, try blaming yourself instead of blaming and discarding the LLM. I think you're missing a skill. This is why adoption curves are a thing, do you want to be a laggard? At the end of the day, you need to realize that agency is going to matter more and more as time progresses and that shackling yourself to a mega-provider who doesn't even know your name is a ticket to inefficiency. I think we've mostly been hoodwinked into thinking that because managing infrastructure is complicated, we need complicated solutions. Now is the time for that pendulum to swing back in the other direction and "un-bundle" IaC.

Cloudflare replaces so many AWS services like API Gateway, Step Functions, Event Bridge with `fetch`. No more proprietary JSON DSL bullshit. Literally, fetch is all you need.

Cloudflare Worker + Queue + Queue Consumer in 10 lines of code. Did I mention it's type-safe?

Cloudflare Containers is built on Firecracker by the way.

Possible supply chain attack happening in the web ecosystem right now all under the guise of supporting node 0.4. Does this GitHub action allow ljharb to inject back doors? Replacing setup-node is very suspicious behavior.

Why do we call kubernetes cloud native? In what way is that native to the cloud? It seems more decoupled from the cloud since it uses only the most primitive abstractions. To me, the cloud is all about depending on other people for operations. Kubernetes is not that.

I'm excited (and grateful) to announce that I've raised $1.06m from @CrucibleCap and angels to fund the development of @alchemy_run. To commemorate this moment, I've also moved the Alchemy repo into its own GitHub organization. Cheers 🍻 github.com/alchemy-run/alche…

I’m only starting to learn kubernetes and might eat my words, but I dunno, doesn’t seem THAT complex. Feels like everything I’ve ever wanted and extremely mature. Why all the fear mongering?

Another perfect example of @Cloudflare reducing a whole AWS service to `fetch`. Load balancing requests across containers with a simple rand() and fetch() 🤯

I re-wrote itty-aws to go all-in on @EffectTS_ It uses a Proxy and types-only, so the entire AWS SDK fits in 34KB. Every AWS API's errors modeled perfectly with TaggedError.

Today I became a Dad! 👨‍🍼

As promised, I'm open sourcing my new project, Functionless - a TypeScript plugin for creating service-to-service (aka. "functionless") integrations such as AWS AppSync Resolvers with beautiful TypeScript syntax. github.com/sam-goodwin/funct…

Why am I leaning into @EffectTS_ ? Because things can go wrong in more ways than right. For example, PutItem has a >10:1 ratio:

Cloudflare's generic API interface is so ugly. HTTP status codes and a `success: boolean` flag? Hides the actual data behind `result` envelope? Why do any of this?

After a month in trade-off purgatory, I'm opening up alchemy-effect 🧪 Still need to build the Resources, but the core functionality is ready: 1. Resources and Bindings 2. Type-checked IAM Policies 3. Plan & Deploy Excited to share more this week. github.com/alchemy-run/alche…

Look at these type safe errors for AWS 😍

D1 over HTTP is 2-4x slower than DO+SQLite3. Here's a test suite ran on different stores: local fs, sqllite3, DO+Sqlite, D1. SQLite3 is blazing 🔥, D1 is a snail 🐌

You can’t assess whether Kubernetes is a poor choice because of “high operational burden” without understanding the problem we are solving. Problem: Big data engineering for cancer tissue analysis and drug discovery via machine learning. You need a system that can process massive scale cancer tissue samples and train machine learning models. A single sample can be well over 100Gb in size, and we have thousands of them. It's clunky, heavy and complex data. Handling this is a multi step process where each step has a wildly different shape. Some need CPU, others need a GPU. Some are low memory, others are high memory. Some are long running and others are short. The language you're stuck with is Python because all of the ML, data science and bio libraries are built with it and the scientists only know Python. As we all know, Python's ecosystem is plagued by dependency hell. One step might require some bizarre library pinned to an old version of Python or a weird native library. Juggling this is hard. Data scientists are, well, scientists. They don't know the right solution and have to explore. It takes many, many tries, so iteration speed is everything. How to give fast feedback on the whole data set (which is massive)? How to handle the resource and dependency complexities? We originally tried EMR. On paper it’s “managed” and we have a tenet to offload operational responsibility (as a good cloud engineer should), but in practice, it is even more complex and feels like a rotting service. Built for a different world where distributed SQL was hard and Spark was king. Good for Spark and SQL, but bad for the heterogenous python environments of the modern ML and AI world. Ok, wrong turn. What we actually need is a container orchestration platform. Data science depends on containers - lesson learned. That means ECS or EKS. If we follow this "pick the most managed service" line of thinking, then ECS should be chosen over EKS because it has “less responsibility”. Problem is that every data engineering framework on the planet is built primarily for Kubernetes, with ECS and EC2 as an afterthought. They don’t work well on those platforms. They especially don't integrate well with each other, if at all. This is what real operational burden feels like. Now I'm responsible for figuring out how to run services on a platform the creators of those projects care less about. Sure, AWS is managing the containers for me, but there is high friction integrating and maintaining different tools. EKS is the right solution. AWS provides a managed control plane with configurable plugins for the shit that was hard 6 years ago. And, now we can just start dropping in the helm charts for Ray, Dagster, Coder, Nessie, etc. Everything follows the Operator Pattern and self-manages for the most part. We only care about ephemeral compute so it doesn't even matter if the services die, we can just "turn it on and off again". All our state is stored off-Kube in S3, RDS, DynamoDb, wherever makes most sense. We even moved our developer environments into it and gave each developer a personal cluster that spins up and down as they log in and out. It's magic, thanks to Coder, Terraform and K8. All together, Kubernetes shaves literal minutes and hours of each development cycle and opens us up to a world of high quality tools and services. Kubernetes has reduced our operational burden, not increased it. Nothing in engineering is ever as simple as “this good” and “that bad”.

Extremely small instances to start.

Working at amazon for 9 years atrophied my muscle of diplomacy and negotiation. Every reasonable argument was trumped by OP1 planning and “calls to authority”. Previous decisions trump all reason. I learned that thinking big was actually not valued. So glad I left Amazon.

I don't understand how this is "Private by design". I'm already spooked by the idea that people I'm meeting with are secretly recording my voice and OCR-scraping any information i screen share. We are entering black mirror era.

It’s classic gaslighting. An appeal to a social value that has no relevance to the issue at hand. Designed to dodge the question he has no answer for. If Putin reneged, there’s no answer. All they can say is “we have no other choice than to try”.

Cloudflare's vision really starting to shine through in this example. 1. Ark type for validating the request body 2. Send to the queue (validated by types) 3. Consume in the `queue` callback `bun ./index.ts` runs it in Region: Earth 🌎

The irony is that the private healthcare is low quality and expensive.

Getting deeper @EffectTS_'s and am really liking Context for type-safe dependency injection. Pairs nicely with @alchemy_run's inferred Worker environment for building multi-worker apps:

For comparison: @alchemy_run vs Cloudflare on Terraform.

This is all it takes to deploy @livestoredev to Cloudflare with @alchemy_run

AWS going hard on generative AI queries in the console! “Show me all non-compliant S3 buckets in my organization”. aws.amazon.com/about-aws/wha…

So excited for the upcoming @_functionless Constructs for #sqs Queues, #sns Topics and #kinesis Streams, all pure #aws #serverless Has the cloud ever been so simple? Here's an entire Dynamo Table, SQS Queue, Express Step Function and a Lambda Function in 35 (!) lines of code

CloudFlare’s API has been by far the hardest to figure out. I now have more respect for AWS’s consistent design company wide. Cf has undocumented APIs. Weird order of operations. Some things are resources, some are not.

Preventative care is the most effective form of healthcare

Notion still doesn’t have find/replace by the way.

Are you getting it yet? IaC-like control over every piece of your codebase. This here sets up Vite, Tanstack, Tailwind, Shadcn, installs the shadcn components and deploys straight to CloudFlare.

Effect or die 💀

Wow, super disappointing! @discord blocks @Cloudflare Workers from connecting to its Websocket Gateway because ... "security". Totally limits our ability to build interactive bots on CF.

CloudWatch Logs is too expensive.

I walk a lot. Like 2 hours a day. It is how I do my thinking. Claude Code via GitHub actions has now become my most valuable tool during this time. I can literally get work done remotely while walking.

Cloudflare Workflows are now available in Alchemy!

Mineflare uses Bun to build the SPA instead of Vite which I did not know was a thing. @EastlondonDev added a BunSPA Resource into @alchemy_run if you want to give it a try yourself.

Alchemy joined the 1 comma club today 🤗

The future of software engineering will be more about problem solving than the labor of writing code. Here, I describe a backend service using a Markdown file and then generate a serverless backend on AWS with support for APIs, Events and Workflows.

I am pretty stoked at how easy retry logic is with @EffectTS_ Schedule.tapOutput to send a note on each failed retry is so fetch 👌

You can now deploy TanStack Start to Cloudflare Workers+Assets with Alchemy.

Alchemy's tests now run in 100s instead of 1-1.5 hours. Fast take-off is imminent 🚀

In the next version of @alchemy_run, you no longer need to `await` resources (but still can) and it just works in @EffectTS_ We're excited to do more with Effect layers soon too.

No more ngrok!

Alchemy now supports binding to a Durable Object hosted in another Worker!

Effect is making me fearless. Alchemy’s new providers are more reliable and take less effort to build. The compiler catches all the edge cases. They write themselves.

Get ready for “SaaS as NPM packages”. Top-level await means we can distribute Infrastructure in ES Modules. Imagine importing an Auth service. Or your Accounting Software.

Finally got Monorepos working in @alchemy_run Here I `dev`, `deploy` and `destroy` a Backend Worker that is imported and bound to the Frontend.

Neon Project + Cloudflare Hyperdrive + TanStackStart in 10 lines of code.

Just hit 1k ⭐️ today on @alchemy_run. Nice acceleration recently!

So thankful for this DX - Cloudflare on @alchemy_run

The holy grail of cloud programming (IAM policy inference) as modeled in @EffectTS_ The type system infers the "Queue.Send" and "KVNamespace.Get" requirements from the `queue.send` and `kv.get` function calls.

cloudflare-typescript keeps growing in bundle size with each major version.

Something, something, @EffectTS_ Like and retweet please.

The end result is a scalable way of declaring cloud programs as Effects. > Infrastructure-as-Effects Type-safety all the way through. Arbitrary references. Least privilege policies. Optimal tree-shaking.

Alchemy now has its own Vite plugin to streamline local development. No more .dev.vars or vite.config.ts boilerplate - control everything from your @alchemy_run code. Manage Secrets and Env variables however you want. Blog post in reply:

A friendly reminder that @alchemy_run supports Cloudflare Containers and you don't need to wait for a Wrangler release to use the new instance types.

Cloudflare is coming to @alchemy_run (effect) 🧪

Psyched for CloudFlare containers. Just saw a demo of MCP with a DO and running Linux container per chat thread.

Who's gonna tell him about @alchemy_run?

Never got access to the Container Beta so we been reverse engineering wrangler like mad today. Containers will be in the next version of Alchemy.

"import backend"

Tell that to the game engine developer

Effect + DOs is going to be a blast. We're focused on this @alchemy_run with alchemy-effect, but aim to take it even further with universal, cross-cloud, type-safe bindings. Cloudflare at the root is perfect. In their words: "the Connectivity Cloud"

I honestly took for granted how well designed AWS is. No one is even remotely close on maturity and consistency. We can debate features, but AWS's platform (IAM, SSO, Orgs, API design) is objectively superior.

"Resumable streams" should be relatively easy to achieve with Durable Objects, yeah?

Minimal IAM permissions guaranteed by the TypeScript compiler! Here, we provide an unnecessary "Put" permission and receive a type error.

lol you zoomed in to fit your narrative. It’s been swinging wildly since 2024, it was on the way down.

What personal autonomy do I have when I get ripped off by the insurance cartels?

I'm excited to share what we've been working on the past few months - meet Eventual. A new cloud framework for building massively distributed systems using APIs, Messaging and Workflows on #AWS, with a #GenerativeAI twist! docs.eventual.ai/blog/hello-…

Did you know that R2's SecretAccessKey is a sha256 hash of the Access Token? Oh, and that you have to compute that hash manually since their UI and API doesn't provide it. There is a severe lack of user-obsession a Cloudflare. So many easy wins fall through the cracks.

We moving fast. @alchemy_run now using capnweb for our remote bindings.

Today I finally found some time to experiment with this concept of a tiny #AWS SDK. Introducing "itty-aws" - a TypeScript AWS SDK with a 40KB bundle size regardless of how many services you use. It was an afternoon side project, so consider it "beta" 😅 github.com/sam-goodwin/itty-…

Classic “appeal to authority”.

Alchemy now supports tunneling requests from the public internet to your local Worker in dev mode. Makes it easy to develop and test things like stripe web hooks locally.

SQLite3 state store changes can be watched in real-time. Makes for a nice IDE experience.

Has Infrastructure-as-Code ever been so clean? TanStackStart server function deployed with Alchemy accessing a type-safe R2 binding. Zero code-gen.

Support for Miniflare with local and hybrid remote is finally available in Alchemy. Hot reloading for all of your infrastructure, and extremely fast ⚡️

Here's what a unit test of a provider looks like in @alchemy_run (effect) The pluggability of @EffectTS_ is next-level.

What if Alchemy Resources existed at runtime? I've been dreaming of this experience for over 5 years.

Does SQL suck though? I like it.

Human rights are just stories. Human minds create and tell stories. Human minds are creations of biology. Human rights are therefore biology and “real”. What he says has no logic.

Alchemy now supports Cloudflare's RateLimit binding! It's simple, but powerful tool for building your own rate limiting logic.

Traction on Alchemy is crazy right now. Hit me up if you're looking for a job and like to work on dev tooling + infrastructure!

#AWS Step Functions are interesting to me primarily because it outsources everything except the business logic to AWS - the Cloud Provider. You don’t have to security patch, manage credentials, client connection pools. Just make sure the business logic is right and “walk away”

Renaming CloudFlare Durable Objects is annoying because you need to manually track migrations in your wrangler.json/toml. Alchemy does this for you automatically. Look at this slick `alchemy.config.ts` for deploying a DO-backed ViteJS site.

Alchemy will now generate physical names based off of your app name, stage and resource ID. Stick to simple resource IDs and you get a consistent file naming convention for all your resources.

Cope. It’s just a compiler. A new abstraction layer. You don’t need to type every letter to learn scalable engineering practices.

Cloudflare has no *egress* fees and AWS has no *ingress* fees. Pay egress fees once to move data to R2 and now you can migrate data to as many AWS regions or cloud providers as you like for $0/GB. Checkmate price gougers.

Did I miss an announcement? Tree View in AWS Console for CDK Apps! Shows my SQS Queue containing its onEvent Function handler. Nice quality of life change, now we just need to allow for trees in logical IDs instead of the awful base64 encoded hash on the end :).

Alchemy now supports 251 AWS services, totaling 1392 AWS Resources thanks to the Cloud Control API. Total bundle size impact <200KB thanks to type-only and Proxy-magic.

A CloudFlare retweet wasn’t on my bingo card today.

You can now deploy Prisma Postgres databases with @alchemy_run! We were so blown away with the speed of their API, we thought our tests were broken. <1s to create a database  🤯

Nearing a working version of Infrastructure-as-Effects. This example: 1. Message schema and Queue 2. Implement consumer biz logic 3. Export the runtime handler 4. Export the infrastructure Tree-shakes optimally. Inferred least-privilege policies. Composes naturally. Testable.

Another day, another bug in the Cloudflare API docs. The browser rendering binding is documented as "browser_rendering" but is actually "browser". Digging through Wrangler's source code saves the day again!

They are highly skilled, intelligent people, but not geniuses. This shows the effectiveness of deliberate education. It does not produce true geniuses like Einstein or Newton.

Looks like JavaScript Resolvers are coming to AWS AppSync. Cannot wait for that! I hope API Gateway follows suit. github.com/aws/aws-appsync-c…