Why AI Keeps Getting Cheaper to Run
Running AI models keeps getting cheaper fast. Here is how I make sense of why that is happening.
The cost of running these AI models keeps dropping, and it's kind of wild how fast. Stuff that would've blown my budget a year ago is basically pocket change now.
A few things are going on at once. There are a bunch of labs now shipping models that are all about as good as each other, so nobody gets to charge a premium, they just keep undercutting each other. The models are also getting more efficient, the smaller ones are catching up to the big ones, and the serving tricks and the hardware keep getting better, so each answer just costs less to produce. And the open models quietly keep everyone honest, because if you can run something good yourself, nobody can charge you a fortune for it.
The takeaway for me is pretty simple. If you dropped an idea because it was too expensive to run, it's worth checking again, because the numbers have moved. And the model itself is turning into a commodity. What you build around it is the part that actually matters, and that never got cheap.