Yesterday, I showed you a radically different kind of AI.
Jev wasn’t designed to write essays or create software. It was designed to make decisions.
And I believe it could be arriving at the perfect time.
You see, the way we use AI computing power is changing.
During the early stages of the AI boom, most of that computing power went toward training the powerful models behind ChatGPT and other AI systems.
But that’s no longer the case.
This year, roughly two-thirds of all AI computing power is expected to go toward actually using AI.
And this is creating a massive new incentive to make AI faster and cheaper.
The Age of Inference
There are two basic ways that AI uses computing power.
The first is training.
That’s the enormously expensive process companies like OpenAI, Google and Anthropic use to teach their models. It requires massive clusters of advanced chips processing huge amounts of data.
But once a model has been trained, it still needs computing power every time somebody uses it.
That’s called inference.
When you ask ChatGPT a question, that’s inference. When an AI writes a piece of software, that’s inference. And when an AI agent searches the web, checks its work or decides which tool to use next, that’s inference too.
And according to Deloitte, inference is quickly becoming the biggest source of AI computing demand.
Take a look…
In 2023, inference accounted for only about one-third of all AI computing power.
By last year, it was roughly half. And Deloitte expects it to reach about two-thirds this year.
In other words, the AI industry is moving from primarily building intelligence to putting that intelligence to work.
And there’s a simple reason why.
A company might spend months training a powerful AI model. But once that model exists, it can be used millions or even billions of times.
Every one of those uses requires inference. And newer AI systems can require a lot more of it.
As I’ve written about before, an AI agent doesn’t necessarily make one request and stop. It might search for information, call a tool, analyze the result, realize something went wrong and try again.
That means a single assignment could require dozens or even hundreds of smaller decisions.
Every one of those decisions requires the AI to run again. And as more companies deploy AI agents, the number of those decisions could explode.
That’s why inference becoming dominant doesn’t mean we’ll need less computing power.
Deloitte expects overall demand for AI compute to continue growing 4X to 5X annually through 2030, even as chips and models become more efficient.
And Gartner is seeing the same transition in where companies are spending their money. It expects global spending on AI infrastructure for inference to reach $23.3 billion this year, compared with $19 billion for training.
That’s the first time inference spending is expected to surpass training spending in Gartner’s AI-optimized cloud infrastructure forecast.
And this brings us back to Jev.
Yesterday, I showed you how Jev was designed to make decisions without generating an answer word by word like a traditional large language model.
That allows it to make certain decisions much faster and cheaper.
And those economics become a lot more relevant when AI systems are making billions or trillions of decisions.
You might need a powerful frontier model to perform difficult research, write software or solve a complicated problem. But simpler decisions don’t always need that much computing power.
Sometimes they just call for a simpler tool.
That’s exactly what Jev was built for.
Here’s My Take
Up to now, the AI race has largely been about building smarter models. But this week’s chart shows us that the balance has flipped.
We’re now spending more computing power using AI than training it.
And the more we put AI to work, the more the cost of every decision will matter.
Regards,
Ian King
Chief Strategist, Banyan Hill Publishing
Editor’s Note: We’d love to hear from you!
If you want to share your thoughts or suggestions about the Daily Disruptor, or if there are any specific topics you’d like us to cover, just send an email to dailydisruptor@banyanhill.com.
Don’t worry, we won’t reveal your full name in the event we publish a response. So feel free to comment away!