The Future of AI is Small, Cheap, and Boring

Earlier this month, I attended Cambridge Tech Week. Cambridge is a hotbed of universities, start-ups, and thought leaders, with a heavy focus on AI. The future of AI came up in almost every conversation.

I don’t have a view on whether AI will take over the world, enslave humanity, or crash the economy (I simply don’t know). But when it comes to where the technology itself is heading, I have a firm opinion: the hype fades, probably within the next year, and giant models that can do everything will make way for smaller, cheaper models made for specific jobs.

AI Has Some Problems

There are valid operational concerns about the current state of AI:

• Energy use: Running the biggest models requires vast amounts of power. One researcher estimated that if every Google search included an AI summary - which it does in 55% of searches currently - it would use as much electricity as Ireland.
• Privacy: Depending on how you access a tool, it may train on your inputs. That means there is a risk that a competitor could ask for your strategy and get a useful answer. Even with enterprise protection, staff at AI vendors may still be able to access your chats.
• Compliance: Most hosted services sit in the US or China, which means your data leaves the UK. You need to understand how this impacts your GDPR position.
• Cost: Spending on hosted AI is unpredictable. Charges are based on input and output tokens, a unit most people don’t intuitively understand. Renting your own AI servers only makes financial sense at huge volume.
• Performance: Speed matters in daily operations. I have had to reject AI on several projects simply because the models were too slow for work where someone was waiting for a process to continue.

The Swiss Army Knife

The current arms race between AI providers is to go bigger to improve accuracy, and scale across more servers to boost speed. The mantra seems to be that bigger is always better.

Therein lies the problem. When I ask an AI for feedback on a blog post, I am querying a model that can also write Python, analyse dinner recipes from a photo of my fridge, recount the entire history of the Roman Empire, and theorise about stock market responses to an IPO. I’m getting a Swiss Army knife, when all I need is a bottle opener.

The model is massive, and any given query is using only a fraction of its capability. That leads directly to higher spending, high energy consumption, and sluggish performance. The model’s breadth is complete overkill for what I actually need.

My Predictions

1: Small Specialists

To make AI faster and cheaper, we will soon see models tailored to specific tasks. Instead of using a single massive system that attempts everything, you will rely on a smaller model optimised for one job - whether that is writing code, searching internal files, or translating data.

This will be a positive change for everyone:

• Reduced cost: Running smaller models costs significantly less, and providers should pass some of those savings down. 
• Faster performance: Smaller models return results much faster than today’s biggest, most advanced models.
• Reduced environmental impact: Less processing power means reduced electricity and cooling demands.
• Privacy and compliance: With smaller models, it becomes feasible for companies to self-host the models, and for local and regional hosting companies to offer hosted AI, which makes GDPR compliance and strong privacy much easier to achieve.

I don’t think this shift will happen while everyone’s still chasing the biggest model. It happens once the excitement has worn off.

2: The Hype Will Die Down

You may be familiar with the Gartner Hype Cycle. It’s a description of the evolution of new hyped technologies. In broad terms, it goes like this:

1. Innovation Trigger: A new technology is invented that captures people’s imagination.
2. Peak of Inflated Expectations: The hype catches, it will be everything to everyone, and everyone involved is going to be filthy rich - if you don’t get in now, your company is as good as bankrupt. (You Are Here)
3. Trough of Disillusionment: The tech fails to meet expectations and the hype dies down.
4. Slope of Enlightenment: Lessons are learnt, we figure out what the tech is really good at, and adoption steadily increases - without the hype.
5. Plateau of Productivity: The technology is broadly integrated into tools and systems we use, but we don’t necessarily talk about it.

A recent example of this is the cloud. Remember that? It was every bit as hyped up as AI is now. Suddenly, every product had “cloud” slapped on it, whether it added any benefit or not. 

I lived through this: During the cloud hype I worked for a software vendor whose entire selling point was that their software was on premises and could control things from within. We were told it had to have a cloud component. It added nothing to the user’s experience or to the company’s bottom line.

After some time, the hype died down, and people started questioning whether all the investment in the data centres and cloud enabled products was actually worth it. Nowadays, you can barely use the internet without a cloud service being involved somewhere. Nobody talks about it anymore, but it is a genuinely useful technology (for certain things!) which is being used to great effect.

My prediction for AI is exactly the same: We’ll all start doubting whether it was all worth it, then the hype will die down, and over time we’ll figure out the real use cases where AI actually works. That’s when we’ll see AI getting integrated into many systems, and soon enough we won’t be able to use the internet without AI being involved somewhere. We just won’t necessarily talk about it, or care.

Why I Think This

I’ve seen this evolution multiple times over my 26-year career, starting with the internet itself back in the dot-com days, and several technologies since. AI is a genuinely useful technology, but the current hype far outweighs its capabilities. 

That said, there are some developments that solidify my opinion that small specialist models are the way this is going to go:

• Large models are getting to a point of diminishing returns. Simply adding more data to the training set is no longer creating the leaps in capability and reliability as it did in the last couple of years.

• Free models that can be downloaded and run on your own PC or laptop are becoming surprisingly capable. This shows that you don’t need a huge model to add value, depending on your use case.

• Two specific new technological developments in AI:

  • Mixture of Experts (MoE) models already split themselves into different parts under the bonnet. Instead of running the whole model for every question, they switch on only the parts needed. That’s a sign even the big players know running everything, every time, is wasteful. 
  • A recent model called Jev is specialised to respond to systems, rather than people. Because it doesn’t have to respond in sentences, the model is much smaller, faster, and cheaper.

• Phones, smart watches and other wearables already ship with AI chips onboard, which can only run small, specialised models.

The Next Year

The dot-com bubble ended in a crash, the cloud one didn’t. I’m not qualified to tell you whether the AI bubble will or won’t. But one thing I do know is that the coming year is going to be interesting. Gartner reckons AI has already tipped into the trough of disillusionment; I predict that the rest of the market will catch up within a year. That won’t stop companies from building new technologies, training new AI models, or quietly getting the hardware ready to support the age where AI is everywhere.

AI is here to stay. You will be left behind if you don’t use it, but not in the way the people caught up in the hype cycle are telling you. You don’t win by bolting it onto everything. You win by using it where it actually earns its keep.

What Should You Do?

As long as the question being asked is “How is AI the solution to this problem?”, we’ll continue to see AI being used for use cases that aren’t a natural fit.

My advice is to adopt AI, but only where it makes sense, and be aware of the risks and limitations. My simple litmus test for a problem that can benefit from AI:

• The process is uncertain, or the data in it requires interpretation. Customer emails are a good fit because everyone writes differently. Standardised invoices aren’t. If the data arrives in a fixed format and you can draw the process as a flow chart, ordinary automation will do it for a fraction of the cost. 
• The cost of failure is low, or failure can reliably be stopped before it causes damage. Having AI read emails or documents and suggest how to file them works, because a person can spot and fix a wrong call. Letting it reply to customers on its own is riskier. Where mistakes could damage relationships, waste stock or hurt people, someone needs to check what the AI is doing before it does it.

So use AI where it genuinely fits, not because someone told you you’ll go out of business without it. Start with a few low-risk jobs, keep a person in the loop where mistakes would have an impact, and judge it on results, not hype.