Will data centers become the bottleneck for Gen AI's growth? Or, are we ready?

Lately, we have probably heard all the noise about Generative AI, and it’s changing the game. This Gen AI technology is everywhere, from crafting stories to assisting in medical research and in other fields. The AI revolution is here, but have we ever pondered what fuels this AI powerhouse? The response lies in data centres. These unsung heroes, i.e., data centres, store, process, and manage the massive amounts of data essential to train and run AI models. But while we marvel at its abilities, a critical question arises as Generative AI (Gen AI) continues to progress: Will Data Centers Become the Bottleneck for Gen AI's Growth?

Let’s understand this. During a huge concert, the backstage crew plays an important role, they aren’t in the spotlight, but the whole show would fall apart without them. Data centres are like these backstage crew, they are responsible for storing, processing and managing data Gen AI is all about BIG Data. Gen AI requires an immense amount of data to learn, improve, and act to generate human-like responses. Due to the continued growth in Gen AI, data centres struggle to keep up with this explosive momentum.

On average, in comparison to a Google search, a ChatGPT query consumes nearly 10 times more electricity to process. This huge difference brings a coming sea change in how the world at large including the US & Europe, will consume power.

Whenever we search ChatGPT/Gemini/Meta, the AI model runs, crunching through terabytes of data stored in a trusty data centre. Folks, we are now talking about petabytes due to the exponential uses of AI search engines.

Gen AI is hungry… The insatiable appetite of Gen AI is not for data only, but for power also. It takes huge amounts of energy to train these massive models, and data centres are already guzzling power. According to a report from Gartner, Gen AI will account for 10% of all data produced worldwide by 2025, up from less than 1% in 2023. Demand for energy is directly proportionate to the scaling of AI, which could become a problem. Goldman Sachs has forecasted a 15% CAGR (Compound Annual Growth Rate) in data centre power demand from 2023-2030, driving data centres to build up 8% of total US power requirement by 2030 from about 3% currently. Goldman Sachs Research estimates that data centre power demand will grow 160% by 2030.

The processing of all data produces heat, and too much heat may slow down the process or even hardware can be damaged. To cool down the servers, data centres have to spend even more energy which, anyone can guess, creates a vicious cycle.

Gen AI flourishes in real-time data processing. Latency can be experienced if a data centre is far away from where the AI operations are taking place— that annoying delay between asking a question and getting an answer.

Though we have advanced GPUs and TPUs (Tensor Processing Units) now, there’s usually a risk that hardware won’t be able to scale as fast as AI models evolve due to hardware limitations. It could be a costly affair to regularly update data centres, and that could become a burden.

By 2025, demands of data storage could skyrocket, as Gen AI models are increasingly data-hungry, thus making scalability a challenge.

But... Are We Ready?

Don’t panic just yet. There’s hope on the horizon!

Companies are moving towards edge computing instead of relying merely on centralized data centres. This means the data distribution process takes place closer to where the data is generated. So, local edge centres can handle smaller, real-time tasks even if the central data centre is a bit flooded.

Liquid cooling systems as new innovations are truly helping data centres manage heat well and minimize energy usage. For the unique needs of machine learning models, companies like Amazon and Google are constructing AI-specific data centers with hardware designed for the distinctive needs of machine learning models.

Many data centres are moving towards renewable power sources with the rise of green energy, wind energy, solar farms, and even underwater data centres (yep, they exist!) are part of this development. So, even if energy consumption increases, we might offset it with cleaner power that can minimize the amount of greenhouse gases emanating into the atmosphere. This balancing approach assists in increased energy demand while curtailing the environmental footprint.

Quantum computing is a bit futuristic and could change the game. These machines might resolve complications much quicker than our current binary systems by leveraging quantum bits (qubits), meaning less data centre strain for complicated AI tasks.

So, techie, after all these challenges and expected breakthroughs, will data centres hold back Gen AI’s growth? The answer is... maybe. Yes, there are challenges, but the tech industry is already trying hard to crack the nuts. Between revolutions in energy efficiency, the upswing of edge computing, and even potential developments like quantum computing, we’re probably not gazing down an unbreakable bottleneck.

That said, the phenomenal growth of Gen AI will force us to rethink and strategize how we design and run our data centres. Since we need not keep up with today’s demands only— rather we need to pull our socks for the explosive growth of AI in the coming days.

In the end, it’s like asking you if your kitchen can be able to manage a 10-course meal. Sure, it is going to be but it’ll take some upgrades, and with the better infrastructure, tools and efficient planning, it’ll pull it off. And, who knows, you might even astonish your dinner guests — or in this case, the world of AI.