AI drives bigger data storage needs, Western Digital study

Western Digital has published research with IDC on AI's impact on data storage. The study found that most organisations are storing more data because of AI.

The survey of 763 IT and business decision-makers across seven countries points to a shift in infrastructure planning as companies retain larger volumes of data and bring older archives back into use for AI workloads.

Recent debate around AI infrastructure has focused largely on processors and computing clusters. These findings instead put storage economics and data retention closer to the centre of the discussion, particularly for operators of large data centres, managed service providers and channel partners that design systems for corporate customers.

The research found that 94.7% of respondents said AI and generative AI adoption had increased the amount of data their organisations stored over the past 12 months. Some 61% said data volumes had grown by 25% or more in the past year because of AI, while 74% expect volumes to rise by more than 25% over the next three years.

Data lakes were among the clearest pressure points in the study. About 85.4% of respondents reported growth in data lake volumes over the past 12 months, and 59.4% identified AI-generated data, including synthetic data, inference outputs and model logs, as the main reason.

Archive retrieval

Additionally, the study found that data once treated as dormant is returning to active use. Some 75.9% of organisations said they were bringing increasing volumes of archived cold-tier data back online to support AI workloads.

That trend has implications for archive system design. Nearly all respondents, 96%, said they expected a need for faster archive retrieval to support AI inference and retrieval-augmented generation applications, suggesting businesses want old data to be available more quickly when feeding models or responding to user prompts.

Retention periods are also changing. Nearly 95% of respondents said the value of their organisation's data had increased because of AI and generative AI adoption, while 74.3% said those technologies had led them to keep data for longer.

The figures indicate that the cost of storing information over long periods is becoming a bigger factor in AI investment decisions. More than 74% of enterprise data sits in warm, cool and cold storage tiers, according to the study, and more than 60% of data lake volume consists of cold or infrequently accessed data.

Cost pressure

Those patterns help explain why 98.2% of respondents said total cost of ownership per terabyte was important or very important when making storage decisions.

For companies running AI systems at scale, the issue is not only how quickly systems can process data, but where that data is held, how often it is retrieved and what it costs to keep it available.

The survey covered the United States, China, Japan, South Korea, India, Saudi Arabia and Germany. IDC said respondents were manager level or above and had direct responsibility for AI infrastructure or data storage decisions.

Western Digital used the findings to argue that storage architecture is becoming a strategic concern rather than a back-office procurement issue. As a supplier of hard disk drives and related storage products to cloud operators and enterprise customers, the company has a direct interest in that argument.

Irving Tan, chief executive officer at WD, said the issue had moved beyond short-term computing demand.

"For the last few years, the AI infrastructure conversation has centered on compute. But AI runs on data," Tan said.

"Organisations are generating more data, keeping it longer, and finding new ways to create value from the information they already have. Compute requirements will evolve over time, but the need to store, manage and access data at scale is only growing. That foundation will play a critical role in determining how far AI can go," Tan added.

The research points a wider industry shift in which AI systems are not only consuming large training datasets, but also creating new stores of information through model checkpoints, logs, inference outputs and synthetic data. Because much of that information remains after a task is completed, storage demand can continue to grow even when compute usage fluctuates.

For data centre operators and service providers, that means infrastructure planning may increasingly depend on balancing faster storage tiers with lower-cost archival options. The survey suggests customers are placing greater emphasis on systems that can preserve large data volumes economically while still allowing historical information to be reactivated when AI applications need it.

Among surveyed organisations, that shift is already under way, with 74.6% of enterprise data now residing in warm, cool and cold tiers.