Why AI requires rethinking the storage-compute divide

"Modern AI pipelines now process large amounts of unstructured and multimodal data, while also generating large volumes of embeddings, vectors, and metadata. At the same time, processing is increasingly continuous, with many compute engines touching the same data repeatedly—each pulling the data out of storage and reshaping it for its own needs."

"The same dataset might be read from storage, transformed for model training, then read again and reshaped for inference, and again for testing and validation—each time incurring the full cost of data transfer and transformation. Data scientists spend up to 80% of their time just on data preparation and wrangling."

"93% of organizations today say their GPUs are underutilized. With top-shelf GPUs costing several dollars per hour across major cloud platforms, this underutilization can quickly compound into tens of millions of dollars of paid-for compute going to waste."

AI workloads differ fundamentally from traditional analytics by processing large volumes of unstructured and multimodal data continuously, generating embeddings, vectors, and metadata. Multiple compute engines repeatedly access and reshape the same data for different purposes—training, inference, testing, and validation—creating redundant work and excessive data transfer costs. Data scientists spend approximately 80% of their time on data preparation rather than model development. This inefficiency becomes economically significant at scale, with 93% of organizations experiencing GPU underutilization. Given that premium GPUs cost several dollars hourly on cloud platforms, this waste translates to tens of millions in unnecessary infrastructure spending, making current separated architectures economically unjustifiable.

Read at InfoWorld