The rise of generative AI applications across enterprises is reshaping not just infrastructure, but the underlying mechanics of data retrieval and search. Organizations deploying retrieval-augmented generation (RAG) systems, AI agents, and other AI-driven workflows are confronting a fundamental problem: managing and searching through massive volumes of vectorized data efficiently.

The core issue centers on data freshness and retrieval speed. As these systems generate enormous quantities of vectors from unstructured data, the challenge becomes locating the correct information quickly enough to power AI applications. "After all, a model can only reason over the context you can find and feed it." Without access to current, relevant context, even sophisticated models cannot overcome stale or incorrect underlying information.

Search has moved beyond keywords

Traditional keyword-based search has given way to more sophisticated approaches. Natural language queries have replaced simple keyword entries, driven by changing user behavior. Agent workloads introduce additional complexity, querying 10 to 100 times more frequently than human users, placing unprecedented demands on search infrastructure.

Modern search strategies now encompass multiple approaches. Full-text search relies on analyzers and relevance tuning to match exact words and phrases. Semantic search moves beyond keywords to understand meaning and intent, allowing users to query with everyday language or synonyms. Vector search uses numerical representations of data to identify semantically similar items rather than exact matches. Hybrid search combines keyword and vector approaches in unified result sets.

For contemporary AI applications that depend on unstructured data to retrieve context, keyword search alone has become insufficient. Organizations increasingly adopt high-performance vector databases, semantic search capabilities, and hybrid search methods to support modern AI workloads. This shift, however, introduces a new constraint: the ability to index vectors at scale.

Why indexing is the new bottleneck

Scaling from gigabyte-level search to petabyte-scale AI operations has exposed indexing as a critical performance limitation. The process of indexing vectors is computationally expensive and time-consuming, often requiring days or weeks before newly created vectors become searchable.

Rachael Wade, global tech alliances manager at Elastic, explains the scope of the challenge: "For customers who want to search across petabytes of data, they may notice that it takes days to weeks to index." While extended indexing timelines may be acceptable during pilot phases, production deployments cannot tolerate such delays without compromising system responsiveness.

The technical root of this bottleneck lies in constructing the vector index itself, specifically the HNSW (Hierarchical Navigable Small World) graph—a layered data structure that connects similar vectors to enable efficient searching. Building this graph requires comparing millions or billions of vectors to determine their connections, demanding enormous computational resources and countless mathematical operations.

When enterprises lack sufficient compute capacity, indexing slows dramatically, and new information remains unsearchable for extended periods. This leaves RAG systems, queries, and agents operating on outdated data. Manas Singh, technical product manager for accelerated ML and vector search at Nvidia, frames the challenge this way: "the new constraint is getting vectors indexed fast enough to keep that search current."

E-commerce product catalogs illustrate the real-world impact. Companies depend on fresh, searchable product data to maintain accurate inventory and pricing information. Without rapid indexing, critical business information becomes stale before it reaches production systems.

It's time for GPU-accelerated indexing

CPUs have traditionally handled vector index construction, but they struggle to meet the scale and data-freshness demands of contemporary AI workloads. Indexing petabyte-scale vectorized data on CPUs can take days or weeks, and the hardware costs escalate rapidly when indexing hundreds of millions or billions of vectors.

GPUs, by contrast, are engineered precisely for large-scale, computationally intensive tasks. Their massive parallelism and high-throughput mathematical capabilities make them ideally suited for accelerated vector indexing at enterprise scale.

Elastic and Nvidia have partnered to bring GPU acceleration to vector indexing in Elasticsearch. The collaboration leverages Nvidia cuVS, an open-source CUDA library for GPU-accelerated vector search and data clustering that accelerates both index construction and embedding retrieval for AI and recommendation systems. Elasticsearch accesses cuVS through cuvs-java, an open-source library maintained by Nvidia.

Elastic contributes semantic and hybrid search capabilities through its enterprise vector database, one of the most widely deployed globally, powering semantic search, AI assistants, and recommendation systems for organizations ranging from startups to large enterprises. Free trials of Elasticsearch are available to developers.

By accelerating vector indexing, the partnership enables enterprises to process massive volumes of vectorized data more rapidly, making it available sooner for semantic search and AI workloads—including generative AI and AI agents—at production scale and speed.

The performance gains are substantial. According to Elastic, moving indexing construction to GPU with Nvidia cuVS delivers approximately 12x improvement in indexing throughput and a 7x reduction in force-merge latency, which measures the time required to consolidate small index segments.

Cost differences between GPU and CPU approaches are significant. Wade illustrates this with a concrete example: a single GPU, depending on the model, can index hundreds of millions of vectors within roughly a 10-minute service-level agreement. Accomplishing the same work with CPUs would require approximately 10 CPUs and roughly two hours—representing a baseline total cost of ownership savings of 78%.

GPU acceleration becomes increasingly valuable as vector dimensions grow. CPUs require substantially longer to process higher-dimensional vectors, making GPU acceleration—whether delivering 5x, 6x, 7x, or 12x speedup—progressively more beneficial at larger scales.

Why faster indexing matters now

For on-premises deployments, GPU-accelerated indexing offers an additional advantage: reduced search latency. By shifting indexing work from CPUs to faster, more efficient GPUs, enterprises can reallocate CPU resources to other operations. As Singh notes, "You free up the CPU resources for other things," which can ultimately reduce search latency across the system.

GPU-accelerated vector indexing is available in self-managed Elasticsearch environments, including on-premises installations. This means organizations can adopt faster indexing without migrating workloads away from infrastructure they control.

For IT leaders focused on maximizing return on investment and maintaining data sovereignty, shifting indexing from CPUs to GPUs delivers multiple benefits: completing more indexing work in less time with reduced infrastructure, meeting self-managed and on-premises deployment requirements, and freeing CPU capacity for search operations.

Broader trends in AI infrastructure reflect how artificial intelligence is transforming data processing and search fundamentals. For RAG systems, AI agents, and other retrieval-dependent applications, making new data searchable quickly is essential to system utility. That requirement begins with accelerated indexing.