Nvidia's Agent Knowledge Foundation aims to give enterprise AI context without replacing data platforms

Nvidia announced Agent Knowledge Foundation at its GTC Berlin conference on Wednesday, adding it to its Agent Toolkit as a collection of open-source components. The bundle includes Nemotron Retriever models, a new Auto-Ontology library, the Kumo family of models for structured data, and CUDA-X data processing libraries. The system allows data to remain in place and continue operating under existing access controls.

Enterprise organizations face a persistent challenge: connecting AI tools to corporate data while navigating a landscape where major vendors like SAP, Microsoft, Salesforce, and Workday all seek to become the central hub for data management and security. Nvidia's approach sidesteps direct competition by providing components that work within existing data platforms rather than replacing them.

The context problem

According to Adel El Hallak, Nvidia's vice president of product for agentic AI, connecting an agent to a vendor like Snowflake does not automatically grant the agent understanding of broader enterprise context. "I wish it was as easy as connected to my Snowflake system, connected to my email exchange, connected to my knowledge base, and then all of a sudden, magically, the AI understands all the data. But the reality is different," he says.

Large language models excel at predicting subsequent tokens but struggle when faced with complex data structures. When presented with a spreadsheet containing multiple tabs, agents have difficulty locating specific information. El Hallak notes that better embedding and reranking models can solve this problem: "In the world of agents, you want the most accurate embedding models or reranking models," even if that introduces minor latency. The result is "much more pertinent information. So the number of lookups at the end, the overall latency actually goes down."

Retrieval improvements

Nvidia's Nemotron Retriever embedding model demonstrated a 65% increase in search relevance during tests on an open-source OpenSearch setup, requiring fewer search steps while adding minimal latency. SAP's fine-tuned version of the reranker achieved nearly three times faster performance, and ServiceNow reported improved reranking quality with approximately 25% less latency.

Moving beyond retrieval-augmented generation

El Hallak questions whether retrieval-augmented generation alone suffices for enterprise needs. Knowledge graphs, he argues, better represent concepts and their relationships compared to RAG systems that retrieve text chunks based on similarity.

The Auto-Ontology library constructs graphs from a company's existing tables and maps raw data to business concepts such as customers, products, and orders. The library is available in Nvidia's NGC software catalog with a REST API and Model Context Protocol server for agent queries. El Hallak illustrates the challenge: "You and I, it's easy for us to understand. A tier is the same thing as a plan when you're looking at a mobile carrier," but LLMs "mess those up."

Nvidia acquired Illumex, an Israeli startup specializing in automated semantic layer and ontology creation for enterprise data, for a reported $75 million in February. An Nvidia spokesperson told Globes at the time that the team's ontology expertise would support the development of "libraries and models that will enable the adoption of Agentic AI in organizations."

Predictive models for structured data

Nvidia acquired Kumo in June in a deal valued by The Information at more than $400 million. The startup had developed foundation models for relational data. Nvidia has since released multiple Kumo models as open weights on Hugging Face, including Kumo Tabular in September.

According to El Hallak, these models eliminate the need for per-task pipelines that data teams previously constructed for individual classification, churn, and sentiment analysis tasks. He explains: "The whole point of the ontology … is to bring harmonization as you create those knowledge graphs, so there's [consistency] across the tables, and then you could use Kumo to drive predictive analytics."

Partnership strategy

El Hallak emphasizes Nvidia's collaborative approach: "I'm not looking to compete with, for example, a Snowflake. On the contrary, I want to partner with them."

Nvidia's own supply chain team deploys Agent Knowledge Foundation through Palantir's Foundry platform, building on prior collaboration between the companies. Databricks integrates a Kumo anomaly detection model with Genie Ontology, its context layer introduced in June. Snowflake customers can execute Kumo models within Snowflake Notebooks.

El Hallak acknowledges the limits of Nvidia's responsibility: "I'm not thinking about ACLs and data access and all those things. That's why I need the partners," referring to access control lists and security features that enterprise partners provide.

Contrasting approaches

Not all vendors follow Nvidia's model. SAP completed its acquisition of Prior Labs, the company behind TabPFN tabular models, in July and committed more than €1 billion to the company over four years. SAP now offers TabPFN-3.5 Plus in SAP AI Core alongside its own RPT models.

Nvidia distributes its models through Hugging Face, which it agreed to acquire for $12.9 billion in September. El Hallak describes the approach simply: "We just build little weights and models, and we publish them on Hugging Face." This arrangement leaves Nvidia focused on model development while data platforms like Snowflake and Databricks retain control over data and associated permissions.