Getting AI agents to work with your data

Enterprises that want AI agents to answer business questions quickly soon discover that raw transaction logs and data lakes are not ready for that kind of access. The gap forces slow answers, security worries and unpredictable spending, three factors that decide whether an AI agent can be trusted in production.

Data readiness matters

TOTVS found abundant transactional and lake data that was not prepared for AI‑agent access. The company showed that precision, security and cost depend on how the data is offered to the model.

Governance through a data‑mesh

A data‑mesh treats each data set as a data product with an owner, a stable interface contract and a quality service‑level agreement. This mirrors the micro‑service model for code and makes data discoverable and governed.

Adding a semantic layer

Applying Resource Description Framework (RDF) triples and domain‑specific ontologies turns vague column names into clear business concepts. In TOTVS trials the semantic layer lifted LLM response precision by roughly forty percent.

Layered platform for speed

Three processing layers deliver the right latency:

Photo by Matheus Bertelli on Pexels

• High‑latency batch jobs on Apache Spark for heavy analytics.
• Medium‑latency queries on Google BigQuery for ad‑hoc reporting.
• Low‑latency sub‑second look‑ups on PostgreSQL with trigger‑driven stored procedures and DuckDB for in‑memory analytics.

The low‑latency layer can answer agent requests in milliseconds, keeping conversations fluid.

Secure tool access with OAuth

Each data product exposes parameterised tools. Embedding OAuth tokens that carry the original user’s identity lets the tool enforce row‑level security automatically, reducing prompt‑injection risk.

Cost‑effective deployment with MCP Fabric

MCP Fabric lets a single service host hundreds of virtual MCP servers, lowering infrastructure cost while preserving isolation between teams.

Token efficiency

Dynamic tool‑search lets the model discover only the tools it needs, cutting token consumption. Returning results as CSV or the newer Token‑Oriented Object Notation (TOON) can shave thirty to sixty percent off output tokens, though TOON may reduce precision because LLMs see fewer examples of that format.

Practical checklist

1. Audit data assets and flag tables ready to become data products.
2. Assign owners, publish contracts and set SLAs for each product.
3. Pilot an RDF‑based ontology for a high‑value domain such as finance.
4. Deploy PostgreSQL triggers that call DuckDB for sub‑second analytics.
5. Wrap access in MCP tools that require an OAuth token.
6. Enable dynamic tool discovery (e.g., Spring AI tool‑search).
7. Test CSV and TOON output formats and measure token savings versus answer quality.

Terms used in this article

• AI agent, an application that combines a language model with code to perform tasks.
• LLM, large language model, a type of AI that generates text.
• RDF, Resource Description Framework, a way to represent data as subject‑predicate‑object triples.
• Ontology, a formal description of concepts and relationships in a domain.
• Data product, a curated data set with an owner, contract and SLA.
• MCP, Model Context Protocol, a framework for exposing tools to LLM agents.
• OAuth, open standard for delegated authentication, used here to pass user identity.
• TOON, Token‑Oriented Object Notation, a compact serialization format designed for LLMs.
• Token, a unit of text that LLMs count when processing prompts and outputs.