How does an AI agent query a database with tens of thousands of tables?

By exploring rather than ingesting. Retrieval surfaces a handful of candidate tables from the catalog, then a probing tool suite previews, counts, aggregates, and text-searches them the way an analyst would before committing to an expensive query. The full schema is never injected into the context window.

Why does injecting a full database schema into the prompt fail at enterprise scale?

It fails on three axes at once. A fifty-thousand-table catalog exceeds any context window, the token cost is incurred on every turn regardless of relevance, and accuracy falls as the model selects from thousands of similarly named candidates instead of a ranked shortlist.

How do you stop an agent pulling millions of rows into a context window?

Materialise large result sets to object storage and return a pre-signed pointer plus a summary. The model receives a reference and aggregate statistics rather than raw rows, so cost stays bounded regardless of result size and the sensitive payload never enters the context.

How an Agent Queries a 50,000-Table Warehouse Without Reading the Schema

Every text-to-SQL demo works against twelve tables. Enterprise catalogs have tens of thousands, and the standard approach — dump the schema into context, hand the model one query tool — fails on cost, latency, and accuracy simultaneously.

The gap between those two situations is where most conversational analytics projects quietly die. It is not a prompt engineering problem, and no amount of instruction tuning closes it.

From the standard

“Pre-trained models with a differentiable access mechanism to explicit non-parametric memory can overcome this issue”

Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — arxiv.org

Why bulk schema injection breaks

Three failures compound, and fixing one worsens another.

It does not fit. A fifty-thousand-table catalog with columns, types, and keys runs to millions of tokens. Even where the window is nominally large enough, retrieval quality degrades badly across that much undifferentiated material.

It is paid for on every turn. The schema is re-sent with each question, including follow-ups that touch two tables. Cost scales with catalog size rather than with the work being done, which is the wrong relationship.

It reduces accuracy. This is the counterintuitive one. Given thousands of similarly named candidates — fct_orders, fct_orders_v2, fct_orders_deprecated, stg_fct_orders — the model picks plausibly rather than correctly. More context makes selection harder, not easier.

What a competent analyst does instead

An analyst handed the same question does not read the catalog. They search for likely tables, open one and look at five rows, check whether it is populated for the period in question, look at the distinct values in a status column, and only then write the query.

That is an iterative loop of cheap probes narrowing toward one expensive operation. Giving an agent the same shape is what makes it work at scale.

Retrieval-driven discovery

Index the catalog once — table names, column names, descriptions, and lineage — into a retrieval store. A question then surfaces a ranked shortlist of perhaps seven candidate tables. Those seven enter the context. The other fifty-one thousand do not.

A probing tool suite, not one query tool

Replace the monolithic query tool with narrow operations the agent can compose: preview a table's first rows, count rows under a filter, list distinct values in a column, compute an aggregate, search text. Each is cheap, each returns little, and each informs the next.

The secondary benefit is auditability. A trace of five named probes is legible to a reviewer in a way that one generated query joining nine tables is not.

The pointer pattern

When a result set exceeds a threshold, do not return it. Materialise it to object storage and return a pre-signed URL plus summary statistics. The model receives a reference and aggregates; it never sees the rows. Cost stays bounded regardless of answer size, and — the part security cares about — the sensitive payload never enters a context window or a model provider's logs.

Three attacks this design has to survive

Iterative exploration introduces failure modes that bulk injection does not have. Each has a specific mitigation.

The blind reasoner. If the model never sees the rows, it can assert things about them that the data does not support. The mitigation is to return summary statistics alongside the pointer — row count, null rates, value ranges, distinct counts — so the narrative is grounded in something checkable rather than in the model's expectation of what a table like that contains.

Egress through the pointer. A pre-signed URL rendered as a clickable link is an exfiltration channel if the destination can be influenced. Validate every URL against an allowlist before it renders, and scope the signature narrowly in both time and object.

Subquery evasion. A rule that inspects generated SQL as a string is defeated by nesting the restricted operation inside a subquery. Parse the query into a syntax tree and validate structurally, so a restricted operation is caught wherever it appears in the tree.

What changes operationally

Exploration costs more round trips per question — five or six tool calls rather than one. In exchange, each call is small, the total token spend falls sharply, and the answer is arrived at rather than guessed. The trace also becomes a review artifact: a reader can see that the agent checked whether the table was populated for Q3 before aggregating it.

The practical test of whether a vendor has solved this is simple. Ask how many tables their system injects into context for a question touching two of them. If the answer scales with your catalog rather than with the question, it will not survive your warehouse.

Axionalytics

Production agentic AI for enterprise engineering, data, and revenue teams.