An agent is only as good as what it can find and trust.
Most disappointing AI pilots are blamed on the model. The model is rarely the problem. Column names are not meaning, a schema stops fitting in a prompt somewhere past a few hundred tables, and nobody wants a system that reads personal data it was never cleared to see. This is the layer underneath every agent we deploy — and the part of the work that outlives whichever framework is current.
› source · warehouse · 50,000+ tables
✓ profile · grain and keys inferred, then confirmed
✓ semantic · 214 measures named in business terms
✓ scan · 31 sensitive spans tokenized
✓ lineage · every field traced to its source
✓ ready · retrieval index built, nothing left the perimeter
The failure nobody logs
A wrong answer and a confident answer look identical.
Point a capable model at a raw warehouse and it will answer. It will join on a plausible-looking key, pick a grain nobody agreed to, quietly exclude the rows that were filtered out three transformations upstream, and return a number formatted to two decimal places. Nothing errors. Nothing is logged. Somebody takes that number into a meeting.
The instinct is to fix this with a better model or a longer prompt. Neither addresses the cause. A model cannot infer that revenue means net of returns at this company and gross at the subsidiary, that the customer table has two rows per customer since the 2023 merger, or that the finance team stopped trusting a column in March. That knowledge exists in people and in transformation code. Until it is written down somewhere the system can read, every answer is a guess that happens to be well formatted.
This work is unglamorous and it is the difference between a pilot and a system. It is also the piece a security review asks about first, because it is where regulated data either gets handled properly or does not.
What ready means
Four layers, each one a thing a reviewer can check.
“AI-ready” is a marketing phrase unless it decomposes into specific artifacts. These are the four we build, in the order they have to happen.
Meaning, not column names
Grain, keys, relationships, and the business definition of each measure, expressed as a layer the agent reasons over instead of guessing at. Where a definition is contested between two teams, that is recorded as a fact about the business rather than resolved silently in favour of whichever one the model saw first.
What a semantic layer isSensitive spans, handled before inference
Personal and regulated values are detected and tokenized before any call leaves your perimeter, and the mapping that would reverse them never does. The detection is deterministic rather than a model judgement — asking a model to redact its own input is not a control a reviewer will accept, because the failure mode is invisible.
See the control it maps toDiscovery past the prompt limit
Injecting the schema into the prompt works until it does not, and the cliff arrives earlier than people expect. Past it, finding the right seven tables out of fifty thousand is itself a retrieval problem, solved with a narrow probing tool set and a pointer pattern for bulk results rather than by buying a larger context window.
How discovery works at scaleLineage that survives into the answer
A citation is only worth something if it resolves to the exact query and result set behind it, not to a document title. Each source also carries an explicit freshness expectation, so an answer built on a table that stopped loading on Tuesday says so rather than aging quietly into being wrong.
Read the provenance teardownWhy separate it
This is the part you keep.
Frameworks and model providers turn over on a timescale of months. A semantic model, a tokenization policy, and a set of lineage contracts do not.
It reads your sources in place
No migration, and no second copy of the data to govern, secure, and keep fresh. A duplicate is a new liability, not a foundation.
It outlives the agent layer above it
Replace the orchestration framework, change model providers, move from a pilot to production — the preparation layer is unaffected, because it describes your business rather than a vendor's abstractions.
It is what makes the rest reviewable
Where regulated data is detected, what the agent may reach, and how a claim traces back to a row are all decided here. A security review that gets a straight answer at this layer stops asking the same question five more times further up.
Asked at every kickoff.
Because column names are not meaning. A model pointed at raw schema guesses at grain, joins on the wrong key, and silently answers a different question than the one asked. It also does not scale: past a few hundred tables the schema no longer fits in a prompt, and past a few thousand the retrieval step becomes the entire problem.
Sensitive spans are tokenized before the inference call, and the mapping that would reverse them stays inside your perimeter. The model sees a token where a name or an account number was. Detection is a deterministic scanning step rather than a model judgement, because a model asked to redact its own input fails invisibly.
No. The preparation layer reads from your existing warehouse, lake, or application sources in place. Moving data would create a second copy to govern, secure, and keep fresh, which is the opposite of the goal.
Yes, and that is the reason to do it as its own layer. A semantic model, a tokenization policy, and lineage contracts are assets that outlive whichever framework or model provider is current. They are the part of an AI programme least likely to be thrown away in eighteen months.
Bring the source nobody wants to touch.
The one with the undocumented join, the column two teams define differently, and the personal data somebody is not sure is still in there. That is the useful conversation, and it is a faster way to find out whether this is worth doing than any demonstration.