Schema-to-Screen. An agentic, RAG-grounded architecture for GraphQL and UI over thousands of tables.
The written reference architecture behind the interactive walkthrough. An enterprise with 3,000 tables does not have a data problem, it has a surface problem — every business question needs a screen, every screen an API, and every API someone who knows which seven tables matter and who may see them. This is the design that makes that surface generative without letting a model near free-form SQL, and the honest account of what it does not yet prove.
Status: reference architecture and build plan. The reference implementation — TypeScript end to end, a scaled public sample database, a hybrid UI — is the next phase and is specified in §11. Nothing here claims a production deployment of this design. The same architecture is available as an interactive walkthrough, and its full TOGAF ADM set is published alongside it.
Status: reference architecture and build plan. The reference implementation (TypeScript end-to-end, scaled public sample database, hybrid UI) is the next phase and is specified in §11. Nothing in this document claims a production deployment of this design.
0. The thesis in one paragraph
An enterprise with 3,000 tables does not have a data problem. It has a surface problem: every business question needs a screen, every screen needs an API, and every API needs someone to know which seven of the 3,000 tables matter and who is allowed to see them. Schema-to-Screen makes that surface generative — a request becomes a typed use-case spec, a RAG-grounded agent retrieves the relevant subgraph of tables (never the whole schema), a composer emits a GraphQL contract slice and a UI spec, deterministic gates verify both, a human approves, and the platform publishes. The model never writes free-form SQL, never sees 3,000 tables, and never produces code that runs unreviewed. Authorization is declared once — on the field, in the schema — and compiled into the query. Everything it produces is a contract behind a gate.
1. The problem, stated honestly
Three things are true at once in most large enterprises:
- The schema is too big for a prompt. 3,000 tables × ~15 columns is ~45,000 columns. Even at a few tokens each, the schema alone is hundreds of thousands of tokens — before comments, constraints, or sample values. Any design that “puts the schema in the context” has already failed at enterprise scale; it just hasn’t noticed yet because the demo used 20 tables.
- Use cases outnumber engineers by two orders of magnitude. A thousand legitimate business screens — approval queues, 360 views, exception lists, reconciliations, dashboards — sit in a backlog no team will ever clear. Each one is a small, well-understood shape over a small subset of tables. The work is not hard. There is just too much of it.
- Text-to-SQL is the wrong abstraction for this. Free-form generated SQL is unreviewable at scale, bypasses every authorization model the organization has, and produces a different answer every time. The failure mode isn’t “the model wrote bad SQL” — it’s “nobody can say what this screen is allowed to show.”
The naive fixes each fail in a specific way:
| Naive approach | Why it fails at 3,000 tables / 1,000 use cases |
|---|---|
| Stuff the schema into the prompt | Context limits, cost, and the model picks the wrong customer_id from twelve candidates |
| Text-to-SQL per request | Unreviewable, un-cacheable, authorization bypassed, non-deterministic |
| Generate a React app per use case | 1,000 codebases to secure, upgrade, and audit; drift within a quarter |
| One generic data grid over every table | Nobody’s actual use case; entitlements still handled after the fact |
| Post-filter results by permission | Recall collapses for narrow-permission users; a control every workflow has to remember |
Schema-to-Screen is what’s left after removing each of those.
2. Six principles the architecture is built to enforce
- The model sees a subgraph, never the schema. Retrieval over a semantic catalog selects the 5–30 tables a use case needs, with join paths. The whole schema is a search index, not a prompt.
- Everything generated is a contract, not code. Agents emit a GraphQL schema slice and a UI spec (a JSON DSL). Resolvers are compiled from the slice by a deterministic query compiler; screens are rendered from the spec by a runtime. Code generation exists only as a reviewed escape hatch (§7.4).
- Authorization is declared once, on the field, and compiled into the query. Classified fields carry a policy directive in the schema slice; the compiler injects row- and column-level predicates from the caller’s validated identity before the SQL executes. Post-filtering is a control you have to remember. A query filter is a control you cannot forget.
- Read and write are different products. Read slices can be published autonomously after gates. Anything that writes to a system of record produces a proposal mutation, and the proposal’s approval workflow is enforced by the platform from the use case’s registration — not implemented per use case.
- The release unit is the whole tuple. Catalog version + schema slice + UI spec + policy version + model + prompt template version. Drift in any element is a release, not a surprise.
- A human approves the first publish of every use case and every write path. Gates catch what can be measured. Review catches what can’t — and it is where the organization’s judgment enters the system.
3. Architecture at a glance
Business request
│
▼
┌─────────────┐ ┌──────────────────────────────────────────────┐
│ Intake agent│────▶│ DETERMINISTIC SUPERVISOR (state machine) │
│ → Use-Case │ │ budgets · gate order · read/write split │
│ Spec │ └───┬───────────────┬───────────────┬──────────┘
└─────────────┘ │ │ │
▼ ▼ ▼
┌───────────┐ ┌────────────┐ ┌────────────┐
SEMANTIC │ Schema │ │ Contract │ │ UI │
CATALOG ───────▶│ Scout │──▶│ Composer │──▶│ Composer │
(hybrid index │ (RAG) │ │ → GraphQL │ │ → UI Spec │
+ FK graph) └───────────┘ │ slice │ │ (+ custom │
└─────┬──────┘ │ component │
POLICY ─────────────────────────────▶ │ │ request) │
CATALOG │ └─────┬──────┘
(classification, ▼ ▼
entitlements) ┌──────────────────────────────┐
│ │ VERIFIER GATES │
│ │ composition · policy 100% │
│ │ cost · bindings · golden │
│ │ tests · PII scan │
│ └──────────────┬───────────────┘
│ ▼
│ ┌──────────────────────────────┐
│ │ HUMAN REVIEW → PUBLISH │
│ └───────┬──────────────┬───────┘
│ ▼ ▼
│ ┌───────────────────┐ ┌──────────────────┐
│ │ Schema registry + │◀─│ React spec │
│ │ supergraph gateway│ │ runtime + │
│ └─────────┬─────────┘ │ component │
│ ▼ │ registry │
└───────────────────▶ ┌───────────────────┐ └──────────────────┘
│ Query compiler │
│ GraphQL → SQL + │
│ entitlement │
│ predicates │
└─────────┬─────────┘
▼
┌───────────────────┐
│ Systems of record │
│ 3,000+ tables │──── crawler ──▶ SEMANTIC CATALOG
└───────────────────┘
Five zones: intake, semantic catalog (the RAG substrate), agentic generation, verification and review, runtime. The runtime is the only zone that touches production data at request time; the generation zone touches sample data only.
4. The semantic catalog — what makes 3,000 tables tractable
The catalog is the platform’s RAG corpus, and it is built about the schema, not from the data.
4.1 What the crawler ingests
| Source | What it yields | Why it matters for retrieval |
|---|---|---|
information_schema / catalog views | tables, columns, types, nullability, PK/FK/unique/check constraints, indexes | The structural truth; FK edges are the join graph |
| Table and column comments | human descriptions where they exist | Highest-quality semantic signal — usually sparse |
| Data profiles (sampled, PII-aware) | row counts, cardinality, null rate, value ranges, top-k values for low-cardinality columns | Disambiguates status columns; identifies enum-like fields; flags identifiers |
| Query-log mining | frequent join paths, filter columns, aggregations actually used | Real join paths beat FK-inferred ones; reveals “the way the business actually reads this” |
| Business glossary / data dictionary | terms → tables/columns; synonyms; owners | Maps “customer” to crm.account not billing.debtor |
| Data classification | PII / PHI / financial / confidential per column; retention | Drives deny-by-default policy generation |
| Entitlement metadata | which roles/groups may read which schemas, tables, columns, rows (by tenant/region/owner) | Compiled into the query, never applied after |
| Existing reports / views / GraphQL types | proven projections | Reuse before regenerate |
The crawler is incremental. Every run produces a catalog version; a diff between versions is the drift record (§9.3).
4.2 The catalog card
Every table gets a card: structured metadata plus a short natural-language description generated once and reviewed by the data owner before it becomes retrievable. Column entries are nested. Join-path documents are separate cards, because use cases are subgraphs and the join is the thing the model gets wrong most.
{
"kind": "table",
"id": "sales.sales_order_header",
"catalogVersion": "2026-09-12T02:00Z#417",
"domain": "sales",
"description": "One row per customer order. Header-level totals, status, ship/bill addresses. Lines in sales.sales_order_detail.",
"owner": "team-order-mgmt",
"rowCountApprox": 31465,
"classification": ["financial"],
"entitlement": { "readRoles": ["sales.reader","finance.reader"], "rowScope": "territory_id IN caller.territories" },
"columns": [
{ "name": "sales_order_id", "type": "int", "pk": true },
{ "name": "customer_id", "type": "int", "fk": "sales.customer.customer_id" },
{ "name": "status", "type": "tinyint", "enumLike": { "1":"In process","2":"Approved","5":"Shipped","6":"Cancelled" } },
{ "name": "total_due", "type": "money", "classification": ["financial"] },
{ "name": "ship_date", "type": "datetime", "nullable": true, "profile": { "nullRate": 0.04 } }
],
"joins": [
{ "to": "sales.sales_order_detail", "on": "sales_order_id", "cardinality": "1:N", "source": "fk" },
{ "to": "sales.customer", "on": "customer_id", "cardinality": "N:1", "source": "fk" },
{ "to": "person.address", "via": "ship_to_address_id", "cardinality": "N:1", "source": "fk+querylog", "frequency": 0.81 }
],
"glossary": ["order","sales order","SO"],
"synonyms": ["orders","order header"]
}
4.3 How retrieval works
Retrieval returns a subgraph, not a list of tables.
- Hybrid search over cards: BM25 (glossary terms, exact column and table names — identifiers must not be embedded away) fused with vector similarity over descriptions, then re-ranked. Metadata filters narrow by domain, classification, and — critically — by the requester’s entitlements, so a use case can only be composed from tables its intended actors can read.
- Graph expansion: from the top-k tables, walk FK and query-log join edges 1–2 hops, weighting mined paths above inferred ones. This is where
sales_order_headerpulls insales_order_detaileven if the request never said “line items.” - Pruning: drop tables whose only connection is a generic dimension (dates, audit columns) unless the spec asks for them; cap at a subgraph budget (default 30 tables).
- Cite-or-refuse: the Scout’s output must reference catalog card ids and join ids. A table the catalog doesn’t know cannot appear. If the subgraph is ambiguous (two candidate “customer” tables with similar scores), the Scout returns the ambiguity to the supervisor, which routes it to intake as a clarifying question — it does not guess.
The semantic catalog is the only thing the generation agents see about the schema. That is the whole scaling story: a 3,000-table enterprise and a 30,000-table enterprise cost the same per use case, because per use case the model sees a subgraph of thirty.
5. The use-case spec — the contract that starts everything
The intake agent’s job is to refuse to start work on an ambiguous request. It converts prose into a typed spec, asks when it must, and hands the supervisor something a gate can check.
{
"useCaseId": "cs-account-risk-360",
"title": "Account risk 360 for Customer Success",
"archetype": "detail-360-with-actions",
"actors": ["role:cs.manager", "role:cs.analyst"],
"questions": [
"Which accounts have overdue invoices AND open high-priority support tickets?",
"For one account: contacts, overdue balance by age bucket, open tickets, last three payments"
],
"entities": ["account", "contact", "invoice", "payment", "support ticket"],
"actions": [
{ "name": "proposeCreditHold", "kind": "write", "target": "finance.credit_hold", "requiresApproval": true }
],
"kpis": ["overdue balance", "ticket age"],
"constraints": { "rowScope": "caller's territory", "excludeClassifications": ["pii.ssn"] },
"nonGoals": ["editing invoices", "closing tickets"]
}
Archetypes matter for scale. Roughly a dozen shapes cover most of a thousand use cases: list-with-filters, detail-360, approval-queue, exception-list, reconciliation, dashboard, wizard-form, timeline, matrix/pivot, map-list, bulk-action, and audit-trail. An archetype fixes the UI skeleton and the query shapes, so the composers fill in a known structure instead of inventing one.
6. Agentic generation
6.1 The supervisor is deliberately not an agent
The supervisor is a deterministic state machine (XState or a plain TypeScript reducer). It holds the run’s state, enforces budgets (steps, tokens, wall-clock, cost), decides the order of gates, and enforces the read/write split by construction: the state that publishes a write path is unreachable without an approval event. An LLM cannot talk it into skipping a gate, because it doesn’t take instructions — it takes typed events.
INTAKE ──spec.valid──▶ SCOUT ──subgraph.cited──▶ COMPOSE_CONTRACT ──slice.parsed──▶ COMPOSE_UI
▲ │ │ │
│ spec.ambiguous │ subgraph.ambiguous │ slice.invalid (retry ≤2) │ spec.invalid (retry ≤2)
└─────────────────────┘ ▼ ▼
VERIFY ◀────────────────────────────┘
│ gates.pass │ gates.fail → back to the failing composer (retry ≤2) else PARK
▼
REVIEW ──approved──▶ PUBLISH_READ ──(if actions)──▶ PUBLISH_WRITE (requires approval event on the write path itself)
│ rejected
▼
PARKED (with reviewer notes → next run's context)
6.2 The agents and their tools
Each agent is an LLM call with a system prompt, a typed output schema, and a small allow-list of tools. No agent has a general-purpose code or SQL execution tool.
| Agent | Input | Tools | Output (typed, validated) |
|---|---|---|---|
| Intake | prose request, actor context, archetype library | askClarifying, lookupGlossary | Use-Case Spec |
| Schema Scout | spec, catalog version | catalog.search, catalog.expandJoins, catalog.getCard | Subgraph: table ids, join ids, per-table rationale with card citations, confidence |
| Contract Composer | spec, subgraph, policy catalog slice, archetype query templates | sdl.parse, sdl.compose (dry run), policy.lookup | GraphQL schema slice (SDL) + operation set (persisted documents) |
| UI Composer | spec, schema slice, operation set, component registry manifest | spec.validate, registry.list, requestCustomComponent | UI Spec (JSON) + optional custom-component requests |
| Verifier (mostly deterministic; one LLM pass for rationale) | all artifacts, sample data sandbox | gate runners | Gate report |
Prompt templates, model deployment, and the archetype library are versioned and form part of the release unit.
6.3 The GraphQL slice the composer emits
The composer produces SDL, never resolvers. Policies come from the policy catalog; the composer’s job is to apply them to every classified field, and the verifier’s job is to prove it did.
extend schema @link(url: "https://specs.apollo.dev/federation/v2.7", import: ["@key", "@shareable"])
"""Use case cs-account-risk-360 · catalog #417 · slice v1"""
type Account @key(fields: "accountId") {
accountId: ID!
name: String!
territoryId: ID!
overdueBalance(asOf: Date): Money @authorize(policy: "finance:read") @cost(weight: 5)
overdueByAgeBucket: [AgeBucket!]! @authorize(policy: "finance:read") @cost(weight: 10)
contacts(first: Int = 20, after: String): ContactConnection! @cost(weight: 3, multipliers: ["first"])
openTickets(priority: [TicketPriority!]): [SupportTicket!]! @authorize(policy: "support:read")
recentPayments(last: Int = 3): [Payment!]! @authorize(policy: "finance:read")
creditHold: CreditHold @authorize(policy: "finance:read")
}
type Contact {
contactId: ID!
fullName: String!
email: String @authorize(policy: "pii:contact") # nullable on purpose: a denied field nulls itself, not its parent
phone: String @authorize(policy: "pii:contact")
}
type Query {
accountsAtRisk(filter: AccountRiskFilter!, first: Int = 50, after: String): AccountConnection!
@authorize(policy: "cs:read") @cost(weight: 20, multipliers: ["first"])
account(accountId: ID!): Account @authorize(policy: "cs:read")
}
type Mutation {
"""Proposal only. Creates finance.credit_hold_proposal; a finance approver promotes it."""
proposeCreditHold(input: ProposeCreditHoldInput!): Proposal! @authorize(policy: "cs:propose") @proposal(target: "finance.credit_hold")
}
Three things the verifier checks on this text: every field mapped to a classified column carries @authorize; protected fields are nullable; every operation the UI will use is in the persisted set and under the cost ceiling. Two things the composer cannot do: emit a mutation without @proposal unless the policy catalog marks the target autonomous-safe, and reference a table not in the cited subgraph.
6.4 The UI spec the composer emits
The UI is a spec, not code. The runtime renders it. Every data binding names a persisted operation, so the screen can only ever run queries the gates already passed.
{
"specVersion": "1.2",
"useCaseId": "cs-account-risk-360",
"archetype": "detail-360-with-actions",
"routes": [
{ "path": "/cs/accounts-at-risk", "page": "AccountsAtRisk" },
{ "path": "/cs/accounts/:accountId", "page": "AccountRisk360" }
],
"pages": {
"AccountsAtRisk": {
"layout": "filter-bar+grid",
"filters": [
{ "field": "minOverdueDays", "control": "number", "default": 30 },
{ "field": "ticketPriority", "control": "multiselect", "optionsFrom": "enum:TicketPriority" }
],
"grid": {
"operation": "op:accountsAtRisk@v1",
"columns": [
{ "path": "name", "label": "Account", "link": "/cs/accounts/{accountId}" },
{ "path": "overdueBalance", "label": "Overdue", "format": "money", "sort": true },
{ "path": "openTickets.length", "label": "Open tickets" }
],
"pageSize": 50
}
},
"AccountRisk360": {
"layout": "header+tabs",
"header": { "operation": "op:account@v1", "fields": ["name", "territoryId", "creditHold.status"] },
"tabs": [
{ "title": "Overdue", "component": "AgeBucketChart", "operation": "op:account@v1", "path": "overdueByAgeBucket" },
{ "title": "Tickets", "component": "DataGrid", "operation": "op:account@v1", "path": "openTickets" },
{ "title": "Contacts", "component": "DataGrid", "operation": "op:account@v1", "path": "contacts" }
],
"actions": [
{ "label": "Propose credit hold", "operation": "op:proposeCreditHold@v1", "kind": "proposal",
"form": [{ "field": "reason", "control": "textarea", "required": true }],
"confirm": "This creates a proposal for Finance approval. It does not place a hold." }
]
}
},
"customComponents": []
}
The DSL is intentionally small. A dozen components, four layouts, typed controls, and bindings by path. Anything it cannot express goes through the escape hatch.
6.5 Hybrid UI — the escape hatch
When the UI Composer needs something the DSL doesn’t have (a Gantt view, a custom map overlay, a domain-specific widget), it does not write it inline. It emits a custom-component request: the component’s name, props contract, the operations it may bind to, and a rationale. A separate generation step writes a React component against that contract into an isolated package, which is:
- compiled in a sandbox with a locked dependency allow-list;
- linted and security-scanned (no network calls except the platform’s GraphQL client; no
dangerouslySetInnerHTML; no dynamic imports outside the registry); - rendered against sample data and screenshot-diffed for the reviewer;
- published to the component registry as a versioned federated remote (Module Federation) only after human review.
The UI spec then references it by registry id and version. The escape hatch produces code, but it produces it once, behind review, into a registry every future spec can reuse — not per screen, forever, unreviewed. The share of screens that need it is the metric to watch: if it climbs, the DSL is missing a component, and the fix is to add one.
7. Verification gates
Gates are deterministic wherever possible. The LLM gets one pass at the end to write the reviewer’s summary, and nothing it says can flip a gate.
| Gate | What it checks | What it catches |
|---|---|---|
| Composition | Slice composes into the supergraph without conflicts; no breaking change to published operations | Type collisions across use cases; silent breaking of an existing screen |
| Policy coverage (100%) | Every field mapped to a classified column carries @authorize; protected fields nullable; deny-by-default holds for the whole slice | The field that leaks because “something else” protects it |
| Entitlement compile | Every operation compiles with row-scope predicates for every actor role in the spec | A role for which the compiler can’t produce a scope — meaning the query would run unscoped |
| Cost & complexity | Every persisted operation under the ceiling with max page sizes; N+1 paths batched | The screen that works on sample data and times out on 30M rows |
| Binding validity | Every UI binding resolves to a field in the slice; every action to a persisted operation; every enum to a schema enum | Screens that render blank; buttons wired to nothing |
| Golden tests | Archetype-derived test operations run against the sample sandbox; shapes and cardinalities as expected; identifier queries return the identifier’s row | The wrong join path — the classic RAG-over-schema failure |
| PII exposure scan | Sample results scanned for classified values in fields not marked as such | Classification gaps in the catalog itself |
| Accessibility lint | UI spec: labels, contrast tokens, keyboard order, table semantics | The screen that ships and fails the audit six months later |
| Groundedness | Every table and join in the artifacts cites a catalog card | Hallucinated tables and columns |
A failed gate returns the run to the responsible composer with the report attached, at most twice. Then it parks for a human. Gates never auto-override.
8. Human review and publish
The reviewer sees one screen: the use-case spec, the subgraph (with rationale per table), the SDL diff against the supergraph, the policy coverage report, the gate report, and a live preview of the UI against sample data. Approve publishes; reject parks with notes that seed the next run.
Two approvals are structurally required and cannot be configured away: the first publish of a use case, and any write path. Subsequent read-only re-publishes (a new filter, a column) can be auto-published after gates if the use case’s registration allows it — that is a decision the owning team makes once, in the registration, and the platform enforces.
9. Runtime
9.1 One graph: registry and gateway
Approved slices are subgraphs of one supergraph. The schema registry (GraphQL Hive or Apollo Studio-equivalent, self-hostable) versions every slice, runs composition checks in CI, and serves the supergraph SDL to the gateway (Hive Gateway / Apollo Router-class). The gateway accepts persisted documents only — the UI runtime sends operation ids, never query text — so the attack surface at the edge is the allow-list, not the language.
Edge concerns (TLS, JWT validation, rate limiting, WAF) live at the API gateway in front. Business authorization does not: it lives in the schema, where there is one declaration to be right.
9.2 The query compiler: GraphQL → SQL with entitlements inside
The compiler is the deterministic heart of the runtime. Given a persisted operation, the caller’s validated claims, and the slice’s mapping to the catalog, it produces SQL (via a query builder such as Kysely) in which:
- row-scope predicates from the entitlement metadata are composed into every table access (
WHERE territory_id = ANY($callerTerritories)), before execution — never as a post-filter; - column-level policy is enforced by resolving denied fields to
nullin the projection (nullable by construction), never by selecting-then-redacting; - batching collapses nested selections into keyed batch queries (DataLoader semantics), so a list of 50 accounts with tickets is two queries, not fifty-one;
- cost is checked against the persisted operation’s precomputed weight with the actual argument values.
Because the compiler is deterministic and the operations are persisted, the same operation with the same claims produces the same SQL. That is what makes the runtime cacheable, explainable to an auditor, and testable in CI.
9.3 Drift and the release unit
The crawler re-runs on a schedule and on DDL events. A new catalog version diffs against the last; any use case whose subgraph touches a changed table is re-verified automatically (gates only) and, if a gate fails, taken to degraded — its screen shows a banner, its owner is paged, and its slice is pinned to the last passing version. A renamed column is a release, not a 3 a.m. surprise.
Every published artifact records its full release unit: catalog version, slice version, UI spec version, policy version, model deployment, prompt template versions, component registry versions. That tuple is what the reviewer approved, and it is what the runtime serves.
9.4 The React runtime
A single React application (Vite, React, TanStack Query/Table, a GraphQL client with persisted-operation support) that:
- loads a UI spec by use case id and version;
- renders it through a component registry (DataGrid, DetailPanel, Form, FilterBar, KPI, Chart, Timeline, Tabs, Wizard, Map — each schema-aware, each accessible by default);
- binds data through the persisted operations the spec names, with the caller’s token — the runtime cannot construct a query the gates didn’t pass;
- renders actions as proposals with status (
proposed → approved/rejected → applied), never as direct writes; - loads reviewed custom components as federated remotes by registry id and version.
Every screen is therefore uniform in security posture, telemetry, and accessibility — and bespoke only where a reviewed component made it so.
10. Scaling: what actually changes at thousands of tables and use cases
| Dimension | Mechanism | Why it holds |
|---|---|---|
| Schema size | Catalog is an index; per use case the model sees ≤30 tables | Cost per use case is independent of schema size |
| Retrieval precision | Hybrid search + FK/query-log graph expansion + entitlement pre-filter + rerank | Identifiers via BM25, meaning via vectors, joins via the graph, and no table the actor can’t read |
| Use-case volume | Archetypes fix structure; composers fill slots; ~80% of screens need no escape hatch | Generation is minutes, review is the bottleneck by design |
| Supergraph size | Namespaced slices; composition checks incremental per slice; unused slices retire on a schedule | 1,000 slices compose in CI in seconds if each is small and namespaced |
| Runtime load | Persisted ops + compiled SQL + batching + response caching keyed by (operation, args, entitlement fingerprint, catalog generation) | The cache cannot serve a result to someone not entitled to it, and invalidates on drift |
| Governance load | Gates deterministic; reviewers see diffs and previews, not code | A reviewer can approve ten use cases an hour |
| Drift | Catalog versioning + automatic re-verification + degraded state | The blast radius of a schema change is computed, not discovered |
The honest bottleneck is human review — and that is intentional. The architecture makes the machine work cheap so that the scarce resource, judgment, is spent on approving contracts rather than writing them.
11. The reference implementation (next phase)
Stack — TypeScript end-to-end
| Concern | Choice | Notes |
|---|---|---|
| Systems of record | PostgreSQL (containers) | Scaled public sample (below); SQL Server adapter later |
| Catalog store | PostgreSQL + pgvector + full-text (BM25-style ranking) + adjacency tables for the join graph | Local-first; Azure AI Search adapter for enterprise |
| Embeddings / LLM | Provider-agnostic client (Azure OpenAI, Anthropic Claude) behind one interface; versions pinned | Model is part of the release unit |
| Agents | Plain TypeScript agents with typed outputs (Zod) + a deterministic supervisor (XState) | No agent framework needed; the state machine is the framework |
| GraphQL | Federation-compatible subgraphs (GraphQL Yoga), schema registry + gateway (GraphQL Hive, self-hosted), persisted documents | Composition checks in CI |
| Query compiler | Custom, on Kysely | Entitlement predicates injected at compile time |
| UI runtime | Vite + React + TanStack Query/Table + urql (persisted ops) + Module Federation for custom components | One app, many specs |
| Sandbox | Isolated worker for component compile/lint/screenshot; sample-data Postgres for golden tests | Generation never touches production data |
| Infra | Docker Compose locally; Azure Container Apps + Postgres Flexible Server + APIM for cloud |
Scaled public sample database. Start from a public sample schema with realistic FK topology (an AdventureWorks/Northwind-class sample, ~70–90 tables across sales, purchasing, production, HR, person). Scale it programmatically with a generator that (a) adds domain modules from templates — finance/AR, support, CRM, fleet & assets, logistics, HR extensions, marketing, compliance — each with 20–60 tables and consistent FK links back to the base entities, (b) adds regional/tenant variants of transactional modules, and (c) injects the realistic pathologies the architecture must survive: sparse comments, ambiguous names (status, type, ref), duplicate concepts (crm.account vs billing.customer), legacy tables with no FKs, and mixed classifications. Target ≥3,000 tables with synthetic-but-consistent data volumes (tens of millions of rows in the hot tables) so cost gates and batching are exercised for real. All of it is synthetic; none of it is anyone’s production data.
Milestones
| # | Milestone | Proves |
|---|---|---|
| M0 | Scaled sample DB + crawler + catalog cards + hybrid search + join graph | Retrieval returns correct subgraphs for 50 seeded requests (precision/recall measured) |
| M1 | Intake + Scout + supervisor; cite-or-refuse; ambiguity round-trips | No hallucinated tables across the seeded set |
| M2 | Contract Composer + composition + policy-coverage gate + query compiler with entitlements | Every operation compiles scoped for every actor role; zero unscoped SQL |
| M3 | UI Composer + spec runtime + binding gate + preview | First end-to-end screen from a prose request |
| M4 | Golden tests, cost gate, PII scan, review UI, publish, proposals for writes | A reviewer approves a use case with a write path end to end |
| M5 | Drift: catalog re-version → auto re-verify → degraded state; escape-hatch component pipeline | A renamed column produces a computed blast radius and a pinned fallback |
| M6 | 100 seeded use cases across archetypes; metrics: minutes per use case, escape-hatch rate, gate failure taxonomy | The scaling claims in §10, measured |
12. Honest limits
- Retrieval will pick the wrong join sometimes. The graph, the query-log mining, and the golden tests reduce it; review catches the rest. The design assumes retrieval is fallible and puts a gate after it, rather than assuming a better model fixes it.
- Semantically ambiguous schemas stay ambiguous. If the organization has never decided whether
crm.accountorbilling.customeris the customer, the Scout will surface the ambiguity every time — which is the correct behaviour, and also a backlog item for the data owners, not the platform. - Spec-driven UI is uniform, not bespoke. That is a feature for 800 of 1,000 screens and a limitation for the rest; the escape hatch exists, and its usage rate is the signal that the DSL needs to grow.
- The compiler is a bounded query language. Deliberately. High-volume aggregation belongs in the warehouse; complex transactional logic belongs in a coded service exposed as its own subgraph. The platform composes those in; it does not replace them.
- Catalog quality bounds everything. Sparse comments and absent classification degrade retrieval and, worse, policy generation. The crawler’s profiling and the owner-review step on descriptions are there to raise the floor, but a catalog with no classification cannot produce deny-by-default policies for fields it doesn’t know are sensitive — so classification coverage is a prerequisite gate for a domain to be enabled at all.
- This is a reference architecture. The numbers in this document are design targets for the reference implementation, not measurements from a production system.
13. Security and governance, summarised
- Entitlements are compiled into the query from validated claims. Never post-filtered.
- Classified fields carry a policy on the schema, nullable, deny-by-default, verified at 100% before publish, readable from an exported schema file in the repo.
- The UI can only run persisted operations the gates passed. Query text never crosses the edge.
- Writes are proposals; approval is enforced by the platform from the registration.
- The model never sees production data during generation; it sees catalog cards and sample data.
- Every published artifact records its full release unit; drift triggers re-verification, not silent change.
- Every generation run is traced end to end — request, retrieval, subgraph, artifacts, gate reports, reviewer decision — so a bad screen can be replayed and turned into a permanent golden case.
Related public write-ups: Making enterprise RAG a platform, not a project · Collapsing a fragmented REST surface into one GraphQL gateway · Module Federation shell · Agentic AI in healthcare