Reference architecture · Agentic RAG

Schema-to-Screen. An agentic, RAG-grounded architecture for GraphQL and UI over thousands of tables.

The written reference architecture behind the interactive walkthrough. An enterprise with 3,000 tables does not have a data problem, it has a surface problem — every business question needs a screen, every screen an API, and every API someone who knows which seven tables matter and who may see them. This is the design that makes that surface generative without letting a model near free-form SQL, and the honest account of what it does not yet prove.

Subgraph
what the model sees — never the 3,000-table schema
Compiled
entitlements in the query, not filtered from the results
v1
reference architecture — design targets, not measurements

Status: reference architecture and build plan. The reference implementation — TypeScript end to end, a scaled public sample database, a hybrid UI — is the next phase and is specified in §11. Nothing here claims a production deployment of this design. The same architecture is available as an interactive walkthrough, and its full TOGAF ADM set is published alongside it.

Status: reference architecture and build plan. The reference implementation (TypeScript end-to-end, scaled public sample database, hybrid UI) is the next phase and is specified in §11. Nothing in this document claims a production deployment of this design.

0. The thesis in one paragraph

An enterprise with 3,000 tables does not have a data problem. It has a surface problem: every business question needs a screen, every screen needs an API, and every API needs someone to know which seven of the 3,000 tables matter and who is allowed to see them. Schema-to-Screen makes that surface generative — a request becomes a typed use-case spec, a RAG-grounded agent retrieves the relevant subgraph of tables (never the whole schema), a composer emits a GraphQL contract slice and a UI spec, deterministic gates verify both, a human approves, and the platform publishes. The model never writes free-form SQL, never sees 3,000 tables, and never produces code that runs unreviewed. Authorization is declared once — on the field, in the schema — and compiled into the query. Everything it produces is a contract behind a gate.

1. The problem, stated honestly

Three things are true at once in most large enterprises:

  1. The schema is too big for a prompt. 3,000 tables × ~15 columns is ~45,000 columns. Even at a few tokens each, the schema alone is hundreds of thousands of tokens — before comments, constraints, or sample values. Any design that “puts the schema in the context” has already failed at enterprise scale; it just hasn’t noticed yet because the demo used 20 tables.
  2. Use cases outnumber engineers by two orders of magnitude. A thousand legitimate business screens — approval queues, 360 views, exception lists, reconciliations, dashboards — sit in a backlog no team will ever clear. Each one is a small, well-understood shape over a small subset of tables. The work is not hard. There is just too much of it.
  3. Text-to-SQL is the wrong abstraction for this. Free-form generated SQL is unreviewable at scale, bypasses every authorization model the organization has, and produces a different answer every time. The failure mode isn’t “the model wrote bad SQL” — it’s “nobody can say what this screen is allowed to show.”

The naive fixes each fail in a specific way:

Naive approachWhy it fails at 3,000 tables / 1,000 use cases
Stuff the schema into the promptContext limits, cost, and the model picks the wrong customer_id from twelve candidates
Text-to-SQL per requestUnreviewable, un-cacheable, authorization bypassed, non-deterministic
Generate a React app per use case1,000 codebases to secure, upgrade, and audit; drift within a quarter
One generic data grid over every tableNobody’s actual use case; entitlements still handled after the fact
Post-filter results by permissionRecall collapses for narrow-permission users; a control every workflow has to remember

Schema-to-Screen is what’s left after removing each of those.

2. Six principles the architecture is built to enforce

  1. The model sees a subgraph, never the schema. Retrieval over a semantic catalog selects the 5–30 tables a use case needs, with join paths. The whole schema is a search index, not a prompt.
  2. Everything generated is a contract, not code. Agents emit a GraphQL schema slice and a UI spec (a JSON DSL). Resolvers are compiled from the slice by a deterministic query compiler; screens are rendered from the spec by a runtime. Code generation exists only as a reviewed escape hatch (§7.4).
  3. Authorization is declared once, on the field, and compiled into the query. Classified fields carry a policy directive in the schema slice; the compiler injects row- and column-level predicates from the caller’s validated identity before the SQL executes. Post-filtering is a control you have to remember. A query filter is a control you cannot forget.
  4. Read and write are different products. Read slices can be published autonomously after gates. Anything that writes to a system of record produces a proposal mutation, and the proposal’s approval workflow is enforced by the platform from the use case’s registration — not implemented per use case.
  5. The release unit is the whole tuple. Catalog version + schema slice + UI spec + policy version + model + prompt template version. Drift in any element is a release, not a surprise.
  6. A human approves the first publish of every use case and every write path. Gates catch what can be measured. Review catches what can’t — and it is where the organization’s judgment enters the system.

3. Architecture at a glance

 Business request
               │
               ▼
         ┌─────────────┐     ┌──────────────────────────────────────────────┐
         │ Intake agent│────▶│  DETERMINISTIC SUPERVISOR (state machine)     │
         │ → Use-Case  │     │  budgets · gate order · read/write split      │
         │   Spec      │     └───┬───────────────┬───────────────┬──────────┘
         └─────────────┘         │               │               │
                                 ▼               ▼               ▼
                           ┌───────────┐   ┌────────────┐   ┌────────────┐
           SEMANTIC        │ Schema    │   │ Contract   │   │ UI         │
           CATALOG ───────▶│ Scout     │──▶│ Composer   │──▶│ Composer   │
           (hybrid index   │ (RAG)     │   │ → GraphQL  │   │ → UI Spec  │
            + FK graph)    └───────────┘   │   slice    │   │ (+ custom  │
                                           └─────┬──────┘   │  component │
           POLICY ─────────────────────────────▶ │          │  request)  │
           CATALOG                               │          └─────┬──────┘
           (classification,                      ▼                ▼
            entitlements)                  ┌──────────────────────────────┐
                │                          │  VERIFIER GATES              │
                │                          │  composition · policy 100%   │
                │                          │  cost · bindings · golden    │
                │                          │  tests · PII scan            │
                │                          └──────────────┬───────────────┘
                │                                         ▼
                │                          ┌──────────────────────────────┐
                │                          │  HUMAN REVIEW → PUBLISH      │
                │                          └───────┬──────────────┬───────┘
                │                                  ▼              ▼
                │                     ┌───────────────────┐  ┌──────────────────┐
                │                     │ Schema registry + │◀─│ React spec       │
                │                     │ supergraph gateway│  │ runtime +        │
                │                     └─────────┬─────────┘  │ component        │
                │                               ▼            │ registry         │
                └───────────────────▶ ┌───────────────────┐  └──────────────────┘
                                      │ Query compiler    │
                                      │ GraphQL → SQL +   │
                                      │ entitlement       │
                                      │ predicates        │
                                      └─────────┬─────────┘
                                                ▼
                                      ┌───────────────────┐
                                      │ Systems of record │
                                      │ 3,000+ tables     │──── crawler ──▶ SEMANTIC CATALOG
                                      └───────────────────┘

Five zones: intake, semantic catalog (the RAG substrate), agentic generation, verification and review, runtime. The runtime is the only zone that touches production data at request time; the generation zone touches sample data only.

4. The semantic catalog — what makes 3,000 tables tractable

The catalog is the platform’s RAG corpus, and it is built about the schema, not from the data.

4.1 What the crawler ingests

SourceWhat it yieldsWhy it matters for retrieval
information_schema / catalog viewstables, columns, types, nullability, PK/FK/unique/check constraints, indexesThe structural truth; FK edges are the join graph
Table and column commentshuman descriptions where they existHighest-quality semantic signal — usually sparse
Data profiles (sampled, PII-aware)row counts, cardinality, null rate, value ranges, top-k values for low-cardinality columnsDisambiguates status columns; identifies enum-like fields; flags identifiers
Query-log miningfrequent join paths, filter columns, aggregations actually usedReal join paths beat FK-inferred ones; reveals “the way the business actually reads this”
Business glossary / data dictionaryterms → tables/columns; synonyms; ownersMaps “customer” to crm.account not billing.debtor
Data classificationPII / PHI / financial / confidential per column; retentionDrives deny-by-default policy generation
Entitlement metadatawhich roles/groups may read which schemas, tables, columns, rows (by tenant/region/owner)Compiled into the query, never applied after
Existing reports / views / GraphQL typesproven projectionsReuse before regenerate

The crawler is incremental. Every run produces a catalog version; a diff between versions is the drift record (§9.3).

4.2 The catalog card

Every table gets a card: structured metadata plus a short natural-language description generated once and reviewed by the data owner before it becomes retrievable. Column entries are nested. Join-path documents are separate cards, because use cases are subgraphs and the join is the thing the model gets wrong most.

{
          "kind": "table",
          "id": "sales.sales_order_header",
          "catalogVersion": "2026-09-12T02:00Z#417",
          "domain": "sales",
          "description": "One row per customer order. Header-level totals, status, ship/bill addresses. Lines in sales.sales_order_detail.",
          "owner": "team-order-mgmt",
          "rowCountApprox": 31465,
          "classification": ["financial"],
          "entitlement": { "readRoles": ["sales.reader","finance.reader"], "rowScope": "territory_id IN caller.territories" },
          "columns": [
            { "name": "sales_order_id", "type": "int", "pk": true },
            { "name": "customer_id", "type": "int", "fk": "sales.customer.customer_id" },
            { "name": "status", "type": "tinyint", "enumLike": { "1":"In process","2":"Approved","5":"Shipped","6":"Cancelled" } },
            { "name": "total_due", "type": "money", "classification": ["financial"] },
            { "name": "ship_date", "type": "datetime", "nullable": true, "profile": { "nullRate": 0.04 } }
          ],
          "joins": [
            { "to": "sales.sales_order_detail", "on": "sales_order_id", "cardinality": "1:N", "source": "fk" },
            { "to": "sales.customer", "on": "customer_id", "cardinality": "N:1", "source": "fk" },
            { "to": "person.address", "via": "ship_to_address_id", "cardinality": "N:1", "source": "fk+querylog", "frequency": 0.81 }
          ],
          "glossary": ["order","sales order","SO"],
          "synonyms": ["orders","order header"]
        }

4.3 How retrieval works

Retrieval returns a subgraph, not a list of tables.

  1. Hybrid search over cards: BM25 (glossary terms, exact column and table names — identifiers must not be embedded away) fused with vector similarity over descriptions, then re-ranked. Metadata filters narrow by domain, classification, and — critically — by the requester’s entitlements, so a use case can only be composed from tables its intended actors can read.
  2. Graph expansion: from the top-k tables, walk FK and query-log join edges 1–2 hops, weighting mined paths above inferred ones. This is where sales_order_header pulls in sales_order_detail even if the request never said “line items.”
  3. Pruning: drop tables whose only connection is a generic dimension (dates, audit columns) unless the spec asks for them; cap at a subgraph budget (default 30 tables).
  4. Cite-or-refuse: the Scout’s output must reference catalog card ids and join ids. A table the catalog doesn’t know cannot appear. If the subgraph is ambiguous (two candidate “customer” tables with similar scores), the Scout returns the ambiguity to the supervisor, which routes it to intake as a clarifying question — it does not guess.

The semantic catalog is the only thing the generation agents see about the schema. That is the whole scaling story: a 3,000-table enterprise and a 30,000-table enterprise cost the same per use case, because per use case the model sees a subgraph of thirty.

5. The use-case spec — the contract that starts everything

The intake agent’s job is to refuse to start work on an ambiguous request. It converts prose into a typed spec, asks when it must, and hands the supervisor something a gate can check.

{
          "useCaseId": "cs-account-risk-360",
          "title": "Account risk 360 for Customer Success",
          "archetype": "detail-360-with-actions",
          "actors": ["role:cs.manager", "role:cs.analyst"],
          "questions": [
            "Which accounts have overdue invoices AND open high-priority support tickets?",
            "For one account: contacts, overdue balance by age bucket, open tickets, last three payments"
          ],
          "entities": ["account", "contact", "invoice", "payment", "support ticket"],
          "actions": [
            { "name": "proposeCreditHold", "kind": "write", "target": "finance.credit_hold", "requiresApproval": true }
          ],
          "kpis": ["overdue balance", "ticket age"],
          "constraints": { "rowScope": "caller's territory", "excludeClassifications": ["pii.ssn"] },
          "nonGoals": ["editing invoices", "closing tickets"]
        }

Archetypes matter for scale. Roughly a dozen shapes cover most of a thousand use cases: list-with-filters, detail-360, approval-queue, exception-list, reconciliation, dashboard, wizard-form, timeline, matrix/pivot, map-list, bulk-action, and audit-trail. An archetype fixes the UI skeleton and the query shapes, so the composers fill in a known structure instead of inventing one.

6. Agentic generation

6.1 The supervisor is deliberately not an agent

The supervisor is a deterministic state machine (XState or a plain TypeScript reducer). It holds the run’s state, enforces budgets (steps, tokens, wall-clock, cost), decides the order of gates, and enforces the read/write split by construction: the state that publishes a write path is unreachable without an approval event. An LLM cannot talk it into skipping a gate, because it doesn’t take instructions — it takes typed events.

INTAKE ──spec.valid──▶ SCOUT ──subgraph.cited──▶ COMPOSE_CONTRACT ──slice.parsed──▶ COMPOSE_UI
           ▲                     │                              │                               │
           │ spec.ambiguous      │ subgraph.ambiguous           │ slice.invalid (retry ≤2)      │ spec.invalid (retry ≤2)
           └─────────────────────┘                              ▼                               ▼
                                                           VERIFY ◀────────────────────────────┘
                                                             │ gates.pass                 │ gates.fail → back to the failing composer (retry ≤2) else PARK
                                                             ▼
                                                           REVIEW ──approved──▶ PUBLISH_READ ──(if actions)──▶ PUBLISH_WRITE (requires approval event on the write path itself)
                                                             │ rejected
                                                             ▼
                                                           PARKED (with reviewer notes → next run's context)

6.2 The agents and their tools

Each agent is an LLM call with a system prompt, a typed output schema, and a small allow-list of tools. No agent has a general-purpose code or SQL execution tool.

AgentInputToolsOutput (typed, validated)
Intakeprose request, actor context, archetype libraryaskClarifying, lookupGlossaryUse-Case Spec
Schema Scoutspec, catalog versioncatalog.search, catalog.expandJoins, catalog.getCardSubgraph: table ids, join ids, per-table rationale with card citations, confidence
Contract Composerspec, subgraph, policy catalog slice, archetype query templatessdl.parse, sdl.compose (dry run), policy.lookupGraphQL schema slice (SDL) + operation set (persisted documents)
UI Composerspec, schema slice, operation set, component registry manifestspec.validate, registry.list, requestCustomComponentUI Spec (JSON) + optional custom-component requests
Verifier (mostly deterministic; one LLM pass for rationale)all artifacts, sample data sandboxgate runnersGate report

Prompt templates, model deployment, and the archetype library are versioned and form part of the release unit.

6.3 The GraphQL slice the composer emits

The composer produces SDL, never resolvers. Policies come from the policy catalog; the composer’s job is to apply them to every classified field, and the verifier’s job is to prove it did.

extend schema @link(url: "https://specs.apollo.dev/federation/v2.7", import: ["@key", "@shareable"])
        
        """Use case cs-account-risk-360 · catalog #417 · slice v1"""
        type Account @key(fields: "accountId") {
          accountId: ID!
          name: String!
          territoryId: ID!
          overdueBalance(asOf: Date): Money @authorize(policy: "finance:read") @cost(weight: 5)
          overdueByAgeBucket: [AgeBucket!]! @authorize(policy: "finance:read") @cost(weight: 10)
          contacts(first: Int = 20, after: String): ContactConnection! @cost(weight: 3, multipliers: ["first"])
          openTickets(priority: [TicketPriority!]): [SupportTicket!]! @authorize(policy: "support:read")
          recentPayments(last: Int = 3): [Payment!]! @authorize(policy: "finance:read")
          creditHold: CreditHold @authorize(policy: "finance:read")
        }
        
        type Contact {
          contactId: ID!
          fullName: String!
          email: String @authorize(policy: "pii:contact")   # nullable on purpose: a denied field nulls itself, not its parent
          phone: String @authorize(policy: "pii:contact")
        }
        
        type Query {
          accountsAtRisk(filter: AccountRiskFilter!, first: Int = 50, after: String): AccountConnection!
            @authorize(policy: "cs:read") @cost(weight: 20, multipliers: ["first"])
          account(accountId: ID!): Account @authorize(policy: "cs:read")
        }
        
        type Mutation {
          """Proposal only. Creates finance.credit_hold_proposal; a finance approver promotes it."""
          proposeCreditHold(input: ProposeCreditHoldInput!): Proposal! @authorize(policy: "cs:propose") @proposal(target: "finance.credit_hold")
        }

Three things the verifier checks on this text: every field mapped to a classified column carries @authorize; protected fields are nullable; every operation the UI will use is in the persisted set and under the cost ceiling. Two things the composer cannot do: emit a mutation without @proposal unless the policy catalog marks the target autonomous-safe, and reference a table not in the cited subgraph.

6.4 The UI spec the composer emits

The UI is a spec, not code. The runtime renders it. Every data binding names a persisted operation, so the screen can only ever run queries the gates already passed.

{
          "specVersion": "1.2",
          "useCaseId": "cs-account-risk-360",
          "archetype": "detail-360-with-actions",
          "routes": [
            { "path": "/cs/accounts-at-risk", "page": "AccountsAtRisk" },
            { "path": "/cs/accounts/:accountId", "page": "AccountRisk360" }
          ],
          "pages": {
            "AccountsAtRisk": {
              "layout": "filter-bar+grid",
              "filters": [
                { "field": "minOverdueDays", "control": "number", "default": 30 },
                { "field": "ticketPriority", "control": "multiselect", "optionsFrom": "enum:TicketPriority" }
              ],
              "grid": {
                "operation": "op:accountsAtRisk@v1",
                "columns": [
                  { "path": "name", "label": "Account", "link": "/cs/accounts/{accountId}" },
                  { "path": "overdueBalance", "label": "Overdue", "format": "money", "sort": true },
                  { "path": "openTickets.length", "label": "Open tickets" }
                ],
                "pageSize": 50
              }
            },
            "AccountRisk360": {
              "layout": "header+tabs",
              "header": { "operation": "op:account@v1", "fields": ["name", "territoryId", "creditHold.status"] },
              "tabs": [
                { "title": "Overdue", "component": "AgeBucketChart", "operation": "op:account@v1", "path": "overdueByAgeBucket" },
                { "title": "Tickets", "component": "DataGrid", "operation": "op:account@v1", "path": "openTickets" },
                { "title": "Contacts", "component": "DataGrid", "operation": "op:account@v1", "path": "contacts" }
              ],
              "actions": [
                { "label": "Propose credit hold", "operation": "op:proposeCreditHold@v1", "kind": "proposal",
                  "form": [{ "field": "reason", "control": "textarea", "required": true }],
                  "confirm": "This creates a proposal for Finance approval. It does not place a hold." }
              ]
            }
          },
          "customComponents": []
        }

The DSL is intentionally small. A dozen components, four layouts, typed controls, and bindings by path. Anything it cannot express goes through the escape hatch.

6.5 Hybrid UI — the escape hatch

When the UI Composer needs something the DSL doesn’t have (a Gantt view, a custom map overlay, a domain-specific widget), it does not write it inline. It emits a custom-component request: the component’s name, props contract, the operations it may bind to, and a rationale. A separate generation step writes a React component against that contract into an isolated package, which is:

  • compiled in a sandbox with a locked dependency allow-list;
  • linted and security-scanned (no network calls except the platform’s GraphQL client; no dangerouslySetInnerHTML; no dynamic imports outside the registry);
  • rendered against sample data and screenshot-diffed for the reviewer;
  • published to the component registry as a versioned federated remote (Module Federation) only after human review.

The UI spec then references it by registry id and version. The escape hatch produces code, but it produces it once, behind review, into a registry every future spec can reuse — not per screen, forever, unreviewed. The share of screens that need it is the metric to watch: if it climbs, the DSL is missing a component, and the fix is to add one.

7. Verification gates

Gates are deterministic wherever possible. The LLM gets one pass at the end to write the reviewer’s summary, and nothing it says can flip a gate.

GateWhat it checksWhat it catches
CompositionSlice composes into the supergraph without conflicts; no breaking change to published operationsType collisions across use cases; silent breaking of an existing screen
Policy coverage (100%)Every field mapped to a classified column carries @authorize; protected fields nullable; deny-by-default holds for the whole sliceThe field that leaks because “something else” protects it
Entitlement compileEvery operation compiles with row-scope predicates for every actor role in the specA role for which the compiler can’t produce a scope — meaning the query would run unscoped
Cost & complexityEvery persisted operation under the ceiling with max page sizes; N+1 paths batchedThe screen that works on sample data and times out on 30M rows
Binding validityEvery UI binding resolves to a field in the slice; every action to a persisted operation; every enum to a schema enumScreens that render blank; buttons wired to nothing
Golden testsArchetype-derived test operations run against the sample sandbox; shapes and cardinalities as expected; identifier queries return the identifier’s rowThe wrong join path — the classic RAG-over-schema failure
PII exposure scanSample results scanned for classified values in fields not marked as suchClassification gaps in the catalog itself
Accessibility lintUI spec: labels, contrast tokens, keyboard order, table semanticsThe screen that ships and fails the audit six months later
GroundednessEvery table and join in the artifacts cites a catalog cardHallucinated tables and columns

A failed gate returns the run to the responsible composer with the report attached, at most twice. Then it parks for a human. Gates never auto-override.

8. Human review and publish

The reviewer sees one screen: the use-case spec, the subgraph (with rationale per table), the SDL diff against the supergraph, the policy coverage report, the gate report, and a live preview of the UI against sample data. Approve publishes; reject parks with notes that seed the next run.

Two approvals are structurally required and cannot be configured away: the first publish of a use case, and any write path. Subsequent read-only re-publishes (a new filter, a column) can be auto-published after gates if the use case’s registration allows it — that is a decision the owning team makes once, in the registration, and the platform enforces.

9. Runtime

9.1 One graph: registry and gateway

Approved slices are subgraphs of one supergraph. The schema registry (GraphQL Hive or Apollo Studio-equivalent, self-hostable) versions every slice, runs composition checks in CI, and serves the supergraph SDL to the gateway (Hive Gateway / Apollo Router-class). The gateway accepts persisted documents only — the UI runtime sends operation ids, never query text — so the attack surface at the edge is the allow-list, not the language.

Edge concerns (TLS, JWT validation, rate limiting, WAF) live at the API gateway in front. Business authorization does not: it lives in the schema, where there is one declaration to be right.

9.2 The query compiler: GraphQL → SQL with entitlements inside

The compiler is the deterministic heart of the runtime. Given a persisted operation, the caller’s validated claims, and the slice’s mapping to the catalog, it produces SQL (via a query builder such as Kysely) in which:

  • row-scope predicates from the entitlement metadata are composed into every table access (WHERE territory_id = ANY($callerTerritories)), before execution — never as a post-filter;
  • column-level policy is enforced by resolving denied fields to null in the projection (nullable by construction), never by selecting-then-redacting;
  • batching collapses nested selections into keyed batch queries (DataLoader semantics), so a list of 50 accounts with tickets is two queries, not fifty-one;
  • cost is checked against the persisted operation’s precomputed weight with the actual argument values.

Because the compiler is deterministic and the operations are persisted, the same operation with the same claims produces the same SQL. That is what makes the runtime cacheable, explainable to an auditor, and testable in CI.

9.3 Drift and the release unit

The crawler re-runs on a schedule and on DDL events. A new catalog version diffs against the last; any use case whose subgraph touches a changed table is re-verified automatically (gates only) and, if a gate fails, taken to degraded — its screen shows a banner, its owner is paged, and its slice is pinned to the last passing version. A renamed column is a release, not a 3 a.m. surprise.

Every published artifact records its full release unit: catalog version, slice version, UI spec version, policy version, model deployment, prompt template versions, component registry versions. That tuple is what the reviewer approved, and it is what the runtime serves.

9.4 The React runtime

A single React application (Vite, React, TanStack Query/Table, a GraphQL client with persisted-operation support) that:

  • loads a UI spec by use case id and version;
  • renders it through a component registry (DataGrid, DetailPanel, Form, FilterBar, KPI, Chart, Timeline, Tabs, Wizard, Map — each schema-aware, each accessible by default);
  • binds data through the persisted operations the spec names, with the caller’s token — the runtime cannot construct a query the gates didn’t pass;
  • renders actions as proposals with status (proposed → approved/rejected → applied), never as direct writes;
  • loads reviewed custom components as federated remotes by registry id and version.

Every screen is therefore uniform in security posture, telemetry, and accessibility — and bespoke only where a reviewed component made it so.

10. Scaling: what actually changes at thousands of tables and use cases

DimensionMechanismWhy it holds
Schema sizeCatalog is an index; per use case the model sees ≤30 tablesCost per use case is independent of schema size
Retrieval precisionHybrid search + FK/query-log graph expansion + entitlement pre-filter + rerankIdentifiers via BM25, meaning via vectors, joins via the graph, and no table the actor can’t read
Use-case volumeArchetypes fix structure; composers fill slots; ~80% of screens need no escape hatchGeneration is minutes, review is the bottleneck by design
Supergraph sizeNamespaced slices; composition checks incremental per slice; unused slices retire on a schedule1,000 slices compose in CI in seconds if each is small and namespaced
Runtime loadPersisted ops + compiled SQL + batching + response caching keyed by (operation, args, entitlement fingerprint, catalog generation)The cache cannot serve a result to someone not entitled to it, and invalidates on drift
Governance loadGates deterministic; reviewers see diffs and previews, not codeA reviewer can approve ten use cases an hour
DriftCatalog versioning + automatic re-verification + degraded stateThe blast radius of a schema change is computed, not discovered

The honest bottleneck is human review — and that is intentional. The architecture makes the machine work cheap so that the scarce resource, judgment, is spent on approving contracts rather than writing them.

11. The reference implementation (next phase)

Stack — TypeScript end-to-end

ConcernChoiceNotes
Systems of recordPostgreSQL (containers)Scaled public sample (below); SQL Server adapter later
Catalog storePostgreSQL + pgvector + full-text (BM25-style ranking) + adjacency tables for the join graphLocal-first; Azure AI Search adapter for enterprise
Embeddings / LLMProvider-agnostic client (Azure OpenAI, Anthropic Claude) behind one interface; versions pinnedModel is part of the release unit
AgentsPlain TypeScript agents with typed outputs (Zod) + a deterministic supervisor (XState)No agent framework needed; the state machine is the framework
GraphQLFederation-compatible subgraphs (GraphQL Yoga), schema registry + gateway (GraphQL Hive, self-hosted), persisted documentsComposition checks in CI
Query compilerCustom, on KyselyEntitlement predicates injected at compile time
UI runtimeVite + React + TanStack Query/Table + urql (persisted ops) + Module Federation for custom componentsOne app, many specs
SandboxIsolated worker for component compile/lint/screenshot; sample-data Postgres for golden testsGeneration never touches production data
InfraDocker Compose locally; Azure Container Apps + Postgres Flexible Server + APIM for cloud

Scaled public sample database. Start from a public sample schema with realistic FK topology (an AdventureWorks/Northwind-class sample, ~70–90 tables across sales, purchasing, production, HR, person). Scale it programmatically with a generator that (a) adds domain modules from templates — finance/AR, support, CRM, fleet & assets, logistics, HR extensions, marketing, compliance — each with 20–60 tables and consistent FK links back to the base entities, (b) adds regional/tenant variants of transactional modules, and (c) injects the realistic pathologies the architecture must survive: sparse comments, ambiguous names (status, type, ref), duplicate concepts (crm.account vs billing.customer), legacy tables with no FKs, and mixed classifications. Target ≥3,000 tables with synthetic-but-consistent data volumes (tens of millions of rows in the hot tables) so cost gates and batching are exercised for real. All of it is synthetic; none of it is anyone’s production data.

Milestones

#MilestoneProves
M0Scaled sample DB + crawler + catalog cards + hybrid search + join graphRetrieval returns correct subgraphs for 50 seeded requests (precision/recall measured)
M1Intake + Scout + supervisor; cite-or-refuse; ambiguity round-tripsNo hallucinated tables across the seeded set
M2Contract Composer + composition + policy-coverage gate + query compiler with entitlementsEvery operation compiles scoped for every actor role; zero unscoped SQL
M3UI Composer + spec runtime + binding gate + previewFirst end-to-end screen from a prose request
M4Golden tests, cost gate, PII scan, review UI, publish, proposals for writesA reviewer approves a use case with a write path end to end
M5Drift: catalog re-version → auto re-verify → degraded state; escape-hatch component pipelineA renamed column produces a computed blast radius and a pinned fallback
M6100 seeded use cases across archetypes; metrics: minutes per use case, escape-hatch rate, gate failure taxonomyThe scaling claims in §10, measured

12. Honest limits

  • Retrieval will pick the wrong join sometimes. The graph, the query-log mining, and the golden tests reduce it; review catches the rest. The design assumes retrieval is fallible and puts a gate after it, rather than assuming a better model fixes it.
  • Semantically ambiguous schemas stay ambiguous. If the organization has never decided whether crm.account or billing.customer is the customer, the Scout will surface the ambiguity every time — which is the correct behaviour, and also a backlog item for the data owners, not the platform.
  • Spec-driven UI is uniform, not bespoke. That is a feature for 800 of 1,000 screens and a limitation for the rest; the escape hatch exists, and its usage rate is the signal that the DSL needs to grow.
  • The compiler is a bounded query language. Deliberately. High-volume aggregation belongs in the warehouse; complex transactional logic belongs in a coded service exposed as its own subgraph. The platform composes those in; it does not replace them.
  • Catalog quality bounds everything. Sparse comments and absent classification degrade retrieval and, worse, policy generation. The crawler’s profiling and the owner-review step on descriptions are there to raise the floor, but a catalog with no classification cannot produce deny-by-default policies for fields it doesn’t know are sensitive — so classification coverage is a prerequisite gate for a domain to be enabled at all.
  • This is a reference architecture. The numbers in this document are design targets for the reference implementation, not measurements from a production system.

13. Security and governance, summarised

  • Entitlements are compiled into the query from validated claims. Never post-filtered.
  • Classified fields carry a policy on the schema, nullable, deny-by-default, verified at 100% before publish, readable from an exported schema file in the repo.
  • The UI can only run persisted operations the gates passed. Query text never crosses the edge.
  • Writes are proposals; approval is enforced by the platform from the registration.
  • The model never sees production data during generation; it sees catalog cards and sample data.
  • Every published artifact records its full release unit; drift triggers re-verification, not silent change.
  • Every generation run is traced end to end — request, retrieval, subgraph, artifacts, gate reports, reviewer decision — so a bad screen can be replayed and turned into a permanent golden case.

Related public write-ups: Making enterprise RAG a platform, not a project · Collapsing a fragmented REST surface into one GraphQL gateway · Module Federation shell · Agentic AI in healthcare