SiddharthThe $100 Million Problem solvable by AI: Semantic Virtualization of Legacy Core Banking
Siddharth
July 27, 202625 min read
Legacy core banking systems cost over $100 million to replace. A new agent-layer architecture can deliver the same multi-tenant, API-first capabilities for $15–20 million — without touching the core. Here's how the full stack works, diagrammed end to end.
Every bank on Earth runs on software built before most of its employees were born. The systems that hold your checking account, process your mortgage, and settle your wire transfers were architected in the 1970s through the 1990s — designed for batch processing, single-institution tenancy, and terminal-based green screens. They are the authoritative ledger for the vast majority of the world's banks, and they are not going anywhere.
When fintechs and embedded-finance providers want to offer banking products — debit cards, savings accounts, lending — they crash into these legacy cores like a speedboat hitting a seawall. The cores expose none of the multi-tenant, API-first, schema-flexible interfaces that modern consumers expect. Bridging that gap has historically cost between nine and thirty-two million dollars per integration, or well over one hundred million dollars for full core replacement.
This is the story of a fundamentally different approach: using meta-orchestrated AI agent layers to semantically virtualize legacy core capabilities — collapsing the integration cost to roughly fifteen to twenty million dollars, without migrating or modifying the core at all.
Before understanding the solution, you need to understand the tax. The integration tax is the total cost — in engineering effort, time, licensing, and operational overhead — of making a legacy core usable by external consumers that expect modern, multi-tenant, API-first interactions. It decomposes into four components:
Multi-tenancy virtualization ($5–15M) is the biggest line item. Legacy cores have no concept of tenant isolation — they treat every account and product as belonging to a single institution. Creating the illusion that multiple fintechs each have their own isolated banking experience, all sharing one core, is enormously expensive.
Protocol translation ($2–8M) covers the impedance mismatch between what cores expose (SOAP/WSDL endpoints, 3270 terminal screen-scrape interfaces, fixed-width batch files, proprietary message queues) and what modern applications expect (REST and GraphQL APIs returning JSON).
Product configuration ($1–5M/year) is the recurring cost of defining new financial products. Want to launch a new CD rate tier or a novel savings round-up feature? That requires core-level parameter changes managed by specialized administrators — a process measured in weeks, not minutes.
Schema reconciliation ($1–4M per tenant) is the cost of building and maintaining ETL pipelines that transform data from the core's rigid schema into each fintech's expected format. Every new tenant needs its own pipeline. Every schema change requires pipeline modification.
Banks have historically faced a binary choice, both painful.
Path 1: Full core replacement. Rip out the legacy system and migrate to a cloud-native core. This is a multi-year, multi-hundred-million-dollar migration — well over one hundred million dollars on average for a mid-to-large bank. It introduces enormous execution risk (core migrations have a notorious failure rate), years of parallel-run complexity, and the certitude that you are trading one vendor's lock-in for another's. Banks that attempt this often find the project consuming five to seven years before completion, if it completes at all.
Path 2: Middleware platform license. Build (or buy) a new platform layer that sits between the fintech and the core, adding abstraction, configuration, and orchestration. This avoids the nuclear option of core replacement, but introduces its own vendor lock-in, certification burden, and operational complexity — at a cost of nine to thirty-two million dollars and deployment timelines of twelve to thirty-six months.
The critical insight: neither approach fundamentally changes what the core can do. Both add a layer of abstraction, configuration, and orchestration on top. The core still settles on its batch cycle. The core still holds the authoritative ledger. The integration tax persists — it just gets repackaged.
The argument is straightforward: the specific job that BaaS platforms perform — interpreting, abstracting, and virtualizing legacy core capabilities for multi-tenant consumption — is now addressable by a different class of technology entirely. Meta-orchestrated large language model agent layers can:
This does not require core migration. It does not require a new platform license. It requires an agent layer with the right architecture, governance, and engineering discipline — at an estimated cost of fifteen to twenty million dollars, if not less.
Here is where it gets concrete. The architecture is formalized as a six-layer stack, each layer with a specific responsibility. The layering enforces a strict separation of concerns between deterministic banking operations (where every transaction must be exact, auditable, and reversible) and probabilistic orchestration (where AI handles interpretation, routing, and composition).
Requests flow top-down, from the tenant's API call down to the core. Responses flow bottom-up, through schema projection and guardrail verification. Let's walk through each layer.
The core stays exactly as it is. It continues to operate as the authoritative ledger, settling transactions on its native batch cycle and maintaining the regulatory audit record. The entire architecture is designed around a single constraint: no modification to the core is required or expected. This is what makes semantic virtualization fundamentally different from core replacement — you are not asking the bank to bet its existence on a migration.
Each legacy core speaks a different dialect — SOAP endpoints, 3270 terminal sequences, fixed-width batch files, proprietary message queues. The core adapter normalizes all of this into a uniform internal interface. It handles protocol translation, authentication, session management, and idempotency wrapping (so a retried transaction doesn't double-post). Building an adapter for a given core takes two to eight months depending on protocol complexity — a one-time investment, not a recurring tax.
This is the layer that makes the entire architecture safe enough for banking. It sits between the probabilistic agent layer (which uses AI to interpret and route) and the deterministic core (which must never receive a wrong instruction). It combines two things:
TXN_CD_03 in a fixed-width batch file to the tenant's expected field called transaction_type. It reads a screen-scrape of a 3270 terminal and extracts the account balance. This is interpretation work that traditional ETL cannot do without explicit, hardcoded field mappings.If the LLM's projection violates any constraint, the operation is rejected and routed to either a deterministic fallback or human review. Every decision — every prompt, every response, every constraint check — is logged with full provenance for regulatory audit.
This is the probabilistic orchestration core. It maintains per-tenant state externally (in an event-sourced, append-only log), performs schema-on-read projection (mapping core responses into each tenant's expected schema at request time), and composes multi-step workflows. Crucially, the LLM is a fallback, not the default — when a deterministic mapping exists, it is used. The LLM only steps in for the 20–30% of cases where the correspondence is ambiguous or undocumented.
A deposit inquiry, a loan origination, a compliance check, and a transaction dispute have fundamentally different requirements. A deposit inquiry needs speed. A loan origination needs thoroughness. A compliance check needs absolute determinism. The meta-orchestrator is a lightweight intent classifier that analyzes each request and routes it to the right executor: a deterministic function for latency-critical reads, a small fast model for schema projection, a large capable model for complex interpretation, or a human reviewer for high-risk operations.
The router is model-agnostic — if a model is deprecated or degrades, it redirects to an alternative without architecture changes. No vendor lock-in at the model layer either.
From the fintech's perspective, it is interacting with a purpose-built banking API. There is no visible indication that the underlying core is shared, legacy, or single-tenant. Each tenant sees its own REST or GraphQL API with its own schema, authentication, rate limiting, and documentation.
The six-layer model above is a conceptual map. Here is what actually deploys — the real services, data stores, protocols, and request paths an engineering team builds and operates.
The conceptual layers translate into concrete deployable components. A production deployment has an API gateway handling authentication and rate limiting, an orchestration tier with an intent classifier and meta-routing logic, a semantic virtualization core with three subsystems (event-sourced state, dual-path schema projection, and workflow composition), a neuro-symbolic guardrail pipeline, and a core adapter with protocol-specific handlers. The LLM inference runs on a dedicated serving tier. Everything is instrumented with audit provenance logging.
The paper formalizes tenant-scoped request processing as a precise algorithm. Every request from a tenant follows this exact sequence — there are no shortcuts or bypass paths:
def process_request(request, tenant_id):
# Step 1: Authenticate against tenant credentials
auth = authenticate(request.credentials, tenant_id)
if not auth.ok:
return HTTP_401
# Step 2: Load tenant scope from event-sourced state
scope = load_tenant_state(tenant_id) # from Kafka + PostgreSQL
# Step 3: Classify intent (rule-based first, small LLM fallback)
intent = classify_intent(request, scope) # read / write / config / dispute
# Step 4: Decompose into ordered subtasks
subtasks = meta_router.decompose(intent, scope)
results = []
for subtask in subtasks:
# Step 5: PRE-VALIDATE through neuro-symbolic guardrail
validated = guardrail.pre_validate(subtask, scope)
if not validated.ok:
return error(validated.reason)
# Step 6: Risk-gate — high-risk operations go to human review
if subtask.risk > THRESHOLD:
reviewed = human_review_queue.submit(subtask)
if not reviewed.approved:
return error("pending human review")
# Step 7: Route to the optimal executor
executor = meta_router.route(subtask)
# deterministic function | small LLM | large LLM | human
# Step 8: Execute (core adapter translates to legacy protocol)
raw_response = executor.run(subtask)
# Step 9: POST-VALIDATE — reconcile against expectations
checked = guardrail.post_validate(raw_response, subtask, scope)
if not checked.ok:
return error(checked.reason)
# Step 10: Project into tenant schema
results.append(schema_project(checked.data, tenant_id))
# Step 11: Log full provenance for regulatory audit
audit_log.append(request, intent, subtasks, results)
return project(results, tenant_id.schema)Steps 5 and 9 — the pre-validation and post-reconciliation — are where the neuro-symbolic guardrail earns its keep. Let us zoom into that pipeline.
The guardrail is not a single check. It is a sequential pipeline of five constraint classes, each catching a different category of error. The critical insight is that stages 1, 2, 4, and 5 are deterministic code — they do not use the LLM at all. Only stage 3 (semantic constraint checking) involves the neuro-symbolic interaction, and even there the symbolic engine has the final word.
The constraint engine encodes domain-specific semantic rules that pure structural validation cannot check. A JSON response can be perfectly well-formed — valid types, required fields present, nesting correct — and still be semantically wrong. The fee might be negative. The balance direction might contradict the transaction type. The interest accrual might exceed what is mathematically possible given the principal and rate. These are not format errors; they are domain logic errors. The constraint engine catches them with formal logic.
Not every request needs an LLM. In fact, the majority do not. The meta-orchestration router sends each request to the cheapest executor that can handle it correctly. A balance inquiry with a known schema mapping is a deterministic function call — sub-2 milliseconds, zero LLM cost. An ambiguous field in a screen-scraped 3270 terminal response needs a small LLM. A complex workflow like account opening (KYC + core transaction + card issuance + notification) needs a large model for decomposition. And a high-value wire transfer above a risk threshold goes to a human reviewer.
The routing distribution matters for cost. If every request went through the large LLM, inference costs would be prohibitive. But because 70–80% of requests use the deterministic path (a simple mapping lookup), the blended LLM cost per request is a fraction of what it would be in a naive "LLM for everything" architecture. This is why the meta-orchestration router is not just a performance optimization — it is a unit economics requirement.
To make this concrete, let us trace a single fund transfer request as it traverses the full stack — every hop, every component, every millisecond:
The total round-trip on the deterministic path is approximately 23 milliseconds of processing overhead, plus network latency. This is fast enough for real-time banking UX — a customer tapping "transfer" in a mobile app sees a response in under 100 milliseconds end-to-end. The core itself settles the transaction on its microbatch schedule (15–30 minutes), and the agent layer reconciles and updates the customer's displayed balance when settlement confirms.
The agent layer maintains all tenant-specific state — configuration, product definitions, customer-account mappings, compliance rules — in an external event-sourced store. The core never sees this state. This is how multi-tenancy is achieved without core modification: the agent layer is the tenant-aware brain, and the core is a stateless (from the tenant perspective) transaction engine.
Every state change is an immutable event appended to a Kafka topic named tenant.{tenant_id}.events. The current state is derived by replaying the event log, with periodic snapshots materialized in PostgreSQL for query performance. Hot paths are cached in Redis with a short TTL. This architecture provides three properties that banking demands:
The core adapter is where the agent layer meets the legacy world. Each core vendor exposes a different protocol surface, and the adapter normalizes all of them into a uniform internal JSON interface. The build effort varies by protocol complexity:
| Protocol Surface | Complexity | Build Effort | What the adapter does |
|---|---|---|---|
| SOAP / WSDL | Medium | 3–5 months | WSDL to JSON, session pooling, mutual TLS |
| 3270 / VT100 screen-scrape | High | 5–8 months | Terminal emulation, field extraction, navigation |
| Fixed-width batch files | Low–Medium | 2–4 months | Record parsing, field mapping, schedule integration |
| Proprietary message queue | High | 5–8 months | IBM MQ / JMS, message transformation, ACK handling |
This is a one-time investment per core vendor. Once the adapter is built, it serves all tenants connecting through that core — no per-tenant adapter cost. The adapter also handles idempotency (generating a unique key for each operation, caching the result, and returning the cached result on retry to prevent double-posting) and circuit breaking (if the core returns five consecutive errors in 30 seconds, the adapter trips a circuit breaker and routes to a fallback).
One of the most expensive line items in traditional BaaS is schema reconciliation — building a separate ETL pipeline for each tenant. Here is how the agent layer eliminates it entirely:
The impact is stark. Under the traditional model, onboarding a new tenant means building and maintaining a dedicated ETL pipeline — a one-to-four-million-dollar engineering project. Under the agent model, the pipeline build cost drops to zero. The agent interprets the core's response at request time and projects it into whatever schema the tenant expects. When the tenant's schema changes, it's a configuration update, not a pipeline rebuild.
Imagine a fintech — let's call it Nimbus Pay — that wants to offer checking accounts and debit cards through a sponsor bank running a legacy core from the 1990s. Under the traditional BaaS model:
Under the agent-layer model:
The single hardest limitation of any abstraction layer over legacy cores is the batch-bound settlement cycle. If the core posts deposits nightly, no abstraction can make deposited funds appear in the authoritative ledger before the next end-of-day run. This is not a software problem — it is a physical constraint of the core's architecture.
In 2024, this exact gap contributed to a catastrophic BaaS collapse. End-users saw balances in their app that did not correspond to real funds at the sponsor bank. When the reconciliation gap was exposed — when users discovered their money wasn't actually there — the resulting crisis destroyed a fintech, froze customer funds, and triggered regulatory scrutiny across the entire BaaS industry.
The solution is elegant precisely because it doesn't fight the core's architecture. Most legacy cores already support configurable batch schedules — they can be reconfigured from a single nightly run to multiple shorter runs per day. By reconfiguring the core to run microbatches every 15 to 30 minutes, the agent layer can submit pending transactions continuously and reconcile against the authoritative ledger within a bounded, auditable SLA.
The result: the at-risk window shrinks by 88 to 94%, from 4–8 hours down to 30 minutes. That 30-minute window is manageable with circuit breakers and provisional-balance caps.
Now for the numbers that matter to the C-suite. The total cost of ownership is modeled over a five-year horizon for three approaches:
Full core replacement carries the highest build cost — well over one hundred million dollars on average for a typical mid-to-large bank. Traditional BaaS middleware splits the difference, with build costs in the nine-to-thirty-two-million-dollar range. Agent-layer semantic virtualization has the lowest build cost — an estimated fifteen to twenty million dollars, if not less.
Beyond raw cost, the deployment timeline advantage is decisive. The agent layer deploys in 3 to 9 months, compared to 12 to 36 months for traditional platforms — three to four times faster. And because the architecture is both core-agnostic and model-agnostic, vendor lock-in is minimal.
| Path | Added Latency | Use Case |
|---|---|---|
| Deterministic | ~15–20ms | Balance inquiries, payment authorization, reads |
| LLM projection | ~100–300ms | Ambiguous field mapping, unstructured output |
| Deterministic mapping | ~2ms | 70–80% of all projections (known field correspondences) |
Here is the deepest insight in the architecture, and the one that matters most strategically. Semantic virtualization does not eliminate the need for a banking-grade platform — it relocates where that platform's value resides.
The most fundamental limitation is that semantic virtualization cannot overcome the core's architectural ceiling for true real-time settlement — instant posting with zero provisional window. If a use case requires sub-second, irrevocable settlement (instant payments networks, real-time fraud detection), the agent layer's 15–30 minute microbatch window is insufficient. Banks requiring true real-time settlement must still migrate to a real-time core.
However — and this is the crucial caveat — the set of banking operations that genuinely require sub-second settlement is smaller than commonly assumed. For deposits, transfers, loan disbursements, account opening, statement generation, compliance checks, and the vast majority of retail banking operations, a 15–30 minute settlement window is functionally equivalent to real-time from the customer's perspective.
LLMs are non-deterministic systems whose behavior may change across model versions. A model update could silently affect schema projections or configuration translations. The architecture mitigates this through three mechanisms: model pinning (specific versions pinned for production use), automated evaluation suites (test suites verify projection accuracy and guardrail enforcement), and model-agnostic routing (the meta-orchestration router redirects to alternatives without architecture changes).
The neuro-symbolic guardrail provides the ultimate safety net: even if a model update changes interpretation behavior, the symbolic constraint engine catches any projection that violates a domain invariant.
Traditional BaaS platforms come pre-certified — they have already navigated the regulatory approval process. An agent layer must be certified from scratch. This is a real cost (time and regulatory effort), though it is a one-time investment rather than a recurring tax.
The build-versus-buy decision in banking infrastructure has been reframed. For two decades, the choice was between a hundred-million-dollar core replacement and an expensive middleware license that introduced its own lock-in. Neither option changed what the core could fundamentally do — they both added abstraction layers at enormous cost.
Agent layers do not eliminate the need for a banking-grade platform. They relocate its value — from the integration tier (which LLMs collapse) to the governance and orchestration tier (which makes probabilistic systems safe for regulated banking). The institutions that recognize this shift early, build the governance layer with rigor, and deploy it with engineering discipline will be able to offer BaaS capabilities at a fraction of the traditional cost — roughly fifteen to twenty million dollars instead of over one hundred million — with greater flexibility, no vendor lock-in, and deployment timelines measured in months rather than years.
The legacy core isn't going anywhere. It doesn't need to. The intelligence layer wrapped around it is what changes everything.
This article is based on my personal and implementation tests using Apache Fineract product. The architecture, cost models, and latency figures are presented as engineering estimates based on industry benchmarks, not measured production comparisons. Cost figures reflect the position paper's parametric model and should be validated against specific institutional requirements.
Thoughts and essays, published with Yokush. See more posts
Siddharth21 min
Siddharth11 min
Siddharth15 min3
Comments 0