This is Part 4 of the series. Parts 1 through 3 covered the what, the trade-offs and the reference architecture. Now let’s put it to work in the industry I know best and where the stakes are highest: banking.
A note on the example. The institution below is a composite — I’ll call it Meridian Bank — built from the patterns, constraints and failure modes that show up repeatedly across large banks. It is illustrative, not a case study of a named firm, and I have deliberately avoided inventing headline ROI numbers.
The starting point
Meridian is a US retail and commercial bank with roughly $150 billion in assets. By 2025 it had dozens of generative AI pilots, almost all running on a single closed-model API. Three problems forced a rethink:
- Data boundary. Information security would not approve sending customer financials, KYC files or suspicious-activity narratives to an external endpoint, which blocked the highest-value use cases.
- Concentration risk. The third-party risk team flagged dependency on one provider, with no ability to pin a model version beyond the provider’s deprecation schedule.
- Unit economics. Document-heavy workflows in commercial lending and financial crimes meant token volumes that made per-call pricing hard to justify at full scale.
The use-case portfolio
Meridian applied the placement framework from Part 3 and ended up with a deliberately mixed portfolio:
| Use case | Data classification | Model zone | Rationale |
|---|---|---|---|
| Commercial credit memo drafting | Confidential | Open weight, on-prem | Customer financials never leave; high document volume |
| AML alert narrative summarization | Restricted | Open weight, on-prem | Regulatory sensitivity; fine-tuned on investigator style |
| Contact-center agent assist | Confidential | Small open weight model | Latency-sensitive, very high volume |
| Developer code assistant | Internal (source code) | Open weight code model | IP protection; open models near parity on code |
| Regulatory change analysis | Public | Closed frontier | Public documents, hardest reasoning |
| Strategy and market research | Public / internal | Closed frontier | Low volume, highest capability need |
The rule that fell out of this was refreshingly simple: customer data stays on open weight models inside the bank; closed frontier models only ever see public or redacted internal data. The gateway enforces it using data-classification tags, not developer goodwill.
Deep dive: commercial credit memos
Credit memo drafting is a good example because it combines unstructured documents, institutional policy and a decision that a human must own.
Commercial credit memo workflow at the composite bank
A loan package — tax returns, financial statements, KYC documents — lands in the origination system. A small open weight model, fine-tuned on Meridian’s own historical packages, extracts and normalizes the key fields. A retrieval step pulls the relevant sections of the credit policy and industry guidelines. A mid-size open weight model then drafts the memo, citing every policy paragraph and source document it relied on.
Automated checks verify that citations resolve and that extraction confidence meets a threshold. Clean drafts go to an underwriter, who reviews, edits and signs. Exceptions go to a senior analyst, who may use a frontier model — on redacted data only, via the gateway — for a second opinion on a complex structure. Either way, a human signs every memo, and every step is logged.
The platform underneath
- Two on-premises GPU clusters (primary and disaster recovery), pooled across use cases so utilization stays high.
- The enterprise AI gateway from Part 3, with routing policies keyed to the bank’s existing data-classification scheme.
- A model registry holding scanned, signed and version-pinned base models plus task-specific LoRA adapters. Each adapter is a separate inventory item.
- An evaluation harness with golden datasets built by underwriters and investigators, run on every new checkpoint before promotion.
Governance: where banks are different
Model risk management is where a bank deployment lives or dies. The regulatory backdrop shifted in 2026: in April, the OCC, Federal Reserve and FDIC replaced SR 11-7 with revised, more principles-based model risk guidance — and explicitly carved generative and agentic AI out of its scope, pointing banks to their broader risk-management practices while the agencies gather input (Orrick).
That carve-out is not a free pass. Meridian, like most large banks, chose to apply MRM-grade discipline to its LLM workloads anyway:
- Inventory everything. Each base model, fine-tuned variant and adapter is registered with an owner, intended use and risk tier.
- Own the validation. With open weights there is no vendor to lean on. The bank validates on its own data, documents the fine-tuning, and records why each customization is justified.
- Change control for checkpoints. A new upstream release is a new model. It goes through the evaluation harness and sign-off before it touches production.
- Explainability through citation. Every generated memo traces back to source documents and policy text, giving reviewers and examiners something concrete to challenge.
- Humans own decisions. The models draft, summarize and recommend. Credit and AML decisions remain with accountable people.
What to measure
Rather than vanity metrics, Meridian tracks: time from complete package to signed memo; underwriter edit distance on drafts; exception and escalation rate; citation accuracy on sampled audits; cost per memo across the full platform; and GPU utilization by cluster.
Lessons that recur
- The gateway is the governance. Policy enforced in code beats policy in a PDF.
- Start with document-heavy, human-in-the-loop workflows. They deliver value while keeping decision risk where it belongs.
- Budget for evaluation, not just GPUs. Golden datasets written by domain experts are the scarcest asset in the program.
- Keep the frontier in reach. Open weights did not replace closed models at Meridian — they gave the bank a choice.
In the final part, I’ll look ahead at where open weight models are heading and what I would put on a CIO’s agenda for the next 18 months.
Featured image designed by Magnific