The best solution is not a single one. Large Language Models (LLMs, ≥30B parameters, often via APIs) You can also find out more about the following: Small Language Models (SLMs, ~1–15B, typically open-weights or proprietary specialist models). Your selection in 2025 should depend on regulatory risk, sensitivity to data, cost and latency, and complexity of use cases.
- SLM-first It is highly recommended for tasks such as structured information extraction, coding, assistance with coding, internal knowledge, retrieval-augmented creation (RAG), and strict guardrails.
- Scale up to LLMs SLMs are ideal for complex synthesis or multi-step reasoning. They can also be used when SLMs do not meet the performance requirements within your latency/cost budget.
- The Government of Canada Both must be treated the same way: LLMs andSLMs should fall under your MRM, aligned to NIST AI-RMF and mapping high-risk applications to EU AI Act obligations.
1. Regulatory and Risk Attitude
The US has developed a mature governance model for financial services. US-based financial services are subject to mature governance models. Federal Reserve/OCC/FDIC SR 11-7 Includes LLMs as well as SLMs. This means required validation, monitoring, and documentation—irrespective of model size. It is important to note that the size of a model does not affect validation, monitoring and documentation. NIST AI Risk Management Framework (AI RMF 1.0) The gold standard of AI risk management, widely used by banks for traditional AI and generative AI.
EU-wide, AI Act It is in place, with a series of compliance dates. (Aug.2025 for models that are general-purpose, Aug.2026 for those systems that pose a high level of risk such as the credit scorers per Annex III.) “High-risk” means that pre-market compliance, risk management and documentation are all part of the definition. The EU is a target market for institutions, and they must adjust their timelines to match.
According to the core sectoral rules:
- GLBA Safety RuleControls for security of consumer financial information and the oversight by vendors.
- PCI DSS Version 4.0: New cardholder data controls—mandatory from March 31, 2025, with upgraded authentication, retention, and encryption.
Supervisors (FSB/BIS/ECB) and standard setters highlight systemic risk from concentration, vendor lock-in, and model risk—neutral to model size.
Note: High-risk use (credit and underwriting), irrespective of the parameters, requires tight control. SLMs, as well LLMs, require a traceable verification, sector compliance, and privacy assurance.
2. Cost, Latency and Footprint Cost, latency and footprint
SLMs (3–15B) now deliver strong accuracy on domain workloads, especially after fine-tuning and with retrieval augmentation. SLMs are a great tool for targeted extraction and classification. They also improve workflows.
LLMs unlock cross-document synthesis, heterogeneous data reasoning, and long-context operations (>100K tokens). LLMs that are domain-specific (e.g. BloombergGPT 50B) perform better than general models in financial benchmarks, multi-step reasoning, and other tasks.
Compute economics: Self-attention in the transformer scales up quadratically as sequence length increases. FlashAttention/SlimAttention optimizations reduce compute costs, but don’t defeat the quadratic lower bound; long-context LLMs can be exponentially costlier at inference than short-context SLMs.
Note: The SLM is best for structured and latency-sensitive short tasks. (Contact center, KYC extractions, Knowledge search, KYC claims) LLMs are the best choice if your task requires 100K+ tokens contexts. “escalation.”
3. Trade-offs between security and compliance
Common risks: These models both have the same risks: prompt injections, insecure output handlings, data leakages, and supply-chain risk.
- SLMs: Preferred for self-hosting—satisfying GLBA/PCI/data sovereignty concerns and minimizing legal risks from cross-border transfers.
- LLMs: APIs present concentration and locking-in risk; supervisors must document exit, fallback and multi-vendor strategy.
- Explainability: LLM reasoning traces can’t replace the formal validation that is required by SR 11-7 / EU AI Act.
4. Deployment patterns
Finance: Three tried and tested methods
- SLM-first, LLM fallback: Route 80%+ queries to a tuned SLM with RAG; escalate low-confidence/long-context cases to an LLM. Cost/latency is predictable; great for call centers, form parsing, and operations.
- LLM-primary tool use: LLM orchestrator of synthesis with DLP protection and deterministic data access tools. Suitable for research and policy/regulatory tasks.
- Domain-specialized LLM: The MRM workload is higher but the gains are measurable for certain niche tasks.
Regardless of the situation, implement all content filters, PII-redaction, connectors with least privilege, output verification and monitoring, as well as red teams and continuous monitoring, in accordance with NIST AI RMF guidelines and OWASP.
5. Quick Reference: Decision Matrix
| Criterion | SLM is preferred | Prefer LLM |
|---|---|---|
| Exposure to Regulation | Non-decisional internal assistance | Use of high risk (credit score) with full validation |
| Data sensitivity | On-prem/VPC, PCI/GLBA constraints | External API with encryption and DPAs |
| Latency & cost | Cost-sensitive, sub-second high-QPS | Low QPS, Seconds latency and batching |
| Complexity | RAG-aided Draft, Extraction and Routing | Synthesis of ambiguous inputs, contexts in long form |
| Engineering ops | Self-hosted CUDA integration | Managed API, vendor risk, rapid deployment |
6. Concrete Use-Cases
- Customer Service For common questions, SLM first with RAG/tools. LLM escalated for complicated multi-policy issues.
- KYC/AML & Adverse Media: SLMs suffice for extraction/normalization; escalate to LLMs for fraud or multilingual synthesis.
- Credit Underwriting: Use LLMs and SLMs as decision-making tools, but always use human reviews.
- Research/Portfolio Notes: The LLM allows for cross-source collation and draft synthesis; it is recommended that you use a tool with citation tracking, read-only access to the database, as well as a logging of citations.
- Developer Productivity: SLM assistants on-premises to improve speed and IP safety. LLM escalated for complex synthesis or refactoring.
7. Performance/Cost Levers Before “Going Bigger”
- RAG Optimization Most of the failures in retrieval are not “model IQ.” Before increasing size, consider improving chunking and ranking, as well as recency.
- Prompt/IO controls: Guardrails for input/output schema, anti-prompt-injection per OWASP.
- Serve-time: Quantify SLMs. Use page-KV caches.
- Selective escalation: Route by confidence; >70% cost saving possible.
- Domain Adaptation: SLMs are best fitted with lightweight LoRA tuning. Use large models for performance improvements that can be measured.
EXAMPLES
Example 1 Contract Intelligence at JPMorgan (COiN)
JPMorgan Chase has deployed a special team to combat cybercrime. Small Language Model (SLM), called COiN, to automate the review of commercial loan agreements—a process traditionally handled manually by legal staff. COiN was trained on legal documents and filings to reduce contract review time from weeks to hours. The bank achieved high accuracy, compliance traceability, and cost reduction by training COiN. JPMorgan was able to redirect legal resources to more complex and judgmental tasks by using this targeted SLM.
Example 2 FinBERT
FinBERT, a language-based model based on transformers, is meticulously trained using diverse data sources such as transcripts of earnings calls, articles about financial markets, and financial news. This domain-specific training enables FinBERT to accurately detect sentiment within financial documents—identifying nuanced tones like positive, negative, or neutral that often drive investor and market behavior. FinBERT helps financial analysts and institutions gauge sentiments around earnings and events in the market. The outputs can be used for market forecasting, proactive decision making, and portfolio management. FinBERT’s focus on contextual nuance and financial terms makes it more accurate than other generic financial sentiment models. It provides practitioners with actionable, authentic insights into market dynamics and trends.
References:
- https://arya.ai/blog/slm-vs-llm
- https://lumenalta.com/insights/hidden-power-of-small-language-models-in-banking
- https://www.diligent.com/resources/blog/nist-ai-risk-management-framework
- https://iapp.org/resources/article/eu-ai-act-timeline/
- https://www.ctmsit.com/it-business-solutions-growing-companies-2025/
- https://www.bis.org/fsi/fsisummaries/exsum_23904.htm
- https://ai.azure.com/catalog/models/financial-reports-analysis
- https://promptengineering.org/bloomberggpt-a-game-changer-for-the-finance-industry-or-just-business-as-usual/
- https://linfordco.com/blog/pci-dss-4-0-requirements-guide/
- https://syncedreview.com/2023/04/04/bloomberg-jhus-bloomberggpt-a-best-in-class-llm-for-financial-nlp/
- https://www.oligo.security/academy/owasp-top-10-llm-updated-2025-examples-and-mitigation-strategies
- https://squirro.com/squirro-blog/state-of-rag-genai
- https://www.evidentlyai.com/blog/owasp-top-10-llm
- https://www.limra.com/globalassets/limra-loma/trending-topics/ai-governance-group/nist-ai-risk-management-framework.pdf
- https://adc-consulting.com/insights/implications-of-the-eu-ai-act-on-risk-modelling/
- https://www.onetrust.com/blog/navigating-the-nist-ai-risk-management-framework-with-confidence/
- https://www.saltycloud.com/blog/glba-safeguards-rule/
- https://securiti.ai/glba-safeguard-rule/
- https://dzone.com/articles/microsoft-reveals-phi-3-first-in-a-new-wave-of-slm
- https://generativeai.pub/from-costly-attention-to-flashattention-a-deep-dive-into-transformer-efficiency-62a7bcbf43d6
- https://www.gocodeo.com/post/inside-transformers-attention-scaling-tricks-emerging-alternatives-in-2025
- https://strobes.co/blog/owasp-top-10-risk-mitigations-for-llms-and-gen-ai-apps-2025/
- https://www.chitika.com/retrieval-augmented-generation-rag-the-definitive-guide-2025/
- https://nexla.com/ai-infrastructure/retrieval-augmented-generation/
- https://www.confident-ai.com/blog/owasp-top-10-2025-for-llm-applications-risks-and-mitigation-techniques
- https://www.linkedin.com/pulse/dawn-ai-powered-compliance-how-llms-slms-transforming-srivastava-rxawe
- https://www.invisible.co/blog/how-small-language-models-can-outperform-llms
- https://www.ibm.com/think/insights/maximizing-compliance-integrating-gen-ai-into-the-financial-regulatory-framework
- https://www.regulationtomorrow.com/eu/ai-regulation-in-financial-services-fca-developments-and-emerging-enforcement-risks/
- https://securiti.ai/glba-compliance-requirements/
- https://www.feroot.com/blog/pci-dss-4-0-compliance-guide/
- https://owasp.org/www-project-top-10-for-large-language-model-applications/
- https://owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-v2025.pdf
- https://blog.barracuda.com/2024/11/20/owasp-top-10-risks-large-language-models-2025-updates

