Description

This practitioner’s guide explains why an AI product’s reliability depends on the software built around a model rather than the model itself. Roughly 80% of a working system is that surrounding harness and only 20% is the model call.

The report works through the practical decisions in order: routing and right-sizing models empirically rather than by reputation; engineering prompts and context as a supply chain with trust boundaries; and grounding answers through retrieval, covering chunking, hybrid search, reranking and when a knowledge graph is warranted.

It explains how to design tools and function calls an agent can use safely, how the agent loop and its harness enforce rules architecturally rather than by instruction, and when multi-agent orchestration is justified, illustrated by a real credit-decisioning incident.

It sets out how to evaluate a system across four layers, including the trajectory most teams skip, how to contain prompt injection and secure agent identity and how to engineer for probabilistic operation once a feature reaches production.

A closing section adds the further obligations facing regulated capital-markets and financial-services firms, on model risk, data residency, third-party concentration and human oversight. Throughout, the guide distinguishes established practice from single-source or self-reported claims.

Focus on AI in Investment Banking: Surveillance

Focus on AI in Investment Banking: Reconciliations

Quantum Computers: In a Store Near You Soon?

The Quantum Threat to Encryption

Our latest reports

All Special Reports

Special Report – Retiring the OTP 2026

AI Engineering Best Practices Guide – Building High-performance Software with AI

FX Trade Settlement 2026

AI Engineering Best Practices Guide – A CTO’s Guide to Transforming the R&D Organisation

How to Use This Guide

1.0 Choosing and Routing Models
1.1 Do Not Marry One Model
1.2 Right-size Empirically, Never by Reputation
1.3 Small Language Models as the Workhorses
1.4 Cost is an Architecture Constraint, not a Line Item

2.0 Prompt and Context Engineering
2.1 Prompt Craft That Survives Contact with Production
2.2 Context is a Supply Chain
2.3 Context Rot and Budgeting Usable Context

3.0 Retrieval and RAG
3.1 Chunking is Downstream of Your Queries
3.2 Hybrid Search and Reranking
3.3 When to Reach for a Graph, and the “RAG is Dead” Noise
3.4 The One Absolute Rule

4.0 Tools, Function Calling and Protocols
4.1 Single-purpose Tools Beat a General Shell
4.2 Design for Outcomes and Mind-the-tool Budget

5.0 The Agent Loop and the Harness
5.1 Reliability is a Harness Property Demonstrated
5.2 The Rules That Follow
5.3 The Harness is a Product You Maintain

6.0 Multi-agent Orchestration
6.1 Borrow from Distributed Systems
6.2 The Reliability Patterns from a Real Incident
6.3 A Production Decomposition That Works

7.0 Evaluation and Observability
7.1 Start with Assertions, Not Tools
7.2 Cover the Four Layers
7.3 Judges, Calibration and the Reliability of a Failing Test
7.4 The Maturity Curve, and the Flywheel

8.0 Security and Guardrails
8.1 Prompt Injection is Unfixable at the Prompt Layer
8.2 Identity, Sandboxing and Continuous Red Teaming

9.0 Productionisation and Reliability
9.1 Engineer for Probabilistic Operation
9.2 Measure Reliability Where it Matters, and Keep it Private

10.0 The Financial Services Overlay

11.0 Appendices
Appendix A: Build Checklist
Appendix B: Tooling Landscape
Appendix C: Evaluation Starter Kit
Appendix D: Lexicon

Related content

Focus on AI in Investment Banking: Surveillance

Focus on AI in Investment Banking: Reconciliations