In Europe, the US Federal Reserve (FED) and the US Office of the Comptroller of the Currency (OCC)’s Supervisory Guidance on Model Risk Management (SR 11-7) is accepted as the global standard for the application of model risk management (MRM). Specifically, this guidance – issued to market participants in the form of a letter – provides recommendations to commercial and investment banks (CIBs)of all sizes on the best practices needed to develop and apply a robust MRM process.
SR 11-7 provides this guidance by introducing stages to MRM, which allows CIBs to garner a common appreciation of the concept of the MRM lifecycle. On the basis of its experience with CIBs of all sizes, GreySpark Partners has developed a comprehensive view of this lifecycle, its requirements and the best means of its practical application.
A Challenging Start
Model identification is considered the starting point in the model lifecycle. In this stage, CIBs outline the definition and the characteristics of the model to be captured, and they establish the process to identify all models and non-models in existence (see Figure 1).
As such, GreySpark has noted a major issue as being a lack of consistency in the implementation of the model identification process for models and model feeders in most banks, which can be attributed to the intentionally-broad US Securities & Exchange Commission (SEC) and OCC SR 11-7 definition of a model to catch all potential sources of models. Specifically, that definition states that:
“A model is a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates. A model consists of three components:
- Information – Delivers assumptions and data to the model;
- Processing – Transforms inputs into estimates; and
- Reporting – Translates the estimates into outputs with inherent uncertainty.”
There is evidence that some banks took the approach that, as soon as something presents elements of a ‘model,’ it will instantly enter the model lifecycle. However, this approach creates conflict between the lines of defence and slows down the process because of the high volumes of assets that have to go through lengthy validation and review processes.
In order to create a shared understanding of ‘model’ definition and remove any debate, some banks determined their own set of criteria to identify models on the basis of the SR 11-7 definition.
Another common challenge GreySpark frequently observes is that most banks do not have a centralised algorithms and models inventory. The absence of a model inventory prevents banks from mapping model interdependencies and correlations between the existing algorithms and their model feeders. This inability to map model interdependencies and correlations creates challenges for institutions as they cannot ensure that algorithms and models are tested and validated to be fit for purpose.
Figure 1: Model Risk Management Challenges & Best Practices
Source: GreySpark analysis
Best Practices & Benefits
As a result of these observations related to the common challenges for CIBs associated with model identification as the first step of the model risk management guidelines implementation process, GreySpark believes that the following four approaches constitute best practices that can yield meaningful benefits over time. Specifically:
Cultural Change
As a best practice, GreySpark advises that firms take a risk-based approach as a complement to the model definition and identification criteria in order to determine the impact of that potential model on the business and on the market.
For example, model identification should not be based only on criteria such as the complexity of the potential model because a mathematically simple model can still present high uncertainty and risk. For that reason, GreySpark views a risk-based approach to be best practice because this constitutes the basis for a flexible and efficient approach to handling model risk.
Increasing Use of Tools to Support the Identification of Models
To determine if an asset is an algorithm or model it is, therefore, necessary to determine if there is any quantitative method in the overall asset. By using a model-specific decision tree, banks can provide a simple and deterministic route to applying model governance proportionately. The use of a decision tree could allow the identification of non-algorithm models, algorithms, model components of algorithms and user tools.
A clear decision tree with the embedded model definition criteria helps banks to accelerate the decision-making process between 1LoD and 2Lod, and it removes the controversy around the model definition.
Some banks have implemented so-called exclusion lists to support the model identification process in combination with the decision tree. Exclusion lists are meant to provide further clarity to all lines on defence as to what falls into the definition of a ‘model’ and what does not. The purpose of the exclusion list should be to allow a risk-based approach to model identification. Types of exclusion include, but are not limited to:
- Materiality Exclusions – Exclusions based on the financial, reputational and / or impact of the potential model; and
- Single-use Exclusions — Exclusion of potential models that were developed and implemented for one-time, specific use.
A Digital & Centralised Inventory
A centralised and digitised algorithm and model inventory provides banks with the ability to map all the models, algorithms and model feeders to the algorithms and to ensure that the right level of risk management is applied with a snapshot of the data lineage and interdependence provided to both the business and risk management functions.
In an ideal scenario, the inventory should be integrated into an automated workflow that provides an audit trail of changes, reviews and approval, and it should allow both 1LoD and 2LoD to create, maintain and validate an enterprise-wide model inventory. The ultimate benefit that banks can achieve with a centralised inventory is to harmonise the MRM framework across functions and lines of defence.
A Flexible Approach
Banks are expected to develop a model identification process with a sufficient level of flexibility to provide evidence-based decisions to take non-models out of the ‘model’ classification so that, downstream, the algorithms or any other asset is not subjected to the scrutiny of model risk due diligence.
There are a multitude of ways to identify models and determine their risk level. Maturity will come as the market evolves. To achieve this maturity, firms must stabilise their processes and then optimise them once they have gained experience. This stabilisation and optimisation of processes will be made feasible through streamlining and automation. Identification of a calculation as a model should make sense and be proportional to the impact of the potential model and the risks present through all the stages in the model lifecycle.
Over a series of five articles, GreySpark will explore these five topics in an effort to assess the challenges associated with their application and to draw out the best practices that can be utilised to manage their implementation.
The second article in the series of five will examine model testing.
