In Europe, the US Federal Reserve (FED) and the US Office of the Comptroller of the Currency (OCC)’s Supervisory Guidance on model Risk Management (SR 11-7) is accepted as the global standard for the application of model risk management (MRM). Specifically, this guidance – issued to market participants in the form of a letter – provides recommendations to commercial and investment banks (CIBs) of all sizes on the best practices needed to develop and apply a robust MRM process.
SR 11-7 provides this guidance by introducing stages to MRM, which allows CIBs to garner a common appreciation of the concept of the MRM lifecycle. On the basis of its experience with CIBs of all sizes, GreySpark Partners has developed a comprehensive view of this lifecycle, its requirements and the best means of its practical application.
An Overarching Challenge
The previous articles of this series introduced GreySpark’s understanding of best practices around the most challenging steps of the MRM lifecycle. However, one overarching principle cannot be overlooked to ensure that the implementation of such best practices actually results in competitive advantage: efficient information sharing across the lines of defence.
This article explores why information can only be valuable if it is carefully selected and if it is aimed at the right talents, and it proposes measures that could be implemented to satisfy these two conditions. As such, the analysis assumes that the model development function within the CIB sits in the first line of defence, and that the independent validation function sits in the second line of defence.
The MRM lifecyle requires lines of defence to communicate and share a number of information on the models and the algorithms they feed into. Notably, the FED and the OCC’s SR 11-7 guidelines states that: “strong governance … includes documentation of model development and validation that is sufficiently detailed to allow parties unfamiliar with a model to understand how the model operates, as well as its limitations and key assumptions.”
Yet, CIBs are not aligned with regard to exactly which information should be shared across the lines of defence and in what amount throughout the whole MRM lifecycle. This is partly due to the fact that, in some banks, developers remain protective of their intellectual property and wish to limit sharing details of their models. In other cases, developers do submit a large number of information on the models, as required by the validators sitting in the second line of defence, but the quality and relevance of the information is not appropriate to allow effective validation.
Additionally, GreySpark observes a lack of collaboration between the lines of defence in those banks that have not spent enough time training or recruiting the necessary personnel in the second line of defence to ensure that they would be able to make the most of the information shared by model developers. In such cases, the first line of defence is reluctant to share information on models because it assumes that the second line could not appropriately challenge and review it, and validators are not confident in their own validation requirements.
However, sharing the correct level of detail and relevant information to the right people is critical at the model development and validation and model change stages of the lifecycle, and it remains important throughout the entire model’s use. In fact, and as an example, how can the risks associated with each model be appropriately mitigated if the stakeholders involved in the validation process are not confident in their understanding of the model at hand?
Without a complete understanding of the model and its testing and other validation requirements, then those requirements assigned to it may either be too stringent or too mild, thus detracting from the preferred, risk-based approach to MRM.
Pertinent Approaches to Information Sharing
GreySpark understands that, for information sharing to be effective, banks must ensure that the correct level of information is provided. In the context of the validation of new models or changed models, the independent validators should be provided with enough information to perform their role correctly. Such information relates to all parameters, assumptions, justifications and scope of the model.
In other words, in the context of new or changed models’ validation, validators should receive enough information on the model to replicate should they want to. In the context of model feeders of algorithms, this is not enough. In fact, a model must be understood in the context for which it was built to ensure that appropriate validation requirements are applied. Therefore, for model feeders of algorithms, actionable information on the algorithm and the controls framework should accompany the information on the model. This information should be used to adjust the testing requirements imposed on the new or changed model (see Figure 1).
Figure 1: The Three Lines of Model Risk Management Defence
Source: GreySpark analysis
On another hand, for information sharing to make sense, the receiving ends of such information must be capable to understand, challenge and act on the information provided to them. Therefore, banks must make sure that the appropriate level of knowledge and expertise sits in the second line of defence.
Some mature CIBs have already taken the approach to hire quants within the second line, thus ensuring that validators can appropriately challenge test results in the validation process. In response to the trust issue between departments, some banks organised training sessions for all three lines of defence to sensitise employees to the work done and to the roles and responsibilities of each line. Finally, some banks gave the opportunity to members of the second line of defence to receive training on the technical aspects of modelling by sitting directly in the first line for a couple of weeks.
The most mature CIBs have put in place automated workflow systems. These are used to fasten and improve the flow of information by allowing it to be shared quickly and to be stored and accessible where and when is necessary so as to avoid the loss of IP. Security around IP can also be improved as these systems enable tracking of records updates and the monitoring of user access rights.
With these factors in mind, GreySpark believes that implementing relevant processes and tools to support the efficient sharing of information across lines of defence will allow CIBs to lower the potential risk of errors in models, in turn decreasing the bank’s reputational and operational risk. In parallel, training and recruiting will improve transparency and trust between the three lines of defence.
GreySpark Partners currently offers several service offerings designed to service the model risk management and regulatory compliance needs of the investment banking industry. For more information regarding these service offerings, please click here and here.
