Model Risk Management: Model Change Management

An Introduction to Model Change Management

Risks incurred by model changes can be of different natures and impact. As such, CIBs cannot rely on a single process for model change management.

GreySpark observes that the application of the model change process varies from bank to bank, but these processes may also be subject to differences within a single institution. More CIBs are now looking for ways to render their model change requirements on a fit-for-purpose and adaptable basis so as to not negatively impact business functions (see Figure 1).

In this article, GreySpark attempts to decompose the approaches leading to effective and proportional change management processes.

Figure 1: An Adaptable & Fit-for-Purpose Model Change Process

Source: GreySpark analysis

Time: The Pain Point of Model Change Management

GreySpark observes that most CIBs suffer from the fastidious processes supporting model changes. Change validation requirements are often so lengthy that the institution is constrained to waiting up to multiple months before a change is validated and put in use.

In such cases, the risk incurred is that, by the time the model change is implemented and validated, it is no longer relevant to the market or to the firm resulting in a potential loss of business. Such time and validation constraints create tensions between the different lines of defence, whose roles and objectives in the change process vary; where the second line of defence works to monitor and manage the risk associated to model changes, the first line of defence’s goal is to benefit from such changes as quickly as possible to better respond to changing market conditions and behaviours.

These challenges are even more visible in the context of model feeders of algorithms. The industry agrees on the general observation that model change and algorithm change timescales do not match. Algorithms changes should be validated in a matter of days, if not hours, so that delays in the validation of model changes have a negative impact on the algorithms they feed.

Best Practices

The key to coping with the constraining MRM lifecycle seems is the application of flexible processes following a risk-based approach to model management.

Allowing different validation requirements depending on the potential impacts of model changes enables validators to better allocate their time and efforts to the riskiest changes and to proportionally mitigate model risk.

In anticipation of the above observation, the FED and the OCC introduced the notion of ‘materiality.’ Specifically, SR 11-7 refers to materiality in the following context:

“As is generally the case with other risks, materiality is an important consideration in model risk management. If at some banks the use of models is less pervasive and has less impact on their financial condition, then those banks may not need as complex an approach to model risk management in order to meet supervisory expectations.”

SR 11-7 anticipates that banks should apply different levels of risk management based on whether a model change is material or not. Although the concept of using materiality to determine validation requirements for model changes is widely understood across the industry, most banks struggle in applying a shared understanding of the definition of ‘materiality’ across the lines of defence.

GreySpark observes that one definition does not fit all purposes if being classified as ‘material’ or ‘non-material’ / ‘immaterial’, and it will influence the way the model change will be treated. Keeping in mind their risk-based approach, banks should look at materiality as a combination of multiple factors and not as a single definition. For example, whether a change should be classified as ‘material’ or ‘immaterial’ could be a combination of:

1. The model risk level;
2. The risk level of the algorithm that the model feeds; and
3. The type of model change – On the parameters or on the scope of the application of the model.

In these three ways the process of applying various factors to determine materiality should be shared, understood and agreed upon across all lines of defence.

CIBs understand that whether a change is deemed material or immaterial will have an impact on its validation requirements and, most particularly, on which validation lines will be involved in the process. Importantly, banks should consider applying different levels of documentation requirements where needed, the end goal being to mitigate the risks brought by the model change, and accompanying documentation should be commensurate with the risk. Banks should thus consider whether the least material changes could rely on a minimum level of documentation.

In order to facilitate the determination of materiality for each model change and its implications on the validation requirements, the most mature CIBs rely on automated workflow systems. These systems are used to allow an optimum level of flexibility in the change process by:

1. Integrating and combining the factors that constitute materiality, thus automatically determining the “material” or “immaterial” nature of the change once submitted in the system;
2. Automatically assigning the appropriate levels of testing and documentation, taking into account that not all material or immaterial changes will require the same validation requirements; and
3. Allowing validators to provide approval quickly.

By applying these best practices, banks can be comfortable applying different validation lines depending on each model and associated model change while maintaining the appropriate level of risk management. The most mature banks are proof that having a flexible approach to change management, shared across the lines of defence, has a positive impact on the communication between first and second lines of defence, and it helps avoiding business loss incurred by validation waiting times.

The conclusion of this series of five articles will examine the challenges and best practices around information sharing in the context of model risk management.