Higher resilience against unexpected market shifts

In the interview with Alla Gil, co-founder and CEO of Straterix, the discussion revolves around how banks manage and analyze stress scenarios, emphasizing the traditional three-step process: risk identification, narrative creation, and quantification of impacts. The current method, heavily reliant on past data and modeler's assumptions, often misses complex market dynamics and lacks early warning signals for proactive action. The introduction of machine learning and stochastic modeling is suggested to automate and enhance the analysis, allowing for a more dynamic understanding of potential outcomes. Artificial intelligence, particularly through machine learning, is highlighted as a key tool for generating synthetic future scenarios and aiding in decision-making by predicting outlier events. Large Language Models (LLMs) are proposed to reverse the traditional stress test sequence, focusing first on data quantification and scenario relevance before narrative construction. The integration of AI and LLMs aims to bolster the banks' resilience against unexpected market shifts by automating scenario analysis and facilitating adaptive contingency planning. Challenges include the need for a hybrid approach integrating human expert input into AI systems to maintain strategic oversight and ensure sensitive handling of data and predictions.

Can you describe typical stress scenarios that banks face today? How do they affect operations and decision making?

Alla Gil: The typical process consists of three steps:

1. Risk identification, i.e., business leaders together with risk managers are putting together the list of risk drivers that can impact their performance.
2. Creating a narrative about what can go wrong in the worl:d in the next couple of years.
3. Quantify the values of scenario variables, corresponding bank specific results as well as major key performance indicators (KPIs).

Stress scenarios are currently used for regulatory purposes. If an institution gets major comments (MRIAs – Matters Required Immediate Attention) from their supervisors about the results, they must fix the respective problems. Otherwise, the current stress testing process is not very constructive in improving institutions' financial health.

Based on your past experience in the financial services industry, what are the major shortcomings in analyzing critical stress scenarios?

Alla Gil: The crisis of regional banks in the US last spring has clearly demonstrated that a few made up scenarios are not enough to be prepared for unprecedented conditions. Such manual scenarios are based on historical experiences or modelers / economists' imagination. In reality, such a process is missing the combinations of market moves that might not be that dramatic by themselves. But combined together, they could be specifically bad for a particular institution. Also, the current process doesn't provide an institution with early warning signals when the probability of adverse scenario is increasing, and institutions must take proactive actions. 

Additionally, selection of risk drivers for banks' exposures is based on stable market conditions and could be wrong during stressful markets so it would require a lot of manual overlays that take a long time and tremendous amounts of resources.

How can mathematics and stochastics support the analysis and projection of relevant stress scenarios?

Alla Gil: Relevant stress scenarios must be identified through analysis of the full range of potential outcomes. But in order to do this the whole process must be automated using machine learning (ML) and stochastic modeling. It requires the following steps:

1. Use ML-based inference analyses to automatically identify which risk drivers to select to project banks' loan volumes, deposit outflows, credit losses, capital markets' fees etc. on the scenarios. These techniques use exhaustive cross validation and backtesting for such risk drivers' selection. They have much better chances of predicting behavioral patterns and exposures in unprecedented scenarios.
2. Interchange standard stochastic models driving Monte Carlo simulations with potential shocks and their ripple effects interrupting the stable markets behavior, implicitly changing correlations, and thus producing unprecedented scenarios.
3. Use cluster analysis to identify relevant scenarios for either adverse or opportunistic outcomes.

How can Artificial Intelligence (AI) help to identify or predict stress scenarios in the banking sector?

Alla Gil: ML is a core technique of AI that entails learning from available data and classifying and clustering it; the data analysis from ML can be used for   robust forecasting - even for outliers.  AI, in addition to ML, includes automated decision making.
So described ML techniques help generate synthetic data in the form of thousands of future scenarios while overcoming historical bias curse. AI can help with selection of optimal contingency plans based on early warning signals.

To what extent can Large Language Models (LLM) be used in risk assessment and in the analysis of stress scenarios in banks? 

Alla Gil: The constructive scenario selection process described here turns the traditional stress testing upside down. Rather than starting with the narrative and proceeding with quantification, it starts with quantifying multiple scenarios, selecting the relevant one and only then assigning the narrative. The last step in this process can be done using specially trained LLM models.

Can you give a concrete example?

Alla Gil: The detailed example is in this article: Exhaustive Scenario Analysis: What Banks Can Learn From the Airline Industry's Flight Simulations

Specific example described there is based on stress testing net interest income (NII) of a sample bank. Analyses of scenarios leading to the worst case NII outcomes show the following particular behavior of the variables that are correlated with such adverse outcomes (red lines)

Figure 01 [Source: Straterix. Time horizon in quarters]

By observing the behavior patterns of identified risk drivers that correspond to the red scenarios, it is possible to build the narrative for the underlying economic environment. In this case, it is  an inflationary environment with a very flat (if not slightly inverted) yield curve and an overheated housing market. The sharp drop in net interest rates is caused by a simultaneous increase in credit spreads – i.e., the correlations between risk-free rates and credit spreads go from negative in stable markets to sharply positive, as might happen in crisis situations.

How will the integration of AI and LLM increase the stress resilience of financial service providers to unexpected market changes?

Alla Gil: Through automatic analysis of thousands of future scenarios bank risk managers will be able to expand their intuition onto unprecedented market changes and prepare for these scenarios with contingency plans based on AI developed early warnings.

What challenges and limitations do you see in the use of AI and LLM to manage stress scenarios in the banking sector?

Alla Gil: This must be a hybrid approach with AI / ML / LLM being "white box" methods that enable interactive communication with business experts so they could input their expert opinions and perform sensitivity analysis challenging the AI recommendations. Such "cyborg" approach will allow for robust balance sheet analysis. Black-box methods are not appropriate for strategic planning though such methods could be used for ALM, cyber risk and consumer marketing strategies.

Can the experience gained from using AI and LLM to manage stress scenarios be transferred to other sectors?

Alla Gil: Yes, such analyses are suitable for financial planning and analysis for all corporate treasurers, CFOs, and risk managers. They can analyze behavioral patterns of their business receivables, potential currency exposures and capital/liquidity needs.

Are there any concerns regarding security and data protection when using AI and LLM? How can these be addressed?

Alla Gil: These analyses can usually use aggregated data and could be performed using public financial statements. They will never need confidential information of banks' clients. Projecting such aggregated segments of balance sheet and income statements enables longer term analysis when some individual loans and deposits are replaced with similar ones.

How can AI approaches and LLM be backtested in practice?

Alla Gil: Backtesting is the most critical part of such analysis. Using the data prior to dramatic market change, projecting it forward and verifying that the actual stress scenario would be included among the projected stress scenarios with realistic probabilities. See examples below.

Figure 02: Back testing of the 10-year Treasury rate

Figure 03: Back testing of S&P 500 – Covid & post-Covid

How do you rate the AI Act as a proposal for a European law on artificial intelligence (AI)? Can mathematics and algorithms be regulated at all? Or should it not rather be about regulating the use of AI?

Alla Gil: I'm not familiar with the details of this proposal but think that the regulation must focus on three key checkpoints: (1) full transparency; (2) ability to incorporate expert opinions; (3) reliable backtesting.

Can you describe a forward-looking project in which AI and LLM are used to revolutionize the handling of stress scenarios in the banking sector?

Alla Gil: Here are a few examples when exhaustive reverse stress testing based on the full-range scenario analysis discovered hidden risk concentrations and helped proactively mitigate them.

Example 1: A major regional rank needed to control increasing delinquency risk in retail lending. They needed to quantify the worst-case scenarios, for business as usual as well as stress environments, and define management actions to control delinquencies levels. Straterix's AI-based analytical solution was applied in less than 2 weeks:

• Loaded bank-specific data, as well as data from the public domain.
• Automatically analyzed delinquencies' patterns on scenarios and interpreted lending segments behavior and associated scenarios using LLM.
• Reduce delinquencies by implementing improved lending practices and optimize consumer segment mix.
• As a result, the bank improved lending practices and reduced exposure to risky segments resulting in the bank's overall exposure reduction by 24% in adverse scenarios. Exhaustive reverse scenario analysis has identified unemployment and credit spreads-dependent drivers as the triggers of contingent management actions ('early warning signals').

Example 2: An international bank subject to CCAR was under regulatory scrutiny because their internal stress scenarios were not severe enough. As a tremendous amount of time and resources was spent on scenario design, consisting of hundreds of variables, the time was running out to develop a challenger model that could solve the matter by the given deadline. Failure to address the problem could have led to failing CCAR but the second and third lines of defense (who are responsible to raise red flags during the regulatory audit), did not have effective analytical tools to adjust the models on time. The bank used Straterix's effective challenge approach to verify their CCAR stress testing process:

• A top-down exhaustive reverse scenario analysis quickly discovered the necessary adjustments to internal scenarios that would produce adequate severity.
• Analysis took less than 4 weeks from start to finish for one of the world's largest banks and the adjustments that were needed for the internal scenarios were discovered using ML/AI technology.
• The bank has learned which environments would be stressful specifically to them and has successfully passed CCAR.

Example 3: An international insurance company struggled to maintain sufficient net income margin in a low credit spread environment and urgently needed to improve earnings while staying within risk appetite boundaries and protecting solvency. They considered high yield investment strategies as a solution. Straterix's toolset was used to analyze and optimize the company's balance sheet and income statement by reviewing the current and various contemplated alternative asset portfolios, under the full range of scenarios. Such analysis has demonstrated that expected returns on high yield indices are lower than the treasury yield. This was caused by low credit spreads not covering even expected losses in high-yield bonds.

Using LLM-based interpretation of the outcomes, the company has arrived at robust, optimal allocation strategies appropriate for a wide variety of market environments. It was able to adjust risk limits dynamically to increase earnings within stated risk appetite boundaries and increased yields by giving up some liquidity in their portfolio while, in special situations, investing in high yield.

Can banks use the Straterix AI generated stress scenarios for reverse stress testing purposes? How are occurrence probabilities being assigned to potential extreme events which have not happened before, yet relevant for the respective institutions?

Alla Gil: The recent regional bank crisis in the US has emphasized the necessity of using more stress scenarios. But even if a bank comes up with a dozen of such scenarios, there is no guarantee that all important scenarios are included. 
Exhaustive reverse scenario analysis is a must, when the stress testing scenarios are selected from the full range of all feasible combinations of market moves, shock event and their ripple effects. Straterix platform is uniquely positioned to perform such reverse stress testing. Only this approach allows risk managers to assign probabilities based on the institution specific key performance indicators outcomes. For example, such reverse scenario analysis might show that the worst-case negative NII is associated with a slightly humped shape of yield curve. This happens because of a mismatch of embedded optionality on the asset and liability sides. While the yield curve scenario is quite possible, it leads to critically adverse outcomes for this particular institution. Even very experienced treasury professionals would not think of such a scenario as a really stressed one. But when discovered through reverse stress testing, it allows for proactive mitigation against such scenario.