Basel Committee guidelines on back testing and stress testing

Basel Capital Accord OF 2003 released by the Basel Committee on Banking Supervision (BCBS) envisages a three-pillar approach to risk management under which the first pillar signifies total minimum capital requirement for credit, market and operational risks. To determine the minimum capital requirement for an approach, subject to certain minimum conditions and disclosure requirement may be assessed by the banks’ internal operational risk management system under Advanced Measurement Approaches (AMA) using the recommended quantitative and qualitative criteria.

Basel Committee guidelines on back testing and stress testing: 

The Basel Committee has further recommended that the applicability of the internal risk models must be verified in a meaningful manner through back testing and stress testing under normal and extreme conditions and these models must also be subject to supervisory review and/or validation by an independent agency. On the kinds of conditions under which the bank’s strategies or positions would be the most vulnerable e.g. abrupt changes in curve etc. In this paper, we present an overview of the Basel Committee guidelines on back testing and being followed by banks in the absence of better alternatives.

It is necessary in order to elaborate the meaning of risk. In common parlance, risk means the possibility or danger of loss. Academic definitions are relatively described as the possibility of the actual return deviating from different from what may be referred to as the expected rate of return which itself will be different from the actual rate of return. The term ‘expected return’ has an element to understand and apply to widen our understanding of risk in quantifiable terms – more precisely, in terms of the mean or expected value of return and the associated of the actual return around the mean value or expected return and measured accordingly.

Credit Risk: Internal ratings based approach:

A lending institution faces a number of risks in which commercial or credit risk is a major risk because it entails not only the potential volatility of loss but also a higher probability of default when the exposure is high. The borrower firm faces different types of risks and its inability to manage anyone or more of these risks may develop into risk of default for the lender. The lender must, therefore, identify all types of risks that the firm may be exposed to in carrying out its business and assess them, in addition to evaluate the risks associated with.

It is crucial for a commercial lending institution to have an internal credit risk assessment system (CRAS) which should be an integral part of its overall the inherent handicap of reflecting subjective preferences of the appraising officials rather than addressing the underlying credit risks of their CRAS in order to develop new rating models which can produce precise results, even on subjective parameters. There is a striking similarity in their approach but in terms of degree of emphasis on to individual accounts. There are wide variances too.

Some institutions use a single method where a single value is placed on each loan which relates to the borrower’s underlying credit quality. Others use a dual system where both the borrower and the credit exposure are rated, which enables attention to be focused on collateral and covenants as well as assessment of the general credit worthiness of of the borrower and loan’s quality is offered and an analytical interpretation of the underlying financials some comfort in its knowledge of the loan asset quality at any degree of standardization of process and documentation is required.

Rating migrations, over time, of such assets may indicate changes in loan quality and expected loan losses from the quality reports referred to above are expected to throw up hasty or timely warning signals in respect of expected loan losses. In due course of time, it should be feasible to build up a reliable database indicating some relationship between credit rating and ex-post default rates. The ground position, however, is otherwise. Lack of available industry data to conduct an appropriate aggregate migration study does not permit an adequately comforting confidence level in the expected loan loss calculations.

For this type of credit quality report to periodically. In addition, any material change in the conditions associated either with the borrower or the will trigger a re-evaluation. GAAP also requires this sort of monitoring. It may be appreciated that with a reliable database emerging from internal loan migration studies, it would be possible to estimate loan loss at a given default probability and then compare the figure with actual provisions made for the diminution in the asset quality. We have not included an analysis of the systemic risk in any of the above mentioned reports indicating industry wise composition of the overall loan portfolio, with a view to limiting their exposure to particular industry groups.

There are no standard benchmarks as yet but banks have been modifying their loan policy document in accordance with the observed variation in marketing indices or individual performance of a study to evaluate the potential downside loss but are rather a subjective evaluation by the Risk Management Department. Concentration reports, leading to value so that only concentrations above a minimum size are recorded for the purpose of exposure setting.

Value at risk or VAR model:

The term VaR is of a banks to calculate their capital requirements for market risk with their coupled with the impetus provided to the use of VaR models in the J.P.Morgan’s Risk Metrics System, have inspired regulators also to put in place an enabling mechanism to promote the use of is used to signify a measure of potential loss from an uncertain but probable adverse level of confidence. VaR is the amount of probable loss when the probability of losing more than the said amount over a given time.

In mathematical notation, we may write it as VaR (99%, 10 days). VaR is essentially the unexpected loss under normal adverse conditions which cannot be known or observed data about the distribution of possible future losses but actual VaR may be different from the of historical data can be invoked as a reasonably. In the case of operational risk, the distribution of operational loss associated with different event categories can similarly be used to estimate both the expected losses as well as the risk losses are the losses that are supposed to occur.

Expected loss (EL) is absorbed as an ongoing cost and managed through internal controls. The unexpected loss (UL) is much less frequent but is of a significantly events and is numerically equal to the difference between actual loss under normal circumstances and expected loss and can be managed through a combination of external firewall of insurance and internal cushion of occurring in excess of the unexpected loss – it is extremely rare but can be really obvious reasons, such losses are usually transferred to an insurance company.

Back testing:

‘Backtesting’ is a statistical testing framework for validating the accuracy and robustness of a risk banks which use internal models for assessing capital adequacy requirement e.g. a bank using a VaR model for computing the risk of a trading portfolio which includes identified basic rates and prices that affect the value of the portfolio. The BCBS released in January 1996 a document detailing a framework for incorporating back testing into the internal models approach to market risk capital requirements which described back testing as the process of comparing daily trading outputs with the internal model-generated risk measures to gauge the quality and accuracy of their risk measurement systems.

In case the differences between the two are significant, then either the model is not accurate or the assumptions of the back test are not pragmatic. In the back testing framework developed by the BSBC, a bank’s daily VaR measures are periodically compared with the subsequent daily profit or loss (actually the VaR estimate is called an exception. The 1996 document referred above requires that VaR should be computed at 99% regulatory capital requirement will be the maximum of previous day’s VaR and three times the average of daily VaR of the preceding 60 trading days.

Stress testing:

The Basel Committee on Banking Supervision (BCBS), while emphasizing the need for each bank to critically assess its internal capital adequacy and future capital requirements in the light of its risk profile and business plan, has recommended that as part of this process, a bank should perform comprehensive and rigorous stress tests to identify possible events or changes in markets that could result in significantly large losses, and to assess its ability to withstand them.

‘Stress Testing’ refers to techniques used for measuring how vulnerable an organization can be in extreme but plausible between late March 2000 and late July 2002. Regulators require that banks prepare themselves to manage not only the normal risks but also the risks resulting from possible violent or exceptionally volatile behavior of key market variables in their day-to-day functioning (and set limits for their VaR exposures and stress losses accordingly). A stress test essentially shows a risk management model’s to define and manage events that are normally not captured in VaR measures but could cause capability to absorb large potential losses (adequacy of capital).

The Basel Committee has described four techniques which are discussed below, along with the information typically referred to as the “result” of that type of stress test.

Scenario analysis:

Scenario analysis involves scenario writing and specifies the shocks that might plausibly affect a number of market risk factors simultaneously assess the probable consequences for a bank of an extreme, but possible, turnout of events. (We have seen above that scenario analysis can be built on a historical event or a hypothetical event. For instance, we may look at a situation where the exchange rate goes down by 6% whereas the rate of interest goes up by 2% and then see whether the bank would be able to absorb the shock. Scenario writing is a complex job and even though extreme events may simultaneously affect more than one market factor. Care should be taken to change only one factor at a time.)

Extreme value theory:

A few banks also use extreme value theory (EVT) as a means to better capture the risk of loss in extreme, but possible, circumstances. EVT is the statistical theory of the behaviour of the “tails” (i.e., the very high and low potential values) of probability distributions. Because it focuses only on the tail of a probability distribution, the method can be fat-tailed is adapting it to a situation where many risk factors drive the underlying return distribution. Moreover, the usually un-stated assumption that extreme events are not correlated through time is questionable. Despite these drawbacks, EVT is notable for being the only stress test technique that attempts to attach a probability to stress test results.

Credit Risk:

The back testing techniques for credit risk have to be different from those applicable to market risk assessment models as described above because of non availability of reliable databases. Most loan exposures are not traceable instruments. No two loan accounts are similar. Daily estimates of profits and losses from a credit portfolio of a loan asset also cannot be predicted from historical experience in the manner of in the credit risk algorithm. Credit risk may vary with the nature of transaction or industry/business segment or exposure/ concentration in a particular location or industry or change in the credit quality or simply the state any set trend and the time period required to capture reliable and representative data may be too long to accept ‘other things remaining unchanged’ as a good estimate of comparable situations.

Information gathered from various banks by the BCBS has confirmed the unavoidable inference – banks do not have a formal back testing programme for verifying their internal credit risk historical loan loss distributions are compared to actual losses to arrive at an estimate of unexpected losses against which economic capital is allocated. In the absence of a better alternative, many banks rely on other methods to validate their credit risk models such as peer group analysis, rate of return analysis or comparisons of market credit spreads with those implied by a bank’s own pricing model. These are presumed to be.

Stress testing can cover a wide range of scenarios and has comparatively easier applicability for credit risk models testing and back testing should in the value of a particular risk factor e. g. credit spread. However, most internal ratings based models (developed to capture pure credit risks from the point of view of pricing) and many external ratings agency models are structured in a manner that does not enable case of counter party risk which includes, in addition to the different risks subsumed into default risk (e.g. interest rate risk, exchange market and operational risks such as legal risk, country risk, reputation risk, the risk of resource allocation strategy etc.

One of the approaches suggested for stress testing uses the following logic:

Risk Capital >= Unexpected loss+ max.[0;(Expected loss – specific loan loss provisions – other general provisions which are not a part of the capital)] Unexpected loss can or estimates of catastrophic loss (in an extreme case) can be added to the equation on the right hand side.

Market Risk:

As already stated, VaR models used by a bank are to be subjected in actual practice are quite similar to the framework developed by the BCBS. The process involves a comparison between the number of times the VaR model under-predicts the is expected. Two approaches are portfolios are constructed to match the VaR exactly. In the second approach, actual trading outcomes are compared with model-generated the last year is recorded. On the average, if losses exceeding VaR have a 1 in 100 chance i.e. 1% of the number of trading days, in a year. We have seen above that more than 1% exception would mean that the VaR model is understating the risk.

Stress testing is of great importance for both general and specific market risks because VaR alone cannot give any idea about the worst cases of the portfolio. General market risks risk relates to individual exposures in stress testing involves identifying the market variables to be stressed or given extreme values, as also deciding the scale and the time horizon over which the identified variables testing scenarios should be used, covering all such low probability events in all major types of risks, as can create extraordinary losses or gains thus making the control of risk in those portfolios very difficult. Quantitative stress tests should aim at comparing the risk capital assessed by the internal model vis-à-vis the potential loss whereas the qualitative tests should suggest the steps to be taken to mitigate the quantum of loss.

Operational Risk:

The constraints associated with validating estimates of economic capital for operational risk at a given soundness standard and risk horizon flow from scarcity of data because of which banks presently are not even able to compare their benchmark parameters and actual losses with those the difficulty banks face in applying formal statistical techniques such as back testing and hypothesis testing because of low amount of available loss data for certain operational risk loss event types – as compared to the market risks.

However, the Committee has suggested that ‘banks may use back testing techniques to assess the reasonableness of qualitative factors such indicators to actual historical loss experience may shed light on the appropriate choice of develop mechanisms to pool their parameter and loss estimates against those of industry peers’. The approach to mitigating operational risks should essentially consist of both the analysis of operational risk losses under normal conditions (high-frequency, low impact or including worst case scenarios (low-frequency, high-impact or Type ‘B’ losses).

The BCBS feels that increased use of automation can transform a high-impact event e.g. a prolonged disruption of services that depend on computers and networks. High frequency events run with one or two standard deviations, catastrophic events lie in the tail of the distribution Operational risks are managed largely through conventional wisdom and this approach has not been supported by any thought recommended by. Nevertheless, banks must put in place appropriate systems to capture the probability and frequency of loss in the case of each category of events and develop contingency plans to mitigate the risks.

The basic indicator approach does not attempt a genuine assessment of operational risk – the risk capital charge is determined merely as a fixed percentage (alpha) of a basic indicator which is average gross income over the previous three years (used as a proxy for exposure). Under the standard approach, the aforesaid eight business lines are each subject to seven multiplied by a factor (beta) to arrive at the risk capital required for that business line and the total of risk capital required for all the eight business lines gives the total risk capital required.

Risk management imperatives:

Before concluding, it needs to be re-emphasised that bank managements must realize before it is too late that stress testing is crucial to critically examine the vulnerability of the organisation of a series of extreme events. Banks, therefore, have to start and experiment with scenario analysis to get an idea as to how much capital would be adequate and results should be reported to management and Board of Directors periodically whose response should be reflected in the contingency plans put in place and policies and limits fixed by them. During a market crisis, historical markets collapse of these vulnerabilities to the management.

The management will also be well advised to take other risk control measures to avoid a total folding up in case these unlikely events occur. At the same time, the downside of scenario analysis is also an important factor. The scenarios are based on an arbitrary combination of stress shocks and quite a few of them may be inconsistent with the basic laws of economics. While constructing a scenario, care should be taken to ensure that the chain of events that follow is logical and it makes economic sense. It must also be borne in mind that stress testing is a more subjective approach than VaR. The choice of scenario itself introduces an element of subjectivity in the analysis.

Also, scenarios are constructed on the basis of past experience but history may not be a good indicator of what lies in future. Crises often span longer time periods during which many market variables may vary from one period to another. The portfolio itself may change. The assumption that events occur simultaneously may not enhance the dynamic characteristics of a scenario. Another major limitation is that scenario analysis does not indicate the probability of occurrence of any particular scenario and it is not possible to guard against every single potential disaster. Nevertheless, stress testing does help in identifying undetected weaknesses in the bank’s portfolio.

  1.  Bank Credit Requirements for Market Risk: The Internal Models Approach by Darryll
    Hendricks and Beverly Hirtle, FRBN Economic Policy Review (December 1997)
  2. Stress Testing by Large Financial Institutions: Current Practice and Aggregation Issues,
    Committee for Global Financial System, BIS, Basel.
  3.  The New Basel Capital Accord (April 2003)
  4.  Principles for the Management of Interest Rate Risk, Basel Committee for Banking
    Supervision (September 1997)
  5.  Supervisory Framework for the use of “Backtesting” in conjunction with the Internal
    Models Approach to Market Risk Capital Requirements, Basel Committee for Banking
    Supervision (January 1996)
  6.  Sound Practices for the Management and Supervision of Operational Risk, Basel Committee on Banking Supervision (December 2001)
  7.  Stress Testing: Risk Management Strategies for Extreme Events by Dimitris N. Chorafas,
    Euro money Books
  8.  Financial Risk Manager: Handbook 2001-2002 by Philippe Jorion, John Wiley &
    Sons, Inc.
  9.  Risk Management by Michel Crouhy et al., McGraw-Hill
  10.  Elements of Financial Risk Management by Peter F. Christoffersen, McGill University
    and CIRANO.

Popular Posts