bat365 Homepage

Rolling Windows and Multiple Models for Expected Shortfall (ES) Backtesting by Du and Escanciano

This example shows the workflow for using the Du-Escanciano (DE) expected shortfall (ES) backtests for rolling window analyses and testing multiple VaR/ES models.

The rolling window workflow in this example is also used for the value-at-risk (VaR) backtests in varbacktest and for the Acerbi-Szekely ES backtests in the esbacktest and esbacktestbysim classes.

The multiple-model workflow in this example is also used for the esbacktestbysim class. For esbacktest and varbacktest, you can create a single object with multiple models and multiple VaR levels.

Rolling Window

The data in the ESBacktestDistributionData.mat file has returns, VaR and ES data, and distribution information for three models: normal, and t with 5 degrees of freedom and t with 10 degrees of freedom. The data spans multiple years from January 1996 to July 2003, for a total of 1966 observations.

To run the test over a rolling window, one esbacktestbyde object must be created for each year (or time period) of interest. In this example, each year from 1996 through 2002 is tested separately. You can test all VaR levels together, but to simplify the output, this example uses a single VaR level. You can also call any test, or the summary report inside the processing loop, but this example calls only the runtests function.

load ESBacktestDistributionData.mat

rng('default'); % For reproducibility

Years = 1996:2002;
TargetVaRLevel = 0.99;

t = table;
for TargetYear = Years

    Ind = year(Dates)==TargetYear;
    VaRInd = VaRLevel==TargetVaRLevel;
    
    ebtde = esbacktestbyde(Returns(Ind),"t",...
    'DegreesOfFreedom',10,...
    'Location',0,... % Always 0 in this data set
    'Scale',T10Scale(Ind),...
    'VaRLevel',VaRLevel(VaRInd),...
    'PortfolioID',strcat("S&P, ",string(TargetYear)),...
    'VaRID',"t(10)");

    t = [t; runtests(ebtde)];
end

disp(t)

    PortfolioID     VaRID     VaRLevel    ConditionalDE    UnconditionalDE
    ___________    _______    ________    _____________    _______________

    "S&P, 1996"    "t(10)"      0.99         reject            reject     
    "S&P, 1997"    "t(10)"      0.99         accept            reject     
    "S&P, 1998"    "t(10)"      0.99         accept            accept     
    "S&P, 1999"    "t(10)"      0.99         reject            accept     
    "S&P, 2000"    "t(10)"      0.99         accept            accept     
    "S&P, 2001"    "t(10)"      0.99         accept            accept     
    "S&P, 2002"    "t(10)"      0.99         reject            accept

For a more advanced approach, you can use arrays of esbacktestbyde objects and then call different functions on objects corresponding to different years as needed.

rng('default'); % For reproducibility

NumYears = length(Years);
ebtdeArray(NumYears) = esbacktestbyde;

TargetVaRLevel = 0.99;

for yy = 1:NumYears

    TargetYear = Years(yy);
    Ind = year(Dates)==TargetYear;
    VaRInd = VaRLevel==TargetVaRLevel;
    
    ebtdeArray(yy) = esbacktestbyde(Returns(Ind),"t",...
    'DegreesOfFreedom',10,...
    'Location',0,... % Always 0 in this data set
    'Scale',T10Scale(Ind),...
    'VaRLevel',VaRLevel(VaRInd),...
    'PortfolioID',strcat("S&P, ",string(TargetYear)),...
    'VaRID',"t(10)");

end

disp(ebtdeArray)

  1x7 esbacktestbyde array with properties:

    PortfolioData
    VaRData
    ESData
    Distribution
    PortfolioID
    VaRID
    VaRLevel

Display the summary for the year 2002.

disp(summary(ebtdeArray(Years==2002)))

    PortfolioID     VaRID     VaRLevel    ObservedLevel    ExpectedSeverity    ObservedSeverity    Observations    Failures    Expected    Ratio     Missing
    ___________    _______    ________    _____________    ________________    ________________    ____________    ________    ________    ______    _______

    "S&P, 2002"    "t(10)"      0.99         0.98467            1.2169              1.1481             261            4          2.61      1.5326       0

Concatenate the conditional tests for all years.

condDEResults = table;
for yy = 1:NumYears
    condDEResults = [condDEResults; conditionalDE(ebtdeArray(yy))];
end
disp(condDEResults)

    PortfolioID     VaRID     VaRLevel    ConditionalDE      PValue      TestStatistic    CriticalValue    AutoCorrelation    Observations    CriticalValueMethod    NumLags    Scenarios    TestLevel
    ___________    _______    ________    _____________    __________    _____________    _____________    _______________    ____________    ___________________    _______    _________    _________

    "S&P, 1996"    "t(10)"      0.99         reject         0.0084691         6.9315         3.8415             0.16265           262           "large-sample"          1          NaN         0.95   
    "S&P, 1997"    "t(10)"      0.99         accept           0.85691       0.032512         3.8415           -0.011161           261           "large-sample"          1          NaN         0.95   
    "S&P, 1998"    "t(10)"      0.99         accept           0.87949       0.022989         3.8415          -0.0093851           261           "large-sample"          1          NaN         0.95   
    "S&P, 1999"    "t(10)"      0.99         reject        2.1168e-50         222.89         3.8415             0.92412           261           "large-sample"          1          NaN         0.95   
    "S&P, 2000"    "t(10)"      0.99         accept           0.89052       0.018948         3.8415          -0.0085367           260           "large-sample"          1          NaN         0.95   
    "S&P, 2001"    "t(10)"      0.99         accept           0.92088      0.0098664         3.8415          -0.0061484           261           "large-sample"          1          NaN         0.95   
    "S&P, 2002"    "t(10)"      0.99         reject        3.5974e-05         17.073         3.8415             0.25576           261           "large-sample"          1          NaN         0.95

Multiple Models

Similar to the esbacktestbysim object, the esbacktestbyde object accepts only one distribution at a time. If you need to test different models side by side, then you must create different instances of the class.

In this example you run the test for a normal distribution assumption and t distributions with 5 and 10 degrees of freedom. You then concatenate the test results to generate a single report.

The data in the ESBacktestDistributionData.mat file has returns, VaR and ES data, and distribution information for three models: normal, and t with 5 and 10 degrees of freedom. The data spans multiple years from January 1996 to July 2003, for a total of 1966 observations. For simplicity, this example uses only data from 1998.

load ESBacktestDistributionData.mat

TargetYear = 1998;
Ind = year(Dates)==TargetYear;

rng('default'); % For reproducibility

Create an instance of an esbacktestbyde object for the normal distribution.

ebtdeNormal = esbacktestbyde(Returns(Ind),"normal",...
'Mean',0,...
'StandardDeviation',NormalStd(Ind),...
'VaRLevel',VaRLevel,...
'PortfolioID',strcat("S&P, ",string(TargetYear)),...
'VaRID',"normal");

disp(ebtdeNormal)

  esbacktestbyde with properties:

    PortfolioData: [261x1 double]
          VaRData: [261x3 double]
           ESData: [261x3 double]
     Distribution: [1x1 struct]
      PortfolioID: "S&P, 1998"
            VaRID: ["normal"    "normal"    "normal"]
         VaRLevel: [0.9500 0.9750 0.9900]

disp(ebtdeNormal.Distribution)

                 Name: "normal"
                 Mean: 0
    StandardDeviation: [261x1 double]

Create an instance of an esbacktestbyde object for the t distribution with 10 degrees of freedom.

ebtdeT10 = esbacktestbyde(Returns(Ind),"t",...
'DegreesOfFreedom',10,...
'Location',0,...
'Scale',T10Scale(Ind),...
'VaRLevel',VaRLevel,...
'PortfolioID',strcat("S&P, ",string(TargetYear)),...
'VaRID',"t(10)");

disp(ebtdeT10)

  esbacktestbyde with properties:

    PortfolioData: [261x1 double]
          VaRData: [261x3 double]
           ESData: [261x3 double]
     Distribution: [1x1 struct]
      PortfolioID: "S&P, 1998"
            VaRID: ["t(10)"    "t(10)"    "t(10)"]
         VaRLevel: [0.9500 0.9750 0.9900]

disp(ebtdeT10.Distribution)

                Name: "t"
    DegreesOfFreedom: 10
            Location: 0
               Scale: [261x1 double]

Create an instance of an esbacktestbyde object for the t distribution with 5 degrees of freedom.

ebtdeT5 = esbacktestbyde(Returns(Ind),"t",...
'DegreesOfFreedom',5,...
'Location',0,...
'Scale',T5Scale(Ind),...
'VaRLevel',VaRLevel,...
'PortfolioID',strcat("S&P, ",string(TargetYear)),...
'VaRID',"t(5)");

disp(ebtdeT5)

  esbacktestbyde with properties:

    PortfolioData: [261x1 double]
          VaRData: [261x3 double]
           ESData: [261x3 double]
     Distribution: [1x1 struct]
      PortfolioID: "S&P, 1998"
            VaRID: ["t(5)"    "t(5)"    "t(5)"]
         VaRLevel: [0.9500 0.9750 0.9900]

disp(ebtdeT5.Distribution)

                Name: "t"
    DegreesOfFreedom: 5
            Location: 0
               Scale: [261x1 double]

Run the tests and then concatenate the results.

testResults = [runtests(ebtdeNormal); runtests(ebtdeT10); runtests(ebtdeT5)];
disp(testResults)

    PortfolioID     VaRID      VaRLevel    ConditionalDE    UnconditionalDE
    ___________    ________    ________    _____________    _______________

    "S&P, 1998"    "normal"      0.95         accept            accept     
    "S&P, 1998"    "normal"     0.975         accept            accept     
    "S&P, 1998"    "normal"      0.99         accept            reject     
    "S&P, 1998"    "t(10)"       0.95         accept            accept     
    "S&P, 1998"    "t(10)"      0.975         accept            accept     
    "S&P, 1998"    "t(10)"       0.99         accept            accept     
    "S&P, 1998"    "t(5)"        0.95         accept            accept     
    "S&P, 1998"    "t(5)"       0.975         accept            accept     
    "S&P, 1998"    "t(5)"        0.99         accept            accept

Display the results for a VaR level of 0.99.

TargetVaRLevel = 0.99;
disp(testResults(testResults.VaRLevel == TargetVaRLevel,:))

    PortfolioID     VaRID      VaRLevel    ConditionalDE    UnconditionalDE
    ___________    ________    ________    _____________    _______________

    "S&P, 1998"    "normal"      0.99         accept            reject     
    "S&P, 1998"    "t(10)"       0.99         accept            accept     
    "S&P, 1998"    "t(5)"        0.99         accept            accept

Rolling Windows and Multiple Models for Expected Shortfall (ES) Backtesting by Du and Escanciano

Rolling Window

Multiple Models

See Also

Related Topics