Measuring trading system risk: Understanding risk outside of historical performance measurements

Trading system risk is one of the most important variables to consider when deciding whether or not to trade a given strategy. Traders generally adopt criteria to measure risk that are often misleading, causing significant under-estimations of risk and often heavy loses when the system is traded under live conditions. On today’s post we’re going to be talking about what risk really means, how it can be measured and why historical performance is often a poor proxy for a system’s real risk. After reading this post you should be able to assess whether you know or don’t know the risk of the systems you’re trading and what this risk might actually be when you look at realistic ways to estimate it. We’re also going to look at historical performance and its relationship to risk and why using historical performance to determine risk can be misleading.

2014-09-22_15-38-45

First of all I want to talk about the concept of system risk. Many of you will probably have notions related with historical performance as ways to estimate the potential risk of a trading strategy. I bet you could say that a system with a 10% maximum drawdown is less risky than a system with a 50% maximum drawdown, right? Well, I think this way of looking at risk is wrong, let me show you why. The problem here lies within the very definition of “risk”, when looking at risk for live trading we are not concerned about the loses a system had in the past but we are concerned about the loses that a system will have in the future. A trading system’s risk relates to a system’s potential for loses going forward. However it is rather obvious that the future is unknown so how can we estimate the risk of a trading strategy if we don’t know the future?

The key here lies in the fact that we make the decision to trade a system and we also make the decision to stop trading it. Therefore, risk is better viewed as the cost that we have to pay before we can decided that a system should not be traded further. A system’s risk is in fact the amount of money it loses before we decide to stop trading it, it is how much we need to lose in order to be sure that the system’s proposition is no longer valid. This way of looking at things brings some immediate consequences that make it such a great way to look at actual trading system risks.

When using this approach it becomes clear that no high risk approach can ever be considered low risk, despite of historical performance. Let us say that you have a martingale trading strategy that only has a 5% maximum draw down within a 20 year period test. However due to the martingale nature of the strategy you cannot be sure that the strategy has failed until you reach a total account wipeout, therefore the system risk is 100% (there is no statistical boundary, the Monte Carlo simulation of any Martingale money management will give a 100% worst case scenario). Even if a martingale has absolutely perfect historical performance, the fact that in order to discard it you require a series of loses enough to wipe your trading account makes it incredibly risky. Instead if you have a highly linear regular trading strategy that always risks 0.5% per trade and you have a 20% maximum historical draw down you can remove the system when it falls out of the expected equity growth pattern – per the linear model – and this risk can even be lower than the historical maximum draw down. In this case you might be able to discard the strategy with just a 5% draw-down, so the strategy can be considered low-risk.

Looking at risk from the perspective of system discarding allows you to avoid falling into the trap of going into highly risky strategies due to good historical performance (real or back-tested). This is because high risk systems can never be discarded before they cause catastrophic problems in the account, while low risk strategies have very predictable growth patterns (linear or otherwise) that can lead to discarding of the strategies with minimal additional loses, as they reach clear boundaries established through formal mathematical means (like linear regressions for example). If you think about risk as the amount of money that it will cost you to remove the strategy from trading you will most probably never go into a trading system that can wipe out your account again because this will be obvious when doing the risk analysis.

2014-09-22_15-45-31

Strategies that have low risk also allow you to better time your trading starting times. For example if I have a highly linear trading strategy – meaning that the log(balance) increases linearly with time, with a correlation higher than 0.95 – with a standard deviation of 3% and I choose the 3.5x standard deviation as a worst case scenario, I can time my trading start to when the strategy is within a 1-2x standard deviation multiple below the linear regression line. This means that I can enter the strategy with a risk of only 3-6% while the historical maximum drawdown may actually be much higher than this. Since the risk is only the amount of money I am putting on the line to discard a trading strategy, if I only need a small additional loss before discarding – because the strategy is highly linear and has an expected tendency to mean revert – then the strategy can be very low risk from the perspective of my particular market entry. The above is impossible to do with a strategy that has an unbounded risk because loses are always going to be 100% before a strategy can be discarded, regardless of the starting point.

From the above it is clear that the best way to estimate risk is to have strategies that fit some clear and concise growth model for which there is a clear statistical reasoning for discarding at some precise points (linear or otherwise). Risk for strategies that do not grow like this needs to be estimated through other methods – such as Monte Carlo simulations – which usually leads to risks that are worse than the historical maximum drawdown values.  In the end risk is the amount of money you need to lose before you can – through some statistical criteria – say that a strategy is not working anymore (above a given level of confidence). The risk is what you are willing to lose before you remove a strategy from trading. 

If you would like to learn more about estimating risk and building systems that can fit a well defined linear growth model please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

Print Friendly, PDF & Email
You can leave a response, or trackback from your own site.

7 Responses to “Measuring trading system risk: Understanding risk outside of historical performance measurements”

  1. Lorenzo says:

    Great work,

    Evidently one can imagine that every system will fall under the red line, my question is what indicator would you use to (re-)enter the trading system ?

    • admin says:

      Hi Lorenzo,

      Thanks for writing :o) I never re-enter a trading system that has failed because a breach of the worst case scenario implies that the system is already too risky to be traded (it failed its growth model), entering that system again would imply an acceptance of a higher risk level because you would be accepting a previous breach as acceptable. Since we have large system mining capabilities (using PKantu which implements our GPU cloud mining) we simply create a new system that is able to give acceptable risk for the whole back-testing period, including the period where the discarded system failed (of course properly accounting for bias sources within our mining process).

      From my point of view it makes no sense to test if an old system that was discarded should be reconsidered, if it failed, it failed forever as failure constitutes a breach of a clear statistical criteria that cannot be “undone”. With modern CPU/GPU capabilities we can generate as many systems as we need and replace systems that have failed with new ones that have better statistical characteristics. Lacking this ability to generate new systems and having the need to “re-evaluate old systems” puts traders at a disadvantage against traders with large mining abilities (provided mining is done with proper accounting of bias sources). I hope this answers your question :o)

      Best Regards,

      Daniel

      • Ryan says:

        Daniel,

        I understand what you’re saying, however, there will be times when a system is out of synch with the market for periods of time that breaches various statistical significance tests (t test, STDEV confidence measures etc.) but I don’t necessarily agree with retirement as the best option.

        I trade the DAX and one of my trades is a gap trade. I have traded this strategy since late 2009 and average about 40-50 trades a year (2002 to 2009 was in sample and out of sample data test period). The trade period 2010-2012 was excellent but found that 2013 broke down rather hard (blew all of my tests of significance) but along came 2014 and there was a full recovery. I also gap trade Euro Stoxx 50 and found that this closely correlated market did not break down as hard as the DAX market during 2013. The edge still apparent in Stoxx 50, coupled with my knowledge of the DAX market, saw me partially ‘hang on’ and ride the DAX gap trade recovery. I mention partially hang on as I did stop trading the DAX market as per my rules when stats backed up my ‘time to stop trading’ antennae and I was able to allocate capital toward my Stoxx 50 gap strategy.

        The key comparison (for me) was between the 2009 – 2012 OOS & trade validation data against the more recent 2013 trade data. While the stats were showing failure, I was not comfortable in writing off a strategy that had one bad year.

        Could 2014 be a strategy dead cat bounce? Always a possibility and something that I will continue to keep an eye on but the mean reverting nature of the gap trade is apparent across a number of global equity markets and I was not going to let stats stop me out of what I considered to be a viable strategy.

        Thanks for your article(s)
        Ryan

  2. Daniel says:

    Hi Daniel

    I am posting here because your website says that My message was posted successfully bu then I receive a failed message send in my email. Is there any other way I can communiacte with you regarding EAs and possibly subscribing to your site?

    Daniel

    • admin says:

      Hi Daniel,

      Thanks for writing. Sure, if you want you can send me an email to dfernandezp, at unal.edu.co,

      Best Regards,

      Daniel

  3. Yoshi54 says:

    Is everything alright Daniel? Been a little over a month since your last post, and hoping you were okay….

    Josh

    • admin says:

      Hi Josh,

      Yes, everything is OK :o) Just doing lots of developments. Some interesting stuff coming! Thanks for posting,

      Best Regards,

      Daniel

Leave a Reply

Subscribe to RSS Feed Follow me on Twitter!
Show Buttons
Hide Buttons