Stress Testing Systems : Learning About Monte Carlo Simulations

When we develop automated trading systems geared towards the long term production of capital we are often faced with a very simple and daunting question. How can we ensure that our systems will remain profitable in the long term and how will we know when our systems fail to be profitable ? The answers to these questions are not very easy and certainly there is no way in which we can ensure a system will remain profitable since we simply do not know the future. However there are several ways in which we can have an idea when a system stops working, with the most robust one being the application of very long term stress-test simulations to see what the systems worst outcomes might be. Today I am going to introduce you to the world of Monte Carlo simulations, a very powerful method which enables us to get accurate estimates of what the “worst possible outcomes” of our strategies might be.

The Monte Carlo simulation is a type of forecasting procedure in which a given “experiment” that depends on a “random factor” is tested a very large number of times. The easiest way to exemplify the Monte Carlo procedure is to imagine a procedure that has two outcomes with equal probability. Imagine we have a coin toss, 50% chance of getting heads and 50% chance of getting tails. When you toss a coin many times it is obvious that you will not get heads-tails-heads-tails type sequences all the time but sequences involving head-heads-heads-tails or something of that sort may in fact arise. A Monte Carlo simulation simply performs thousands of coin tosses and gives you the expected runs of heads and tails you would expect if you did this in the long term. We can say that Monte Carlo simulations allow us to see the extent to which a given series of individual random probability outcomes may temporarily differ from their long term statistical outcome (which we assume to be certain).

-


-

When we evaluate trading systems this means that we will take our system’s winning percentage, average profit and average loss and execute a Monte Carlo simulation of about X trades evaluated X times ( for example 100K separate runs of 200 trades). Using this very large number of trades we will be able to know what the “worst consecutive losing run” and “worst draw down” of our strategy might be given the statistical randomness of short term results. This gives us some very important information as we now know what the “worst behavior” of our system might be before it deviates from its true long term statistical characteristics. For example if you have a system which has a predicted 24 maximum consecutive losing run within a Monte Carlo simulation and it eventually reaches 26, then you know that your system has become too risky to be traded because it has started to deviate dramatically from the long term expected Monte Carlo deviations.

Up until now in Asirikuy we have used a “double rule” to make up our worst case scenarios. For systems that have risk to reward ratios near 1:1 (from about 1:2 to 2:1) the “doubling rule” aligns pretty well with Monte Carlo simulations as the worst draw down scenario usually corresponds to twice the maximum draw down seen in back-tests. This means that our current approximation is certainly good although Monte Carlo simulations give us additional refinement in this regard. The number of consecutive losing trades is very high in Monte Carlo simulations and indeed a system may take up to 22-25 consecutive losses in the very long term, as shown by these simulations. (above, the results of a 200 trade subset of a 100K-200 trade run Monte Carlo Teycanani EUR/USD simulation are shown (the green line is the peak value for the given trade number while the red one is the lowest peak for that trade number across the 100K runs)).

A very interesting observation comes when you evaluate very skewed systems using this type of simulation. Systems like Megadroid – with very high winning rates and bad risk to reward ratios – give overall numbers in Monte Carlo of 7-9 consecutive losses across 100 K trades with a very high probability of having at least 4 consecutive losing trades within the first 10K trades and three consecutive losses being almost a certainty every 1K trades. Although such runs are quite infrequent they do happen, showing why – as I have said before – I keep my stand against using this type of systems as everything points out to the fact that they will wipe your account in the longer term. Since you cannot stop running them before they do (as you only know they stopped working when they wipe your account) these are very unsound all-or-nothing type strategies. Take into account that this assumes that the simulations of these systems are reliable, something very unlikely due to their low take profit targets meaning that wiping scenarios may be even much more common than what is portrayed by the simulations discussed above.

A very important part of Monte Carlo simulations is to ensure that the worst case draw down remains within a small portion of equity such that you can ensure that  if a worst case happens you will be able to stop trading the system and move onto other strategies that are still working. In systems with sound trading tactics and controlled risk this is always the case as you can determine the system stopped working regularly just at twice the historical maximum draw down seen in backtests, meaning that you can actually stop and change systems with a very good chunk of your equity still in place. Using a portfolio further increases your chances of doing this as you may be able to change a losing strategy without any hard losses in capital as the other systems which “keep working” take the account out of the draw down. As the probability of many systems failing at the same time is very small, the chances of just “fixing the parts that fail” becomes much bigger.

There is a truth we need to understand here: trading systems can and do fail sometimes as market conditions change beyond their ability to adapt. The important thing is that their failures don’t imply equity wipe outs enabling us to simply change the systems that have stopped working for new systems that work or for older systems that continue to perform accurately. It is not about having a system that will “last forever” (since presuming this would be unrealistic) but about having many robust systems with a low probability to fail coupled with a clear criteria that lets us know whenever a strategy starts to under-perform so that we can change it. Monte Carlo simulations allow us to do just that, develop a clear criteria of when a system might fail and what behavior we might expect in the longer term regarding consecutive losing, winning trades, etc.

Next week I will probably do a video for Asirikuy explaining how to run Monte Carlo simulations and what conclusions we can derive from them about our trading systems and their long term profit and draw down characteristics.  If you would like to learn more about my work and how you too can use and develop your own likely long term profitable systems based on sound trading tactics please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach to automated trading in general . I hope you enjoyed this article ! :o)

Print Friendly
You can leave a response, or trackback from your own site.

5 Responses to “Stress Testing Systems : Learning About Monte Carlo Simulations”

  1. Maxim says:

    Daniel,

    A very interesting article!
    Could you please explain what is low and high PnL?

    Maxim

    • admin says:

      Hello Maxim,

      Thank you for your comment :o) The green line represents the peak net profit for that trade (for example trade No.2 on each 200 subset) on a 100K – 200 trade simulation while the red line represents the peak loss for that trade. The Blue line represents the last 200 trade subset. You could take the green/red lines as “best” and “worst” case scenarios respectively for each 200 trade sample, an “envelope” that determines the system’s natural long term statistical boundaries for “any” 200 trade subset. Of course I will explain all these concepts much better during next week’s videos. Thank you very much again for your comment :o)

      Best regards,

      Daniel

      PS : I changed the text a little bit to better explain the above :o)

  2. andrea says:

    Hi Daniel,
    ok now i’m very pleased to see you have talked about montecarlo simulation. As i told you a few days ago, i use this simulation to understand not only which are the top and the bottom line of my system, but also to investigate the 95% (+- 2 dev.st from the mean) or 99% (+-3dev.st from the mean) probability of the possible results.
    So, in my graph i have 4 lines, like those you have drawn above, that help me to understand if the behavior of my system is within confidence interval of 95% or 99%.
    A very interesting aspect of this technique, is that you can see that if you have a system with profict factor >1 (or in other words a positive mathematical expectation), in the long time, you can always recover from a DD.
    Thank you very much for your posts. I’m very tuned with your methologies about trading.
    Andrea

    • admin says:

      Hi Andrea,

      Thank you very much for your comment :o) I am glad you liked this article ! The lines shown are in fact for a 95% confidence interval, which – as you mention- covers 2 standard deviations. On next week’s Asirikuy video I will cover many of these concepts also teaching Asirikuy members how to run their own simulations. It may also be plausible to start a section of the website in which we will include the Monte Carlo simulations of the different strategies. As you mention Monte Carlo simulations are powerful tools to understand what the future may hold if the system remains true to its long term statistical characteristics. Monte Carlo simulations allow us to understand our systems to a much better degree, therefore greatly increasing our chances of success. As you say, it is also interesting to observe that – as a Casino – a system with a positive edge always wins, regardless of the short term “negative deviations” arising from the randomness of short term statistics. Monte Carlo simulations tell us that short term results are random but long term statistics are not :o) I certainly hope you will enjoy my future articles on Monte Carlo simulations as well as all the content I will be developing for Asirikuy. Thank you very much again for your comment !

      Best Regards,

      Daniel

  3. [...] help you evaluate the worst case characteristics of your system then you might consider reading  this post where I explain some of the basics of the Monte Carlo procedure and how this leads to the [...]

Leave a Reply

Subscribe to RSS Feed Follow me on Twitter!