Mechanical Forex

This past week I received an email where a regular blog reader asked me if Monte Carlo simulations could be used to determine whether a system is more or less likely to fail in the future (how robust the system is) as there seem to be several software packages that offer Monte Carlo simulations as a way to measure strategy robustness. Today I want to talk about Monte Carlo simulations, the types there are, what they do tell us and what they cannot tell us about trading strategies. I will also talk about robustness, what this actually means and whether or not Monte Carlo simulations can or cannot help us discriminate strategies according to their level of robustness. In the end you’ll see that Monte Carlo simulations are no magic bullet and offer only limited insights into possible trading system robustness.

–

–

First of all let us talk about what Monte Carlo simulations are. In general these simulations randomize some measure of trade returns in order to obtain alternative equity curves that share the same distribution of returns. The idea is that a system that behaves as expected will take the same types of trades in the future, only that the order of these trades will be randomized as market conditions change. The strategy will remain exactly the same at its core, the distribution of returns will be the same, but the equity curves will vary significantly depending on each random draw. In general Monte Carlo simulations are limited by some fundamental assumptions – they do not work for every strategy type – and they give some important information related to worst case boundaries for strategies as we are able to tell whether a strategy’s results are likely to be draws from the back-testing distribution of returns at a given confidence interval.

But does this have anything to do with robustness? Robustness is the ability of a system to work, to survive to future market conditions. Systems that are more robust are systems that belong to a group which has a smaller tendency to fail under new market conditions than another system group. For example strategies developed over 10 years of data are more robust than strategies that are developed over 1 year because the first group has a much larger probability to survive a year of unknown market conditions. This can be shown to be true for at least the past 30 years across many different Forex symbols. But do Monte Carlo simulations can in any way improve robustness? In reality, they do not.

–

–

Monte Carlo simulations describe the boundaries of trading system behavior, these boundaries have little to do with future probabilities of survival. If a trading system can be said to fail by Monte Carlo simulations if it has faced a 5% drawdown over 1 month it is not more or less robust than a trading system that would fail at any different drawdown level within this same period. These different systems would only show different levels of risk – different amount of losses needed before discarding – but their probability of future failure is unrelated to this. A Monte Carlo simulation with better overall distributions or statistics only points to a better system under in-sample conditions – a system with a better distribution of returns – but it tells nothing about future survival. The boundaries determined by Monte Carlo simulations are boundaries for the past, they do not foresee future failure but only help determine when it happens.

There are also Monte Carlo simulations that attempt to vary price in order to see how price distortions affect trading simulations. The idea here is to see how robust the strategy is to market changes and whether it could survive to some shuffling of the underlying trading instrument. These distortions go from  small increases/decreases in the OHLC to bar shuffling and trade skipping. The results show how much the system relies on the underlying market data to provide its result but again, there is no relationship between these and future failure since the market conditions being tested are not known to be market conditions that the strategy will face in the future. There is no relationship between systems that are less affected by distortions and systems that survive under real out-of-sample trading conditions, because there is simply no relationship between these distortions and future market behavior. If future market behavior could be expressed as some sort of perturbation over existing data then we could create trading systems from first principles that would always survive under new data.

A system may show to deteriorate heavily under a Monte Carlo simulation based on distorsions and may yet prove to be very robust. Imagine a system that trades profitably because a company always performs a purchase that moves price after two days of consecutive higher highs, let’s suppose that this has happened and always will happen, the system will always be profitable. This behavior may be constant within the market but random distorsions would make this systems appear to be much worse than a system working on a much less robust inefficiency – say for example an MA cross strategy – because these distorsions only show how precisely the systems rely on past data to look like past data to be profitable, which has nothing to do with how the data will look like in the future. These distorsions favor strategies that apply heavy smoothing and do not rely on precise points within charts, but these strategies really have no higher likelihood to survive.

–

–

The above does not imply that Monte Carlo simulations are not useful. On the contrary they can be a wonderful tool to assess statistical strategy boundaries provided that the key assumptions of Monte Carlo simulations are respected and care is taken to avoid cases where Monte Carlo simulations simply fail to provide useful information (I suggest this article to learn more about them). However Monte Carlo simulations are not useful for robustness measurements since behavior under distorsions and worst case boundaries are simply not related to behavior under unknown market conditions — at least as far as I have been able to experiment.

Of course if you can program a Monte Carlo simulation to behave exactly like your trading system, expressing for example money management changes, it could tell you some obvious facts related to robustness – like how all Martingales and grids will with certainty always wipe out an account – but if you’re already coding strategies that have no trade chain dependency and sound trading principles they will do nothing to help you discriminate systems according to real robustness (probability of failure under new market conditions). If you would like to learn more about Monte Carlo simulations and how you too can diagnose your strategies and remove strategies that fail according to the boundaries then define please consider joining Asirikuy.com, a website fiAsirikuy.comlled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.