On yesterday’s post I talked about the evaluation of system characteristics through Monte Carlo (MC) simulations and how these results are better indications of system quality than those derived from simple system analysis. I also talked about the fact that Monte Carlo simulations could prove useful for the evaluation of broker dependency, something which I intend to evaluate more in depth on today’s post. First of all I am going to talk about what changes in a system from one broker to another and how these changes can be at least approximated through a Monte Carlo simulation. You’ll see how MC simulations allow us to detect how a system behaves against distortions in its distribution and how – in the end – we need to carry out an “MC simulation of MC simulations” to get a better look at what broker dependency might generate.

The first thing we need to understand is how changing a broker feed to another affects systems. Whenever you use a broker you subject a system to a particular feed which depends on the liquidity providers used by the broker, each broker has its own set of execution conditions and its own set of specific characteristics (such as ending point of daily candles, week starting time, etc). A system’s results will change significantly between brokers due to these reasons in mainly three ways. The first one is that a given percentage of positions will not be taken, the second is that a large percentage of the positions taken will mismatch between brokers by a small percentage (execution differences) and the third one is that some positions taken will have a completely different outcome (for example a very small amount of positions may miss the TP by a few pips and then go to SL and vice versa).

–

Once we understand how the results of a system might change we can use an MC simulation to carry out a distortion of the distribution of the system such that we take into account the above scenarios. Of especial interest for us is to take into account the second and third scenarios which are the ones that most importantly affect profitability in an MC simulation since missing trades would just reduce the sample size of the simulation (which can be done simply by changing the input). These two changing scenarios can be easily implemented by adding distortions to trades that match our specific dependency “guess”.

In order to simulate random broker dependency we introduce a random distortion to a certain percentage of trades – chosen randomly – which oscillates between X and -X% with a set maximum of X%. For example if we choose that 20% of trades will have a +/-0.1% maximum distortion the MC simulator will affect 20% of trades taken by a random factor between -0.1 and 0.1% effectively simulating the effect of a random distortion. We could also simulate for worse trading conditions by simply making this distortion “negative only” meaning that 20% of trades will be affected with a loss in profit between 0 and -0.1%. The second characteristic we want to emulate follows a similar idea, we take a given percentage of trades – for example 5% – and completely inverse the outcome of a random selection fitting this size. Of course this has the problem of overestimating possible trade reversals in systems with reward to risk ratios higher than 1 but it is a very good idea for systems with ratios close to this value.

Of course since the effect of the above distortions affects a random selection of trades equal to the specified percentages every run will have a different overall statistical shape which will change continuously as the actual distorted trades change across different MC runs. The solution – which will be implemented in the future – is most likely to run an MC simulation of MC simulations where we simply make many iterations of distorted Monte Carlo simulations to see if we can find some convergence towards a given set of numbers. However running individual distorted runs does give us important information about broker dependency which we can use to evaluate the robustness of our systems against this problem.

When I ran several Asirikuy portfolios with distortions it quickly became evident that the effect of distortions depends strongly on how the trade distribution of a portfolio exists. Whenever your profitability depends on a larger percentage of smaller profitable trades distortions are negative since a big percentage of these trades is easily moved into losing territory by the small random distortions. Inversions also seem to have a net negative effect as they often have the effect of making the WC scenario deeper as randomly shifted losing trades fall from time to time within strong draw down periods that make them stronger. Of course if you include only negative distortions things become even gloomier. However it is worth noting that all profitable systems remain profitable under moderate distortions and only become unprofitable when the distortion is very strong (100% of trades affected by random distortions and more than 30% trades reach inverse outcomes).

Although I have yet to explore this in further depth it is already clear that this will be a good way to evaluate systems against broker dependency. For example we could discard systems that do not survive to a certain level of distortion or compare systems based on how much random or inverse distortions they can survive to. This is the first tool we have to evaluate some results of broker dependency against our systems so it is bound to be very useful, adding another dimension to the way in which we compare systems to each other.

The above mentioned strategies have already been implemented into v0.6 of our Asirikuy Monte Carlo simulator which will be released this weekend. If you would like to learn more about my work in automated trading and how you too can learn to create and evaluate strategies using sound techniques please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

Daniel,

do yo have data to compare the trading in Alpari with respect to other brokers?. I ask because of the historic data used come from Alpari. Do you know whether trading in Alpari fits better backest/MC than trading in other brokers?.

The systematic errors also come into play, we might find out that there are brokers better fit to trade some EAs than other. This would be very worth information.

Thanks.

I was also wondering the same thing about Alpari when looking at the backtests and the obvious differences between brokers when looking at live tests on myfxbook.

Did you already make a distorsion analysis between Alpari and other freely available quality feeds such as Dukascopy, for the recent years ?

[…] April 15th I wrote a post about simulating the effects of broker dependency using Monte Carlo simulations, introducing […]