The Homework Corner : Understanding Simulation Limitations, Part One– Assignment No.3

Certainly one of the most important aspects of algorithmic trading system development is the evaluation of trading strategies across past market conditions. Testing a strategy’s past performance is critical as this is the way in which we measure if an edge is present, if the strategy is adaptive and the way in which we gather adequate information over the expected long term statistical characteristics of a strategy. Without some type of historical evaluation, algorithmic trading wouldn’t be possible since we wouldn’t be capable of knowing whether or not our systems have an edge which is worth pursuing. On today’s homework assignment I will attempt to teach you a little bit about the limitations of simulations in MT4, particularly pertaining to the inherent limitations of simulations and what is needed to get reliable results.

Most people who develop expert advisors online are very unaware of the faults within their simulations and the possible overestimations of profitability their systems may suffer due to these problems. You will often see people trading systems that have taken 100 USD to billions on backtests when in reality such backtests are obviously flawed, with the reasons becoming very clear at a close inspection. If you’re interested in reliable strategy development then it becomes critical for you to understand what is required for reliable simulations.

This is your homework assignment for this week :

1. Read this article on

2. Run the first system proposed on a demo account for one week

3. Run a backtest for that same trading week

4. Read these mechanical forex articles (No.1, No.2, No.3)

After you have carried out the above assignment please answer the questions below :

1. What overall results did your one week backtest have against your forward trading results ? Did the results match ? What differences could you see on the tests ?

2. What is one minute interpolation ? How does it affect testing quality and how can its effects be avoided ?

3. What does the modeling quality number on MT4  mean ? How does it relate to simulation quality ?

4. Even if tick data on one broker was available for 20 years, would that information be useful to trade a system on another broker on a market with no central exchange ? Explain why you think this might or might not be the case.

5. How can computationally cheap simulations using only End of day data provide reliable results with little complexity ? How do you know such results are reliable ?

6. What do you think – after carrying out this assignment – is required for reliable strategy evaluation within MT4. How does this differ from the vision you had before ?

7. Let’s suppose I want to develop a system to capture 100-200 pip movements based on entry/exit points on the 15 minute time frame. Write a list of the requirements that would be needed to evaluate such a strategy reliably, what the limitations of the evaluation might be or -if you think this cannot be done – why you think this system could never be evaluated in a meaningful way.

The above reading/practical exercise will provide you with some insight into trading simulations and their limitations but there is clearly still a lot of things which must be done in order to grab a good grasp of what needs to be done to develop systems in a reliable manner. Within the next few weeks I will continue to do some practical exercises on reliable strategy evaluation with some additional interesting insights into systems which are unreliable for reasons different to those explored within this assignment. As always feel free to leave a comment within the post with your assignment results and answered questions once you finish your homework :o)

If you would like to learn more about my work in automated trading and how you too can learn to build systems based on reliable simulations please consider joining, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

Print Friendly, PDF & Email
You can leave a response, or trackback from your own site.

Leave a Reply

internal_server_error <![CDATA[WordPress &rsaquo; Error]]> 500