Is my system a good system? This is perhaps the question that all trading system designers should ask themselves when they end up the design of one of their trading strategies. I have written several posts on curve fitting during the past few months dealing with this subject and the development practices which need to be used in order to avoid getting curve fitted results. However, it is true that regardless of the level of care we put into the design of our strategies, there will always be the possibility to get curve fitted results or simulations that are not reliable for one reason or another. On today’s post, I am going to talk about the stress-testing of systems and the different types of tests I put all my systems through to ensure that they have the highest possibility of working under future market conditions. To learn the tactics that other traders don’t want you to know, you can click links like this Kiana Danial review.
When you design a trading system you should put a lot of thought into your design procedure to ensure that your end results are as reliable as possible. However, it can be the case that you end up with unrealistic results because you somehow introduced a “bad element” into your trading system which caused it to become a very unreliable strategy with unsound results. It is important to understand that even when we take great care not to introduce any unsound tactics into our trading systems we can – often unconsciously – introduce certain characteristics that may make our systems lack robustness and even lack adequate simulations.
–
–
This is precisely why it is extremely important to stress-test strategies to ensure that any elements which may have come from such involuntary introductions are found. I generally use two types of different stress-tests to evaluate my strategies to search for both spread dependency and curve fitting which are the two most dangerous enemies we encounter when developing strategies, especially within MT4/5.
The first type of test is conducted to verify that a strategy is not dependent on the assumption of constant spreads for its profitability. Sometimes you may create a strategy that looks apparently sound but upon a closer look at its internal logic you will find that most profits are gained through trade closed within spread values (or close to that). In order to stress-test a system to ensure this is not the case I generally run a backtest with 3 times the spread and compare it to the original tests. If there is a drop of more than 50% on the average compounded yearly profit’s value then I consider that the system is not robust enough and too dependent on the assumption of a constant spread value.
The second characteristic I evaluate – curve fitting – is tested through three different procedures which allow us to know how robust the strategy is. The first test involves the use of out-of-sample tests to verify that the system trades (out of the period used to optimize it) with good results. I generally use a 10% out-of-sample test and evaluate it against the long term characteristics of the strategy per their Monte Carlo simulation results but certainly longer out-of-sample tests (such as 20-100% of the original testing period) might be better. When it comes to out-of-sample tests, the longer the better.
The second procedure I use evaluates the symmetry of the strategy and the presence of a symmetric edge. I evaluate long-only and short-only versions of the strategy and ensure they are both profitable so that I can have certainty that the strategy has been designed with a symmetric long term edge in mind and not through the exploitation of some underlying market characteristic (like a very long term trend) which is definitely bound to change in the future. Having overall profitable long and short only back tests is definitely an important part of system robustness evaluations.
The third test I use evaluates the system’s robustness to change, evaluating individual +/-1% variations of all of the system’s parameters in order to ensure that the neighbors of the strategy are also profitable. If a sharp decrease in profitability happens after a 1% variation then the system is not robust but merely “curve fitted” to give good results. A good strategy is always very robust to change and small variations will not introduce extreme losses in profitability. Stress-testing a system against small variations of its parameters is therefore very important to evaluate possible curve-fitting that might have been introduced within the optimization procedure.
Definitely the above stress-tests do not give you a 100% guarantee that your systems will work in the future but you can be certain that if a system fails any of the above tests there is a good probability that it will fail (or under-perform) in the future when compared to its simulation results. A system which can withstand a backtest with 3 times the spread, has > 10% out-of-sample tests within Monte Carlo statistical characteristics, has short-only and long-only profitable backtests and profitable outcomes within 1% variations is a robust strategy which has a very good probability of surviving to changes in future market conditions.
Last but not least it is always important to remember that your objective in forex trading is not to get “pretty simulations” but to get systems which can give you profit with a high probability in the future. Assuming all your systems are “bad” until proved otherwise and putting them to as many stress-tests as possible ensures that you will only run systems which are robust and have the highest like hood of performing up to your expectations. If you would like to learn more about my work in automated trading and how you too can develop your own strategies please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)
