Optimizing Without Curve-Fitting : Six Tips to Avoid Over-Optimization

Coding successful trading strategies can be a long and frustrating process, especially when it comes to the part where you improve your trading strategy to make it perform better on past market data. The process by which a given system has its parameters adjusted to give better performance in the past is denominated “optimization”, a process by which the results with many different parameter sets are compared and the best ones amongst those are chosen. Optimization is a natural part of system development since changes in certain things – like indicator periods, stop loss and take profit values – can dramatically affect the performance of a trading strategy. However one of the main problems of performing optimizations is the dreaded word : curve fitting. You can read more about definition of curve fitting on this post I wrote earlier this year and you can also read this post to learn about five very common mistakes people run into when performing optimizations.

Obviously avoiding curve fitting should be a very important part of any system developer’s efforts as we don’t want to generate trading strategies with absolutely astonishing results that will not be achievable going forward. Since our goal is to produce systems that achieve good performance in the past with the highest possible guarantee that that performance will be repeated in the future it becomes vital to take steps in order to ensure that optimization does not deliver curve-fitted strategies. Here are six important tips you should implement to avoid curve-fitting strategies to the past :

1. Avoid unreliable simulations. Perhaps the most important thing you need to do in order to avoid curve-fitting a strategy is to completely avoid the use and optimization of systems which cannot be simulated accurately. Simulating systems that trade on time frames lower than 30 minutes or systems with very small take profit and stop loss targets (below 10 times the spread) should be absolutely avoided as the results will not be viable and a lot of curve fitting to past data will most likely take place under optimization. Not only will the results be meaningless but exploitation of backtesting interpolation errors and broker dependency will play a primordial role.

2. Code simple systems. Complexity is the mother of curve-fitting.  Whenever you give a strategy enough degrees of freedom an optimization will yield curve fitted results. The less complexity and less parameters available within a given strategy the less probable it is that it will ever be curve fitted as systems that don’t have complex criteria tend to be unable to “fit” to the data if a true inefficiency is not present. It therefore becomes extremely important to code simple “elegant” strategies in order to avoid added complexity which will result in curve fitted solutions.

3. Long testing periods. Optimizations should be carried out for long periods of time, ideally 9-11 years of data should be used for the process in order to ensure that a large amount of market conditions become available. If a simple strategy yields profitable results across a ten year period then the probability of curve fitting is greatly reduced as the system has limited degrees of freedom to artificially “fit” all those different market conditions.

4. Keep your system symmetric. One of the first ideas new traders have when they start analyzing system development and mathematical expectancy results is to have a separate criteria for entering and exiting short and long trades (for example using an indicator cross at 20 for long entries but 15 for shorts). Although it is true that under past data up and down trends might have developed differently in currencies this cannot be guaranteed to continue in the future as these differences rely on interest rate differentials or such similar macro economic variables that inevitably change through economic cycles. Adding separate criteria for longs and shorts automatically increases the strategy’s degrees of freedom and makes it excessively prone to curve-fitted solutions.

5. Out-sample testing periods. A very common practice in system development is to have a certain amount of historic data “outside” the optimization set – usually one year or two -  in order to perform a simulated “forward test” (commonly referred to as out-sample test) to see how the strategy behaved under new market conditions without being able to artificially “fit” into this data. Certainly the out-sample test doesn’t have to be profitable (as all strategies have profit and draw down periods) but it must at least hold correlation to the draw down depths and profitable periods seen in the past. When an out-sample test shows a loss higher than twice the previous maximum draw down then the strategy is certainly curve fitted.

6. Avoid common optimization mistakes. There are many different small things you can do wrong when performing optimizations, such as using highly correlated procedures, fine grid tests or ignoring the “surroundings” of your intended results. A very important part of avoiding curve-fitting is to avoid these common mistakes to perform coarse and efficient optimizations that do not predispose your system towards curve-fitted solutions. (read this post to learn about five very common mistakes people run into when performing optimizations).

By following the above five tips you will have a much higher like hood of developing strategies that are not curve-fitted. Even though there isn’t any guarantee that a strategy is not fitted to past data when optimizations are done it is obvious that the system development and testing tips suggested above reduce the possibility of this happening by a large amount. By developing simple symmetric strategies with limited degrees of freedom and reliable simulations over long periods of time with one or two years of out-sample testing the possibility to find a curve fitted solution will be extremely unlikely.

If you would like to learn more about system optimization and how you too can develop your own long term profitable strategies based on sound trading concepts please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach to automated trading in general . I hope you enjoyed this article ! :o)

Print Friendly
You can leave a response, or trackback from your own site.

2 Responses to “Optimizing Without Curve-Fitting : Six Tips to Avoid Over-Optimization”

  1. Winter says:

    Hi Daniel,

    Just wonder which of the current Asirikuy system fit into these strategies ?

    • admin says:

      Hello Winter,

      Thank you for your comment :o) All Asirikuy systems have been developed with the above criteria in mind. All of them have reliable simulations, long term tests, coarse optimizations and about 1 year of out-sample testing. I hope I have answered your question !

      Best Regards,

      Daniel

Leave a Reply

Subscribe to RSS Feed Follow me on Twitter!