Destroying a Mechanical Trading Myth : Frequent Short Term Optimization Works

When less experienced people start to develop automated trading system they soon realize that their strategy faces losing periods which they view as the strategy not being able to “adapt” against ever-changing market conditions. The answer to this problem given by most amateur system developers is to do a constant optimization of a strategy against a previous short period of trading so that the system can remain “current” and “updated” against present market conditions without any regard for the past. On today’s post I am going to share with you why this behavior almost never yields long term profitable outcomes and why long term optimizations are much better at improving systems than simply “chasing” the market with constant small optimizations. I will first talk about changes in market conditions and the actual edge necessary for successful trading following into my argument as to why it is impossible to always have a system adapted to the “current” market.

To have a winning trading system you simply need to have one thing : a system with a positive long term statistical edge. A system that lacks this edge – through any means – will not be successful simply because any profit and losses are generated by chance and eventually trading costs will make the system run into ruin in a speed which is directly proportional  to the amount of lots traded and the frequency of trading operations.

When you develop a new system and you don’t know if you have a long term statistical edge -or you lack one for that matter – it is easy to say that “the past doesn’t matter” and take an attitude in which the present takes top priority without any regard for how the system behaved in the past. Such system developers then start to do frequent optimizations against very recent market conditions in the hope that they will be able to take into account changes that will grant them an edge for the near future. The result of these attempts are always the same, since a true statistical edge is not present the system effectively only “chases” market conditions without ever achieving long term profitability.

What I have found is actually quite interesting regarding short term optimizations.  Whenever you take a strategy that succeeds in the long term by doing an optimization over a short period and then an even shorter walk forward (for example optimizing for the last month before every one week forward run) you get account wipe outs in the longer term unless the system already has a true statistical edge. If the system has an edge in this way but it fails to achieve straight ten year profitable tests on some settings then this is usually caused by an absolute assumption about the market (like a fixed SL or TP in pips) and it is usually easily fixed by making this criteria adaptive to the past few days or weeks of market conditions.

However most of the time this is not the case and people neglect any type of long term evaluation of their strategies with the excuse that the past is irrelevant as they want to profit from the present. This argument is very lacking and just an excuse in nature since having information about successful past performance based on reliable simulations makes the future profitable trading of the strategy – although not guaranteed – far more likely. Any strategy you come up with will not be successful unless you have a true statistical edge and any true statistical edge will not be apparent unless you do long term simulations.

Short term optimization usually have only a “chasing effect” for systems that do not have a true long term edge, they make the system use the optimal setting for conditions which have already passed and the system tackles current market conditions with a random degree of success as the conditions are different and the system holds no robustness or true edge. There might be some successful and losing periods after these short optimizations but they usually end with a drastic loss when the system uses conditions that were extremely different from current ones (for example when sharp trends start). The key to profitability against changing market conditions is not to make your system attempt to “chase the present” but to build a system with a robust true statistical edge that adapts against some global variables of the instrument or market you are trading.

It is definitely a much more robust and sound idea to evaluate the longer term statistical edge of a system using 10 year simulations and then optimize the system to these 10 year conditions walking  the system forward for 1 or 2 years after that to ensure that the statistical characteristics of the system are preserved (note how I do not mention that the system should be profitable but that the monthly distribution of returns is preserved as the next two years could easily be  or include a few standard draw down periods for some systems). This way of development ensures that a long term edge is present and that this edge is robust enough to withstand large changes in market conditions.

Building systems with short term optimizations hoping to adapt to current market conditions is a very dangerous and deceptive game that can make people trade strategies lacking an edge without realizing this until it is too late. If you want to test a strategy with such a short run optimization technique you should still do the ten year analysis simulating your optimization criteria to make sure that what you are doing truly holds a long term edge. However if this is the case the most probable thing is that the adaptability of your system can be hard-coded to make the system evolve against some changing aspect of the market, like price volatility.

In the end the short optimization game doesn’t usually yield good results and moreover – in every case I have tried – it has been bested by simple adaptive techniques that shape the system against changes in market global conditions. If you would like to learn more about how you too can build automated trading systems with sound trading techniques in mind please consider joining, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach to automated trading in general . I hope you enjoyed this article ! :o)

Print Friendly, PDF & Email
You can leave a response, or trackback from your own site.

Leave a Reply

WordPress › Error

The site is experiencing technical difficulties.