A few days ago I wrote a post about the inherent problems of walk forward analysis (WFA) and why this technique – in itself – does not constitute a holy grail for automated trading. Since this post I have done deeper research into the matter and – by comparing results for several systems – I have been able to find interesting relationships between the number of degrees of freedom and the survivability in walk forward analysis (using simple selection algorithms). Within this post I will expand on this topic, attempting to explain why systems that are given more freedom are able to exercise a better ability to adapt and why this ability causes fundamental problems that lead to unprofitable walk forward analysis. I will also go into why this tells us something fundamental about the inability of systems to evolve towards unknown market conditions and how we may possibly deal with this.
As I went deeper into the area of walk forward analysis, it soon became clear that systems with successful WFA results have some very clear characteristics in common. The first obvious relationship between them was a very simple trading strategy setup, a higher trading frequency and a low number of possible parameter selections. Upon a closer analysis of the results it became clear that these systems were not changing dramatically over the course of the WFA but they were simply having small changes across a wide variety of market conditions. A closer look also revealed that the systems that give profitable WFA results tend to work only on pairs for which the inefficiency they trade seems to be practically ever-present, only failing under very specific market conditions and for short periods of time.
The above is important because it spoke to me about a general lack of adaptability. What we have here are systems that trade in a very fixed way – like a volatility breakout strategy does for example – and the ability to survive the WFA comes from the fact that the efficiency exploited by the system changes little across the instrument it is trading. For example volatility breakout systems will find very profitable results in symbols like the EUR/USD but they will fail bluntly on symbols where this inefficiency is not ever-present , such as the USD/JPY. More clearly, these systems will be unable to fend periods when the market has changed, for example in the case of the GBP/USD after 2009, where the market changed to make volatility breakouts almost obsolete. Although WFA may show reduced drawdown such periods, it does become clear that the drawdown in itself is unavoidable and if lasted long enough it would potentially destroy the account. If a good ability to adapt was present, the drawdown would have been easily avoided at least after the first part of the drawdown period made changing market conditions evident. A system that adapts to changing market conditions should easily avoid drawdown periods longer than a few WFA window lengths, especially if there is a complete and dramatic market change.
In my view, it would be foolish to believe that such systems are truly adapting, because they never need to adapt to a dramatic change that removes the inefficiency they trade from the market and when they have to, they fail. However this poses a big question which is how we can give a system a larger ability to adapt in order to see if it can truly tackle dramatic changes in market conditions. At this point I decided to try systems with larger degrees of freedom, especially those systems that could generate dramatic adaptations to changes in market behaviour. The most obvious initial test is to try a system that can “switch” the way in which it trades in a very dramatic manner; a breakout strategy that can either fade or trade breakouts.
The results of this experiment were very interesting because they showed that under competing opposite market strategies there are optimization periods where both can give profitable results but only one gives profitable results in the subsequent walk forward trading period. However there were also times when one of the two system switches was dominant, achieving profitable trading through several different periods at a time, only failing when the other strategy started to become dominant. In essence what we have when we introduce freedom that allows for a completely different trading logic to enter the picture is a system that is “split” in duality between what it can achieve through both trading techniques. Such a system has enough freedom to adapt to two opposing inefficiencies and it only fails when the ground is neutral between the two (as the result is equivalent to a guess since none of the two techniques is dominant).
Perhaps the most dramatic effect is that the trading logic in fact changes in periods where you would expect it to, as the breakout inefficiency becomes less effective, the fadeout inefficiency becomes more predominant and the system starts to trade in a completely opposing manner. This is alike what traders generally call the “switch” a flip between two opposing market views that happens under changing market conditions. Clearly you have losing periods while this happens – while the change takes place – but if the change is long lasting you actually get a few periods of very good profitability. Obviously if the change is short lasting you hesitate between the two logics and you end up with drawdown accumulated on either case. The market is very efficient in this case because it fails to imitate its past behaviour.
The above is only the beginning of the story but it does show that walk forward analysis success only seems to make sense when you give your system enough freedom to completely change the way in which it’s trading. Using WFA to adapt a system that is “stuck” in a box is not a good idea because you’re fooling yourself into thinking you have a true ability to adapt to market changes when you are simply observing a positive effect because the inefficiency you want to trade is practically constant under the pair you’re analysing. By giving your system the ability to tap into logic that trades in a naturally opposing manner you ensure that you give your system the “ultimate choice” regarding the way in which it should be trading.
Granted, the above is not easy to achieve and you will see that WFA fails bluntly under many different conditions when you increase degrees of freedom. However, profitable WFA is indeed possible for systems with very complex makeups (even genetic frameworks) but we will get into this topic on future posts. If you would like to learn more about trading systems and how you can learn to trade and analyse them please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)