The design and evaluation of trading systems for financial markets is limited to the use of historical data. This means that we look for things that worked in the past that we believe might work in the future but we often forget about the inherent limitations of this process and how it can inevitably lead to failing systems. On today’s post I am going to be talking about the limitations of historical data and how this constitute a real brick in the world of trading system design that simply cannot be overcome. We will talk about how historical data is used and why none of the creative ways in which traders manipulate historical data can ever lead to the breaking of its limitations.
–
–
Every trading system is inevitably doomed to fail. There is a significant portion of traders who believe that this can be completely avoided using some tools but the reality is that every trading system, no matter how well designed and tested, is inevitably subjected to an unavoidable destiny of failure. In the same way that we are all going to face the grim reaper, every system will face its eventual demise at the hands of the market. This is because every system is born and raised using historical market data and eventually historical market data ceases to be a good predictor of future market behavior because the number of market conditions is for all intents and purposes, virtually infinite.
This is the great limitation inherent to the use of historical data, the amount of historical data available is limited in nature while the possibilities in the future are infinite. We assume that the data we use will be a decent proxy for future data but the reality is that at some point in the future the market will behave in a manner that the historical data did not account for, that is the point where your systems will fail. When faced with this problem some traders then interpret this as an issue to adapt to new market conditions but deriving a methodology to adapt from historical data is useless as well.
–
–
Since you believe that your system will fail in the future because it will face conditions not present in the historical data you come up with a very clever thing to avoid this, you’ll design your system using walk forward analysis so that it can “adapt” to changes in market conditions. Surely if it was able to successfully adapt in the historical data then it will be able to do so in the future when new conditions come in. Wrong. The fact that a system succeeded in a walk forward methodology in your historical data does not mean that it will face the same success in the future. This is because market conditions also include how often, at what speed and exactly in what manner market conditions change and the variability in these changes is also potentially infinite in future data. There is actually no evidence in that walk forward analysis even increases chances of survival under unseen data versus a system that was designed to be profitable over long term data. As a matter of fact – in my research at least – systems mined to trade over a 10 year period often out perform systems that were designed using walk forward analysis over a 10 year period with any given window distribution, optimization technique or selection method. Trying to adapt with a predefined frequency and within a given set of parameters is often a worse leash than simply optimizing a strategy for a very long set of market conditions.
No matter how you try to adapt or design a system so that it survives to very long market conditions you are still using known market conditions to achieve this, you’re using historical data. If you learn to adapt, you learned to adapt to what the historical data knows as change, if you learn to adapt to adapt, then you’re doing the same thing at a higher level, always limited to the historical data you have available. To whatever degree of adaptation you subject your system using historical data, it will still be limited to the information present in the historical data which is finite. The more data you use the better your system will be able to tackle the market – as it has more information – but this does not mean that your system won’t finally have to cash its chips.
–
–
The brick wall we all face when designing and evaluating systems using historical data is that we don’t know the future. No matter how you slice and dice the historical data and come up with ingenious ways to make your systems survive to changes within it, no matter how much effort you put into the analysis of this data, it will always come down to the fact that your data is finite and therefore your system’s survival scope will be too. If you want to make it in the long term then you need to ensure that your systems survive for as long as possible, can be discarded quickly when they fail and can also be replaced very rapidly with systems that are able to survive to what caused your initial strategy to fail.
To achieve the above does not mean you need to use any complicated techniques – as I said before in my research this often proves to be the opposite – it means that you should use lots of historical data to design and evaluate your systems (try 20-30 years), that you should have a clear statistical methodology for discarding strategies and that you should have some way to come up with strategies rapidly in case that one system needs to be replaced with another. If you would like to learn how we do all these things please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.
