Last month I began a new series of posts dealing with the world of neural networks and how these could be used for the creation of automated trading strategies. One of the first problems someone who wants to derive profitable Neural Network (NN) based strategies has to tackle is the inherent complications within the evaluation of this type of strategies. Unlike regular transparent rule based trading systems, neural networks cannot be evaluated in a simple in sample/out of sample test basis since their characteristics make this type of evaluation dangerous. On today’s post I am going to share with you why NN derived strategies need to be evaluated with special care and how this evaluation can be carried out. I will also share with you a very cool video showing how I am currently performing this analysis to derive our first Asirikuy NN based systems :o)
First of all we need to understand how neural network systems are different from traditional rule-based strategies. A neural network is merely a set of functions with very intricate relationships that predicts (hopefully) some sort of non-linear relationship within a data set. The main problem here is that the actual rules used to make decisions are not transparent and therefore we don’t have any idea of how they are being used for trading. This poses a very dangerous problem if we attempt to build a long term NN based strategy (for example simply attempting to forecast some variable using a 10 year training period) as the NN does not know what constraints are needed to aim for strategy robustness. The aim of an NN is only to match the data as perfectly as possible during the testing period and this may lead to implicit asymmetric rules which we know nothing about.
No video? Download the DivX player for free.
Since we cannot know anything about the rules used by the NN it becomes difficult to evaluate their robustness. In order to do this it becomes necessary to come up with an evaluation methodology which can exploit the ability of the NN to fit data and – additionally – show that this can be done in a way which exploits a market inefficiency without the problems from asymmetry that could be generated if we attempted to train the network over very long periods of time. The best solution to this problem – which favors the NN strengths – is a walk forward analysis.
The video above shows you the type of methodology I’m currently using to evaluate my neural network strategy ideas. I take a training period (150 days in the above example) and use it to train the network. The network is then put to the test on the following 5 days and then these results are saved. The training window is then moved by 5 days and the process is repeated to get the results for the following 5 days. In this way we are actually evaluating the network in a continuous walk forward “moving window” type of way which ensures that any type of asymmetric rules will be contrained to the very short term.
By evaluating a 10 year period using the above methodology we can know if the NN does exploit some inefficiency and its able to adapt against changes in market conditions or if it simply is not able to carry out this task. In the end – when net profits are achieved – what we get is a strategy which is completely based on a neural network and constant retraining, a strategy that not only succeeds in the long term but successfully readapts to constant changes in market conditions. Certainly this doesn’t mean that the NN is a “holy grail” or never losses but it simply means that the NN retraining allows for the exploitation of some type of inefficiency.
In the end NN strategies will also have draw down periods and will be as hard to trade as other long term profitable strategies but they do increase the degree of confidence and adaptability against changes in market conditions. As the NN is able to continuously adapt against changes in the market it has an advantage which other strategies do not have, the ability to “learn” from what happens in the market and take that information into account for the prediction of future values. The “moving window”, walk forward strategy provides a very good way to evaluate the robustness of an NN train/trade strategy based on the prediction of a given future price characteristic.
Hopefully within the next few months I will be able to start sharing some of these finding with the Asirikuy community :o) (still lots of things to research!). If you would like to learn more about my work in automated trading and how you too can learn to build and design systems with robustness in mind please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)