Does automatic trading system generation in Forex really work? Some live trading evidence.

On one of my recent posts I described some problems with manual trading system generation and why automating the generation of trading strategies can lead to a much more robust trading methodology that allows for the efficient creation, evaluation and implementation of alpha generating strategies. Right now most of my system generation efforts are devoted to the automatic creation of strategies using GPUs through a cloud collaboration effort at my online community. On today’s post I want to share with you a bit of how this methodology has been working for our community since its implementation. I will also be sharing with you the up-to-date results  of a real money live trading Oanda account where I have been executing a portfolio of algorithmically generated strategies during the past 4-5 months through direct API execution. This account is a small sample from our live trading accounts library, which are currently mostly trading price action based strategies created using Kantu/pKantu.

2014-11-22_19-47-56

I will begin by saying that automatic system generation is not easy. Using a computer to create strategies leads to a variety of statistical problems that – although present in a manual system generation process as well – are compounded by the intensity with which an automatic system generation approach uses the back-testing data. When generating a strategy using things like machine learning or genetic programming it is very easy to arrive at solutions that look extremely good on historical data but fail to perform in the same manner under live-trading market conditions. The reason why this happens is related with data-mining bias, some techniques – in particular genetic programming – make such intensive use of the data that finding anything that you ask for will be warranted, just because you’re torturing the data until you find what you want. The systems produced therefore have a high probability to come from random chance and a very low probability to be the result of a real historical market inefficiency.

At Asirikuy we studied this problem through our first year of using our initial implementation of a price action based system generator (kantu) and we decided to create a methodology that allowed for the clear evaluation of data-mining bias and the creation of strategies that had a very low probability to come from random chance. To allow for this we created an improved version of our software (pkantu) that exploits the power of GPU computing (about 100x faster than the original kantu) and we also created a methodology to evaluate for data-mining bias using bootstrapped random data series derived from the original back-testing data. This is only possible due to the speed of our GPU approach and the use of a cloud mining GPU system which allows many Asirikuy members to contribute computational power to our system generation effort. In this manner we are able to find systems and determine the probability that these systems come from random chance. In this way we do not trade any systems for which there is a very large data-mining bias but we only trade strategies for which the probability to come from chance is less than 1% (in many cases less than 0.001%).

2014-11-22_20-07-54

Using these methodology we have produced many systems so far, however back in the middle of July we had only generated 4 trading strategies which I decided to use within a portfolio traded in a real live account to see whether our trading methodology did lead to the expected real out-of-sample trading results. With this I wanted to see how the strategies behaved when put together (if the correlations between them changed significantly) and what the lifetime of the strategies is. Clearly all trading systems eventually fail – as potential market conditions are infinite and the strategies are created within the scope of limited historical data – but we still don’t know what a reasonable “life expectancy” is for a strategy generated within this methodology. Obviously the systems should work for a considerable amount of time for them to be worth trading.

The first image above shows you the +25 year back-testing results of these portfolio including swaps and spreads. The portfolio trades at an average system risk of 0.25% per trade, with the four systems at a rather even balance (this was optimized using a simple Markowitz portfolio optimization to get to the efficient frontier). The back-testing results show a maximum drawdown of 9.89% with an Average Annualized Return (AAR) of +18.06%. This means that the portfolio has an AAR/MaxDD of +1.83, which I consider quite good. As you can see risk is really controlled and the portfolio follows a rather ideally linear equity curve (note that the left axis is logarithmic). The increase of 100x of capital in 25 years look quite great considering the very low risk. Since bias is also measured and controlled and the strategies are extremely simple – they are very simple price action based strategies – it is very interesting to see whether these results hold-up across real trading.

2014-11-22_20-09-44

Results during the past 4-5 months have been quite encouraging. The second graph shows a comparison of the live trading (blue) and back-testing (black) data within our portfolio. There are some differences which can be mostly attributed to the use of a virtual order wrapper in our account (since Oanda is NFA compliant and we therefore need to handle positions virtually and then consolidate through a net positioning approach). This means that if we’re long 0.1 lots and we get a short signal for 0.05 lots on the same symbol we end up with two virtual positions (one for 0.1 lots and the other for 0.05 lots) while our net positioning changes to 0.05 lots (0.1-0.05) in the end – when all trades are closed – our results are the same but while trades are open there are some differences as we might take profit/loses sooner due to the net positioning issue. This is for example why the blue graph goes so high during the end of july while the black graph remains flat, because entering of contrarian positions caused us to partially close very profitable positions on the account (while we keep opposite virtual trades).

Some difference have also happened due to issues with execution. Some of the Oanda trades have been exited at values worse than the stop-loss or they have been entered at more unfavorable prices than anticipated by back-testing (mainly due to spread widening during news). All these execution issues end up generating the small difference you see in the end, where live results are slightly worse. The difference is not that big because the back-test attempts to account for this including worse-than-actual spreads (about double of the real market spreads).  In the end we expect our long term back-testing and live-trading results to be a very good match.

Gladly I must say that broker dependency has not been an issue here. Although the back-testing and live trading feeds are different (Alpari UK vs Oanda) all the effort we have carried out within the F4 framework to eliminate structural data differences (such as weekly starting times) has paid off. From the signals generated within the back-testing results all the same signals were generated within the live trading account. Beyond the execution issues mentioned above, all signals were exactly the same, despite both results coming from two different feeds.

So far the systems have behaved precisely as expected from a statistical perspective, with none of them breaching their statistically determined worst case scenarios. So far the results of the portfolio are perfectly in line with the yearly profit target and the drawdown level attained so far is within our trading expectations. If you are an Asirikuy member, a verified myfxbook link to the live account mentioned above is included within the live account’s page. Access to the myfxbook pages of our live trading accounts is only available to Asirikuy members.

Surely the above does not prove that our system generation methodology works in the long term yet or that all of the systems on this account won’t fail next month (I clearly don’t know the future and past performance is never a guarantee of future results). However watching the systems trade live within their statistically expected behavior, observing low broker dependency, watching the portfolio reach new balance highs and watching system correlations stay stable suggests to me that we are at least on the right path. If you would like to learn more about our system generation methodology and how you too can help us mine strategies using a GPU please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

Print Friendly, PDF & Email
You can leave a response, or trackback from your own site.

5 Responses to “Does automatic trading system generation in Forex really work? Some live trading evidence.”

  1. Dim says:

    “Results during the past 4-5 months have been quite encouraging.”

    is this with real money or simulation through a broker?

  2. steve says:

    Hi, interesting article as always. I just finished reading your ebook and found a wealth of useful information so thanks heaps!

    I do have some profitable EA’s running live, and back-testing others. I have a question which I can find the answer to after reading your ebook or scanning your archives.

    “Is an EA not robust if it doesn’t work on multiple currency pairs?” My thoughts are no as each pair is unique and has its own idiosyncrasies. For example, a London breakout might work better on GBPUSD but not EURUSD even though those pairs are correlated.

    Would be interested in hearing your thoughts :)

  3. Chris says:

    Hi Daniel,

    You indicated above that the drawdown is slightly deeper on live results, compared to backtesting. How much in percentage points do you estimate the difference is? (1% risk account). Thank you.

    Chris

    • admin says:

      Hi Chris,

      It’s actually almost the same when you look at equity (less than a 0.5% difference), the difference in balance appears due to the difference in closing sequences caused by the use of the virtual order wrapper. When all positions are closed it leads to the same result. I hope this answers your question,

      Best Regards,

      Daniel

Leave a Reply

WordPress › Error

There has been a critical error on this website.

Learn more about troubleshooting WordPress.