Cutting Out the Fat at Asirikuy: Changing To Higher Requirements for Trading Strategies

This past month has been very interesting at Asirikuy as the use of a new back-testing data feed (forex-historical data) has revealed several problems with some of our strategies related to feed dependency and robustness. These problems have led to the development of several important modifications to Asirikuy systems in order to enhance their robustness and make them more reliable (see my last blog post). After this analysis it has become clear that some strategy characteristics are necessary for robust/reliable trading and for this reason I have decided to re-think our criteria for strategy trading at Asirikuy in order to enhance our chances of long term profitability. Through this post I will go through the changes I’ve decided to implement and what this will mean for Asirikuy.

Let me start by talking about our previous requirements for system trading. Previously the only criteria we had in order to allow a system to trade live (to test it on cent accounts at least) was for the system to show profitable and reliable back-testing results on a 3 year out of sample period after a 9 year regular optimization. If the system didn’t show good results we would not trade it. However we could often find profitable parameter sets for most strategies devised in this way. This procedure allowed us to develop several strategies but as time went by it became obvious that there were several things wrong with this way of developing trading strategies (especially when building portfolios with them).

The first problem we found was related with the evaluation of portfolios because most strategies were not designed to be trade chain independent. This means that strategies easily fell out of sync and historical correlations were nothing more than a wild guess when you moved outside of a very rigid trade chain defined by the back-testing starting dates and data structure. The second problem we found – related to the actual feeds – was that some systems can show different long term results on slightly different trading feeds due to the way in which they take their trading signals. The last post describes this problem extensively and how I found some solutions using alternative statistical measurements. However there was still another problem related to the way in which strategies were optimized. We found that 9/3 regular optimization was often not the best solution while rank analysis – where a parameter set is chosen when it is a “constant best” in the parameter space – was a much better alternative. Obviously walk forward analysis is also the way to go for evolving optimization, but this will be discussed further on future posts.

After all this experience it then became clear that we need to develop a criteria for trading systems which is very demanding, in order to have the highest possible chance of success. This becomes even more obvious when we look at strategies that have been profitable for the past 2/3 years, where all of them comply with some very specific trading characteristics. With this in mind I have decided to implement a much more rigid threshold for strategy trading at Asirikuy that will give us a much better chance of having successful results going forward (and to make less mistakes when implementing new strategies). Any new system or existing system traded on a live account within Asirikuy will need to comply with the following :

  • Reliable back-tests compliant with open-price only simulations (this means large profit/loss targets, use of only past closed candle data, etc)
  • Lack of trade chain dependency (system should act on all entry/exit signals by updating or closing positions)
  • Only one trade opened at a time per instance  (this allows for clear control over risk)
  • Rank-stable results derived from a 2000-2009 rank analysis with a 2009-2012 out of sample using our Alpari data.
  • Profitable results in 1987-2012 forex-historical data (this means we have an additional 13 year out of sample). Note that even if the 1987-1995 data represents a lower liquidity market (therefore lower likehood of accurate simulations on the 1H time frame), surviving it does represent a higher level of inherent robustness. Since this data does not take place in the optimization process it does not affect parameter generation.
  • Have profitable walk forward analysis results (although we will have to wait for the strategy tester module to carry these experiments out).

It is clear that the above will not guarantee any future success (as no one can see the future) but it does guarantee that we will not fall into some pitfalls that exist in the world of algorithmic trading. First of all we will not have exceedingly feed dependent systems since our systems would have been tested on a data set which is different from the data set in which they were optimized. In addition to this our strategies would have also survived to 13 years of additional out of sample, meaning that we are covering a much wider spectrum of market conditions. This protects us from trading systems that may have failed under past market conditions, highlighting that the inefficiency they trade is more sensitive to market changes. The above generates strategies that are robust to feed changes, robust to changes in market conditions and also stable amongst different market conditions (meaning they have more stability in their income generation capacity). Since we now know that systems outside of this regime are prone to failure before profitability can be achieved (meaning that exiting with a profit is unlikely) it doesn’t make sense to trade strategies that fall outside of these definitions.

For Asirikuy this will mean that we will need to “cut out the fat”. We will need to eliminate from our trading repository all systems for which the above conditions are not met. This will clearly mean that we will stop trading several strategies which we have been developing during the past year, some of which even have profitable results on live accounts up until now. This is obviously because we need to think about the long term stability of our results and how we should only trade strategies that are as reliable as possible. However this doesn’t mean that we will only trade daily strategies aiming for longer term trends, on the contrary the majority of systems that comply with this are in fact hourly systems which we have been trading for the past 2/3 years.

Up until now there are already 13 instances of Asirikuy strategies (besides Quimichi and Comitl which were designed with a similar criterion in mind from the start) showing all the above results to be true. This includes genetically generated strategies as well as Sunqu (our neural network trading strategy). This means that after the next F4 update we will be able to trade with much more confidence on our trading strategies, knowing that what we are trading has the highest possible chances of success going forward. All future strategies developed in Asirikuy will have to comply with this same criteria.

If you would like to learn more about trading strategy design and how you too can improve the reliability of your trading systems please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

Print Friendly, PDF & Email
You can leave a response, or trackback from your own site.

4 Responses to “Cutting Out the Fat at Asirikuy: Changing To Higher Requirements for Trading Strategies”

  1. Joe says:

    “Reliable back-tests compliant with open-price only simulations (this means large profit/loss targets, use of only past closed candle data, etc)”

    Good work! Questions:

    Which open price?
    Why large profit/loss targets?
    What is “past closed candle data”?

    Thanks

    P.S. I think you assume your readers are familiar with what you imply but this is not always the case and you may have ot be more specififc.

    • admin says:

      Hi Joe,

      Thank you for your comment :o) Let me now answer your questions:

      1. By “open price” I mean the open price simulation mode of the MT4 platform. These are simulations that only use open/high/low/close data of past candles. It is called “open price” because the information available for simulation is only the open price of the current candle plus the open/high/low/close of all previous candles

      2. Because if you use low profit/loss targets your results can be greatly influences by slippage, requotes, historical spread variations, etc. You need to have profit/loss targets of at least 10 times the spread to make sure you don’t have an influence from these factors in historical results that may show you an edge where none exists.

      3. It means the data obtained from candles that are not the current candle. So it means you do not use any information from this candle but you only use information from candles that are already in the past. This is fundamental for reliable simulations.

      If you want more information on the above you might want to checkout the free ebook available (link on the right), that contains some previous blog posts dedicated to explaining these concepts in more detail. I hope this answers your questions :o)

      Best Regards,

      Daniel

  2. Fd says:

    Hi Daniel,

    I have a slightly different point of view:

    – on open price simulations:
    while it is true that open price simulations on H1 using indicators on D1 with shift 0 are not accurate, I think the conclusion should be to use control point simultions or tick instead. Not using D1 indicators with shift 0 inhibits adaptive measurements that take advantage of intraday data. Using ATR(lookback, PERIOD_D1, 0) will give you better volatility estimates during H1 trading than ATR(lookback, PERIOD_D1, 1). Therefore my conclusion is that I have to use control point simulations, restrict trading to candle open, but use shift 0. This does give reliable simulations.

    – on profit targets:
    sure, a high the profit target combined with a high trading frequency will give you some confidence that your back testing results are not attributable to chance. However it is difficult to draw the exact line. I think it all depends on how well you can explain why your strategy is making some pips – only stating it has been profitable is not sufficient regardless of your profit target. The bound you are drawing is completely arbitrary and makes some implicit assumptions on the nature of the price generating process, your abilities to measure and to trade it. Here again the problem is that you will exclude some good strategies because of this rigid rule.

    Best regards,
    Fd

    • admin says:

      Hi Fd,

      Thank you for your comment :o) Well sure, you can use control points or Every tick simulations to take advantage of additional information but the problem is that these simulation style is simply not fast enough for extensive parameter space evaluations or walk forward analysis, two areas we want to be able to fully explore with all systems going forward.

      On profit targets, you’re right in that I have no reason based on an analysis of the data for these limits (because I lack the Bid/Ask historical feed to say what the actual lowest limit is). My reasons are mainly that with a distance of 10x the spread results have always been back/live testing consistent (using the same data) while with lower targets I have observed significant problems. Obviously I am not saying that there couldn’t possibly be reliable simulations below this threshold but I am simply not willing to risk having unreliable results because of this.

      This is especially important when you consider that historically spreads have been higher, if you do not have significantly high profit targets I would doubt the accuracy of the historical results without a proper Bid/Ask feed. Another very important point is spread widening on news events (this is why I have the 10x rule), lower targets usually allow you to curve fit a strategy to trade during times of high volatility with unrealistically low spreads (a dangerous way of fitting since it would be very hard to find out, except after a lot of live trading or with accurate Bid/ask feeds). If you have a proper Bid/Ask feed to make the historical analysis it would be great :o)

      Thanks again for posting!

      Best Regards,

      Daniel

Leave a Reply


Fatal error: Call to undefined function mcsp_html() in /home1/danielfp/public_html/mechanicalforex/wp-content/themes/modern-style/comments.php on line 71