The truth is that there is no universal measurement of system quality. Although people may believe that things such as profit and draw down are “no brainer” ways to examine a trading system against another, with time I have been able to see that such measurements are not relevant if they are not examined within the context of robustness. However since robustness is a future quality of systems (the ability to survive future market conditions) it becomes almost impossible to come up with a factual way to evaluate performance in the context of robustness and therefore it is up to each trader to draw a compromise between what might be considered robust Vs expected profit and draw down levels. On today’s post I will talk about system quality, robustness, expected profits and how each trader needs to find a balance between all these different fields.
Our problem – which cannot be solved – is that the future is not known. When you do not know the future you therefore do not know if a trading system will succeed or not. Since we do not know if a system can succeed or not then in order to decide which strategies to use we must look at extensive past performance and at the systems which have been used in the past which have indeed survived into extensive true live trading periods. This helps us draw an idea of what might be successful in the future and therefore allow us to pick systems that have a very high probability of future survival. However the added certainty about future profitability – for example when using a multi-currency 20 year unoptimized trading strategy like Comitl – comes at the price of lower expected profitability. This is also seen in a wider market perspective, investments that carry a very reduced level of risk (like T-bonds) have a very low yield while higher risk investments have a higher yield (like stocks) but a higher tendency to fail.
The market punishes additional certainty in future profit with psychological pain (higher Pain and Ulcer index values) and lower expected profit to draw down ratios . Obviously we would all want to trade a system with very high returns, low draw downs and a very high expected probability to profit into the future but as increases in robustness naturally decrease degrees of freedom the ability of a system to be more profitable in simulations is reduced as you go into territory which implies a higher chance of future long term profits. The market has a natural and expected tendency to demand systems with higher expected profit and draw down targets to have higher expected failing rates. The more you want to be sure you won’t be failing, the lower you need to aim regarding the profits you want to make.
However each trader’s personality and risk aversion determines to some extent what level of compromise between robustness and expected profitability they would want to have. For example some traders might wish to have a higher expected profit level with expected lower draw downs and psychological pain but the cost is that the probability of reaching the Monte Carlo worst case scenarios in the future becomes higher (a higher probability to fail). Therefore a trader who wishes to trade systems that have higher profit to draw down targets is effectively saying that he or she prefers to have to remove his or her system/portfolio to come up with a new one more frequently with the expectation that at least a few of these setups will reach their full potential (match the expected profit to draw down ratio). Psychological pain is paid in the form of failed setups and not in the form of losses on the systems which do make the expected profit and draw down targets.
In the opposite case a trader might choose a system which has very high robustness with the already known idea that profit to draw down ratios will be quite low and psychological pain will be high. For example a 20 year unoptimized strategy trading a large basket of currencies might have a profit to draw down ratio from 0.3-0.7 (which is low compared to system developed with lower robustness standards) but in the future the chance of this system failing is very low as the system has been able to tackle wide array of market conditions and instruments without ever needing a change in parameters. A trader who chooses something like this will pay for his or her added probability to reach profit and draw down targets in psychological pain when using the system.
Since the future is not known there is actually no way of telling which system type is better and therefore there is NO absolute factor to determine system quality. There may be a system which has very high robustness, achieving its profit and draw down targets within the next 10 years while a trader using a system with far less robustness might also achieve these targets and get much more profit with much less psychological pain, however a different system choice of an equally less-robust system might cause Monte Carlo worst case scenarios to be reached and therefore lower overall profit to draw down ratios and a high psychological toll. In the end it is up to each trader to choose systems according to the scenario they want to face, if they want to face a higher probability to face Monte Carlo worst case scenarios with the idea of having much higher profits and less draw down or if they would rather settle for a system that has a much smaller chance of failure with less profit, more draw down and psychological pain.
As you see the matter of system quality is not an easy one and therefore the newbie question of “what is the best system?” losses all of its meaning. The best system is inevitably the one that will trade into the future with high profits and low draw downs but the more you are certain about one (future profits), the less certain you are about the other (high profit to draw down ratios) (maybe Heisenberg also had something to say about trading?). If you would like to learn more about my work in automated trading and how you too can learn about algorithmic trading system evaluation and design please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)