Mechanical Forex

With today’s article I want to start a new series of blog posts where we will be using the R statistical software in order to analyse several aspects pertaining to algorithmic trading, especially system generation. These blog posts will show you how to go deeper into the analysis of results obtained from an automated system generation process using powerful and easy-to-use statistical analysis tools. Most of these posts will use data derived from the Kantu Parameterless System generator, which you can also obtain by using the program’s demo (available here), I also recommend R studio, which you can get here. On today’s blog post we are going to find out how we can explore the correlations between in-sample (IS) and out-of-sample (OS) variables, in a group of strategies created using Kantu. For this analysis I have used EUR/USD 1D data from 1986 to 2012 (included within the Kantu demo download, this data is also available for free at forex-historical.com). If you want to reproduce my analysis without generating the systems yourself you can get the data used within this post here. 

–

–

A key aspect of algorithmic trading is to be able to pick a system to trade from a pool of available strategies. You would certainly want to trade a strategy with a high chance of success under unknown market conditions and – since the future is unknown – the best we can do is to draw some correlations from past data and determine the past correlations between IS and OS variables. In order to do this we must create a series of strategies that match what we consider acceptable criteria for historical performance in the IS and we then examine the data to see what relationships have been valid historically. In order to do this we will generate systems with an acceptable trading frequency and linearity on the EUR/USD 1D data. We setup Kantu filters and options as showed above. Note that the “LR Through Origin” (Linear Regression Through Origin) option is set to true and the R^2 filter is set to a minimum of 0.98. The Trades/Year is also set to a minimum of 10 in order to have statistically significant results. The above setup generates systems through 4000 days of data and then out-of-sample tests them on the next 600, a maximum of 1000 systems will be generated.

After loading the data for the EUR/USD you can then run the simulation by going to “Quota Simulation”->”Randomize Dates”. The randomize dates simulation type selects random 4000 IS/ 600 OS periods through the whole 1986-2012 data set, in order to find strategies that comply with our required filters across several different regions of the requested data. This random selection of roughly 11 years through the 26 year period allows us to obtain conclusions that are more general and not merely belonging to a single trading period selected. This is important because there might be a period where high Reward to Risk strategies were favored but we don’t want to find this correlation unless it’s been a historical constant on the EUR/USD through the majority of IS/OS pairs.

The system generation process will take a while since the filter is highly demanding, the generation of 1000 strategies can take a bit more than one week using the 8 available threads on an intel i7 4770k, amounting to a bit more than 30 million tested systems. This shows that finding highly linear systems is no walk in the park. Once we find the systems required we can then perform an analysis of our results by first saving the csv file (right click on the results grid and select the save to csv file option). After this we can then run some R scripts in order to discover the correlations between our IS variables and the OS Profit and OS Profit/trade variables.

–

dataset <- read.csv("C:/PathToCSV/CSVFileName.csv")
dataset$Custom.Criteria <- NULL
dataset$No. <- NULL
dataset$P <- NULL
dataset$Symbol <- NULL
cor(dataset$OSP,dataset)

–

–
The first thing we want to do in R is load our data from the csv file into an appropriate dataset and eliminate variable which are not meaningful for our analysis. For this reason we eliminate the Custom.Criteria (in this case 0), the system number, the portfolio identifier and the symbol. After this we can now carry out a correlation analysis using the cor command, allowing us to see a correlation matrix between the OSP variable (the OS profit) and all the in-sample variables. This already gives us some valuable information, such as high correlations between the OSP and the in-sample profit (0.717) as well as a highly inverse correlation between the OSP and the in-sample profit factor (can you think why the PF is negatively correlated with the OSP? Leave a comment!).

The next thing we want to do is to plot some variables against one another in order to look at the overall relationship between our IS and OS variables. Note that the high correlations between some IS/OS variables might make you think that you would find a straight line when plotting them together, this is often not the case. In order to make things easier to analyse we also color positive OS results red while negative results are colored black. This allows us to easily see which IS variable values have led to bad OS results. Below we can see the graph obtained for the PF (first line of the code below).  The probability of having a positive OSP result decreases exponentially as the in-sample PF grows larger.

-
plot(dataset$PF,dataset$OSP,col=((dataset$OSP >0) +1),pch=19)
plot(dataset$Abs..Profit,dataset$OSP,col=((dataset$OSP >0) +1),pch=19)

–

–

However we might also want to take a look at the overall correlations between all IS variables, among themselves and with OS variables. In order to do this we can use the corrgram library that allows us to create very useful correlation diagrams that display – in an easy to analyse fashion – the correlations between all relevant variables (you need to install the corrgram variable in R in order to use it, very easy to do in R studio). The stronger the blue color the more positive a correlation between two variables is (closer to 1) while the redder the color the more strongly inverse (close to -1) the correlation is.

–

 library(corrgram)
corrgram(dataset, order=TRUE, cex.labels=0.6)

–

–

We can also see some interesting things here, such as a strong inverse correlation between the Winning Percentage and the Reward to Risk ratio (not surprising). However what is interesting is that the multiplication of both of these variables isn’t constant, but increases as the Reward to risk ratio increases, meaning that IS expectancy increases as the Reward to Risk ratio increases. You can analyse the relationship between this (Winning%) X (Reward to Risk) variable with the OS profit as an exercise.

The above results reveal that OS expectancy for highly linear systems (generated through 4000 days) on the EUR/USD daily charts for a 600 day horizon have been positively and strongly correlated with the Absolute Profit and the Total Trades while the results have been inversely correlated with the PF and the Sharpe Ratio. How would you pick a system from a pool of highly linear systems if you wanted to start trading tomorrow? Feel free to leave your criteria within the comments below :o)

If you would like to learn more about Kantu and how you too can generate and trade strategies generated with it please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)