If you want to use the OpenKantu trading system generator to create trading strategies you will soon find that the first thing you need to successfully carry out this endeavor is to have a source of trading data that you can use to generate trading strategies. When you find such a source you need to ensure that the data is adequate for trading system generation and this implies carrying out a variety of tests and ensuring that there is a match between the characteristics of the historical data and the data that you will actually be facing in live trading. On today’s posts we will be talking about data sources, some notions about data quality for automatic system generation and some the matching between the data used for live trading and the historical data used to make simulations.
–
–
Before continuing it is important to note that OpenKantu functions by making OHLC comparisons of bars within your trading data. This means that the relative position of bars in the data is very important for the trading systems built using OpenKantu. Therefore if you want your live trading of an OpenKantu system to follow the same logic as in the generation process you must make sure that the positions of bars within your live data and the historical data you used to create the systems is the same. This means that your data must have the same candle structure in both cases. If the week starts at 0:00 GMT on your historical data, then it must start at exactly the same time within your live data. The same goes for the weekly ending times.
It is very easy to throw all system generation efforts out the window by failing to comply with the above. Imagine that your historical data is GMT 0 and your broker’s data is GMT +2/+3. When you perform the system generation process the beginning of the week is 0:00 GMT 0 while when you live trade it is 0:00 GMT +2/+3, this means that your entire data is shifted 2 or 3 hours and all comparisons that you derived when you were generating strategies are completely useless. This is a certain way to make all your system generation efforts futile because you’re trading something live that is completely different from what you generated in the system creation process.
–
–
It is also very important to note that it is not enough to use your own broker’s data to solve this problem. Many MT4 brokers have the nasty practice of changing their DST offsets through the historical data. For example Alpari changed their GMT offset from GMT +1/+2 to GMT +2/+3 in 2011 but they did not change the timestamps of the historical data before this change was made. Therefore if you downloaded historical data for Alpari today and you went on and generated systems you would in fact be using GMT +1/+2 for data prior to 2011 and then +2/+3 for data after that. You can easily imagine how this can heavily distort your system creation effort, even if you are actually live trading with Alpari as well. Your current GMT/DST changes must match the entire series, not just the recent data. Verifying this is extremely important to achieve good results (you can read my post on using the NFP to perform checks on DST changes).
If you’re looking into freely available online historical data sources then you also have this problem. Some sources are extremely vague about the GMT and DST changes within the data which makes them practically useless. Take for example the FXDD 1M data available from FXDD. They make 15 years of 1M data in hst format directly available to you but there is no information about the exact GMT and DST changes within the set. If you look into the data you’ll notice that the data is actually GMT +2/+3 but the actual time of the changes is mostly random as it does not match any usual timezone structure. Gain capital 1M data has similar problems as they never let us know how their data actually changes. Many Forex brokers have this exact same problem – even with their live data – their actual DST changing time does not match any known time zone and it changes apparently randomly through the years.
–
–
In this sense it is mostly better to get a broker for which there are no DST changes – or for which these changes match a very specific timezone – and make sure that your historical data does the exact same thing. If you have a broker without DST or for which these DST changes are perfectly aligned with a timezone then you can use TickStory to download historical data from Dukascopy that is directly formatted to match your broker’s timezone. You can then export this data from MT4 into an MT4 formatted csv file (or you can do so directly from tickstory) so that you can use this data for generation within the OpenKantu. However you must still make sure that the weekly starting and ending times are exactly the same for the historical data you have downloaded and your broker’s live data, reason why some data processing after this is usually needed.
It is very important to stress how critical the whole data matching process actually is and how wrong things can go if you fail to ensure that the whole thing matches. If you have a different DST change or GMT offset, if your data starts at a different time or ends at a different time each week, etc all your system generation efforts will be lost. Ensuring that all weekly starting and ending times , DST changes and GMT shifts match between your historical data and your live trading data is critical and not a trivial task. This is the reason why we created the F4 framework at Asirikuy and have 1987-2015 historical data where all the DST/GMT offsets are known to make all these corrections as swift as possible. If you would like to learn more about our programming framework and how you too can use our tools for automatic trading system generation please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.
