Mechanical Forex

Certainly one of the first things newbies are told to look at when evaluating the results of Metatrader 4 backtests is a number called “modelling quality” which is a measurement Metaquotes came up with to attempt to measure the quality of the simulations a trading system is subjected to. While you start your journey in automated trading using MT4 you will hear things such as “only 90% modelling quality works” or “90% quality is useless, you need 99% quality”. But what is this modelling quality number, how is it calculated and what validity does it hold against the accuracy of the simulations being done ? Through the following paragraphs I will explain to you what modelling quality is, why by itself it is a useless number and why understanding about how a trading system works and the inherent limitations of simulations is FUNDAMENTAL to understand when simulations are accurate and when they are not.

–

–

When you run a simulation in Metatrader 4, the program gives you a number called modelling quality. This number was invented by the Metaquotes people to measure the percentage of lower time frames which are taken into account when evaluating a strategy. Their idea was simply that the more lower time frames you take, the “finer the grain” of your simulations and the better their quality. A higher modelling quality means that the system was simulated under a “higher resolution” data close to tick data. The modelin quality number is calculated according to the following formula (as detailed on this link) :

As you see the number only takes into account the “completeness” of the time frames as used by the Metatrader 4 platform. This means that using 1 minute data gives you a 90% modelling quality and using tick data gives you a 99% modelling quality while on the 1 minute time frame (since there is no lower time frame) you can get 20% modelling quality at a maximum. Does this mean that a system with n/a modelling quality gives inaccurate results while a 99% modelling quality gives accurate results ? No.

You need to understand here that depending on the characteristics of your strategy certain things need to be taken into account to have accurate simulations and this may need just very little modelling complexity. For example a system that trades exclusively on daily time frame EOD (end of day) data without any SL, TL, TP or pending orders will be able to give accurate simulations on a simple Control Points method with n/a modelling quality as the system only needs a very limited amount of data to be simulated successfully (only the Open/High/Low/Close data at the end of each day). This shows that having a low modelling quality does not imply that the simulations of a system are inaccurate, it merely points out that the simulations where carried out using less data from the lower time frames. How accurate simulations are without this data depends on the very nature of the system.

The same thing applies to the other side. Having a 99% modelling quality does NOT imply that the system is simulated accurately, as I have pointed out on several posts during the past few weeks you can create low time frame systems that give inaccurate simulations because they need appropriate distinctions between Bid/Ask tick feeds to give reliable estimations of past performance. So having a 99% modelling quality does NOT mean that your simulations are accurate, as a matter of fact you can develop systems that give extremely inaccurate simulations with this seemingly extremely high modelling quality simply because the MT4 platform has some simulation limitations that are important for adequate consideration about a system’s performance.

By itself, modelling quality therefore tells you NOTHING about whether or not the simulations of a system are or aren’t accurate. You can have systems with accurate simulations that give an n/a modelling quality and you can have extremely inaccurate simulations with a 99% modelling quality. What does modelling quality depend on then ? Real modelling quality depends on the characteristics of the system being tested and whether or not the system is able to adequately address the limitations of the testing software being used. A system that is adequately programmed and designed to take into account the limitations of Control Point simulations will give reliable results on very low modelling quality while a system built without taking into account the inherent limitations of MT4 will be able to give very unrealistic results EVEN on 99% modelling quality.

To sum it up, you simply need to understand what you are trading, the system you are simulating and make sure the data you are using for simulations and the simulation method/program used correctly addresses the needs of your strategy to arrive at accurate past historical performance. Blindly relying on numbers such as the MT4 “modelling quality” is a sure recipe for failure, as always you need to DEEPLY understand the system you are trading, simulations and their limitations in order to know when something is accurate and when it is not.

If you would like to learn more about my work and how you too can develop systems with reliable simulations (even very computationally cheap ones) please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach to automated trading in general . I hope you enjoyed this article ! :o)