Understanding the Parameter Space: Some Frequently Asked Questions (FAQ)

When you want to build trading strategies, the first thing you will find in your path is a little monster called the “parameter space”. This little critter generates many traps for algorithmic system developers and is often to blame for strategies that can look very good in reliable historical testing and later fail under live trading conditions (note that I am talking about sound strategies to begin with). Understanding how the parameter space works and what information it gives us is of fundamental importance to reach profitable trading outcomes in real live trading. Although I do not claim to understand everything about the parameter space, I do believe the knowledge we have gathered in Asirikuy within the past few years is enough to draw some light into the role played by the parameter space in profitable system design.

With the above in mind I decided to create a small FAQ today in order to condense the most basic aspects of the knowledge we currently have about the parameter space and how this relates to profitable system building. The answers to these questions may change as we find out more about system design and evaluation but for people new to trading – or interested in understanding more about the parameter space – I trust it will be a good source of basic information for at least a few years to come. Let me know get into answering some basic questions about our little friend:

What is the parameter space?

When you build a trading strategy you need to make some choices that will determine how your strategy will trade. For example, if I build a system that trades the cross of a moving average I will need to choose what type of moving average, what period of the moving average, what shift of the moving average, etc. Additionally if my positions are being exited by a stoploss or a takeprofit I will also need to choose some values for these as well. All the things I can change are called “parameters” and the group of all possible parameter variations is called the “parameter space”. In the above example the moving average period, the stoploss and the takeprofit are parameters while the parameter space is made up of all the possible choices of each of of these parameters. A single combination of all parameters is also called a parameter set (this is what you load to trade).

Why is the parameter space important?

It is important because the parameter space is the single entity that describes your trading strategy. Following the above example, if you have a strategy that trades a moving average cross and has a take profit and a stop loss how will you know if your strategy is good or bad as a whole? If a system with parameters (200-MA, 50-SL, 100-TP) gives you profit, does it mean that your strategy is likely to succeed ? Analysing how all parameter variations (the parameter space) is constructed will allow you to obtain a better idea of how robust your strategy is to changing market conditions.

What do I want to know about the parameter space?

The foremost important characteristic of the parameter space is its topology. Image if you drew the parameter space as a map where each point in the XY plane represents a parameter set and each Z point represents the system’s profitability, a system that has a lot of spikes is not good because it means that small variations in the parameter space cause large variations in profitability which means that the system is simply tightly fitted to some past market condition and greatly loses this profitability when the market changes. What you would want to have is a very flat parameter space where every parameter set has very similar results. This means that your strategy works, no matter how much you vary its inputs. It is also good to consider that the above analysis becomes harder when  larger numbers of parameters are used because the spatial closeness of parameter sets needs to be determined.

What do you mean by closeness?

Well, since we are interested in measuring how variations in parameters affect our trading system, we need to ensure we know how a parameter set compares with another. Is there more variation from set A to set B or from set A to set C ? The closeness in the parameter space can be determined by using measures of Cartesian distance in the n-dimensional space. Getting a good measure of distance is possibly a very important aspect of doing a thorough topology analysis.

What are the risks of ignoring the parameter space?

When you ignore the parameter space you run the risk of developing systems that are very dependent on a large number of characteristics of the historical data. This means that small variations in the future will cause your system to fail to some extent. If you’re unlucky this can mean that your strategy can become useless very quickly under live trading conditions. Performing a full analysis of the parameter space allows you to have a lot more peace of mind regarding the ability of your system to remain profitable under market changes.

How do I need to consider the parameter space when designing a strategy ?

The foremost important advice, something we have learned through experience, is to keep the size of the parameter space small. This means that you should try to reduce the number of parameters to an absolutely bear minimum. Strategies that have less degrees of freedom are easier to analyse and it becomes easier to know whether there is a problem with your trading results. If you have a small parameter space you can easily get back-testing results for the whole space and analyse how your system performs. For systems with large numbers of parameters this becomes almost impossible because the computational effort – and the memory required – becomes too large. For example a system with 2 parameters (that go from 0-50) can be evaluated easily in a few minutes, a system with 20 parameters following the same variations is – in practice – impossible to fully analyse. Don’t forget that adding degrees of freedom increases your chance of stumbling upon some profitable historical settings through optimization but finding out the relevance of this profitability within your strategy will be extremely difficult.

Can you have a system with no parameter space?

There will always be a parameter space because there must always be some quantitative rules to trade. However parameter spaces can become reduced and even taken to higher levels of a trading strategy. The parameter space determines the lowest level of a trading strategy that is deterministic in nature. A neural network strategy, for example, can use no parameters on the lowest level of the strategy but it will use parameters (network topology characteristics for example) on a higher level. This means that the parameters of the neural network are controlling the way in which the network adapts but they are not controlling actual trading at a ground level. The higher the decision level of the parameter space, the more dynamic a strategy is.

Obviously there are many more questions related to the parameter space which I will try to cover on future posts. However I hope that the above answers give you some idea about how the parameter space affects trading and why it is so important for you to completely analyse it and consider it when developing your trading systems. A trading strategy with a small parameter set is ideal while strategies with higher level parameter spaces are also desirable. For practice I would suggest you to take a simple strategy, analyse the parameter space and see how variations in parameters affected the strategy along past market conditions.

If you would like to learn more about system design and how you too can learn to build and analyse trading strategies please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading in general . I hope you enjoyed this article ! :o)

Print Friendly, PDF & Email
You can leave a response, or trackback from your own site.

Leave a Reply

Subscribe to RSS Feed Follow me on Twitter!
Show Buttons
Hide Buttons