Yesterday one of the traders taking my Forex Robots Course asked me an interesting question: “What’s the difference between Optimization and Curve-Fitting in Forex Backtesting?” Let’s talk about that for a minute…

The article

The question was prompted by an article from onestepremoved.com. It’s a good read, but I’ll give you a break down here to save time.

Basically, the author defines optimization as the process of finding “the best collection of entry signals that in conjunction with the exit signals maximize some objective”. Curve fitting, on the other hand, is referred to as merely finding “a curve that best fits the historical data”.

What doesn’t sit right with me, is that this definition severely undermines any Optimization we do in MetaTrader 4. The article is effectively saying that if you use the MT4 strategy tester, you are not putting enough thought into your entry/exit signals, and hence you are curve-fitting your results. Your system will fail.

I completely disagree with that. I truly believe in Forex Optimization in MetaTrader 4 and this is why.

Forex Backtesting in the Real World

In the academic world, perhaps, optimization can be defined as the process of searching for the best signals, timing, entry points, etc. However, this is a very “theoretical” approach. In the real world we use machine-learning to find the best possible result and it is still called optimization.

Here’s an example which will help draw similarities in other fields of finance. For instance, any respected bank creates propensity models for its customers when giving out credit – to see who is more reliable, who is more risky, who will pay on time, etc. Many years ago, in the previous century, banks would hire statisticians to create segmentation models of their customer base using various statistical methods. Business knowledge played a massive role in this process, because it is very timely to go through all possible variables and their combinations. So the statisticians would look at the existing processes, ask bankers about their intuition, and prior experiences with customers, to build the most “proper” model. This is what the article refers to as “Optimization”. And that’s an academic approach.

On the other hand, what banks do today is they use Big Data systems like Hadoop in conjunction with machine-learning techniques where they just throw ALL the variables they have (1000’s++) and let the machine do the work and find the best possible model that describes credit risk of their customers. There are caveats to this (penalty factors to avoid over-fitting), and, of course, some business knowledge is still used to override parts of the model. However, in the big scheme of things, banks no longer care if they can explain how the model ranks their customers. All they care about is that it predicts credit risk.

Same thing here. What we do, the author would call “curve fitting”. I call it optimization, because frankly, it can be called whatever – as long as it works. Moreover, as we discuss in the Backtesting course, Forward tests help detect and avoid curve-fitting. Banks use the same technique, it’s just called different – model verification.

To conclude, don’t be afraid to throw as many variables into the MT4 Optimization mix. Modern machine learning is at a very high level and it would be unwise not to take advantage of that.

Till next time – Happy Trading!

Kirill