The Illusion of Backtests and Drivers of Price ft. Richard Brennan

[00:00:00] Intro

You’re about to join Niels Kaastrup-Larsen on a raw and honest journey into the world of systematic investing and learn about the most dependable and consistent yet often overlooked investment strategy. Welcome to the Systematic Investor Series.

[00:00:23] Niels

Welcome and welcome back to this week’s edition of the Systematic Investor Series with Richard Brennan and I, Niels Kaastrup-Larsen, where each week we take the pulse of the global market through the lens of a rules-based investor. Rich, it is great to have you back this week. How are you doing down under?

Any big news from where you are in the world?

[00:00:41] Richard

Nothing really, Niels. It’s a bit cold down here. We’re going into spring now, so things are starting to warm up slightly, but it’s great to be back on the podcast.

It seems like forever that I was here last time.

[00:00:55] Niels

So many things happened, which we’ll get to in between our conversations at the moment. As you know, before we dive into all that good stuff, one thing I always love to do with all of you is to just kind of hear what’s been on your radar, what you think have been interesting, none of the topics that we’re going to talk about, but just something that’s come across your desk that you thought, yeah, this is actually kind of interesting.

[00:01:19] Richard

Well, I suppose what’s grabbed me this month is we had this very high level of volatility early in the month. There were pundits out there professing gloom and doom. Of course, the trend followers, we were hit with the trends that we were riding.

We got slapped around by the Japanese yen, and the NASDAQ coming off, and the S&P 500 coming off. But now it’s almost a forgettable event, what happened earlier in the month as we’ve now all sort of starting to recover again and going back into trending as normal. So that’s interested me because people do forget these events and they do pop up from time to time.

Luckily, for this particular event, for most people, it was a short, sharp event and we’ve had a reversal back into trend. So, it wasn’t the ensuing collapse that many people were professing. But it’s just interesting that you just never know what’s around the corner.

And these events hit you from left field and sometimes you recover well, like most of the trend following community has this month. And if, of course, it had turned into a significant downturn, I think our models would have quickly adjusted and we’d probably be in a good position as well. So, the good thing about our models is they’re very quick to adapt and I’m very fortunate to be a trend follower.

[00:02:55] Niels

Indeed. I mean, I agree with what you said here. For me also, it’s been a couple of things. One is exactly as you said, there’s so many things going on. But then you look and if you zoom out, like for a few weeks at least, not a lot has actually changed in most of the markets. So, part of me thinks, well, this could be the quiet before the storm when it really gets hairy. And in that sense, we’re kind of lucky, to some extent, that we’ve had this quick, volatile event because that usually makes us become a little bit more defensive in our positioning. I think that’s interesting.

I think this month we’re going to see some interesting things when it comes to maybe performance, dispersion, speed of models will be interesting to analyze to some extent at the end of the month when we know how it all pans out. I also think that some of the news coverage, in the early part of the sell-off, again, the usual articles came out, how much equity CTAs have to sell. And all these big brokers and investment banks came out with their analysis and saying, oh, CTAs have to sell $170 billion, I think was the number of equities. But what are they going to say now?

I mean, markets have bounced back. We’re within a fraction of all-time highs in the US. And does that mean we’re going to buy it all back, the things we didn’t sell or whatever it is? And also, people don’t realize that actually, even if the markets bounce back, we might be selling still because it’s more volatile. So, we need to adjust positions. Well, some of us, I know you don’t, but some of us would need to adjust positions accordingly. So, I think all of that is interesting.

Then I think about what else has been on my radar. Not much actually this week, except for last night where there was nothing on the television. And so. I was looking around and then I suddenly see this program, a documentary that has been done about the recent turbulence, severe turbulence (let’s call it that), that some airplanes have experienced. In particular one, I think Singapore Airlines, going towards Singapore from London, a few months ago, unfortunately also causing a fatality, but a lot of people got really badly hurt when it hit some severe turbulence. And I know this is a little bit of a stretch, Rich, but I think between you and I, I think it’s okay to always try to tie things back to trend following.

So, I was looking at this program and I noticed that a captain, not the captain on the plane, but a captain comes on to explain things and she said, it’s very important for you to notice that every time you see pictures from these airplanes and who have been through some really severe weather conditions, all the damage is internal. It’s from people bumping their heads or their bodies into the ceiling, trolleys flying around, which is of course horrible in itself. And I certainly wouldn’t have wanted to be on any of these airplanes. However, she said, but also noticed none of the planes have any structural damage.

To me that’s kind of a little bit like how I see trend following, right? That we go through these turbulent times, which we’ve just been through, and of course we’re going to see some bad performance, we’re going to see some volatility and turbulence in the performance, and it’s not going to feel nice. But I don’t see it breaking the models. I don’t see it breaking the programs or the managers that we all know and follow. And I think that’s very interesting.

And I think, obviously, with something like trend following that has been around for so many decades, we’ve seen so many different turbulent times, that I feel very comfortable saying that it’s very hard for me to imagine that any market event would break a trend following system, unless you start doing things you shouldn’t do. But if you follow rules, and if you have sensible rules, of course, if you’ve been around for a while, so you’ve tried a few things (not any AI optimized, what have you type stuff), I think you’re in very good shape with that.

Other strategies that I see do break, of course, tend to be convergence strategies, and even “the systematic macro strategies”. And maybe (and this is something we’re going to find out in a few weeks), maybe part of the difference between managers’ returns this month will be those who implemented non-trend trading models in their programs during the period where there were fewer trends and where you needed a little bit extra return if you could get it, and if you were prepared to deviate from the core trend, pure trend, classic trend, whatever we call it, methodology. Because we know that there’s always going to be a pressure from investors to either return better when there are no trends, but also to smooth out returns in general.

And this is why some of these other strategies, including Carry, have snuck their way into some trend programs, for sure. So, in that sense, I think it’ll be an interesting time to look back at.

[00:08:19] Richard

It’s interesting, Niels. You talk about how the planes were structurally fine, and it was only the things inside the plane that were disturbed. And when we look at the psychology needed to be a trend follower, you can understand this, that our models are incredibly robust, but our ability, our psychology, to hold and respect what these models do, that’s where we get thrown around.

It’s where our psyche kicks in and we start going against what our models are saying, and then we make the errors, and we make the mistakes. So, coming with these robust trend following systems is the need to be very disciplined, patient, and just respect what the model does. If you second guess it, or you try and override the rules that have been generated in this robust process, you’re going to come a cropper, and you’re going to be one of these victims inside these planes that sort of hit the roof, you know, without the seatbelts on.

[00:09:23] Niels

You know, we’ve talked about this a lot over the years, and I’m a big fan of sort of medium to long-term. I think all the data that I’ve seen suggests that it’s more robust in the long run and has performed better the shorter term. And again, as I’ve said many times before, I don’t really consider short-term trading short-term trend following. I think there’s a difference.

Anyways, the other thing that was fascinating about this program was they also looked at people studying turbulence and people trying to find solutions for turbulence. And one of the solutions that has actually been implemented on some small planes, but not the big airlines yet, is where they have technology now.

I think maybe AI helped some ways, but anyways, they somehow are able to, in less than a second, analyze the wind flow just in front of the wings, and then send a signal back to the flaps, and then they’ll adjust for it every second to offset the turbulence. And I kind of see also, from a research point of view, it’s kind of what we’re trying to do. I mean, we’re trying to find things that will dampen the volatility of the program.

[00:10:37] Richard

Which is what volatility adjusting is doing, and dynamic position, that is adjusting.

[00:10:42] Niels

Sometimes, yeah, sometimes. But also, in a sense, that you wouldn’t want to do this if it meant that the plane would get off course, right? You want to maintain the same course.

And I think with good research, and I think we’re going to be talking about some of these things today, with good research also comes this idea that, yeah, it’s fine if we can find things to improve and, say, lower the volatility or reduce the drawdown as long as we don’t change the profile and the long-term performance of the program. Because otherwise, we just end up lowering expectations, and in a sense, lowering volatility, but at the cost of performance, or even style drift, which obviously most trend followers don’t want to do.

So, in that sense, I ended up watching an interesting program. I wouldn’t recommend it for anyone who has a little bit of fear of flying, or is a frequent flyer, including myself. I have to say, I have a little bit more doubt for my next upcoming trips.

[00:11:39] Richard

I’m going to keep my seatbelt on next time.

[00:11:41] Niels

Yeah, oh my god. I tell my kids all the time, do not take your seatbelt off.

Anyways, speaking of trend following, let’s do a quick update. I mean, as we already talked about, it has been a really interesting month.

Of course, we focus on the trend followers, and it’s been a rollercoaster month, I think we can say that. Still another week to go before we know the end result, but it had a weak start, had a pretty good, strong middle section. We now have to wait and see how the last week or so ends up.

So far, as far as I can tell, tell me if I’m completely wrong here, it seems like equities have been the most challenging sector for trend followers by quite a margin. I think other sectors, like currencies, have been somewhat difficult. We know, of course, the yen has.

But I think fixed income has been a little bit divided, whether it’s US markets or European markets, maybe even Australian markets. So, I don’t think there’s a clear picture of that.

And then you have some single markets that have done okay. Sugar, coffee has been okay, trending well. Grains, trending well. So, I think, for me at least, what I take away from so far, the month of August, nothing new, of course, and that is just the importance of true diversification in a portfolio.

If you’re too heavy on financials because you want to have massive AUM, maybe this is going to be a little bit more difficult as a month compared to those who are fully diversified and can actually deploy real risk in all of these markets, so to speak. Any insight so far from your own side?

[00:13:21] Richard

Well, I think you’re right. I think it’s going to be very interesting looking at the dispersion at the end of the month. That’s going to give us an indication of how trend followers are positioned with their market universes.

So, I’m certainly happy with the commodities at the moment. They’re still continuing to trend very nicely. The things that crippled me early in the month were, of course, the Nikkei, the Japanese yen and the Nasdaq, and the equity indexes, but the equity indexes have all pretty well recovered.

The yen and the Nikkei are still down, but it’s been a very interesting month. The interesting thing is that mostly we find industry statistics are collected on a monthly basis. So, if you analyze things on a month-by-month basis, you won’t see this intra-month volatility and that’s a bit deceptive because it has been a bit of a quake this month.

There were certain times when I woke up and kicked the cat, but at the end of the month, everything seems fine and dandy. When you go back in history, and reflect back on these records, you’ll say, what volatility event did we have this month? Yet, I think the VIX had its biggest one-day move, did it? It was a historical move.

[00:14:39] Niels

Well, there’s a little bit… I discussed that last week, I think, where I came across this blog post that talked about whether the VIX really did hit 65, because the futures didn’t hit 65, not anywhere close. But anyways, it was certainly a vol spike, without a doubt. But also, yeah, the VIX is down.

[00:15:05] Richard

But look how quickly it subsided. Look how quickly it’s come back to low vol conditions. So, that’s very impressive.

[00:15:12] Niels

But it’s a little bit like the news cycle, right? You see these big headlines and you think like, in a US presidential period, there’s going to be some news out there that’s really damaging for someone, right? And then two days later, we’re not really talking about that anymore.

It’s kind of scary how quickly things disappear. But maybe it’s a good thing that we don’t dwell too long on a negative story. I don’t know. Who knows?

Speaking about negative stories, my trend barometer and my daily market score apps broke. This is old software. And they seem to have put themselves in retirement this week. So, people have written to me many times this week saying, are you aware that your trend barometer and your daily market score has not been updating itself for the last week?

And sadly, it has not. Now, I am, of course, working on finding a solution, but to anyone listening, you have to bear with me. Now, I think there will be a short-term solution for the trend barometer, which, by the way, finished because I do know how to… I can’t calculate it. It finished at 61 yesterday.

And you know what? It picked up quite early in the month, relatively. It started to move higher. And we’ve seen this recovery now from a trend-following perspective. Of course, this is still using somewhat shorter-term timeframes, but still. So, interesting to see if it continues like this and the month actually turns out to be okay.

[00:16:42] Richard

Well, let’s hope for next month that we have a real cracker.

[00:16:46] Niels

You never know. So, that’s good.

But as I said, people need to be patient and not rely on any data. As you were joking with me earlier when I told you about it, you said, well, I hope nobody’s using this as an input in their trading system.

[00:17:03] Richard

But the replicators…

[00:17:04] Niels

Yeah, I’m sure none of the replicators would use this system, but they are using other people’s data. And it just shows you that sometimes these things are not to be relied on, unfortunately. Anyways, that’s it about.

So, now we get to the performance part. I will say the industry has not recovered quite as strongly as maybe the managers, you and I know well. BTOP50 is down still 1.7%, up 3.4% for the year. SocGen CTA Index down 2.6% in August, up 3.4% for the year. No, sorry, I’m repeating the same number here. Up 2.11% for the year for the SocGen CTA Index. Trend Index is down 3.55% and up 2.3% for the year. And the Short-Term Traders Index, not surprising, up 61 basis points for the month and up 61 basis points for the year.

Now, in the traditional world, MSCI is up about 1% for the month now and up 13.84% for the year. And the S&P Global Development Sovereign Bond Index up 1% in August, up 1.58% for the year. And the S&P 500 Total Return up about 1% in August, up 17.8% so far this year.

All right, we have no questions, Rich. That’s fine, although we always like questions, so do send them in. But we are going to talk about three distinct topics, I think you can say. You’ve given me a little bit of a teaser.

The first one is about backtesting. I think it’s super interesting. And it discusses the pitfalls and limitations of relying solely on backtesting, I think it’s fair to say, when you assess trading strategies.

But without me taking any of the thunder from you, why don’t you start out laying out the topic and then I’ll hopefully be able to ask you one or two questions.

[00:19:07] Richard

All right, Niels, let’s get into it. So, I suppose what’s prompted me, with this particular article, is we’re in a pretty intense research phase at the moment. And I was looking at the comparisons between backtest results versus what I’m calling walk-forward results or out-of-sample results. And the differences are marked.

I’ve always known that backtesting introduces a degree of overfitting into your models. But I think people, and trend followers particularly, would be surprised how the backtest process actually significantly introduces overfitting into the process.

So, this article was inspired by the fact that industry has got to do better than just presenting backtests, because backtests are sitting in a landscape which sort of goes against the physics of things. For instance, we don’t have a time machine that allows us to go back in time and look at things from back to front.

When we’re live trading or when we’re trading, we’re trading on what you call the right-hand side of the chart. We’re looking into an abyss of blackness. We don’t know where we’re going. In our testing processes, we need to mimic that effect.

But with a backtest, when you’ve got the history laid out, you know every deviation, all in hindsight. You’ve got the benefit of this magnificent hindsight position. And you’ll know when you’re talking to a lot of podcast people and people, for instance, in Global Macro and things like that, when they’re talking about the reasons for things occurring, it’s always from the benefit of hindsight.

In other words, when you’re in the fray, in the middle of an event, you’re typically not thinking, what are the causes of this? You’re just trying to survive. It’s only when things settle down, a month later, that you can analyze what were the causal implications of that significant volatile event. That’s sort of from the privilege of a hindsight bias.

But when you are trading on the right-hand side of the chart, you seriously are sitting at the edge of the void. You’re sitting not knowing if price is going to go up or it’s going to go down and for how long or whatever.

So, I think a far better way to undertake a testing… Now, don’t get me wrong. Backtesting is useful for doing things like undertaking sensitivity tests on your models. For instance, you know, if you throw in different simulated regimes that might have occurred, historically, in the past, the backtest is a good way to assess how your model actually faced those regimes. But the backtest should never be used as a basis to predict future expected returns. Because what you inevitably find is that they’re always overfit.

You can’t help yourself. There are two things being introduced here. One is selection bias, which occurs. And just to understand what selection bias is, let’s say you undertake a backtest with a model over, you’ve got 10 parameter choices and your results are 10 different return streams over a defined period, 20 years. Now you produce these 10 different return streams because you’re testing over 20 or 10 different parameter settings. And then what you say to yourself is, right, I’m going to choose this return stream because it has performed better than these other return streams.

Now, as soon as you take that into the forward environment, you are biasing your result. That’s what we call a statistical bias associated with a hindsight selection process. You can’t do that.

You can’t do that. That’s using the benefit of hindsight to select from alternatives. You never, in the real world, when you’re looking into the abyss on the right-hand side of the chart, have the ability to make that selection. So, you shouldn’t do it in your backtest process because that’s introducing a statistical bias.

Another thing, of course, is overfitting to noise and overfitting to every event that has occurred in the past. Now, because we’ve got the past laid out in front of us and we know exactly how it occurred, yes, your backtest is inevitably going to overfit to that perfect scenario that’s laid down.

But the reality is that, when we go into the future, we never have that roadmap of what the events are going to be, what order of events, how they’re going to occur. So, we don’t have that luxury. That introduces another bias.

So, a far better way to undertake testing for anticipating what your future performance is going to be is what I’m referring to as a walk forward using out-of-sample data. So, there are two ways we can do this. We could either develop our backtest models using traditional backtest processes up to a defined point; an in-sample component of testing up to a defined point.

But then from that point going forward, we want to simulate the live trading environment, so, the right-hand side of the chart. We don’t want to know what’s coming up, but we want to assess how our models cope with that out-of-sample performance. That is something you can take to the bank. That is something that gives you an idea of what you’re going to face when you are dealing with future uncertainty.

So, when you do this, and a good example might be, let’s say you get 50 years of data. So, let’s say we’ve got futures data going back to, say, the 1970s. So, what you could do, for instance, is over a 30-year period, over from 1970 up to the year 2000, you could use that for your in-sample development process robustness testing using all of the evaluation methods using that data. But you have tainted that data once you’ve used it. You can never use it again.

So, you develop your models, and you statistically analyze your models and all of these things. But from the year 2000 on, you could either, for instance, say, I’m going to take this walk forward out-of-sample from 2000 up to 2024, where we are now, and evaluate how that performs.

That’s like saying, all right, let’s step back in time, back to the year 2000. Now what I’m going to do is, I’m going to pretend we’ve got the future ahead of me, which is uncertain, and I’m going to see how these models fare over that. Now that result you get from that process, without the benefit of cherry-picking, without the benefit of hindsight bias, that process actually gives you something that allows you to use it as a good benchmark for how much of a drawdown you’re going to get. What is your likely level of performance?

That’s something that hasn’t been tainted through this over-optimization process, and it’s very useful. And so what you do when, for instance, you do a process like that, so you have an in-sample component up to 2000, and then you test out-of-sample up to 2024, when you’ve got that process undertaken, and you’ve got the equity curve of your portfolio result, and all of the results of your different return streams, you can benchmark that against the live track records of other fund managers. Lo and behold, you’re going to find it’s very similar.

It’s not the glorious back-test that outperforms them, but something that approximates how the other people in the industry are faring. That is a much more reliable process to evaluate the efficacy of your strategy.

And I gave you an example where you had from 1970 up to the year 2000, and then you might decide to go out-of-sample from that period on. What you’ll find is that with your trend-following models, it is likely that they will be very robust, and they will stand up to the test of time.

But what you can’t expect, for instance, just like we can’t expect in real life, you can’t expect to say, in one year’s time, my trend-following models have got to be performing, because that’s not how trend-following works.

You’ve got to have a favorable trend-following regime for your trend-following models to be working. If, for instance, you’ve got an unfavorable regime, you must expect stagnation. You must expect drawdowns over that period of time.

So, it’s not like the expectations that I’m going to take my trend-following models now into the uncertain environment. I must expect this straight linear equity curve from this point on. That’s not the reality.

You want to have an equity curve that is actually representative of industry’s equity curve, and then you’ll see it’s fairly volatile. You’ll have positive regimes. You’ll have adverse regimes. You’ll get all of these things that allow you to, A, develop much better realistic expectations of future performance. B, have a true realistic benchmark.

You’ll have a better idea of what your maximum drawdown over the last 20 years is going to be using this process, and you can say to yourself, right, knowing that this is my maximum drawdown, and it’s much bigger, trust me, it is much bigger than what the backtest drawdown is going to tell you.

The backtest drawdown is going to say 10%, 5% or something. The out-of-sample drawdown is going to say 35%, 40%, something. A much more realistic assessment of drawdown.

But then you can use that as a much more reliable benchmark for future expectations, knowing that is the more realistic level of drawdown you’ve got. This is a more realistic level of compound annual growth rate you’ve got. This is a more realistic level of volatility that you get.

All of these things come out of a much better process of undertaking testing for trend following models. And I just know how many people use backtests blindly. You see it on Twitter, Niels, you see a myriad of these beautiful equity curves, these magnificent equity curves that you know are generated from a backtest environment. Looking back into the past, everything’s mapped out. Of course, it’s fantastic.

And I know often when you have Mark Rzepczynski on, he talks about, you never see a bad backtest. And the reality is you want to see a bad, you want to see realism, you want to see realism.

And so, the processes I’m outlining, this out-of-sample component, perhaps you might have an adaptive process built in where you might use an in-sample component from 1970 up to 2000, and then periodically you might do a walk forward to that point in time and then reassess your models.

In that case, it’s just like the live environment. You would undertake your backtesting over your in-sample data, over the past data, and then, going forward, once again, we put it into the out-of-sample environment going forward.

So that tells you, ah, with these adaptive models or workflow processes, how does that compare to something that I haven’t tampered with in the adaptive way, using these models from 2000 right up to current day? So, there are all of these different comparisons and benchmarks. What you find is that when you do it this way, what you can do is you can generate a pure backtest equity curve, which is the pink unicorn, I call it that. That’s what you’ll never get, but that’s the backtest equity curve.

You will then get your out-of-sample sample equity curve over that period of time. And you could get your adaptive equity curve over that period of time. And then you can compare the three, recognizing that the backtest equity curve is always going to be far superior than the other models, but the other model is going to be much more realistic.

So, this is why I refer to this topic as the illusion of backtests and how it can get you into deep water if you’re relying too heavily on these backtests as a guide for future expected performance. Inevitably these backtests are overfit, they’re curve fit, and they are not a representative of true trading going forward.

I know that in mean reversion modeling, for instance, where backtests are used, what they typically do is they have an in-sample, they divide their data up into in-sample and out-of-sample components.

The intent of mean reversion is, if you’ve identified a profitable pattern, a mean reverting pattern, you want to see that immediately reflected in your out-of-sample performance. So, let’s say you’ve got an in-sample component from 2000 up to 2018, and then you are trialling that model for 2019 and 2020, you want to see that there is no deterioration in the performance for the mean reverting modeler. That’s how they do it.

But in our world of trend following, we never know when the next trends are going to be. So, we don’t have that luxury of assuming the continuous trajectory going on into the future. We’ve got to develop a better way to integrate the volatility that’s inherent in trend following and associated with different favorable and unfavorable regimes.

So, what do you think so far about that?

[00:32:48] Niels

Yeah, and I think that everything you said, of course, is correct. This is why, if you want to do a proper backtest, it is more complicated than what a lot of people do and what a lot of people think. I think most of the larger trend followers, obviously, are fully aware of these pitfalls and so on and so forth, but you’re right. I mean, if you think about what goes on with social media, and people who want to attract attention, yeah, clearly they’re going to be showing some type of backtested results which are unlikely to be reproduced in the future, for sure.

But I think big managers are aware of these pitfalls and they do some kind of walk-forward process as well. I think that makes sense. But no, other than that, the only thing I was going to ask you, I think, regarding this, because we want to get to the other topics as well, is just if there is kind of a balance between in-sample and out-of-sample data that you think one needs to adhere to, because obviously, if you’re using 30 years of in-sample data and you have one year of out-of-sample data, of course, the in-sample data will be dominating.

[00:34:08] Richard

Yeah, well, the key criterion is in your in-sample component, you’ve got to develop robustness in your models. That’s critical. And so, you need a lot of data to do that.

But you might decide that multi-market testing is the way to develop robustness over 30 years of data, over 100 markets or whatever, that’s a lot of in-sample data to get robustness out of your models. But once you take them into that walk-forward environment, out-of-sample environment, you can’t afford to touch them. That’s the crucible to test the efficacy of all of your processes, how you’ve developed your models, all of that sort of stuff.

But as soon as you start selecting from that out-of-sample component, or you start changing up models, et cetera, that’s really starting to interfere with that process and contaminating your process with statistical bias, which inevitably causes this overfit result at the end. And it’s amazing how sensitive it is to overfitting. People don’t realize how sensitive things are to overfitting.

And you often find a lot of these off-the-shelf data mining packages that you might be able to purchase here and there. Once you’ve done your data mining, then there’s this extensive test used to extract the overfitting components out of it. You might do Monte Carlo, other forms of walk-forward, all sort of designed to sort of remove that level of overfitting from it.

But they’re still overfit at the end. So, the way to avoid that whole process and not have to go through that extensive extraction of the overfitting from your process is never get into that situation in the first place. Never touch that out-of-sample data.

That’s gold to you. So at least that’s from my viewpoint, Niels.

[00:36:13] Niels

Maybe it’s a marketing ploy, Rich, where the people who develop this software, the data mines, first they’ll make you pay for having something that gives you complete over-optimized and overfitted results. And then the next module you have to pay for is the one that takes it all out again. That might be a good strategy.

Anyways, we’re going to move on to the next topic. We’ve got two more to go through. And this is about dynamics of financial markets by analyzing how collective trader behavior influences price movement.

So, I’m very curious about this and where we’re going to go with this. And so, again, I’m going to let you guide us through it and hopefully I’ll have a couple of questions along the way.

[00:37:02] Richard

All right, Niels. So, we often hear the question about whether AI is going to be responsible for there being no more trends in the future. AI making this market so efficient that we won’t see trends existing in the future.

But I’d like to challenge that through looking at how markets work in a different way to conventional viewpoint. Like, for instance, in some of the theories we’ve got about trend following, a lot of it tends to center around behavioral bias of humans. Basically, the tendency of fear and greed in humans causing this behavioral bias. This is prospect theory.

But I’d like to draw things back and say, well, that’s a symptom of something that I think is a bit deeper than that. Because these trends that we see in the financial markets are not unique to financial markets. Trends are omnipresent in any complex adaptive system.

So, there’s something going on in complex adaptive systems that’s more than just human behavior. Human behavior isn’t the reason for trends existing in the financial markets. And I’d like to suggest that it’s actually trader impacts that is responsible for trends and mean reversion in the financial markets. So, we often hear about theories of using fundamental analysis as a way to assess future performance. But I think there’s a more primeval way.

A lot of people you hear just use price as a way to assess future performance. But I’d like to draw that back to something even deeper. And this is the trader impact associated with buying and selling. That’s all.

So, in other words, what actually moves price? There might be countless reasons behind the decision that a trader uses to buy or sell a certain quantum of amount in a financial market. But they’ve got to be matched with a seller on the other side who’s buying and selling or desires that same quantum amount on the other side to match that price of the buyer for a price movement to occur. So, this is a matching process. But the fundamental dynamics working there is more a mechanistic viewpoint of how markets work.

It’s the trader impact caused by someone buying in the market which exerts a pressure in the market – a buy, and someone selling in the market exerting a pressure in the market to sell. And when you look at microstructural research like Jean-Philippe Bouchaud’s done with his products, he talks about the square root law where the size of the order, there’s not a linear relationship between the size of the order and the impact it has on price movement.

There’s actually a nonlinear relationship there. It follows a square root law. Larger orders have more gravitas, more impact in the market than smaller orders. So, when you look at the price moves that occur in a financial market as a more mechanistic way to look at it rather than human behavior or fundamental reasons, they’re all sort of symptomatic or derivative reasons for why price moves.

But the actual reason price moves is the trader impact associated with buyers and sellers coming in together in the market and the quantum of the amount that they’re trading, that is responsible for applying an impetus or a force into the market that moves price either up or down. It’s as simple as that. But it’s not just the individual trader’s decisions.

The way the market aggregates all of these orders together is we find that it’s the aggregated effect of all of these trader impacts, the collective impact of all of that, which is responsible for moving price. So, it’s bringing things back to a physics way of assessing price moves, mechanistic way. And when you look at it through that perspective, you can get a deeper understanding of what is causing price moves and the complex adaptive nature of financial markets.

Because we understand this when you read works like Andrew Lowe’s Adaptive Hypothesis. We start realizing that when these trader impacts are exerted into the market, it depends on the model that’s being applied. Now we might have discretionary traders, we might have systematic traders in there.

They’re all applying an impetus into the market and the model that they have, whether it’s a model of their mind, as a discretionary trader that makes them buy or sell, or the model of a systematic rules-based process that tells them to buy or sell, they’re influencing, they’re putting a force into the market, a small force that approximates a square root law.

And when you get collections of groups doing the same thing, you can start seeing that the signal exerted by that force gets bigger. And when you get the collective groupings of different styles of trader behavior or trader impact, you can start segregating the market into these collective groupings of things like trend followers, mean reverters, fundamental investors, high-frequency traders. They’re all exerting forces, as a collective group, in different ways and how all of those different forces are coming together is responsible for moving price either up or down.

So, to explain what we mean by the markets being non-stationary and complex and adaptive in nature, we recognize that this trader impact changes over the course of time. The relative mix of participants doing a similar thing changes over the course of time. We get traders coming into the system and leaving the system. That influences the relative proportion of these different groups who are exerting these particular pressures in the market.

So fundamentally, we can start looking at two different pressures that are working in the market. Divergent pressures; those that are amplifying each other through positive feedback loops, and there are convergent pressures which apply negative feedback loops and actually dissipate or destroy a trend following signal.

So, if you aggregate all of these different trader styles as little ecosystems and you look at the gross force or the cumulative force that occurs, if they’re all working in tandem together, you can start seeing the market as these different directions of force being applied. Some are accelerating trends or diverging trends away from the market equilibrium. Some are converging price back towards the equilibrium. Whoever is dominating in that particular environment tends to win the day and that sort of tends to be in a sum total way where price tends to move with these two different forces.

So, in this manner, we can treat the market as comprising three broad classes of participants, the divergent participant, the convergent participant, and whoever dominates tends to win the day. But there is a third class of market structure which is noise, which is when the convergent and divergent neither win.

There’s a battle going on and we’re not getting a winner out of any of those two situations. That’s creating a very noisy environment where we’re not getting any directional price move at all, either up or down.

So, when you look at it like this, it really simplifies understanding how these markets behave. Rather than having it to go into these derivative processes of, oh, these fundamental factors are causing price to move at this time, different fundamental factors will affect the market at a later date. We recognize that all happens but when you look at things mechanistically, you realize that, in a simplistic viewpoint, there are convergent, divergent, noise, and there are two forces going on, what we call positive feedback forces that amplify price movements and there are negative feedback forces that dissipate price movements.

So, to understand these feedback loops, let’s imagine that we have a trend commencing and the trend gets very strong. We often find that the participants, the different ecosystems, they change in their mix because some people say, there’s a massive trend I don’t want to miss out on. I’m going to change my style from mean reversion or high-frequency trade or whatever. I’m going to jump on board that trend.

They’ll then do something that amplifies that trending signal. So, these participants often change their mind, change their models, flip to being different forms of participants in these different ecosystem groupings and you can see the flow in and flow out of different trader impact behaviors in this composite view of how markets tend to behave.

It becomes quite easily understandable when you understand that a lot of people say, oh, there are too many trend followers in the market. That means that trend following is going to die. Well, under this framework, no, that’s actually wrong. If there are too many trend followers in the market, trends get more extreme. Because everyone is doing the same thing. And so, when you look at the collective trader impact is amplifying that directional trend.

So, I can understand the edge being dissipated from mean reverting systems because with a convergent signal, the way that tends to dissipate the signal is that to apply convergence, you’re usually commencing from a divergent premise. In other words, prices made an excursion away from an equilibrium. And at this point, the mean reverter says, it’s got to come back to equilibrium. It’s gone too far.

So, they will apply this negative force to that prior positive directional force that’s got that trend to that particular position. And that of course, is going to dissipate that trend signal. It’s going to conflict with that trend signal and it’s going to dissipate that trend signal if they win.

And if there’s enough of them doing that, of course the trend signal dissipates. And we now go back into a mean reversion, swing back to an equilibrium and vice versa. You can see the ebbing and flowing occurs through this way of viewing the markets.

[00:48:04] Niels

Just to visualize this for the audience, a really good example, I think very easy to understand example from this year is cocoa. So, cocoa had clearly, I wouldn’t say a dominance of trend followers, but trend followers, generally speaking, were getting long during 2023. And the price did move higher.

But as we got into 2024, trend followers were actually getting out. And I would say that between January, and I can’t remember the exact month, April, May-ish, trend followers were probably (I mean, I’m generalizing of course) selling 80%, 85%, 90% of their notional position simply because the volatility was picking up, etc., etc. Now, I’m not talking about people who don’t dynamically adjust their positions, right?

Okay. Now, what we also know from the press is that another group of people got interested in cocoa in Q1. Certainly, and I don’t know about producers and suppliers, I have no idea.

But we know some other hedge funds, like global macro managers, and the most vocal one was Pierre Andurand, I think, that has been on a few podcasts and come out with this analysis saying, oh, they think cocoa can go to 20,000. So, they started buying cocoa in March, right? At the same time as we were selling.

But initially, that maybe with other people carried the price higher, actually, even though trend followers were selling. Now, the price has then subsequently come down and corrected maybe 40% from the high, and now we’re selling.

So, this is a very easy example to kind of visualize what you just explained. And I wonder whether the work you referred to, and I wonder if this is another side of this discussion that Jean-Philippe Bouchaud wrote about, this exogenous versus endogenous forces in the markets, because clearly what you’re saying, and I think that’s what he concluded, is that endogenous forces are actually much more responsible for big moves than the exogenous, which is like news events and stuff like that.

[00:50:22] Richard

Yes, well, if you can imagine the exogenous forces that probably commenced the cocoa run was the situation in Africa and the cocoa producers. But the acceleration of that trend was certainly, I think, contributed to endogenous forces. What we’re talking about here, the flow of different participants jumping on board the bandwagon, accelerating that trend. It was an extreme trend to start with. It got quite large in 2022, as you were saying, or 2023, what was it?

[00:50:54] Niels

2023, yeah.

[00:50:57] Richard

And then more eyes were looking at it. They started jumping on board. We got different participants coming in for the ride.

This just accelerated that trend. And of course, some which weren’t dynamically positioning, like, for instance, me, but there are certain other ones that we do know that were holding a very, very, very large cocoa position in April. When they started selling, that might’ve been a significant change in trader impact causing that significant sell-off that we saw in April. That might’ve accentuated that factor.

I think it’s a more intuitive way of understanding the mechanics of how things are moving rather than having to worry about all the details of knowing your fundamental analysis and all of these things, which is very complex. And there’s no guarantee that it’s actually causing that result.

But I think when you look at things from trader impact, it’s much more intuitive to understand. But also, the question, will AI ever stop trends and mean reversion? Well, I think when we look at these sorts of agent-based models that we’re talking about here, where every individual trader or agent exerts this pressure, AI is just like us. They exert an impact in the market.

I think trends and mean reversion are here to stay for eternity because I think that they’re a universal symptom of how collective dynamics work in complex adaptive systems. So, I don’t think, no matter what we get as agents participating in this market, we’re always going to get the trends and we’re always going to get mean reversion.

I think they’re here to stay.

[00:52:45] Niels

Great stuff. All right, last topic. Maybe we’ll do a short one on this one, if possible. But it is an important topic, actually quite interesting as well when I read your outline.

And this is kind of a challenge, I guess, of the conventional wisdom in trend following that highly correlated assets should not be traded together due to the perceived lack of diversification benefit. So why don’t you briefly talk us through that point as well so that people have something to think about when they listen to this on their Saturday or Sunday?

[00:53:23] Richard

Okay, so what this is, it is getting into a fairly challenging discussion here, but I’d just like to say it’s challenging because you need to view this from the perspective of an outlier hunter as opposed to a different form of trader because it is unique to outlier hunters. So, what I’m suggesting is that very small variations in price movement are sufficient to create an outlier in one market and no outlier in another market. And a good example is there is a conventional wisdom that says that diversification benefits start dissipating after a certain number of markets are included.

Now, for nearly all models except the outlier hunter, I would say 100% correct. However, for the outlier hunter, I’d say incorrect. And this is why.

If we take two very highly correlated markets like Brent oil and crude oil, and this is also specific for outlier hunters who use ensemble models to trade those outliers, not single systems, but ensemble models; groups, collections of systems. So, if you look at the price structure of Brent oil and crude oil, they are very highly correlated. And conventional wisdom would say, well, pick one or the other, don’t pick both because, really, they’re just doing the same thing.

And from the ability to improve diversification benefits, you should be looking at uncorrelated markets as opposed to correlated markets. That would be the conventional wisdom. And what I’d say is that, well, it’s not so cut and dried, especially with ensemble models. The reason is that, let’s say I’m applying 10 different forms of trend following model to both Brent oil (they’re the same models), and crude oil. The important thing for the outlier hunter is not the price correlations, it’s the trade correlations. So, what I mean is it’s the performance results from trading these correlated markets.

What are the returns? How are the returns generated from trading crude and Brent? Are they similar or are they different when you’re trading these ensemble models?

Now, when I do apply ensemble models, the same ensemble models to Brent and crude, lo and behold, I find that their performance results or return streams are really quite uncorrelated. And this is because when we’re dealing with these outliers, these anomalies in the market, for an outlier hunter who’s, they’re investing all of their efforts in riding these anomalies.

I’d like to use the term anomalies rather than trends here, because I think when I’m referring to outlier hunters, they’re the people that are looking for the cocoa move, you know, the significant material move, or the orange juice move, or, you know, we occasionally get these massive, unpredictable, nonlinear trends. These are what I’m referring to as outliers.

What you find is that when you are in this nonlinear territory, very small variations are sufficient to cause significantly dispersed results. And so, if we’re applying, let’s for instance, 10… Now the reason I apply an ensemble of models to these outliers is that any single model, it’s almost crossing your fingers, hoping it works on those outliers. They’re very volatile, ugly beasts to trade.

You need an ensemble to maximize your chances of extracting as much opportunity from that price move as possible. If you apply a single system to an outlier, you’re going to fail, because it might catch a one outlier, but it’s going to be snagged out of the volatility move of one outlier, and it’s going to take forever to get back in.

But that’s why you apply an ensemble model, because it’s maximizing your chances to extract the fruits out of that anomaly. But the very small variations between, say, Brent oil and crude oil are sufficient to significantly alter the return profiles that you achieve from trading both of them together.

So, when, as a portfolio, you’re putting quite highly correlated markets in terms of their price correlations together, when the trade correlations are very uncorrelated, you’ve got no problem bundling these all up together as legitimate markets to trade and increasing your levels of diversification.

So, I’d just like to say that for our particular outlier hunters, the small variations do matter. And importantly, this comes to a physics concept, Neil’s, called the butterfly effect, which we’re all aware of that in nonlinear mechanics. The flap of the butterfly wing over in Brazil or whatever might cause a hurricane over in Australia or something like that.

You know what I mean by the butterfly effect here. This is the very small differences in initial state are sufficient to create significant dispersion in the nonlinear mechanics that arises from that result. And this is why with outlier hunters, we’re a rare breed, a bit different to say those that focus more on the bulk of the distribution of returns.

We’re focusing on the tail properties of the distributions that are very unpredictable, nonlinear in nature. These very small variations are sufficient to create significantly different correlated properties than what you get when you’re trading more tamed subdued markets, et cetera. So, you know, you were saying how cocoa, a lot of trend followers, most trend followers were selling into cocoa in 2024.

Well, I wasn’t. And I’d say that who I’d class as an outlier hunter wouldn’t have been selling into that. They would have been holding on for dear life, like me, and probably pyramiding more.

When I say pyramiding more, with the ensemble of trend following systems, more and more systems are getting into that trend. Yes, we find that there’s a big slap in the face when that trend turns. That’s inevitable with our process. But yeah, I wouldn’t have been selling into trend on cocoa in 2024.

[00:59:53] Niels

No, and this is what makes the market, right? That not everyone is doing the same thing. And this came up in another discussion that I was talking to maybe Katy about last week or maybe it was the week before, where we talked about one thing that really got tested during the mini event, let’s call it, I wouldn’t even call it a crisis, but mini event we saw this month was, of course, risk management systems or approaches, right?

And I think a lot of managers today use some kind of valued risk-based model. Maybe not the classic managers, but certainly a lot of the bigger managers do. And so, the natural question to ask is, well, if everyone is using the same type of risk management system that is sensitive to changes in volatility, aren’t we all going to be selling and buying at the same time?

Well, the answer is, of course, not everyone uses value at risk. A lot of people do, but not everyone. And therefore, it still works. And that is, of course, an important point to remember.

This was great, Rich. I think three very interesting topics. I’m sure people would have found that super useful, as always, when you come on giving us a lecture, trend following lecture, Professor Brennan, as I like to call you.

So, if people out there listening to us today appreciate what Rich puts into this as much as I do, why don’t you go to your favorite podcast app and leave a rating and review? That certainly helps us, and it shows Rich and everyone else involved in producing the podcast that it is appreciated.

Next week, I’ll be joined by Andrew Beer. And that’s also going to be an interesting timing, at least, to be talking to Andrew, because clearly, they have a different approach when it comes to trend following and because they do replication. So, we’ll see how they have experienced August and what their takeaways have been.

So, if you have questions for Andrew, I hope you do, then send them to the usual email address, info@toptradersunplugged.com, and I will be happy to bring them up with him. So, from Rich and me, as always, thank you so much for listening. We look forward to being back with you next week. And until next time, take care of yourself and take care of each other.

[01:02:31] Ending

Thanks for listening to the Systematic Investor podcast series. If you enjoy this series, go on over to iTunes and leave an honest rating and review. And be sure to listen to all the other episodes from Top Traders Unplugged.

If you have questions about systematic investing, send us an email with the word question in the subject line to info@optradersunplugged.com, and we’ll try to get it on the show. And remember, all the discussion that we have about investment performance is about the past, and past performance does not guarantee or even infer anything about future performance. Also understand that there’s a significant risk of financial loss with all investment strategies, and you need to request and understand the specific risks from the investment manager about their products before you make investment decisions.

Thanks for spending some of your valuable time with us, and we’ll see you on the next episode of the Systematic Investor.