Finding the Optimal Trend Following Rules ft. Richard Brennan

[00:00:00] Intro

You’re about to join Niels Kaastrup-Larsen on a raw and honest journey into the world of systematic investing and learn about the most dependable and consistent yet often overlooked investment strategy. Welcome to the Systematic Investor series.

[00:00:23] Niels

Welcome and welcome back to this week’s edition of the Systematic Investor series with Richard Brennan and I, Niels Kaastrup-Larsen, where each week, we take the pulse of the global market through the lens of a rules-based investor. Rich, wonderful to be back with you this week. How are you doing? What’s going on down under? Any big news for you to share?

[00:00:40] Richard

Not really any big news at the moment, Niels. It’s just getting colder down here. We’re approaching 10 degrees Celsius, which to us is freezing, and so we’re starting to rug up. But I know that’s a bit of a different climate to what you’re experiencing up there.

[00:00:56] Niels

Well, unfortunately, it’s pretty close to 10 degrees where I am and it’s the middle of summer, so I’m not sure which part is the best. Anyways, we’re going to be tackling a few important topics, actually some really interesting new ones. But before we do that, I always love to hear what’s been on your radar. Anything you picked up since we last spoke?

[00:01:19] Richard

Not really. It’s just that it’s been a very tough week or so, I found, with trend following models. I’m referring to it as a pogo stick environment at the moment, lots of whipsaws happening. We’re getting some success in some assets and some failures in other assets. Quite a difficult period for us.

But on a brighter note, we have our regular State of Origin football match over here in Australia. There are three games played by Queenslanders vs. New South Wales. The individual players, they line up with their associated states of origin and then they battle it out on the field for three games. It’s a pretty exciting match, and I’m proud to say that in the first match, we gave New South Wales an absolute drubbing. So we’re very happy.

[00:02:10] Niels

Yes, absolutely. Since it is June 14th as we speak, it also means that over here in Europe, we are going to be looking forward to our kind of football at least for the next few weeks, namely the European Championship. So that will be exciting as well, maybe not as rough as what you’re used to, but it’ll be exciting. Now, for me this week, yeah, you’re right. We’re going to talk about a little bit of a rough time in trend.

But the things that I came across and I thought I could not pay attention to, I guess, is the small paycheck that Elon Musk last night seemed to have been awarded, although it’s smaller than it was in value when the stock was at its high. $56 billion is the number that’s being thrown around, which is a pretty sum to most of us. It kind of made me think, I was going to ask you, if you had a paycheck of $56 billion, Rich, how would you invest it?

[00:03:10] Richard

I think we all know, Niels. A 100% trend following.

[00:03:12] Niels

It would be the biggest boost for the assets of trend following that’s for sure if suddenly you did that. That’s great. It also reminds me of a thing when you were a kid and you were playing lotto (I still do from time to time), you were always thinking, “So what if I really won this big prize?” When I was a kid at least, the prices weren’t that big. But nowadays, some of them are quite meaningful, or extraordinary I should say. So you’d always think about, “What if I won? What would I spend the money at? What would I do first?” Did you ever have those dreams as well, Rich?

[00:03:54] Richard

I certainly did, Niels. I still do.

[00:03:57] Niels

I think we all do.

All right, let’s move on. There was a couple of more market-related things that I picked up as well this week. One thing I will say, everybody will be familiar with what’s happened with the Fed this week of course, I’m sure. I do think it’s interesting and I think it shows the unpredictable nature of the world and certainly of financial markets.

When we started the year, we were talking about several rate cuts fixed in stone. Now with the latest dot plot, they’re thinking about maybe one towards the end of the year. I think that’s a major shift in sentiment for sure. Maybe that’s also why the markets have been kind of in flux recently, because there’s no real clear path at the moment it seems.

So yeah, I just think it’s interesting, and also, I think it actually aligns pretty well with my own thoughts for the last period of time that it’s not going to be that easy to get rid of inflation and higher for longer is probably not a bad way to describe it. We’ll just see whether it’s still high enough or whether we might even get a complete U-turn from the Fed, which we’ve seen before, and the next change is another hike. I wouldn’t rule it out even though it’s certainly not the consensus at the moment.

You mentioned it’s been a rough time for trend following. I completely agree with that. It certainly has been also on our side. It looks to me like maybe half of Q1’s returns have been given back by managers, which is fine. It’s not unusual whatsoever, but it’s pretty broad-based. Metals, fixed income, currencies, obviously some standout markets have been challenging like sugar that seemed to have turned around.

So maybe we’re just setting ourselves up for a really good entry point for those who have missed trend as an allocation up until now or those who just don’t have enough. In an uncertain world with equities at all-time highs, I certainly remember that a lot of people rebalanced out of trend after 2022 into 2023 because it had done so well relative to stocks and bonds.

Maybe it’s time to reverse that equation and rebalance back into trend now that stocks are at all-time highs or close to, and bonds have done a bit better and trend is having a little bit of a wobble, let’s put it that way. So, it may not be only you putting in $56 billion into trend, Rich. There are a few other people. But that would be a small change compared to your allocation, I’m sure.

Anyways, my Trend Barometer finished at 27 yesterday, so that’s a pretty weak reading completely in line with the returns we see. BTOP 50 down 2.23% as of Wednesday, up 7% still for the year. SocGen CTA down 2.16% for June as of Wednesday, up 7.15% for the year. SocGen Trend down 2.5% for the month, up 8.89% for the year. SocGen Short-Term Traders Index is having a bit of a rough time actually. Down 2%, which is a lot for that kind of volatility in June, up 69 basis points for the year now.

MSCI World doing well, up 1.72% for June, up 10.58% for the year. Finally, the S&P 500 Total Return up 3% in June, up 14.65% for the year. For some reason, I couldn’t find the World Government Bond Index. My page has stopped updating. Maybe they have figured out a way to charge people for that kind of information.

Anyway, maybe a quick comment to one of your Twitter friends, Dennis, who has a question about CFDs. He writes in, “Keep up the great work. A couple of questions for Rich when he’s back on the show again. I’m looking to trade CFDs and had some questions regarding costs. I look at dated CFDs but prefer cash CFDs due to the increased number of markets available to trade. Given the nature of lengthy hold periods, do you recommend trading cash CFDs accounting for the overnight cost? Thanks, mate. Regards, Dennis.”

[00:08:22] Richard

Good old Dennis. I know him well.

[00:08:23] Niels

I know. I could hear that from the tone of the email. Do you have a quick comment?

[00:08:30] Richard

Yes. Dennis, I trade cash CFDs as well. He raises a very good point, which we do find when trading these products and it’s something to bear in mind. So the costs associated with these products can be quite large. What I’m doing is I’m saying relative say to futures. In our CFD world, it’s a decentralized landscape as opposed to running through centralized exchanges.

So we’re typically dealing with brokers who are margin makers, or they create the market makers. So there’ll be specific broker costs such as broker spreads. There will be typical slippage, which is another cost, and there’ll be holding costs, which in our world of CFDs we call swap costs. S-W-A-P, swap costs. Now, swap costs can either be negative or positive dependent on the market you’re trading.

A positive swap means that you earn interest when you’re holding an asset for a period of time, and a negative swap means you’ll pay interest over the course of time. So with the structure of these products by these brokers, we can find there’s a large array of assets, either long or short, and the swap will vary depending on the direction and the type of asset.

Now, we often can have positive swap, which is good for our models. That means that there’s an element of carry in our trades. So when we’re trading an instrument in the direction of a trend that has positive swap, on top of it, the trend value, we’ve got the benefit of the carry trade associated with a positive swap and holding it for a period of time. But then if we have a positive trend in the markets, it might possess negative swap, which means that if you’re holding for extended periods, the cost of that swap can become quite large over the course of time, and it can materially impact your performance results.

So for Dennis, I’m just saying you need to monitor these costs. You’ll need to have a little dashboard up on your trading screen which monitors these costs on an ongoing basis. It’s wise to always look at the particular impact of these costs when you’re doing your backtesting process.

So when you do your backtesting process, the first aim is to identify whether the models you’re developing are appropriate for trending markets. And then, once that test is done, you need to import back the costs associated with your broker to say, what is the practical application of me trading knees with trend following models?

You might find that some of the markets and models get ditched because, for practical purposes, you simply can’t trade them because of the onerous impact of the costs. That’s something that Rob Carver has mentioned before with you, Niels. So yes, it’s wise to monitor these costs. As part of your process, you do need to include these into your process.

[00:11:39] Niels

Cool. Great stuff.

So the next question is from Ramki, which we definitely wanted to bring because he is very kind to us in the beginning of the email, as people can hear now. “Thank you very much for the great content you and your team produce every week. It’s simply the best podcast out there. Nothing else comes close. The show has been invaluable in my trend following adventure.” So of course, with an intro like that, Rich, Ramki is welcome to ask as many questions as he wants.

Now, here it goes. “I have a question for Rich on position sizing. I’m working on backtesting my own trend following system based on all the excellent inputs over the years from TTU. I have run into an issue that has stumbled me. Most assets have gone up several folds in price over the past 20 years. For example, WTI was trading at $25 a barrel in 2001. The ATR at that time was $1. Another more extreme example is gold. It’s up 10x in price since then, and the ATR has increased from $2 to $40 per contract. I’m using ATR for position sizing, so I end up with a very large number of contracts for the position size in those early periods when the contract size was much smaller. This skews the results and doesn’t feel right. How do I get around this? I’m grateful for any comments and feedback on this issue. Thanks in advance, Ramki.” I’m sure there’s a good fix for that, Rich.

[00:13:08] Richard

Look, the thing for Ramki is don’t worry about it. It’s actually not an issue and it’s something you do find. For instance, when you look at your extensive backtest over 30 years, the price of that underlying asset, in certain instances, can significantly change. And yes, that does impact the number of positions or lots you apply to that process. So with the models that I deploy, I’m finding exactly what Ramki finds with his models, and it’s nothing to be concerned with.

The key thing to keep in mind is that, for instance, for my models, back in say 30 years ago when I’m doing backtests, I risk per trade let’s say an example of $500 per trade. So what I do is I assess the ATR using say a 25-period ATR, and then I look at the multiple of ATR that positions my stop at a particular logical place, and then I assign $500 risk to that, which decides the number of positions to trade to risk that $500.

That is exactly the same back then as it is that I’m applying now. But the position sizes are significantly different because the price of the asset has significantly changed, and the underlying volatility has significantly changed. Provided you’re still risking that $500 risk bet, you are today as you were in the past, that’s a good sign. But yes, recognize that your position sizes will change. So it’s not something to avoid, it’s something to embrace.

[00:14:40] Niels

Good stuff. I hope that was useful, Ramki, and that you fixed that in your backtest. Rich, you brought along, obviously, a very topical question for two people like us who have spent years in search of the optimal way of doing trend following. We would of course need to find the optimal rules to use in different kinds of market regimes. Incidentally, that is the topic that you wish to discuss sprinkled with a bit of AI at the end, let’s put it that way. So I can’t wait to jump in.

It’s based on a paper which I have to admit, I have not read. I’ve done what I could to catch up on your notes, but it is more in your hands, so to speak. So without further ado, Rich, let’s get into this topic.

[00:15:33] Richard

Okay, Niels. Here we go. So, this is a paper that was produced in 2023 called “Optimal Trend Following Rules in Two State Switching Regime Models”. What this is about is developing a mathematical process that can develop, using maths, as a way to optimize a trend following portfolio. The paper was produced by Valeriy Zakamulin and Javier Giner.

Where it’s interesting is that a gentleman by the name of Dr Tom Starke, he’s a well-known quant in his world, he had a YouTube video where he examined the paper in detail, and he explained to his audience in layman’s terms what the paper was about and the different processes that were being described. I found it very educational and informative because as you know, Niels, when we get to some of these white papers, they’re full of mathematics.

Often, it’s very, very hard for the lay reader, who isn’t skilled in these particular mathematical concepts or these physics concepts or whatever, to actually navigate through those papers. So this is where Dr Tom Starke on his YouTube video, and it’s a great video, he examined these papers in detail. In over a course of about an hour, he explained it in detail. I’d like to just go through some of the observations and findings from this process.

But before I do, I also want to suggest that for those people that are comfortable and fluent in using their Chat GPT or their AI, they can use Chat GPT, for instance, to review white papers, and then they can discuss with Chat GPT in a layman’s viewpoint how to step through that white paper and understand it. I use that now fairly heavily, Niels.

When I’m examining a white paper, the first thing I do is I download the PDF from that white paper into Chat GPT and then I say with a prompt to Chat GPT, “Read this white paper, and then to a layperson with no experience in the topic, please explain what the principle in this paper is about.” And then you step through it in a very educational way, and it teaches you as you go some of the things you might not have known.

Do you use this at all, this method of using Chat GPT to help you get through some of this hot content?

[00:18:22] Niels

I have not used it for a white paper like that. As I said to you before we pressed record, I did take your extensive notes and put it into Chat GPT just to kind of summarize it for me. But I’m intrigued by the technology. Of course, I am. I’m sure I’ll find time to become better at it. Exciting technology, for sure.

[00:18:47] Richard

This paper, that was reviewed by Tom Starke, produced by Zakamulin and Giner, it looked at an alternative way to assess the optimized models used for trend following. What they did, they basically suggested in their abstract there’s a common problem with trend followers in that the methods that they deploy to come up with their models tend to be what they refer to as ad hoc.

They’ll use this extensive backtesting process over very large datasets. Through that process, they’ll do heavy optimizations across the different parameters to ultimately come to models they feel meet their performance requirements on an ongoing basis. But there are no mathematical formulas associated with that. It’s a process that we deploy with this extensive backtesting.

What they’re saying is that they believe that there are mathematical formulas that can be applied to this process. So this white paper is about using what we call Markov processes and also processes of autoregression. Using these mathematical concepts, they evaluate the efficacy of three different types of trend following models: one being a simple moving average crossover model, another being what you call a moving average convergence/divergence model using the MACD (you’ll be familiar with that, Niels. I’ve certainly heard of it), and another using a momentum model.

When they look at these models, in the simple moving average crossover model, there are three different types of simple moving averages. There’s either the simple moving average or a linear moving average. What the linear moving average does is it weights the moving average in a linear way such that when you’re using a lookback of, say, 50 days for your moving average, the weights at the beginning of the 50 days is zero, and then, in a linear progression, the weights slowly increase over that course of time. Or uses what they call an exponential moving average, which is like a linear moving average, but it puts the greatest weight of that towards the most recent data in an exponential way.

Now, what they’ve done is they’ve used those models and then they’ve undergone these processes using Markov processes and autoregression, which can be used to define these mathematical formulas that evaluate the correct or the optimal model to use in the optimal market regime.

Rather than just talk about Markov processes and autoregression in my conclusion, we probably need to step through a few examples of what a Markov model is, what an autoregressive model is, and look at it before we start looking at the conclusions of what they found in their research. But what I can say in the conclusions of their research is that the optimal models that they come to actually agree with the optimal models that, for instance, you and I come to with our backtesting process.

In other words, they come to models that deploy these simple models that have a medium to long-term hold, and they define the parameter sets with fairly what we call loose pants. Now, this is using this mathematical process that has not got any assumptions in it apart from the mathematical process that has been deployed. But it comes to the same optimal levels that we do through this time-intensive crunching that we do through our backtesting process.

So the benefits of this mathematical route that they’re talking about is that once the mathematics has been defined, it allows you to very quickly use it in other market landscapes. What Dr Tom Starke did was (this research undertaken by Zakamulin and Giner was on equities), what he did, he took the values that came up with the optimal models to use based on the stock market into the futures market and found that on the same data, there were very effective models that got him a Sharpe of about 1.2 Sharpe over about 20, 30 years, which is pretty impressive using the power of mathematics alone.

So let’s step through the style of maths that they were using. The first thing that they were using was what they call a Markov model. Now, what a Markov model is, is it’s a model that examines the state of a market without needing to have any information about what the preceding market states were. It says, if we understand the current market state, then what is the probabilities of it staying in that current state for a period of time or over a period of time transitioning to a different market state?

So what a Markov model is trying to do is it’s assessing two things. It’s looking at the probability that a market will exist in a particular state. A good example would be, let’s say for instance we were examining a price series and saying, I want a Markov model to identify (you’ve got to give a number to the market states), let’s say three market states. You could say 12 market states. The process that it uses, uses a process called Bayesian inference and machine learning, basically. It comes up with what the 12 market states are.

So you give the particular number of market states. In an example, let’s say we have three markets states. Those three markets states don’t necessarily need to be states that you and I think of as market states. So we might say there’s a bull market state or a bear market state, but the way the Markov process works, let’s say you say I want it to define three states, it might not necessarily come to the three states that agree with our definitions of what the market states are because it’s using the data as opposed to our assumptions as to what those states are.

So the states, let’s say we want to examine three states and it comes up with a solution where it says, one state is what we call a normal market state, the state that the market exists in in most frequent terms. That, for instance, might be a market that’s exhibiting a slight trending positive bias or a market that’s exhibiting a slight negative bias, but that is considered, by the Markov model, as one particular state.

Another state might be a very fast retracement, which is identified as a significant market state that’s different to the alternative market states. Another state might be not a retracement, but an explosive volatility surge in the market. So, what happens? When you say let’s examine three possible market states, it then does a process of Bayesian inference where you say, now apply training data to it, the market data.

Let’s say you initially have to say, what is the probability assignment for the three hypothetical market states? You might say, I want 33% for this market state, 33% for this market state, 33% for this market state. It then goes through a process of Bayesian inference using the data to then ultimately come to the probabilities and the states that those markets exist in with a couple of key things.

First is the probability of that state existing at any point in time, and two what is the probability assessment of it transitioning to another state? So with a Markov model, the whole idea is to say, once we’ve done this process of Bayesian inference using basically machine learning algorithms, where it gets better and better at its probability assessments, then it comes to a conclusive process. From the analyzed data, I’ve identified these three market states.

Now, to go from let’s say we’ve got state A, state B, and state C. So what it says is it says, right, the possibility of a market being in state A might be 40%, the possibility of being state B will be 20%, the possibility in state C might be the balance up to 100%. So that’s the first thing to know. The second thing is, how is that state going to transition? Does it plan to stay in its state? So we say let’s take a defined period into the future.

What is the probability that that state stays where it is? What is the probability that state A transfer relates to state B? What is the probability of it going from B to A, A to C, C to A, or just staying in this existing state? That creates what we call a transition matrix of probabilities. So we’ve got two pieces of information there: how frequently that state is there, and, secondly, what the transition probability is. So that’s basically how a Markov process works.

Now, a hidden Markov process is something that us trend followers use without really realizing it. So a hidden Markov process says, let’s say we don’t know the underlying state that a Markov model produces. What is another way we can define those market states? We can use trend following models as a bit like thermometers. Now, what I mean here is that when we are assessing the temperature of an environment, we’ll use a thermometer. That gives a gauge of how it correlates with that external environment. So it’ll correlate with the environment and tell us the temperature of that environment.

Our trading models are very much like thermometers in that if I’ve got a particular trend following model that’s long-only, or a trend following model that’s short-only, or a mean reverting model, they will correlate to the market with a degree of certainty and give us a signal, a strength in that signal. So if we’re using these different models, like thermometers, we can measure the market state in relation to the returns generated by those models.

So what happens when we’re using a hidden Markov process as opposed to a Markov process? With a Markov process, we’re analyzing price series. But now what we’re saying is, now we’re applying these models and we’re going to use the returns generated for those models because they’re correlated to that price series. The returns generated will be able to map out what those market states are based on the performance of those trend following models. That’s what you call a hidden Markov process.

So you not only have the market states and the transition probabilities, but then you also have effectively a regime filter to say, in this particular market state, how is this model going to perform? So with a Markov process, it’s a very clever process in that it not only develops an ability to be able to assign a weight of how much to allocate towards that model for a particular market state, it also gives you the ability to predict in the short term what the next market state is going to be and how to dynamically adjust your models to that next market state.

So, that’s a Markov process that they talk about. Hopefully, I’ve explained. I probably haven’t done justice to it, but that’s the general gist of the Markov process. So with a Markov process, when we’re using a strict Markov model, it just tells us the probability to transition to the next stage or back again or to what state. It’s a jumpy process. It jumps from one state to another at a particular time, or it stays in that state.

That’s probably not the best way to assess our trends. Because what we want to recognize, we need to recognize that the strength of a trend changes over the course of time. These persistent features called trends which are attributed to autocorrelation and series, they wax and wane over the course of time. The longer you are in a trend, the greater the chance that trend is going to end effectively, as ultimately, these signals die out over the long term.

So instead of using a Markov model or a hidden Markov model, they use what they call a semi-Markov model. What a semi-Markov model does is it not only assesses the probabilities of the existing state and the transition probabilities, but also how those transition probabilities change over the course of time as that trend endures.

So then we’ve got a more definitive mathematical process that basically can be used to assess not only the allocation to our trend following models but when to switch models, when to switch models from here, to here, to here, to here, and the strength of trend in a probability assessment.

So what these two researchers found was that there was a correlation between these Markov processes and what you and I know is autocorrelation in a price series. In fact, they found that using autoregression, a Markov process uses a probability landscape to assess how it transitions to the next probabilities. But an autoregression analysis looks at a linear projection of basically a regression based on history and how the strength of the progression changes in a linear way over the course of time.

It’s not looking at a landscape of probabilities. It’s looking at a regression that extends over a period of time. So the way we understand trends, especially our time series or our absolute momentum trends is that we get these periods of time where price series are correlated. We’re talking about serial correlation here. What we’re talking about is, the impact of a price series in a prior time influences the price at a future time.

There’s a serial correlation there. But it’s not necessarily the price at the prior day that influences the current day. It might be the price last week that influences the price now, or it might be the price last month that influences price now. So there are all of these different lags in these autoregressive models that need to be considered. When you bundle up all of the lagging effects of these autocorrelations, lo and behold, you’ve got a probability landscape very much like a Markov model.

So what they talk about is a symmetry between autoregression and Markov models. They’re the same mathematical processes, looking at the same problem from different angles. Your autoregressive problem is looking at a time series over the course of its history and how correlation waxes and wanes over the course of time. But a Markov model looks at a landscape of probabilities at any point in time. They’re mathematically equivalent.

So what that’s allowed them to do in their paper is it’s allowed them to therefore look at the simple trend following models, like the simple moving average model, the EMA model, that MACD model, and the momentum model to work it out. Each one of those models has a variety of different variables in it.

The way to understand this is when we look at a state of a market at any particular point in time, the state of a market will be defined by many different observables. The state of a market might be defined by the level of volatility in the market, the price direction of that market, up or down. There’ll be numerous factors that actually are defining the state of the market. Those variables, what we call observables, they are tied to the models that are being used to attack that trending price series.

So these models that are being applied to these states of the market, they can use an optimization process to then define what are the optimal settings to use for the parameters used in those models that map to that mathematical equation that’s produced. What they’ve found through this entire process is that they for instance found that just using a Markov model, the EMA was the best model to use in trending environments, in what they call a two-state switching regime method.

However, when looking at the impact of the changing nature of trends over time where they used the semi-Markov model, which is a much more realistic practical implementation of it, they found that actually the MACD model was the better model. When they looked at the optimal parameters for the MACD model, lo and behold, it fell into the same optimal parameters that you and I and a lot of other trend followers might come up with in their extensive backtesting process.

But now they’ve got a mathematical formula that can be applied to define that optimal setting or optimal configuration for that formula using this process. So the advantage of it is that we now have mathematical rigor to the validity of our trend following process.

Previously, before this study, there probably wasn’t much mathematical rigor. You couldn’t really put a mathematical formula to our process to define it. We could explain it in narrative form about behavioral biases and these sorts of things that create trends. But now there’s a paper suggesting that we can actually get a mathematical formula using this process of mathematical models with machine learning to develop the optimal parameter sets and how those parameter sets change over time.

So it’s not only a model to define what is optimal based on history, but it’s also a model to define, in the very short term, how I should dynamically adjust those models or what’s coming up quickly. As you and I know, Niels, in our models, we’re often late to enter a trend and early to leave a trend because it takes time to identify a material trend. And then our models enter that trend, and we take a segment of that trending opportunity not right from the bottom and right from the top. We wait for it to occur.

But with this Markov process, it gives you the chance, using these models in knowing what the probability of the next transition state is, it allows us to almost preempt what the market is going to do with a bit of predictive forecasting in the short term to therefore dynamically adjust our models, or at least prepare our portfolios with models that are most likely, from a statistical assessment, are likely to be the next step in that market regime effectively.

So it’s a very powerful process. I hope I’ve done justice to at least give an overview of what the paper is about and why it’s worth listening to Dr. Tom Starkes’ review of that paper, because it’s incredibly interesting. What grabbed my attention was that, for the first time, I think there’s a real bit of mathematical rigor coming into trend following that’s really substantiating our case and I think we should all be jumping on it.

One of the things that clearly comes out of this is that its predictive certainty about what’s the next market regime is a very short-term finding. In other words, the more they try and extend their predictions out into the future, the more uncertain it becomes. They find that there’s a very finite range over which these Markov models work. It’s a bit like trying to assess the weather. They found a similar thing with weather being these complex adaptive systems.

They found that their forecasting potential was limited I think to about 14 days with a degree of predictive certainty. After that period, it really decayed quickly. There were so many contingent factors all influencing these states of the weather. In a similar way, with the financial markets, we do find that with these machine learning methods and with these AI methods, etc, they’re certainly becoming much more adept at being able to provide a degree of predictive certainty into the very short-term future, maybe the next one day to one week, but they’re not very good at predicting beyond that.

Now, there’s two things I see here Niels that we probably need to discuss. Firstly, I think they probably give us a bit of mathematical substance to justify the approach we do, but they’re not the be-all and end-all. They’re not the savior of our process, but they’re certainly helpful additions, because it does tell us perhaps how to possibly consider ways to dynamically adjust our portfolios as markets evolve over the course of time.

But what it does say to me is that it tells me, if I want to be successful in trading and investing, and I have a hope of trying to out-compete AI, keep away from that short-term space, because that short-term space, with AI advancements in these mathematical models, etc, I certainly think is going to become a much more intensely competitive market space.

The good thing about this is, for those people that capture the medium to long-term trends, I think it’s going to be a very long time before AI starts out-competing us with our processes. But I think these things are very helpful. What do you what do you think so far?

[00:41:16] Niels

I think it’s super interesting. Of course, anything that helps with evidencing what we’ve been doing for a long time is incredibly helpful. Obviously, I’ll leave it to people like yourself and our own research team to find out how we could use this. Of course, on our side, we have been using dynamic parameter selection for quite a while. So I’m not entirely sure whether we discovered this a while ago or whether we’re using a different methodology, but it is very interesting.

Now, as you rightly alluded to at the end, it almost sounds like the holy grail, right? Everyone who uses these could then end up with the optimal parameters, which of course we know is not going to be the case. So that’s interesting. But I wonder whether again (and I’m completely out of my depth here), I wonder, again, once we get more of this power technology methods, you could even extend it and say, well, if we know what the optimal parameters might be, maybe we could calculate what would the optimal position size be at any given time, for example.

There are lots of things that you start to wonder about when you get some of these new breakthroughs at your disposal, for sure. But I do agree with you. I think this is the important bit maybe to always go back to, and that is that trend following has been around for a very long time and we’ve seen inventions of technology, even the internet being born whilst we’ve been doing this as trend followers.

It has been predicted every time that this would change trend following, it would make maybe trend following obsolete in some ways, etc. But here we are 50 years into at least some of the firm’s history, and trend following, following the core principles that we always come back to, still seem to be working very well.

So do I believe at this stage that AI will revolutionize what we do? No, I don’t. But will it provide some helpful tools along the way? I’m sure it will. But I think that whoever came up with the foundations of all the things that we call the golden rules, the core rules, I think that’s where the real gold lies still and will continue to be the case.

[00:43:53] Richard

Yeah, I totally agree with that. You know, it’s very interesting. I read that book about Jim Simons. As we know, Jim Simons passed away recently. “The Man Who Solved the Market”, written by Greg Zuckerman. I also spoke to Greg Zuckerman. Through the course of that reading and talking with Greg, it became clearly evident that they were using these advanced mathematical processes at the Medallion fund.

For instance, the book goes into detail about how from the 1960s, Jim Simons and some of the key people that worked at Renaissance, Lenny Baum and a few other people, they were involved in this research project with the Department of Defense in the USA trying to crack Russian code.

So they had to use these mathematical models to basically evaluate their code, to look at the market states of that code and identify what the next letters in that code are or what are the sequence of letters in that code to try and crack that code. That early stage of cracking code in the Russian code basically filtered its way into Medallion.

The Medallion fund, as we all know, operates, particularly in the short-term timeframe, from one day to one week. They’re not high-frequency traders, but they operate over a short-term timeframe. They’ve got so many different data sources that they analyze: alternative data sources, price data, and volume data. we’re not talking about alternative data sources, talking about things like satellite imagery, all of these different things.

What all of that analysis does is it’s using these Markov models to then say, we’ve identified all of these significant opportunities that offer alpha or an edge. Now, let’s develop an overarching model that says how best to allocate all of those based on the market state. This is basically what this research is getting to now. It’s probably something that’s been implemented by Medallion for many decades now. But now with the advent of AI, these secrets that are coming out.

With the advent of AI, for instance, I can understand or take research with Chat GPT to understand what a Markov process is and how to apply it and go through all of these examples. So effectively, I’ve now got a quant research division at my disposal with Chat GPT, and it suddenly gives me this immense ability to expand my education and awareness of financial markets using this.

A decade ago, Niels, I never had this opportunity, and I didn’t understand a lot of these things. But now, this is a marvelous way to teach you, empower you in some of these very advanced scientific and mathematical processes.

[00:46:58] Niels

I agree with that. Although, I guess I’m a little bit skeptical as well, because I did read that book a while back. Of course, it sounds fantastic and interesting and intriguing and secretive when you hear all of this. However, I also have to question, why is it that they could make so much money in Medallion but they couldn’t make any money for their external clients?

I have to be skeptical about that, because clearly, they could use the same type of technology of some sort and at least keep up with benchmark returns. But as far as I understand, they didn’t.

[00:47:44] Richard

I’ve asked that same question to Greg and we had a bit of a chat about that. The prime motive of Jim Simons and the Renaissance fund was never for the external client. It was always for their employees and the researchers that work for them. So people like Lenny Baum, who was responsible for what they call a Baum-Welsh algorithm, which is necessary for these Markov processes we’ve talked about, getting together these incredible intellectuals all through the history of Jim Simon’s life into it, a team that was working together to basically crack the market.

I think that certainly, they could operate within up to I think 10 billion, but they knew that they had to keep the fund at that particular certain level. And then they thought themselves, all right, we’re doing so successfully in Medallion internally with our employees, this is where Jim Simons said, now let’s start bringing in external clients. But how do we do it?

He said, he knew he couldn’t add any more AUM to his existing models because the market impact of those models would get too significant, and they needed to operate with lower market impact. So they decided to start the I think it was called their long equity, or they started investing in stocks using medium to long-term effectively trend following models.

The performance of that Renaissance equity fund was marginal at best. It was no better than any of the funds that you and I are aware of in the trend following space. In fact, it pretty well underperformed for a long period of time. But that was scalable. They could certainly go up to 100 billion or 200 billion with that fund, so they didn’t have this market impact limitation. This is one of the key things I’d like to bring up here.

When you’re dealing in the medium to long-term space, for these mathematical processes to work, I’m a tiny bit doubtful of it because there is so much uncertainty in these markets. This is why I think that, in the short-term space, I’ve got no doubt that AI and these advanced models is going to help in making it much more competitive. But in the medium to long term space, I honestly think that if there’s anyone out here who’s got a competitive advantage, Niels, it’s us the CTAs that have been doing this for their lives.

I don’t think we’re going to go away that quickly even with the advancements of AI. I think we’re here to stay. That’s something that I’m very excited about. Because I think to myself, with the rapidity of change that AI is bringing into the workplace, do we have jobs in the near future?

I’m quietly confident we do as trend followers, but I’m not so certain about the mean reverters, I’m not so sure about the high-frequency traders, I’m not so sure about those people that are dabbling in the short-term space, because I think that AI is going to quickly get its talons into that short term predictive space.

[00:50:43] Niels

I agree with that. I was not aware. Of course, I don’t think any of us can completely substantiate whether they were indeed using some kind of trend following for their equity products. If they were, all I would say is that is incredibly naive, because all the data shows that if there’s one thing you shouldn’t do, is to do trend following on a single sector or single market. It doesn’t work. It didn’t work either for them, I guess.

So maybe they did use trend following, but I’m not going to make that claim. But I think it’s interesting. It’s obviously a space and it’s obviously a technology that we’re going to be talking about a lot. But I think I would agree with you about the whole idea of having access to ways of learning and explaining and all of that. That’s pretty cool.

Now, since I knew you were a little bit of a Chat GPT fan, I did before we went online, because I knew we were going to talk about the optimal way of doing trend following, so I did ask Chat GPT the following. “What is the optimal way of doing trend following (a systematic and rules-based investment method carried out by commodity trading advisors). What would you say?”

Chat GPT very promptly defined seven things that it lists. I’m not going to read all of them, but here are the bullet points that it came back with. “Define clear rules and parameters, risk management,” and then it talks about position sizing, blah, blah, blah. “Backtesting and optimization, adaptation to market regimes, execution and cost, continuous monitoring and evaluation, and psychological discipline.”

So, clearly, it does understand what we’ve been talking about for the past many, many years, which is exciting. Anything else you want to leave us with? This was a big mouthful. I’m sure people will need a bit of time, maybe listen to this episode a couple of times to get all the details. It’s technical stuff, it doesn’t make it less important. So yeah, people should take their time to fully digest and appreciate this kind of stuff. Anything else, Rich, you’re thinking of?

[00:51:36] Richard

No, Niels. I could go into how you use your models like these thermometers. But look, I might leave that for another time. I think probably people have had a gut full of all of the mathematical associations.

[00:53:22] Niels

I think that’s fair. When you were talking about thermometers a lot in the beginning, I was always in my mind thinking, “Trend barometer, trend barometer.” But I don’t think of my trend barometer as a Markov model, I have to say. But you never know. There could be something that I’ve missed.

[00:53:41] Richard

The good thing with this I’ve found is that I was always daunted by maths. When people would talk about these things like Markov processes, I’d fear them. But now, when you start getting into it with the assistance of things that can train you and educate you in that area, seriously, it does unpack it a lot. A lot of that resistance I had to it, I’m now curious. It’s changed my mind to these things. So I think it’s very helpful.

[00:54:12] Niels

I think it is as well. You know what else is helpful is the new weekly “Week in Trend” updates you and I post every week, not using Chat GPT, except maybe for creating an image. But it is written by real people. Maybe that should be what we need to do with it.

[00:54:29] Richard

Written by real people.

[00:54:36] Niels

Written by real people with lots of scars from being trend followers for a while. Anyway, people should go and check them out. I try to post them every Saturday, midday, my time. Obviously, we need to work through them and all that. So anyway, hopefully that’s another useful addition to the line of information that we provide.

Also, I think there’s a monthly trend following update coming out soon where people can check out what the industry managers’ indices are up to. We’re working on that. So lots of good stuff coming out.

Rich, thank you ever so much. As usual, I always learn something new from you every time we speak, so I’m sure lots of people do.

That’s it for today. If enjoy these conversations, the best way to show that you appreciate what Rich is doing, you should go to the podcast on your favorite podcast player, leave a rating and review. It really does help. Spread the word to friends, colleagues, family. Hopefully, we can expand the learnings and lessons about trend following. There we are.

Next week, I’ll be joined by another trend following expert, Rob Carver. So that will be another real person talking about trend. If you want him to answer some of your questions, by all means, send me an email info@toptradersunplugged.com, and I will do my best to bring it to Rob’s attention.

That’s it from Rich and me. Thanks so much for listening. We look forward to being back with you next week. Until such time, take care of yourself and take care of each other.

[00:56:24] Ending

Thanks for listening to the Systematic Investor podcast series. If you enjoyed this series, go on over to iTunes and leave an honest rating and review and be sure to listen to all the other episodes from Top Traders Unplugged. If you have questions about systematic investing, send us an email with the word “question” in the subject line to info@toptradersunplugged.com, and we’ll try to get it on the show.

Remember, all the discussion that we have about investment performance is about the past and past performance does not guarantee or even infer anything about future performance. Also understand that there’s a significant risk of financial loss with all investment strategies and you need to request and understand the specific risks from the investment manager about their products before you make investment decisions.

Thanks for spending some of your valuable time with us and we’ll see you on the next episode of the Systematic Investor.