The new reality of high inflation has changed many things for fund managers. With the rise of disruptive technologies there has come an acceptance from many market participants of all things AI. RL or reinforcement learning is one of those disruptive technologies. We spoke to Aitor Muguruza Gonzalez about RL in AI-driven Alpha Strategies to find out more.
Aitor is Head of Scientific Research at Kaiju Capital Management Ltd. He is also a Visiting Lecturer at Imperial College where he finished a PhD in Mathematical Finance. In his role he both works with business as well as with academia. He has a unique view of the unique role of academia in supporting the development of investment strategies.
During his career, Aitor has been commended for his academic spirit:
In his current role at Kaiju, Aitor focuses on U.S. equities. Stocks as well as options.
Aitor explained how in his role he works predominantly with an in-house platform to generate trading strategies. This platform has access to sophisticated markets data from exchanges like ICE and CBOE markets, for stocks and options respectively.
Unlike leading fund managers like Ray Dalio, Ken Moelis, Warren Buffet or Cathie Wood, Kaiju is different. The alpha strategies are predominantly created using the help of AI.
The rise of in-house platforms
Over the last few years Aitor explained how he has noticed a shift whereby firms are now developing platforms in-house. This has especially been driven by the amount of data that AI or machine learning algorithms require.
Most of the solutions available on the market today are not yet able to manage the throughput needed. “We go directly to the source of the data and then pipe it real time to our infrastructure. And then distribute the data the way we want and the way we need,” Aitor shared.
Aitor used the analogy of a self-driving car. “Why would you want a camera with a lag right on your steering wheel when you can have the latest picture of where you are driving?” he asked. Why make a bottleneck out of the lag?
“Fortunately AI technology has been democratized,” Aitor added. This allows small or medium sized hedge funds on the market, who don’t necessarily have the big resources of firms like JP Morgan, to stay competitive. Through the use of data and AI.
How good is the data available?
Aitor shared how a few years ago he spoke at Quant Strats on the topic of market regime detection. He used this example to show how he works with data.
Over the last 15 years the markets have experienced some of the biggest turbulence in history. Aitor believes that this period is the worst ever period to back test. Just consider how the S&P 500 has performed since December 2022 with the backdrop of spiking interest rates.
“I think these are very interesting challenges,” Aitor explained. “Especially if you have to build AI models or machine learning techniques. Hedge funds have to watch so that they don’t bias their algorithms towards what has happened in the past 10 years because that’s not a real reflection of how our economy functions.”
In-house Platforms become Intellectual Property
One of the firms on Wall Street who originally pioneered a highly automated platform was D.E. Shaw. This was in 2013 and the company launched to offer this platform to the market was Arcensium.
Today other firms have emulated the work pioneered at D.E. Shaw. In the case of Kaiju, Aitor explained how the platform was treated as intellectual property (IP). How it solved business problems for the markets.
At Kaiju, the platform is called ‘Leviathan’, and is described as “the inevitable evolution of holistic asset management systems.”
Using the example of Google’s algorithms and how they are able to play video games, Aitor explained how the IP of these algorithms is very powerful. “So powerful that it can be extrapolated to self-driving cars or trading in the markets,” he explained.
“On the one hand investors see what hedge funds do with money. But on the other hand there’s IP which you can patent and more. And I see more and more of these trends. Just like at Kaiju where Kaiju Worldwide is the parent company of Kaiju IP,” Aitor shared, pointing out how Kaiju Worldwide and Kaiju Capital Management Ltd were separate entities.
The Popularity of AI driven strategies
Talk went on to market appetite for AI driven strategies. Aitor is of the belief that people are willing to invest in this type of product. Whether the SEC is able to adapt and learn how to judge these strategies is another question.
Instead Aitor raised the traditional portfolio management analogy. In the analogy people don’t question when Cathie Wood decides that she wants to buy Meta. The reaction is often that its going to be a great move for Ark. Some people might question it, but most will agree with her decision. People can imagine Cathie sitting on a board and making a decision to buy Meta and discussing the pros and cons.
But then you translate that into a Tesla and Elon Musk making a decision about whether the car detects a car or if it can detect a truck. For instance it says the car in front is a truck when it isn’t. It’s wrong. But you wouldn’t imagine Elon Musk going and change the source code if he notices this error, would you? Aitor asked. Highlighting the difference.
You would imagine a lot more people involved in the change of code at Tesla, wouldn’t you? They would probably follow a much more scientific path. They would look to how to solve the problem.
Questions that should be used to test the AI driven strategy by regulators like the SEC should be more scientific too, Aitor believes.
“Show me what you would do if this happened?
“What would you do if that happened?
“How would your strategy perform under this situation?”
These are questions that would help with regulation of not only AI-driven strategies, but also with traditional strategies, Aitor believes. He also highlighted how AI-driven funds are built on understanding the answers to these types of questions. Especially in his role at Kaiju. Through due diligence and constant checking of positions.
Reinforcement learning, the future of AI driven strategies
Reinforcement learning or RL has been around since the 1960s. But it was only a few years ago that the team at Kaiju started to look at the machine learning training method in more detail. Aitor explained why as well as sharing more of his insights into actor-critic methods whilst using RL
“Reinforcement learning is data hungry,” Aitor shared. “How the Atari games paper helped with this was by asking how can I make this work with the least data I can. And that’s when they published the rainbow paper.”
“When we realized less data was needed,” Aitor added. In the case of the Rainbow paper this reduced the need for data by approximately 100 times. “This is when it became interesting for us to start exploring reinforcement learning.”
This was of particular interest when it came to options. “Because options have interesting profiles. You can hold it. Because of the way you pay for the option and how the premium works. How it is uncorrelated from the market. It’s attractive from an investors’ point of view,” Aitor explained.
“Options felt like a very natural framework for reinforcement learning. It feels like a game where you can buy, hold, exit. There’s a stop-loss. There’s profit taking. There are limits,” Aitor added. “But the biggest benefit of options, or futures as well, is that there’s an expiration date. It all makes it a very good reinforcement learning problem because it is fixed, you have a time horizon.”
Thanks to RL today, Aitor believes that his team is solving the core problem.
How Mathematics can Improve reinforcement learning
Sutton & Barto published a famous book on reinforcement learning in 1998. Aitor is very complimentary about the book. However, Aitor believes the book is very focused on computer science. The book lists lots of examples that differ marginally from each other. What Aitor thinks is missing from all actor-critic reinforcement learning algorithms is a bit of mathematics.
“I think more and more mathematics are being used to understand how it works,” Aitor explained. “It’s like with Chat GPT. They created it and it works. They don’t really know why. So they make it bigger and it keeps working. But eventually it will stop growing at the pace it does. At this point you go back to mathematics. Then the science comes and that is when the opportunities will really start to appear.”
In order to facilitate this approach to trading strategies at Kaiju, Aitor explained how the firm encourages PhD students and post-graduates to become interns at the firm. He also shared more about the rest of the team. The larger part of which is focused on data science. Quants are still needed but at Kaiju they focus on simple and reliable models, not on very complex ones.
Aitor added how “getting the RL concept more accepted could be the biggest revolution in investing. It could be as important as the mean variance portfolio theory by Markowitz.”
Join the Conversation at Quant Strats
There are two lecturers amongst the speakers at Quant Strats, not including Aitor. Saeed Amen is Visiting Lecturer at Queen Mary University, London. Whilst Margaret Holen is Lecturer at Princeton University, she is also an Advisor at Thinknum. Aitor agrees that the role of academia at events is key. Just like the presence as speakers of academia at Quant Strats.
“Many of the business ideas we see in the world today come from a research paper written by academia,” he added.
“The panel sessions at Quant Strats focus on actual problems that people are facing. I found people to be quite open and keen to answer questions,” Aitor shared. “I feel like it’s a good compromise between what we can share with our peers and what you can. There is always a healthy discussion and the solutions being presented to the problems I find myself in are usually spot on.”
You can see Aitor presenting at Quant Strats at 12.10pm on October 24th at Park Plaza Victoria London. Click here for ticket details.
Author: Andy Samu
