Unlock the Editor’s Digest for free
Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.
The writer is a former global head of research at Morgan Stanley and former group head of research, data and analytics at UBS
The late Byron Wien, a prominent markets strategist of the 1990s, defined the best research as a non-consensus recommendation that turned out to be right. Could AI pass Wien’s test of worthwhile research and make the analyst job redundant? Or at the very least increase the probability of a recommendation to be right more than 50 per cent of the time?
Well, it is important to understand that most analyst reports are devoted to the interpretation of financial statements and news. This is about facilitating the job of investors. Here, modern large language models simplify or displace this analyst function.
Next, a good amount of effort is spent predicting earnings. Given that most of the time profits tend to follow a pattern, as good years follow good years and vice versa, it is logical that a rules-based engine would work. And because the models do not need to “be heard” by standing out from the crowd with outlandish projections, their lower bias and noise can outperform most analysts’ estimates in periods where there is limited uncertainty. Academics wrote about this decades ago, but the practice did not take off in mainstream research. To scale, it required a good dose of statistics or building a neural network. Rarely in the skillset of an analyst.
Change is under way. Academics from University of Chicago trained large language models to estimate variance of earnings. These outperformed median estimates when compared with those of analysts. The results are fascinating because LLMs generate insights by understanding the narrative of the earnings release, as they do not have what we may call numerical reasoning — the edge of a narrowly trained algorithm. And their forecasts improve when instructed to mirror the steps that a senior analyst does. Like a good junior, if you wish.
But analysts struggle to quantify risk. Part of this issue is because investors are so fixated with getting sure wins that they push analysts to express certainty when there is none. The shortcut is to flex the estimates or multiples a bit up or down. At best, taking a series of similar situations in to consideration, LLMs can help.
Playing with the “temperature” of the model, which is a proxy for the randomness of the results, we can make a statistical approximation of bands of risk and return. Additionally, we can demand the model gives us an estimate of the confidence it has in its projections. Perhaps counter-intuitively, this is the wrong question to ask most humans. We tend to be overconfident in our ability to forecast the future. And when our projections start to err, it is not unusual to escalate our commitment. In practical terms, when a firm produces a “conviction call list” it may be better to think twice before blindly following the advice.
But before we throw the proverbial analyst out with the bathwater, we must acknowledge significant limitations to AI. As models try to give the most plausible answer, we should not expect they will discover the next Nvidia — or foresee another global financial crisis. These stocks or events buck any trend. Neither can LLMs suggest something “worth looking into” on the earnings call as the management seems to avoid discussing value-relevant information. Nor can they anticipate the gyrations of the dollar, say, because of political wrangles. The market is non-stationary and opinions on it are changing all the time. We need intuition and the flexibility to incorporate new information in our views. These are qualities of a top analyst.
Could AI increase our intuition? Perhaps. Adventurous researchers can use the much-maligned hallucinations of LLMs in their favour by dialling up the randomness of the model’s responses. This will spill out a lot of ideas to check. Or build geopolitical “what if” scenarios drawing more alternative lessons from history than an army of experts could provide.
Early studies suggest potential in both approaches. This is a good thing, as anyone who has been in an investment committee appreciates how difficult it is to bring alternative perspectives to the table. Beware, though: we are unlikely to see a “spark of genius” and there will be a lot of nonsense to weed out.
Does it make sense to have a proper research department or to follow a star analyst? It does. But we must assume that a few of the processes can be automated, that some could be enhanced, and that strategic intuition is like a needle in a haystack. It is hard to find non-consensus recommendations that turn out to be right. And there is some serendipity in the search.
