The Morningstar Investment Conference at which Mo was introduced.
The MIT AI and robotics expert Rodney Brooks once coauthored a well-known paper—also memorialized in an Errol Morris documentary—about the virtues of being fast, cheap, and out of control. Brooks believed that no complex technology could be simultaneously fast, cheap, and in control. But Morningstar, the independent investment research company, seems to have achieved all three goals with respect to its development of Mo, an investment research assistant based on generative AI. Mo is intended to provide Morningstar’s investment knowledge to both investment advisors and its “informed investor” individual clients in the U.S. (you can sign up for a free trial that includes Mo usage here.)
Some of the other companies and organizations that have launched specific content domain versions of generative models have been experimenting and collaborating with vendors for a year or more. But Morningstar was able to introduce its Mo system after only a month or so of development. Mo has now been up and running since late April of this year, and it’s “in production.” James Rhodes, Morningstar’s CTO, said that technically it would always be in beta test mode because it is always evolving. He attributes the speed of implementation to his team of engineers and data scientists, although he add that it’s hard to find people who already understand generative AI at a detailed level.
It is generally believed to be quite expensive to train a generative AI model from scratch—in the millions of dollars for a single training run. But it apparently doesn’t have to be terribly expensive to modify one to support an individual company’s content, at least in the way that Morningstar went about it.
Rhodes said that thus far Mo has answered over 25,000 questions and the average cost per question answered is an astounding $0.002—two tenths of a cent. The total expense devoted to Mo thus far—not counting the compensation of its creators—is $3000. Now that’s cheap. Rhodes attributes the low cost to the way in which Morningstar tailored the system to its own content—more about that later—and to its use of the multi-tenant GPT model available through Microsoft Azure OpenAI Services.
The speed and low cost with which Mo was implemented doesn’t mean that Morningstar cut corners with regard to privacy, security, and legality. Microsoft agreed, for example, that it would clear GPT’s memory after each chat session so that Mo’s users would not have worry about their questions becoming public. Morningstar’s own data also stays protected in the Azure cloud. Morningstar’s lawyers reviewed every aspect of Mo to ensure that it doesn’t violate any ethical or regulatory issues about investment advice. In fact, before you enter Mo it clearly states that the system does not provide investment advice.
I certainly found that Mo hedged on any specific investment recommendation. For example, when I asked Mo, “Should I buy Nvidia stock?” it said in response:
Morningstar’s most recent rating for Nvidia stock is 2 stars, which means it is considered overvalued and may be an unattractive investment. However, Morningstar also believes that Nvidia’s data center business will prove more resilient to macroeconomic challenges and that the company has enviable exposure to secular trends in artificial intelligence and cloud investments. Ultimately, the decision to buy Nvidia stock should be based on your own investment goals and risk tolerance.
One of the more interesting aspects of Mo’s design is a set of statements about how it should be used that are prepended to every user prompt. I’ll say more about this in the next section, which describes how Mo works.
How Mo Goes
Mo was trained using prompt-tuning of an existing large language model (LLM)—OpenAI’s GPT-3.5. Instead of training a new version of the model with new content, this approach keeps the original model frozen, and it is modified only through prompts in the context window that contain domain-specific knowledge—in this case, Morningstar’s investing content. This approach is relatively economical, as it does not require training of an LLM from scratch. The resulting system retains all the capabilities of language use and reasoning in the original model, but it can also answer domain-specific questions about investing and specific investments.
One technical challenge with prompt-tuning is that the text (or tokens) used as input to an LLM—the Morningstar content in this case—is too large with too many important attributes to enter it all directly in the context window for the LLM. The alternative is to create vector embeddings—arrays of numeric values produced from the text by another pre-trained machine learning model—also from OpenAI in Morningstar’s case.
The vector embeddings are a more compact representation of this data that preserves contextual relationships in the text. When a user enters a prompt question into Mo, a similarity algorithm determines which vectors are most similar to the prompt, and they are loaded into GPT. This is all invisible to the user, of course. Context window token limits are increasing rapidly, but it may be a while before the entire knowledge base of a big content producer like Morningstar can be entered into the context window.
As I mentioned above, another interesting aspect of Mo’s operation is that a set of instructions are prepended to user prompts. They ensure that the GPT system understands the larger context in which the question is being asked. There are roughly twenty such instructions, including “Mo must not answer questions that are not finance-related” and “If it’s not possible to answer questions using Morningstar data, just politely say you can’t provide an answer.” The instructions also include a series of prompts to help Mo understand terms which may be confusing or unique to finance. Another set explains Morningstar’s rating systems and how they differ for stocks vs. bonds.
Without them, GPT might “hallucinate” more often and delve into issues that are irrelevant to Morningstar’s business and interests. As it is, user questions that stay within the limits set by the instruction prompts are almost always answered in a reasonable fashion, although Morningstar tells users that “While we’ve designed Mo to provide answers and content that are relevant, the system may generate incorrect or misleading responses” just in case.
All in all, Mo should make many more companies and organizations comfortable with the idea of embedding their own knowledge into a generative AI model. Many will undoubtedly realize that they can afford the time and cost to prompt-train their own models. And it is easy to see how this would be useful for a wide variety of business purposes.
The only real limitation that some organizations may face is the volume, accuracy, and structure of the content that they have to incorporate into a model. If the relevant knowledge is still in employees’ heads, if it is sprawling and poorly organized, or if there is no definitive source on any given topic, even the smartest generative AI system won’t be able to make sense of it.