On Microsoft's most recent earnings call, CEO Satya Nadella offered an upbeat assessment of the company’s progress with generative AI. The strategic partnership with OpenAI has proven a highly successful venture, potentially providing Microsoft with a significant advantage over rivals like Google and Amazon.
But Nadella's message went beyond just large language models (LLMs). He also emphasized the importance of small language models (SLMs) in Microsoft’s growth strategy. He pointed out adoption from companies like AT&T, EY and Thomson Reuters.
SLMs generally are five to 10 times smaller than LLMs. However, the reduced size is not necessarily a disadvantage. SLMs still possess considerable capabilities and, in certain cases, can perform on par with their larger LLM counterparts.
The category of SLMs is relatively nascent and subject to rapid innovation. Consequently, most businesses are currently experimenting with these models in pilot phases. But this technology carries substantial potential.
The Problems With LLMs
While LLMs are a new technology, they have already become a major force in the enterprise sector. They excel in processing, summarizing and analyzing large volumes of data and offer valuable insights for decision-making. Then, there are the advanced capabilities for creating compelling content and translating foreign languages.
Yet LLMs have major disadvantages for the enterprise. One is the accuracy and quality of model outputs. This includes not only the bias within the models but also addressing the issue of “hallucinations.” These are instances where the model generates plausible but factually incorrect or nonsensical information.
Next, LLMs can be too generalized. The reason is that the training data is mostly from the public internet. The lack of customization can lead to a gap in how effectively these models understand and respond to industry-specific jargon, processes and data nuances.
Then, there are the concerns with security and privacy. When an enterprise uses an LLM, it will transmit data via an API, and this poses the risk of sensitive information being exposed.
But with SLMs, it's possible to help mitigate these problems.
SLMs
Often, LLMs offer capabilities that are extraneous to enterprise needs. After all, an energy company does not need detailed information about the Middle Ages, classic novels or anthropology.
An SLM offers a more concentrated training set. The model is tailored to the unique enterprise-specific datasets. These can range from product descriptions and customer feedback to internal communications like Slack messages. The narrower focus of an SLM, as opposed to the vast knowledge base of an LLM, significantly reduces the chances of inaccuracies and hallucinations.
With a smaller model, creating, deploying and managing is more cost-effective. This is important given the heavy expenses for infrastructure like GPUs (graphics processing units). In fact, an SLM can be run on inexpensive commodity hardware—say, a CPU—or it can be hosted on a cloud platform.
The advantages of SLMs go beyond cost-effectiveness. They are more adaptable, allowing for easier adjustments based on user feedback. These models also show lower latency. This is a crucial feature for applications where responsiveness is key, such as in chatbot interactions. This blend of adaptability and speed enhances the overall efficiency and user experience.
Regarding security, a significant advantage of many SLMs is that they are open source. This allows for deployment within a private data center, offering enhanced control and security measures tailored to an organization's specific needs.
Granted, this does not mean SLMs are not without their drawbacks. Again, the technology is fairly new, and there are still issues and areas that require refinement and improvement.
One of the challenges is evaluating the appropriate SLM. There are many available—which you can find on sites like Hugging Face—and new ones seem to come onto the market every day. While there are metrics to make comparisons, they are far from foolproof and can be misleading.
There are also various ways to customize an SLM, which require specialized expertise in data science. For example, fine-tuning involves adjusting the weights and biases of a model. Next, there is retrieval-augmented generation (RAG). This is an advanced technique that enhances the functionality of the SLM by incorporating external documents, usually from vector databases. This method optimizes the output of LLMs, making them more relevant, accurate and useful in various contexts.
Conclusion
While LLMs are powerful, they often generate responses that are too generalized and may be inaccurate. These models are also susceptible to security and privacy risks.
But SLMs have been able to effectively address these problems. Some of the main benefits include:
• Grounding in a company’s proprietary data, which greatly improves accuracy.
• Lower costs, as SLMs can be run on commodity hardware.
• Easier to customize the model.
• Lower latency, which can improve chatbot performance.
• Deployment in private data centers, which enhances security and privacy.
SLMs are still evolving quickly. But so far, they have shown to be an effective way for enterprises to leverage generative AI.