That creates immediate design questions for IT teams. The model size is only one constraint; architects still need to validate memory headroom, sustained token throughput under concurrent use, platform-specific runtime support, and how tool-calling behaves when embedded into endpoint or departmental applications. Apache 2.0 availability lowers licensing friction, but production use still depends on packaging, update cadence, observability and policy controls around on-device prompts, outputs and model versioning.
The more interesting trade-off is operational, not just technical. Ternary compression may preserve much of benchmark performance, but teams should treat this as a workload-selection tool rather than a universal replacement for larger models. The strongest near-term fit is likely bounded tasks such as summarization, classification, code assistance, translation and private knowledge retrieval at the edge. More complex agentic workflows will still need fallback paths, routing logic and evaluation thresholds to decide when local inference is sufficient and when cloud escalation is worth the cost.
In other words, compact models like this are most valuable when paired with an explicit hybrid AI control plane, not when treated as a drop-in substitute for frontier-scale systems.
Prism ML Inc. announced Thursday the launch of Bonsai 2 27B, the second generation of its ultra-compact multimodal generative artificial intelligence small enough to fit on PCs and some high-end mobile devices.
The company said it used ternary, which uses three parts, to scale down its Qwen3.8 27B-based model. Qwen3.8 weighs around 56 gigabytes at its full 16-bit uncompressed size, and Bonsai 2 reduces it to around 5.9 gigabytes while retaining around 98.2% of its capabilities.
Although it is possible to shrink AI models using other compression techniques called quantization, these methods usually strip away accuracy, knowledge and other systematic capabilities. Qwen3.8’s minimal memory footprint is around 9.4 gigabytes.
Ternary provides an interesting compression method when shrinking the “weights,” or parameters that make up the model. Weights are the model’s numerical dials that control how it processes information and generates outputs. In full-size models, these are represented by 16 bits; with PrismML’s approach, these are simplified down to ternary, or three bits, represented by +1, 0, and -1. This lets the company store information in a much smaller memory footprint while still holding onto reasonably high intelligence.
In essence, this allows Bonsai 2 to punch well above its weight class at a very small size.
On benchmarks, Bonsai 2 showed close performance on agentic and tool calling compared to Qwen3.8 within 3 points, at 77.6 and 79.8 respectively; with aggregate scores of 81.6 and 82.2 for coding across HumanEval+, LiveCodeBench v6, MBPP+ and BigCodeBench; and 82.7 and 81.3 for knowledge and reasoning across MMLU-Redux, GPQA Diamond and AA-LCR.
The model can run on an Nvidia GeForce GTX 5090 card without quantization, reaching 143 tokens per second and 46.8 tokens per second on Apple Inc.’s M5 Max chip. The company said the model consumes extremely low power per token at 0.714 megawatt-hours, making it 40% more energy-efficient than other 8B models running at full precision, meaning uncompressed.
Ultra-small models let users run AI locally on their own machines without sending inference to the cloud. Any time data is sent across the internet, there can be a delay in receiving a response, or sensitive information might be sent to a third party. Bringing intelligence onto a local machine eliminates third-party data sharing, keeps prompts and responses local, helps meet strict privacy regulations, and can improve security.
For example, simple translation, summarization, and search organization could run on device, while long-horizon task comprehension and research might need to be sent to an expensive cloud model. For an everyday user, or even an enterprise use case, running a local model that is far less expensive and respects privacy when a task is simple and involves sensitive information, while scaling up to highly intelligent, cloud-based models to handle complex, high-touch, goal-oriented work.
The new model runs on Nvidia graphics processing units via CUDA and on Apple devices, including Mac, iPhone and iPad, via MLX, through low-bit kernels. Model weights are available today under Apache 2.0 licenses.
Original Postrismml-launches-bonsai-2-27b-a-high-intelligence-ai-model-so-small-it-fits-on-consumer-hardware/">PrismML launches Bonsai 2 27B, a high-intelligence AI model so small it fits on consumer hardware
Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.