Inkling 2 Mini: A Local Multimodal MoE Model with About 26B Parameters (3B Active per Token), Supporting transformers, llama.cpp and MLX
On November 20, Hugging Face introduced Inkling 2 Mini from Thinking Machines Lab. It is a multimodal looped MoE model built to run locally, with support for transformers, llama.cpp, MLX and SGLang.
News date: About 5 min read
Contents
On November 20, 2026, Hugging Face published a post introducing Inkling 2 Mini, developed with Thinking Machines Lab. It describes the model as a mid-sized model designed for local setups. Inkling 2 Mini is a MoE (mixture-of-experts) model. It contains many sub-networks called "experts" and switches on only a few of them for each token (the smallest unit a model splits text into). That is why it has about 26B total parameters but uses only about 3B at a time. The post explains how to use the model with several inference frameworks and does not mention a preview, waitlist or regional restrictions. For developers who want to run models on their own computers, this is a smaller option. However, most of the performance comparisons are Hugging Face's own claims, and the company also says more validation is needed.
How does Inkling 2 Mini differ from Inkling 2 Small?
According to Hugging Face, Inkling 2 Mini is a looped transformer. Among its 24 decoder layers, layers 6 through 17 run three times and reuse the same weights, so it computes like a model with 48 logical layers (6 + 3×12 + 6). In other words, the file only stores the weights of 24 layers, yet the model performs 48 layer visits of computation. Information moves between passes through "cross-pass messages," and a small gating controller decides how much of the current input to forget and how much of the message to add.
The post states that Mini and Small both make 48 layer visits per token, but Mini's hidden size (the width of the vectors the model uses to represent information internally) is 2,304 (Small: 4,608). Mini selects 8 experts from 128 per layer (Small: 8 from 384). Each six-layer block contains five sliding-window layers (which look only at nearby text) and one full-attention layer (which looks at the entire context), with a local window of up to 1,024 tokens.
| Item | Inkling 2 Mini | Inkling 2 Small |
|---|---|---|
| Hidden size | 2,304 | 4,608 |
| Experts per layer | 128 (8 selected each time) | 384 (8 selected each time) |
| Layer visits | 48 per token | 48 per token |
| Positioning | Designed for local setups | The post says Mini is much smaller |
How does it perform, and what are the limitations?
Hugging Face says Inkling 2 Mini is state-of-the-art in its class on the Artificial Analysis Intelligence Index v5, leading on 32 of 48 benchmarks when compared with Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. The post also says it leads on FrontierCode v2.0, ties with Qwen3.8-27B on Terminal-Bench 4.0, and trails Qwen on AutomationBench-AA v1.0.6. See the official model card for full scores.
Which platforms and inference methods are supported?
| Framework | Support described in the post | Notes |
|---|---|---|
| transformers | AutoModelForMultimodalLM; supports fixed-loop and variable-loop versions | Reasoning effort can be set with reasoning_effort |
| Llama App / llama.cpp | GGUF 4-bit (IQ4_NL) and 8-bit (Q8_0) | Llama App runs on macOS and Windows and provides a local OpenAI-compatible API |
| MLX | Upstreamed to mlx-lm, with image and text input | Also works with the original checkpoint in its native nvfp4 + mxfp8 format, with no reconversion |
| SGLang | Day-0 support (available from release day), aimed at data center deployment | Requires SM103; DSPARK speculative decoding can be added (a small draft model proposes tokens and the main model verifies them, for extra speed) |
How do I get started?
GGUF is the model file format used by llama.cpp. 4-bit and 8-bit refer to the quantization level, meaning the weights are compressed to fewer bits: 4-bit uses less memory, while 8-bit keeps more precision. On Linux, or if you prefer the command line, the post gives the following command for the fixed-loop model. To load the 8-bit weights, change the suffix to :Q8_0.
llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NLIn transformers, the post uses the model ID thinkingmachines/Inkling-2-Mini with AutoProcessor and AutoModelForMultimodalLM. Reasoning effort can be set to none, minimal, low, medium, high (the default, 0.9), xhigh and max, or you can pass a numeric value from 0.0 to 0.99.
Frequently asked questions
Can Inkling 2 Mini handle images?
Yes. Hugging Face says it accepts image and text input, making it a multimodal model, and supports a 1M-token context. Context refers to how much content the model can read at once.
What is the variable-loop version?
It is a research version with the same architecture (TML-d24-varloop-research), trained to accept different loop counts. Setting it to 1 or 2 loops makes inference run a lot faster, but output quality may be slightly lower. Hugging Face says it has not been evaluated in depth.
Does the post give pricing, licensing or regional availability details?
No. The post does not mention pricing, licensing details or availability in specific regions. Check the official model page for these.
Can these benchmark results be treated as independent verification?
No. The information comes only from a single Hugging Face post, which reflects the company's own claims, and the post says more validation is needed.
Found this guide helpful? Share it with someone who needs it
Claude Code 2.1.330: Enterprise Admin Console Sign-In, easel Chart Module, and Scheduled Tasks Extended to 30 Days
Anthropic's Claude Code 2.1.330 changelog lists Enterprise Admin Console sign-in, a built-in easel chart module, scheduled tasks of up to 30 days, and stricter approval for rm commands.
AWS Quick adds chat-built flows, multi-agent answers and Meta ads/Amplitude connectors, available today in all Quick regions
AWS Distributed KV Caching Agent Skill for Self-Hosted LLMs: AWS Says Cost Drops Up to 40% and Time to First Token Up to 80%
Amazon SageMaker HyperPod Slurm Adds Cluster Extensions: Managed Observability and User Management, Available in All HyperPod Regions
Sources
Compiled from the references and verification dates listed above; fares, schedules, product features, and terms can change, so please confirm the latest details on the relevant official website before travelling, booking, or purchasing.