Norwegian language models compared: Od1n, NorMistral and the international models on Norwegian

Norwegian language models compared: Od1n, NorMistral and the international models

On Norwegian text, purpose-built Norwegian models outperform the large international models. In our internal benchmark, NorMistral-11B (0.541 BPB) and NorMistral-7B (0.627) lead the field, followed closely by Od1n 3B (0.632). Od1n 3B is #3 of seven models – but it beats every international model tested, including Google's Gemma-2-9B (0.657), Meta's Llama-3.1-8B (0.703), Mistral-7B (0.839) and Alibaba's Qwen2.5-3B (0.894), despite several being two to three times its size. The two NorMistral models still score better overall, and they are considerably larger. Lower BPB is better.

The benchmark

BPB (bits-per-byte) measures how efficiently a model predicts raw bytes of text. The lower the number, the better the model models the language.

What the numbers actually say

Two facts sit side by side, and both matter. First, Od1n 3B beats all four international models on Norwegian – including models with 2–3x more parameters. A 3B model built for Norwegian is a more efficient Norwegian reader than an 8B or 9B general-purpose model trained mostly on English. That is the core result, and it is what dedicated language work buys you.

Second – and we state this plainly – both NorMistral models score higher overall than Od1n. NorMistral-11B is roughly 3.7x larger and NorMistral-7B is 2.3x larger than Od1n 3B. At those sizes, a stronger aggregate score is expected. We are not the best Norwegian model overall; on the full corpus we are third, behind two larger Norwegian models from the research community.

Where Od1n beats a model more than twice its size

The aggregate score hides an important sub-result. On Norwegian public administration text, Od1n 3B outperforms NorMistral-7B – a model more than twice its size:

• Government text (GOV): Od1n 0.779 vs NorMistral-7B 0.799
• Municipal administration: Od1n 0.530 vs NorMistral-7B 0.549

To be precise: this advantage is on administrative and government prose, not on the overall score. For organisations whose Norwegian workload is dominated by public-sector, regulatory and municipal language – the register where much Norwegian business actually happens – this is the most relevant slice of the benchmark, and a 3B model leading a 7B one there is a meaningful signal about training focus.

Same size, different result

The cleanest apples-to-apples comparison is against Qwen2.5-3B, which has the same parameter count as Od1n. Here Od1n scores 0.632 versus Qwen's 0.894 – roughly a 29% lower (better) BPB at identical size. This is the difference a Norwegian-first training corpus makes when compute and model size are held constant.

How this was measured

BPB is a byte-invariant, tokenizer-independent measure: because it is computed over raw bytes rather than tokens, it does not reward a model for having a tokenizer tuned to the test language, which makes cross-model comparison fair. Lower is better. All models were evaluated on a fresh, contamination-free Norwegian 2026 corpus – parliamentary proceedings, municipal administration, Nynorsk and code – published after the competitors' training cut-off, so none of them could have seen this text during training. Every model was run with the same context length (2048) and the same evaluation code.

One asymmetry is worth naming: the competitors are finished, published models, while Od1n is measured at completed pre-training, before instruction-tuning and alignment. And to be transparent about provenance – these are internal measurements, not third-party verified. We publish the methodology so it can be reproduced and scrutinised; we do not present it as an independent audit. Read the full methodology and per-domain results on our technology and full benchmark page.

Who builds these models

Od1n V5 is a 3B model trained from scratch, a product of EZ-Fix AS (Oslo). NorMistral comes from the Norwegian research community (NORA / University of Oslo). The international models are general-purpose: Gemma (Google), Llama (Meta), Qwen (Alibaba) and Mistral (Mistral AI). The comparison is therefore between three groups: dedicated Norwegian research models, a dedicated Norwegian product model, and large international generalists.

What this means in practice

If you operate in Norway and your workload is Norwegian text – especially public-sector, legal and administrative language – model size is a poor proxy for quality. A 3B model built for the language reads Norwegian better than an 8B or 9B generalist, at a fraction of the inference cost. If you need the strongest possible aggregate Norwegian score and can run a larger model, NorMistral-11B leads. If you need efficient, on-premise-friendly Norwegian with an edge on administrative text, a 3B model like Od1n V5 covers that ground while staying small enough to run cheaply and privately.

The larger point is why a country builds its own models at all. Norwegian is a small language in global training data, and control over the model – its weights, its data provenance, its deployment – is control over the infrastructure that increasingly mediates Norwegian text. That is the argument for what a sovereign language model is, and this benchmark is one piece of evidence for it: with focused training, a small Norwegian model competes with, and on the right text beats, models many times its size.