Danish speech model, measured: Pianissimo on 930 FLEURS clips

On 7 November 2026 I ran the Danish version of Pianissimo, an open speech model from Klang AI in Helsingborg (CC BY 4.0), through the same Core ML path that Sæga uses on a Mac. The question was simple: is it good enough to ship as a Danish option, and how does it compare with Parakeet Ultra, the multilingual model that is already in the app?

I did the same for Swedish in September. That article has the longer reasoning about method. This one is shorter, because the method is the same.

The method

The material is FLEURS, Google’s open speech set, Danish (da_dk), test split: 930 clips, 176 minutes of read sentences. FLEURS is CC BY 4.0. Klang publishes a number for the first 200 clips of that split, so I could check my setup against theirs.

The normalisation is Klang’s, from the model card: lowercase, punctuation replaced with spaces, whitespace collapsed. Numbers are left as they are. Word error rate is computed at corpus level (all errors divided by all reference words), not averaged per clip. The reference is FLEURS’ own normalised text field; the raw field gives the same score after normalisation.

Everything ran through Sæga’s own Core ML code, on a MacBook with Apple Silicon, with no phrase boosting. Parakeet Ultra was run with the language set to Danish.

The result

Model	First 200 clips	Klang published	All 930 clips
Pianissimo Danish, 8-bit	6.95%	6.9%	7.80%
Pianissimo Danish, fp16	7.04%	not published	7.85%
Parakeet Ultra (multilingual), language Danish	16.29%	not published	16.3%

Corpus-level word error rate, lower is better. 8-bit is the version Sæga would ship.

What the numbers say

The Danish Pianissimo is about twice as accurate as Parakeet Ultra on Danish. 7.8% against 16.3%. Parakeet Ultra is a good multilingual model, but Danish is not what it was built for, and the gap is what you would expect. This is the reason Sæga does not ship a general fallback model: a language is listed when there is a model that is good at it.

8-bit and fp16 are the same in practice. 0.05 points apart over 930 clips. The smaller one is the sensible choice.

7.8% means roughly one word in thirteen is wrong on read, clean sentences. That is good, not perfect. Real speech with background noise, mumbling or crosstalk will be worse. I have not measured that.

Speed

Warm, on a MacBook with Apple Silicon: about 183 times real time. The 176 minutes of audio took about 58 seconds. Loading the model cold took 7.4 seconds. The model runs on the Neural Engine. These figures are from one Mac, so treat them as a size, not a promise. Klang quote much higher figures for their own server hardware; I do not use those.

A long file

I also put 23.7 minutes of FLEURS clips, joined into one file, through the app’s file transcription, which cuts the audio at quiet moments and transcribes it piece by piece. Result: 8.47% word error rate and no gap over 10 seconds, so nothing was dropped silently.

Details that came out right, and wrong

In 20 synthetic Danish sentences (made with a text-to-speech voice, so they are clean) I checked the things a reader notices. Right: percent (5%), kroner (249 kr), decimals (3,2 millioner), dates (14. december), times (17:45) and the letters æ, ø and å. Wrong: e-mail addresses, phone numbers and domain names. If you dictate those, check them.

Names

Sæga lets you add your own words to a dictionary, which nudges the model towards them. On 50 clips that contain names, with a list of names to boost (alpha 1.0), individual names came out right 52.1% of the time against 28.7% without. On 100 clips without names the error rate went up by 0.09 points. It is a small test, one run and one setting, so read it as a direction.

What I did not test

• Danish TV and radio. DR was not measured. FLEURS is read speech, so these numbers say little about fast, overlapping or noisy speech.
• Danish users. There are no human Danish testers yet.
• Dialects, children, strong accents, and other Macs than the one I used.
• Meetings in Danish with two or more speakers.

Recompute it yourself

The hypotheses from the run are published, one JSONL file per model with the clip id and the text the model produced, together with the references from FLEURS (transcription is the normalised field I scored against):

• hyp-8bit.jsonl (Pianissimo Danish, 8-bit)
• hyp-fp16.jsonl (Pianissimo Danish, fp16)
• hyp-ultra.jsonl (Parakeet Ultra, language Danish)
• refs.jsonl (references and clip durations)

Normalise as described above and compute word error rate at corpus level, and you should land on the table. The audio is in the FLEURS set itself: google/fleurs, Danish, test split, clips numbered in file order.

Thanks

To Klang AI for Pianissimo and for publishing the model and its method openly (CC BY 4.0), and to Google for FLEURS (CC BY 4.0, Conneau et al.). The speech data and its reference texts belong to them; I republish only the references needed to recompute the score and my own model output.