How to Run Whistle Locally: Cactus's 16.9MB On-Device Speech Recognition Model

Whistle is an open-weight speech-to-text model from Cactus Compute built for phones, wearables, robots, smart home hardware and microcontrollers rather than servers. The entire model fits in a single 16.9 megabyte file, needs no GPU, and runs through the same CPU engine and quantization scheme as Cactus's Needle text model, so a device that already runs one can run both from one binary.

Written by Priya Raghunathan, local-AI hardware reviewer. I check parameter counts and hardware requirements against the primary source before recommending an install path.

A new top-20 entrant on Hugging Face's trending list

Whistle entered the top 20 of Hugging Face's overall trending list. At time of writing it has 267 likes and 5,558 downloads on the Cactus-Compute/whistle model page, and the cactus-compute/needle GitHub repo that runs it has 13,886 stars. The top linked community Space is tardellirs/model-pulse (199 likes).

Specs at a glance

What it actually does

Whistle handles three jobs on the device, with nothing sent to a server. Transcription turns 16 kHz mono audio into text in any of its seven languages, detecting the language automatically unless you name one, and it returns an empty transcript for silence instead of guessing at words that were never said. Word timestamps attach a start time, end time and probability to every word, pulled from the decoder's own attention rather than a separate alignment pass, which is what lets an app highlight or seek to a specific word. Speech embedding skips transcription entirely and returns the encoder's raw output, one row per 80 millisecond frame, for matching or retrieval tasks that do not need readable text.

Hardware: no GPU, one shared runtime

At 16.9 MB, Whistle is built to run on the same CPU budget as the rest of a device's firmware, not a dedicated accelerator. It reuses Needle's .cact container and quantization, so if a device is already running Needle for text, Whistle adds speech recognition without a second runtime, a second set of kernels or a second model format to manage. The decoder is also laddered the way Needle's is: every depth from 2 layers up is a separate deployable model, selected at load time with the --audio-depth flag, so a constrained microcontroller and a phone can both run Whistle at a depth that fits their budget from the same download.

Cactus benchmarked speed against OpenAI's Whisper and Useful Sensors' Moonshine by running 10 seconds of audio on an Apple M4 Pro, each model on its own official runtime at default settings: Whistle's C++ engine at 5 beams, openai-whisper, and moonshine-voice running non-streaming over the whole clip. Precision differed by design across the three: Whistle ran at 2 to 4 bit, Whisper at fp32 in memory, and Moonshine at int8. Word error rate was scored with the Whisper normalizers over 86,174 utterances for Whistle, against the figures Whisper and Moonshine published themselves, so treat the comparison as directional rather than a single controlled benchmark run on identical hardware and precision.

One engine, text and speech together

Because Whistle and Needle share a runtime, needle_load reads whichever model a given .cact file carries and the same binary can transcribe on its own, answer text on its own, or do both in one call. Handing the engine both a text model and Whistle at once means it transcribes an audio clip, runs the transcript against your tool definitions, and returns one JSON object containing both the tool calls and the speech fields, the audio-specific ones prefixed audio_. The audio never leaves the engine as a separate transcription step.

Install it

The Python package covers transcription, word timestamps and embeddings. A 16 kHz WAV file or raw samples work out of the box; microphone capture and other sample rates need the optional mic extra.

import needle

print(needle.transcribe("clip.wav")["text"])
# turn off the kitchen lights

Every call returns the text, the detected language, the milliseconds to the first token and the decoder's tokens per second after that. Set word_timestamps=True to get per-word timing, pass keywords=[...] to bias toward specific vocabulary, or force the language with language="de" instead of letting it auto-detect. The engine and weights download once and are cached after that.

For deployment outside Python, the engine and its platform builds live in the companion Needle repo. Each platform folder ships a needle binary plus the C library and header, and loads any .cact file you point it at, Whistle, Needle, or both.

FAQ

What is Whistle?

Whistle is an open-weight speech-to-text model from Cactus Compute that ships as a single 16.9 MB file and runs entirely on-device, with no GPU, for transcription, word-level timestamps and speech embeddings.

Which languages does it support?

Seven: English, German, French, Spanish, Italian, Dutch and Polish. It detects the language automatically unless you pass one explicitly.

Do I need a GPU to run it?

No. Whistle runs on CPU through the same engine as Cactus's Needle text model, and is built specifically for phones, wearables and other devices that do not have one.

Can it run alongside a text model in one call?

Yes. Loading a text model and Whistle into the same Needle engine lets one call transcribe audio and reason over the transcript against your tool definitions, returning both results in a single JSON object.

What license is it under?

Apache 2.0.

Where to go from here

• Need speaker labels instead of just a transcript? Run Nemotron 3 Diarization locally covers a CPU-friendly model that tags who spoke when.
• Comparing it to another streaming speech model? Run Audio8 ASR Infinite locally covers a larger, GPU-oriented alternative built for 24/7 transcription.
• Browse other speech and audio models: the local AI model directory lets you compare specs side by side.