How to Run Hy-SimulMT-30B-A3B-FP8 Locally: Tencent's Real-Time Speech Translator

Hy-SimulMT-30B-A3B-FP8 is Tencent Hunyuan's 30.39 billion parameter model for real-time Chinese-English speech translation, with an FP8-quantized mixture-of-experts core that cuts the weight footprint roughly in half next to the BF16 release. It runs through Tencent's own SimulInfer server on an NVIDIA GPU with vLLM 0.31 or newer, and streams translated text and cloned speech while the speaker is still talking.

Written by Priya Raghunathan, local-AI hardware reviewer. I check parameter counts, licenses, hardware requirements and install commands against the primary source before recommending a setup path.

A new top-20 entrant on Hugging Face's overall trending list

Hy-SimulMT-30B-A3B-FP8 entered the top 20 of Hugging Face's overall trending list in late October 2026, four days after Tencent published it. At time of writing the model page shows 236 likes and 165 downloads, and the model's GitHub repository has 163 stars. No Hugging Face Space links to it yet, so the fastest way to try it is still a local install.

Specs at a glance

What it actually does

Despite the "translation" pipeline tag, Hy-SimulMT isn't a text-in, text-out machine translation model. It listens to a continuous audio stream in Chinese or English and produces a running translation in the other language, as text and as cloned speech in the original speaker's voice, before the speaker finishes talking. Tencent built it for meetings, speeches and interviews, where a simultaneous interpreter normally has to choose between waiting for more context and speaking early. Voice cloning for the spoken output needs a text-to-speech service wired in separately; without one, you still get the streaming text translation.

Tencent publishes six checkpoints from the same underlying model. -FP8 and -GGUF are quantized versions of the base BF16 weights, and the separate -Interleave line adds the source-language transcript alongside the translation in its output instead of returning the translation alone.

Benchmarks

Tencent evaluated Hy-SimulMT against open-source simultaneous-translation models and commercial APIs on RealSI, a public long-form audio benchmark, then on four shorter public benchmarks. The long-form result is the headline number: a StreamLAAL (a measure of streaming translation lag, lower is better) of about 3 seconds, the lowest of the systems Tencent compared it against, with a GEMBA quality score ahead of the open-source models and four of the five commercial APIs in the comparison.

These results suggest that lower latency does not necessarily require a substantial sacrifice in translation quality.

These are Tencent's own benchmark numbers for the base model, not independently reproduced figures, and the model card doesn't break them out separately for the FP8 checkpoint. Treat them as a starting point for evaluation on your own audio rather than a guarantee.

Hardware you need

Tencent's deploy script asks for a Linux machine with Python 3.12 or newer, Git, an NVIDIA GPU with CUDA, and vLLM 0.31 or newer (the version that added support for the model's HYV3ForCausalLM architecture). For the full BF16 model, Tencent states roughly 57 GiB of weights, about 61 GB, which fits on a single H200 and little else. The FP8 checkpoint is smaller: its safetensors metadata reports 28.65 GB of FP8 weights plus 3.49 GB of BF16 weights for the non-quantized layers, about 32 GB total before you add room for the KV cache and audio-streaming buffers. Tencent doesn't publish an official minimum GPU size for the FP8 build specifically, so budget for a 40 GB card at minimum and a 48 GB or larger card, an RTX 6000 Ada or an A100 80GB, for comfortable headroom.

Install and run with SimulInfer

Tencent ships a deploy script inside the GitHub repository that installs vLLM, downloads the model and starts both the model server and a web UI in one step.

deploy.sh downloads the model, installs a matching vLLM into its own environment, and brings up the web UI on port 8080 and the local model server on port 8800. Logs and config land under SimulInfer/.simulinfer/, including config.toml, app.log and model.log. Wait for a "Ready" message in the terminal before opening the UI.

If you already have a model already downloaded, or a vLLM environment you want to reuse instead of letting the script build a new one, point deploy.sh at it directly:

Multiple GPUs are a flag away. --gpus pins which devices the server uses; --tensor-parallel-size splits the model across them:

Sending audio through it

Once the server reports "Ready", open http://127.0.0.1:8080 in a browser, pick a translation direction, and either upload an audio file or speak into your microphone. The same endpoint also takes a file over curl, and the repository's command-line client wraps that same call:

With a TTS service configured, the CLI client saves the translated speech to outputs/translated.wav. Full request and response shapes are documented at the server's own /docs endpoint once it's running.

License and use

Hy-SimulMT-30B-A3B-FP8 ships under the Apache 2.0 license, which permits commercial and non-commercial use, modification and redistribution, including the FP8 weights themselves.

Known limitations

This checkpoint only speaks Chinese and English, despite the 32 languages listed in the model's metadata tags; those tags describe a broader multilingual intent for the Hy-SimulMT line, not a capability of this specific release. Voice cloning on the output side depends on a TTS service you configure yourself, Hy-SimulMT doesn't bundle one. Tencent's own SGLang-Omni path and the in-progress llama.cpp support both skip the FP8 checkpoint, so SimulInfer on vLLM is the one supported way to run this exact model today. And the benchmark numbers above come from Tencent's own evaluation of the base model, not an independently reproduced result for the FP8 build specifically.

FAQ

What is Hy-SimulMT-30B-A3B-FP8?

A 30.39 billion parameter model from Tencent Hunyuan that translates spoken Chinese and English in real time as the speaker talks, producing both translated text and, with a TTS service attached, cloned speech in the original voice. The FP8 suffix marks this as the 8-bit quantized checkpoint of the base BF16 model.

How much VRAM does it need?

The FP8 weights total about 32 GB per the model's safetensors metadata, roughly half the 61 GB the full BF16 checkpoint needs. Tencent doesn't publish an official minimum for the FP8 build, so plan for a 40 GB card at minimum and more headroom for the KV cache and audio buffers.

Can I run the FP8 checkpoint with SGLang or llama.cpp?

Not directly. Tencent's SGLang-Omni integration explicitly serves only the base BF16 model, excluding the FP8, GGUF and Interleave checkpoints. The llama.cpp support in progress targets the separate GGUF build. For this exact FP8 checkpoint, use Tencent's SimulInfer server on vLLM 0.31 or newer.

Is Hy-SimulMT-30B-A3B-FP8 free for commercial use?

Yes. It's released under the Apache 2.0 license, which allows commercial use, modification and redistribution.

Where to go from here

• Browse other local speech and translation models: the local AI model directory lets you compare specs side by side.
• New to running quantized models locally? the how-to guide index has setup guides for other local models that cover the same vLLM-based workflow.