How to Run Nemotron 3.5 Parse Locally: NVIDIA's 630M Document Parsing Model

Nemotron 3.5 Parse is NVIDIA's 630.8 million parameter vision-language model built to turn document images into structured text with layout classes, bounding boxes, and reading order. It runs through Hugging Face Transformers or vLLM, needs roughly 1.3 GB of bfloat16 weights, and hits 15.42 pages per second in NVIDIA's single-pass benchmark on an H100.

Written by Priya Raghunathan, local-AI hardware reviewer. I check parameter counts, licenses, hardware requirements and install commands against the primary source before recommending a setup path.

A new top-20 entrant on Hugging Face's image-text-to-text list

Nemotron 3.5 Parse entered the top 20 of Hugging Face's image-text-to-text trending list in October 2026. At time of writing it has 95 likes and 2,535 downloads on the nvidia/Nemotron-3.5-Parse model page, first published on Hugging Face September 3, 2026 under the OpenMDW 1.1 license. The model card lists no GitHub repository for the model itself. The most-liked Space linking to it is tardellirs/model-pulse (201 likes), a Hub-wide stats dashboard rather than a demo built around this specific model.

Specs at a glance

What it actually does

Nemotron 3.5 Parse is a document image parser, not a general chatbot. Feed it a scanned PDF page, a slide, a form or a report, and it returns structured text broken into layout classes such as titles, paragraphs, tables and charts, each with a bounding box and a position in the reading order. A downstream pipeline gets not just what the text says but where it sits on the page and in what sequence. A control-token prompt decides the output shape: ask for markdown, HTML, LaTeX, JSON or CSV tables, and the model switches format. A dedicated chart token means it treats charts as their own layout element instead of flattening them into garbled plain text.

Under the hood it pairs NVIDIA's own C-RADIOv4-SO400M vision encoder with a 10-layer mBART decoder that treats the image as a causal prefix. NVIDIA built it for document intelligence, retrieval-augmented generation and data-curation pipelines that need machine-readable text out of document images, rather than just a transcription.

Benchmarks

NVIDIA ran its own benchmark, ParseBench, across 2,078 unique document pages using a triple-pass setup with region union. It also reports two OCR-focused benchmarks: IndicVisionBench across 10 Indic languages, and MOSCAR, a synthetic multilingual test spanning 163 language and script entries.

These results do not guarantee accuracy on complex scanned documents, handwriting, degraded images, or other real-world variations.

That's NVIDIA's own caveat, and it matters: MOSCAR uses simple synthetic text renderings to check basic OCR capability, not real-world scans. Treat the headline numbers above as a ceiling under easy conditions, not a guarantee for a stack of crumpled receipts.

Install and run with Transformers

The model card doesn't publish a minimum GPU memory figure or a CPU benchmark. Do the arithmetic yourself: 630.8 million parameters at two bytes each in bfloat16 is roughly 1.3 GB of weights, small enough to leave headroom for activations on most CUDA GPUs, even though NVIDIA only benchmarks the model on an H100.

import torch
from PIL import Image
from transformers import AutoModel, AutoProcessor, GenerationConfig

model_path = "nvidia/Nemotron-3.5-Parse"
device = "cuda:0"

model = AutoModel.from_pretrained(
    model_path,
    trust_remote_code=True,
    dtype=torch.bfloat16,
).to(device).eval()
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)

image = Image.open("path/to/your/image.jpg").convert("RGB")
task_prompt = "</s><s><predict_bbox><predict_classes><output_markdown><predict_text_in_pic>"

inputs = processor(images=[image], text=task_prompt, return_tensors="pt", add_special_tokens=False).to(device)

generation_config = GenerationConfig.from_pretrained(model_path)
generation_config.max_new_tokens = 8744

with torch.inference_mode():
    outputs = model.generate(**inputs, generation_config=generation_config)

generated_text = processor.batch_decode(outputs, skip_special_tokens=False)[0]

The task_prompt string is doing the real configuration here. predict_bbox and predict_classes ask for bounding boxes and layout classes, output_markdown picks the output format, and predict_text_in_pic asks the model to transcribe text inside embedded pictures too. Swap output_markdown for one of the tokenizer's other format tokens to get HTML, LaTeX, JSON or CSV output instead.

postprocessing is a helper module NVIDIA ships inside the model repository, not a PyPI package. You don't need the full repository for it: pull in beautifulsoup4 and the three files the module depends on with a scoped download instead.

from pathlib import Path
from postprocessing import postprocess_document

document = postprocess_document(generated_text, image_size=image.size)
Path("prediction.md").write_text(document["markdown"], encoding="utf-8")

Serve it with vLLM

For production traffic, NVIDIA publishes a vLLM plugin that serves Nemotron 3.5 Parse over an OpenAI-compatible API and turns on the one-step speculative decoding head automatically.

The huggingface-hub pin and the version range on vLLM both matter: NVIDIA's plugin needs huggingface-hub 1.24.0 or newer (but below 2.0) and a vLLM release between 0.25.0 and 0.33 (exclusive), so installing a newer vLLM on its own will not pick up the speculative decoding plugin. The --chat-template flag points at chat_template.jinja, a file that ships inside the model repository itself, which is why the command downloads the full repo first instead of just pip-installing transformers.

License and use

Nemotron 3.5 Parse ships under the OpenMDW 1.1 license, which NVIDIA states covers both commercial and non-commercial use. The bundled tokenizer is licensed separately under MIT; check NVIDIA's tokenizer attribution file if you're redistributing the tokenizer on its own.

Known limitations

NVIDIA hasn't published a CPU inference path, a quantized build, or a minimum GPU memory figure for Nemotron 3.5 Parse, so budget conservatively if you're trying it on a GPU smaller than the H100 used in the published benchmarks. The MOSCAR language-coverage numbers come from simple synthetic text renderings rather than real scanned documents, so don't expect the same accuracy on handwriting, degraded scans, or other messy real-world pages. The postprocessing helper also lives inside the model repository rather than being pip-installable, so a transformers-only install can generate raw token output but can't turn it into clean markdown or JSON without those extra files.

FAQ

What is Nemotron 3.5 Parse?

A 630.8 million parameter vision-language model from NVIDIA that converts document images, scanned PDFs, slides, forms and reports, into structured text with layout classes, bounding boxes and reading order. It's built for document intelligence and RAG pipelines, not general chat.

Does Nemotron 3.5 Parse run on a CPU?

NVIDIA doesn't document a CPU path. The published benchmarks and usage examples all target a CUDA GPU in bfloat16, and no quantized or CPU-specific build is listed on the model card.

Is Nemotron 3.5 Parse free for commercial use?

Yes. It ships under the OpenMDW 1.1 license, which NVIDIA states covers commercial and non-commercial use. The bundled tokenizer carries a separate MIT license.

How big is Nemotron 3.5 Parse?

630,800,504 parameters per the Hugging Face API's safetensors metadata, about 0.631B. NVIDIA's model card separately counts 630,800,496 trainable parameters. In bfloat16 that's roughly 1.3 GB of weights before accounting for activations.

Where to go from here

• Browse other local vision and document models: the local AI model directory lets you compare specs side by side.
• New to running Hugging Face Transformers models locally? the how-to guide index has setup guides for other local models that cover the same workflow.