Chips & Supply · United States

Nvidia's Nemotron 3.5 Super VL is an open model built to run enterprise AI agents nonstop

Credited to The Decoder · the-decoder.com · 21h ago

Nvidia has released Nemotron 3.5 Super VL, an open multimodal model with 120 billion parameters built for enterprise agents. Two compact Bolt models designed to bring AI agents to local PCs and edge devices are launching alongside it.

Nvidia built Nemotron 3.5 Super VL for specialized agents that run continuously in the background, processing text, images, documents, and videos.

As the company explains in a technical blog post, it uses a Mixture-of-Experts architecture where only 12 billion of the 120 billion parameters are active per token. Nvidia developed the model with the Nemotron Coalition, a group that includes Mistral AI, Thinking Machines Lab, Perplexity, and Black Forest Labs. Ad

Two smaller models for local hardware ship at the same time. Nemotron 3.5 Bolt 74B-A9B 24GB runs agent tasks on local machines, while the smaller Bolt E4B 4GB targets fast voice assistants and will follow later this month. Both models offer the "highest intelligence per gigabyte of memory," Nvidia claims. Ad

Super VL and Bolt join the existing Nemotron lineup. Lightning, released in August, is built for high throughput, while Ultra tackles tough reasoning tasks. Every variant is optimized for Nvidia hardware.

Nvidia claims its biggest model's accuracy at one-fifth the size

Nvidia distilled Super VL from Nemotron 3 Ultra, its largest reasoning model. On tasks where an agent follows instructions, calls tools, works through multi-step business processes, or analyzes images, Super VL is supposed to match Ultra's accuracy with only about one-fifth the parameters. In agent tests from Hermes and LangChain, it actually scores higher than Ultra, according to Nvidia, and it also beats its predecessor Nemotron 3 Super on the coding benchmarks SWE-bench and Terminal-Bench. Ad

On a single GPU, Super VL delivers up to five times the throughput of other open models of similar size on agent tasks, according to Nvidia, citing the Artificial Analysis leaderboard. In an independent test by EXO Labs on the DGX Spark, no other model was both more accurate and faster. That speed advantage pays off when agents need to grind through large task volumes over days or weeks.

Against closed frontier models, Nvidia is pitching Super VL to companies that want to fine-tune with their own data and run the model on their own servers, free from a provider's terms of service. Nvidia says that could appeal to an insurer that continuously screens claims for fraud, or a financial services firm that scores creditworthiness by its own criteria. For customization, the company provides its NeMo framework, Python pipelines, and open datasets. Ad

Compression and caching cut compute overhead

Nvidia compresses Super VL with its 4-bit NVFP4 format. On Blackwell GPUs, the model runs entirely in 4-bit precision, while older cards store only the weights at 4 bits and handle math at 16 bits. Sensitive parts of the model keep higher precision. To prevent quality loss, Nvidia trained the compressed version to match the uncompressed original's outputs. Ad

A built-in draft component proposes several tokens ahead of time, which the main model then verifies in a single forward pass. On the DGX Spark, Nvidia recommends its own LiLiCorr method instead, which evaluates all draft tokens together for more coherent output. Nvidia says this approach speeds up text generation by up to 2.3x compared to running without speculative drafts.

Agents send the same instructions, tool descriptions, and conversation history with every request, so Super VL caches the computations for these recurring parts rather than rerunning them each time. Nvidia's inference software Dynamo also routes requests from the same session to the GPU that already holds that session's cache. On 16 B200 GPUs, this lets Super VL serve eight times as many concurrent users, and time to first token drops by 85 percent.

Super VL parses tables, charts, and PDFs without a separate vision model

For vision, Nvidia focused on the kind of material agents encounter in daily work, including OCR, tables, charts, multi-page documents, and videos. Super VL can interpret a chart, link it to a table elsewhere in the same document, and summarize both without needing a separate image model.

On the chart benchmark CharXiv, Super VL scores 80.8 percent, according to Nvidia, while the older Nemotron 3 Nano Omni hit 63 percent. When the model can also run Python code, the score rises to 84 percent. On the OCRBench v2 leaderboard, it ranks among the best open models, Nvidia says.

Local agents still demand expensive hardware

Bolt 74B-A9B 24GB is a stripped-down version of Super VL built as a hybrid Mixture-of-Experts model where only 9 of its 74 billion parameters are active per step. To squeeze it into 24 GB of memory, Nvidia applies aggressive 2-bit quantization to the weights.

Bolt handles everyday local tasks where an agent needs to work through multiple steps, call tools, or search the web. Users can have it write and debug code or figure out which PC components fit their system, according to Nvidia.

The model is optimized for Nvidia's RTX Spark, a new chip for AI-capable Windows laptops, and the DGX Spark, a compact AI workstation with up to 128 GB of shared memory. It also runs on GeForce RTX PCs, DGX Station, and RTX PRO workstations, and supports llama.cpp, Ollama, LM Studio, and vLLM.

Even with heavy compression, 24 GB is a high bar. Only pricey high-end hardware offers that much memory, and at current GPU prices, most consumers can't justify the cost. For now, the local agent is aimed at developers and owners of powerful workstations.

For weaker devices, Nvidia has Bolt E4B 4GB planned, though it hasn't shipped yet. It's designed to run on GeForce RTX cards, Jetson modules, and RTX/DGX Spark, powering voice assistants, interactive characters, or device interfaces. The model is supposed to hold natural conversations while using tools across multiple turns and staying aware of what's happening in the current app.

Early adopters span finance and cybersecurity

Dozens of organizations are already fine-tuning Super VL for their own needs, Nvidia says, mainly in industries with heavy document loads and sensitive data. Scale AI adapted the model for an agent that handles vendor credits at media company Skydance, with costs expected to drop to one-fortieth and response time to one-fifth. Ramp uses the model to turn photos of receipts into structured data and expects ten times faster responses. EY uses it for acquisition due diligence, and Hebbia for analyzing credit agreements.

A second wave of adopters comes from cybersecurity. AI21 fine-tuned Super VL for security firm Alice, which uses it to analyze threats and vulnerabilities, with accuracy improving by 40 percent according to Nvidia. Dream uses the model to hunt for vulnerabilities, and CrowdStrike is building agents to defend enterprise networks. Salesforce plans to automate government workflows, R1 is auditing medical billing, and Perplexity, Dassault Systèmes, and Mercor are also using the model.

Nvidia publishes datasets and training recipes

Nvidia is releasing the weights in a base version, a post-trained version, and a 4-bit compressed version, along with the datasets and training recipes and reinforcement learning environments. That makes Super VL more open than many open-weight models, which typically only share the weights.

The model was trained on over 25 trillion tokens during pre-training and continued pre-training, Nvidia says, with post-training and reinforcement learning adding roughly four million examples across about 100 open training environments.

On Hugging Face, Nvidia is also releasing several datasets that other developers can use to train or fine-tune their own models. They cover how agents use tools, follow instructions, handle long texts, and understand images, while other datasets deal with finance, safety, and fairness.

One dataset targets languages spoken in India. Nvidia built it with the Indian Institute of Technology Madras and its AI lab AI4Bharat, and it contains about 2.83 trillion tokens of text in 22 Indian languages.

Open models sell Nvidia hardware

Super VL runs on desktop systems like DGX Spark, DGX Station, RTX PRO, and RTX, on Jetson modules in edge devices, and in data centers and the cloud. Through Nvidia NIM, enterprises get the model as a ready-to-deploy microservice with validated configs and support, with updates planned every two weeks. Weights are available on Hugging Face for Nemotron 3.5 Super VL 120B-A12B and Nemotron 3.5 Bolt 74B-A9B 24GB.

Nvidia releases the models openly but tunes them for its own products, from data center servers to workstations. Anyone who wants to get the most out of Nemotron ends up buying Nvidia hardware.

The strategy behind Nvidia's open models is self-serving. The more companies run their own open models, the greater the demand for compute hardware, and open models will likely stay optimized for Nvidia for the foreseeable future. Meanwhile, companies like Meta, Google, Anthropic, and OpenAI are building their own chips and optimizing models for those instead. To push the spread of open AI models further, Nvidia recently invested $12.9 billion in Hugging Face, one of the most important platforms for developing and sharing them.

AI News Without the Hype – Curated by Humans

Chips & SupplyUnited States

Dow jumps 556 pts while Nvidia keeps bleeding, our AI spotted the pivot weeks ago

Credited to Investing.com · investing.com10h ago

Useful

View the articleInvesting.com

Front Method