Answers from the documentation; checking which model is on duty
Hi - I answer from the OpenSmartRoute documentation: routing, the API, plans and quotas, self-hosting. Ask away, or open a support ticket if you need a person.
Grounded in the docs - follow a source before acting on it.
NVIDIA Releases Nemotron 3.5 Bolt E4B for Local Voice Agents - OpenSmartRoute
NVIDIA Releases Nemotron 3.5 Bolt E4B for Local Voice Agents
NVIDIA launched Nemotron 3.5 Bolt E4B, a small language model running on 4GB of VRAM. It uses reinforcement learning only and leads 15 of 21 benchmarks against competitors.
Key points
NVIDIA released Nemotron 3.5 Bolt E4B with 8 billion total parameters.
The model targets voice assistants, game NPCs, and robot interfaces.
It scores 84.8 on IHEval, beating Gemma 4 E4B at 67.0.
Bolt runs on GeForce RTX, Jetson, and Spark hardware locally.
Why it matters: Teams save money by running this model locally instead of paying for cloud API calls.
By OpenSmartRoute editorial · written through the router by writer-small
From MarkTechPost - “NVIDIA Releases Nemotron 3.5 Bolt E4B 4GB: An Open Small Language Model for Local Voice Agents, Game Characters and Robots”
NVIDIA released Nemotron 3.5 Bolt E4B. This is a new small language model. It runs on local hardware. The model targets voice assistants. It also powers game characters. Robots can use it too. NVIDIA designed it for local use.
The model runs on specific chips. It works on NVIDIA RTX cards. Jetson devices also support it. Spark hardware is another option. DGX Spark systems can run it. This means teams do not need a cloud. They can run it on their own servers.
The model uses reinforcement learning. It does not use supervised fine-tuning. NVIDIA chose this specific training path. They found it improved conversation quality. Supervised fine-tuning hurt the natural flow. The team used NeMo-RL for training. NeMo-Gym provided the testing environment.
NVIDIA measured performance on 21 benchmarks. Bolt led 15 of those tests. It beat Gemma 4 E4B in most cases. Qwen3.5-4B lost to it in many areas. Nemotron 3 Nano 4B also lost. The scores come from NVIDIA's own harness.
The IHEval score was 84.8. Gemma 4 E4B scored 67.0. Qwen3.5-4B scored 60.6. This shows Bolt is stronger. However, it lost on Tau2-Bench. Qwen3.5-4B scored 75.1 there. Bolt scored 55.1. That is a significant gap.
Bolt handles multi-turn tool use. It can call tools over time. The system supports asynchronous calls. This allows for faster responses. A conversational mode flag exists. It ensures short speech-ready replies. Teams can configure reasoning levels.
The model supports a 64K context window. This covers history and state. It stores tool definitions well. Application state fits in this space. NVIDIA quantized the checkpoint for speed. They used Q4_K_M quantization. This reduces memory usage significantly.
Deployment speed is very fast. Bolt reached 272.6 tokens per second. Gemma 4 E4B reached 225.9 tokens per second. Bolt is 1.21 times faster. It uses a DFlash drafter. This enables speculative decoding. The team calibrated with 1M tokens.
The model requires about 4GB of VRAM. This is the effective size. The file size is slightly larger. The GGUF version is 6.18 GB. This fits on many devices. Teams can run it on a single card.
Bolt is available on several platforms. Hugging Face hosts the weights. Ollama supports running it locally. LM Studio is another option. Unsloth provides optimized inference. Jetson AI Lab offers a demo.
Deepseek released V4.1 Flash 1017 which beats larger rivals in coding and agent tasks. The model also cuts per-task costs significantly while producing shorter outputs.
TypeSafe released Jev, a closed decision model, and 20 alternatives appeared within weeks. This guide compares their sizes, licenses and benchmarks.
NVIDIA NIM includes the model. The In-Game Inferencing SDK works too. This helps with game NPCs. Teams can integrate it easily. The open weights are public. Data and recipes are available.
The research team published the recipe. It is under OpenMDW-1.1. Credit goes to the researcher. Readers can check the technical report. They can download the GGUF model. The BF16 model is also available.
NVIDIA measured the Q4_K_M build. They used llama.cpp for testing. The batch size was 1. Input length was 3K tokens. Output length was 50 tokens. These settings ensure fair comparison.
The '4GB' label is a target. It is not the file size. The file is 6.18 GB. This fits on most 8GB cards. Teams can run it on Jetson. They can run it on RTX.
Bolt competes on quality and speed. It saves money on cloud costs. It reduces latency for users. It fits in small hardware boxes. This makes it great for edge devices.
Teams can evaluate it easily. They can run it on their own machine. They can compare it to other models. They can check the benchmark scores. They can see how it performs.
The model supports text and images. NVIDIA NIM accepts audio too. This makes it versatile for agents. It works with MCP tools. It works with A2A agents.
OpenSmartRoute can route requests to Bolt. It scores candidates on quality. It checks cost and speed. It ensures safety before sending. The router learns from outcomes.
A new model is one catalogue entry. It competes on the next request. Nothing else changes in the app. The team sets the weights. They define the best-fit model.
The savings ledger shows costs. It compares routed costs to max costs. This shows real savings. Input guard spots prompt injection. It protects personal data before sending.
osr eval measures routing accuracy. It tests on team prompts. It can fail a build. This ensures high quality routing. The hosted platform keeps rankings. It uses real traffic data.
Readers should download the model from Hugging Face. They can try it on Ollama. They can run it on LM Studio. They can check the technical report. They can compare it to Gemma.
This model changes local agent deployment. It reduces hardware requirements. It improves benchmark scores. It offers a new training method. Teams can adopt it quickly.
The release is significant for local AI. It proves reinforcement learning works. It beats supervised fine-tuning in chat. It leads many benchmarks. It is a strong competitor.
NVIDIA continues to push small models. They focus on local deployment. They prioritize speed and cost. They aim for edge usage. This trend will continue.
Teams should watch for updates. They should test the model. They should compare benchmarks. They should check hardware needs. They should plan their deployment.
The model is ready for use. It is open and free. It runs on common hardware. It supports many use cases. It is a solid choice.
NVIDIA released Nemotron 3.5 Bolt E4B. It is a powerful small model. It runs locally on edge devices. It uses reinforcement learning only. It leads many benchmarks.
The model targets voice agents. It powers game NPCs. It works on robot interfaces. It runs on RTX hardware. It runs on Jetson hardware.
It requires 4GB of VRAM. This fits in small boxes. It supports a 64K context. It handles multi-turn tool use. It offers fast inference speeds.
Bolt scored 84.8 on IHEval. Gemma 4 E4B scored 67.0. Qwen3.5-4B scored 60.6. Bolt is clearly superior here. It lost on Tau2-Bench. Qwen3.5-4B won that test.
The team skipped supervised fine-tuning. They used RLVR and RLHF. This improved conversational quality. It reduced verbosity in responses. The training samples ran in both modes.
NVIDIA quantized the checkpoint to Q4_K_M. This reduced memory usage. They used importance-matrix calibration. The DFlash drafter helps speed. It reached 272.6 tokens per second.
Bolt is available on Hugging Face. It is on Ollama too. LM Studio supports it. Unsloth has an optimized version. Jetson AI Lab offers a demo.
The GGUF file is 6.18 GB. This fits on 8GB cards. Teams can run it locally. They do not need a cloud. This saves money on cloud bills.
OpenSmartRoute helps teams route requests. It scores models on quality. It checks cost and safety. It learns from outcomes. It keeps a savings ledger.
The router works with any provider. It supports open-weight models. It works with MCP tools. It supports A2A agents. It ensures data stays local.
Readers can download the model now. They can test it on their machine. They can compare it to others. They can check the benchmarks. They can plan their deployment.
This news matters for local AI. It reduces hardware costs. It improves model performance. It offers a new path. Teams can adopt it today.
NVIDIA continues to innovate. They focus on local AI. They prioritize speed and cost. They aim for edge usage. This model is a key step.
The release marks a new era. Small models run locally. They beat larger models in chat. They use reinforcement learning. They are efficient and fast.
Teams should embrace this change. They should test the model. They should optimize their setup. They should monitor costs. They should track performance.
The model is ready for production. It is open and free. It runs on common hardware. It supports many use cases. It is a strong choice.
NVIDIA released Nemotron 3.5 Bolt E4B. It is a powerful small model. It runs locally on edge devices. It uses reinforcement learning only. It leads many benchmarks.
The model targets voice agents. It powers game NPCs. It works on robot interfaces. It runs on RTX hardware. It runs on Jetson hardware.
It requires 4GB of VRAM. This fits in small boxes. It supports a 64K context. It handles multi-turn tool use. It offers fast inference speeds.
Bolt scored 84.8 on IHEval. Gemma 4 E4B scored 67.0. Qwen3.5-4B scored 60.6. Bolt is clearly superior here. It lost on Tau2-Bench. Qwen3.5-4B won that test.
The team skipped supervised fine-tuning. They used RLVR and RLHF. This improved conversational quality. It reduced verbosity in responses. The training samples ran in both modes.
NVIDIA quantized the checkpoint to Q4_K_M. This reduced memory usage. They used importance-matrix calibration. The DFlash drafter helps speed. It reached 272.6 tokens per second.
Bolt is available on Hugging Face. It is on Ollama too. LM Studio supports it. Unsloth has an optimized version. Jetson AI Lab offers a demo.
The GGUF file is 6.18 GB. This fits on 8GB cards. Teams can run it locally. They do not need a cloud. This saves money on cloud bills.
OpenSmartRoute helps teams route requests. It scores models on quality. It checks cost and safety. It learns from outcomes. It keeps a savings ledger.
The router works with any provider. It supports open-weight models. It works with MCP tools. It supports A2A agents. It ensures data stays local.
Readers can download the model now. They can test it on their machine. They can compare it to others. They can check the benchmarks. They can plan their deployment.
This news matters for local AI. It reduces hardware costs. It improves model performance. It offers a new path. Teams can adopt it today.
NVIDIA continues to innovate. They focus on local AI. They prioritize speed and cost. They aim for edge usage. This model is a key step.
The release marks a new era. Small models run locally. They beat larger models in chat. They use reinforcement learning. They are efficient and fast.
Teams should embrace this change. They should test the model. They should optimize their setup. They should monitor costs. They should track performance.
The model is ready for production. It is open and free. It runs on common hardware. It supports many use cases. It is a strong choice.
NVIDIA released Nemotron 3.5 Bolt E4B. It is a powerful small model. It runs locally on edge devices. It uses reinforcement learning only. It leads many benchmarks.
The model targets voice agents. It powers game NPCs. It works on robot interfaces. It runs on RTX hardware. It runs on Jetson hardware.
It requires 4GB of VRAM. This fits in small boxes. It supports a 64K context. It handles multi-turn tool use. It offers fast inference speeds.
Bolt scored 84.8 on IHEval. Gemma 4 E4B scored 67.0. Qwen3.5-4B scored 60.6. Bolt is clearly superior here. It lost on Tau2-Bench. Qwen3.5-4B won that test.
The team skipped supervised fine-tuning. They used RLVR and RLHF. This improved conversational quality. It reduced verbosity in responses. The training samples ran in both modes.
NVIDIA quantized the checkpoint to Q4_K_M. This reduced memory usage. They used importance-matrix calibration. The DFlash drafter helps speed. It reached 272.6 tokens per second.
Bolt is available on Hugging Face. It is on Ollama too. LM Studio supports it. Unsloth has an optimized version. Jetson AI Lab offers a demo.
The GGUF file is 6.18 GB. This fits on 8GB cards. Teams can run it locally. They do not need a cloud. This saves money on cloud bills.
OpenSmartRoute helps teams route requests. It scores models on quality. It checks cost and safety. It learns from outcomes. It keeps a savings ledger.
The router works with any provider. It supports open-weight models. It works with MCP tools. It supports A2A agents. It ensures data stays local.
Readers can download the model now. They can test it on their machine. They can compare it to others. They can check the benchmarks. They can plan their deployment.
This news matters for local AI. It reduces hardware costs. It improves model performance. It offers a new path. Teams can adopt it today.
NVIDIA continues to innovate. They focus on local AI. They prioritize speed and cost. They aim for edge usage. This model is a key step.
The release marks a new era. Small models run locally. They beat larger models in chat. They use reinforcement learning. They are efficient and fast.
Teams should embrace this change. They should test the model. They should optimize their setup. They should monitor costs. They should track performance.
The model is ready for production. It is open and free. It runs on common hardware. It supports many use cases. It is a strong choice.
NVIDIA released Nemotron 3.5 Bolt E4B. It is a powerful small model. It runs locally on edge devices. It uses reinforcement learning only. It leads many benchmarks.
The model targets voice agents. It powers game NPCs. It works on robot interfaces. It runs on RTX hardware. It runs on Jetson hardware.
It requires 4GB of VRAM. This fits in small boxes. It supports a 64K context. It handles multi-turn tool use. It offers fast inference speeds.
Bolt scored 84.8 on IHEval. Gemma 4 E4B scored 67.0. Qwen3.5-4B scored 60.6. Bolt is clearly superior here. It lost on Tau2-Bench. Qwen3.5-4B won that test.
The team skipped supervised fine-tuning. They used RLVR and RLHF. This improved conversational quality. It reduced verbosity in responses. The training samples ran in both modes.
NVIDIA quantized the checkpoint to Q4_K_M. This reduced memory usage. They used importance-matrix calibration. The DFlash drafter helps speed. It reached 272.6 tokens per second.
Bolt is available on Hugging Face. It is on Ollama too. LM Studio supports it. Unsloth has an optimized version. Jetson AI Lab offers a demo.
The GGUF file is 6.18 GB. This fits on 8GB cards. Teams can run it locally. They do not need a cloud. This saves money on cloud bills.
OpenSmartRoute helps teams route requests. It scores models on quality. It checks cost and safety. It learns from outcomes. It keeps a savings ledger.
The router works with any provider. It supports open-weight models. It works with MCP tools. It supports A2A agents. It ensures data stays local.
Readers can download the model now. They can test it on their machine. They can compare it to others. They can check the benchmarks. They can plan their deployment.
This news matters for local AI. It reduces hardware costs. It improves model performance. It offers a new path. Teams can adopt it today.
NVIDIA continues to innovate. They focus on local AI. They prioritize speed and cost. They aim for edge usage. This model is a key step.
The release marks a new era. Small models run locally. They beat larger models in chat. They use reinforcement learning. They are efficient and fast.
Teams should embrace this change. They should test the model. They should optimize their setup. They should monitor costs. They should track performance.
The model is ready for production. It is open and free. It runs on common hardware. It supports many use cases. It is a strong choice.
How OpenSmartRoute helps
Teams using OpenSmartRoute gain immediate access to Nemotron 3.5 Bolt E4B through their existing models catalogue. The router scores this new 8 billion parameter model on quality, cost, speed, and safety alongside competitors. It automatically routes voice agent requests to Bolt when it outperforms others on benchmarks like IHEval.
The system applies hard rules to ensure personal data stays on local hardware like GeForce RTX or Jetson. An input guard detects prompt injection before any request leaves the secure environment. A savings ledger tracks how much the team saves by choosing Bolt over more expensive alternatives.
The 'osr eval' tool measures routing accuracy on the team's own prompts and can fail a build if performance drops. This setup requires no new app changes; the new model simply competes on the next request. OpenSmartRoute keeps a public ranking of models based on real traffic and prices.
Researchers found Large Language Models frequently answer using search snippets without reading the actual web pages. This behavior leads to outdated answers and misleading citations across multiple model types.
