Introducing Nemotron 3.5 Lightning
Ollama now supports NVIDIA Nemotron 3.5 Lightning, a 30 billion parameter open model designed for agent systems. The model is built for continuous operation, including context gathering, tool invocation, and handling multi-step processes. It is developed by the Nemotron Coalition and trained on datasets used by developers across coding, tool calling, instruction following, and multi-turn workflows.
Technical Specifications
The model utilizes a Mixture-of-Experts architecture with 3 billion active parameters per token. It supports a context length of 1 million tokens, facilitating extended tool histories within multi-turn interactions. Inference is optimized through speculative decoding with multi-token prediction (MTP), DFlash, or DSpark, achieving up to 4x higher throughput compared to other open models of similar size. The model is available for Apple silicon with state-of-the-art performance: nemotron-3.5-lightning:30b-mlx.
Performance and Use Cases
Testing indicates Nemotron 3.5 Lightning achieves 4x higher throughput and 30% faster task completion time compared to leading open models of similar size. It excels in workloads such as long-running personal assistants, coding sub-agents for tasks like testing and refactoring, and security operations involving alert enrichment and log querying. The model is designed to operate locally, maintaining data privacy and reducing latency.
Deployment Options
Users can deploy Nemotron 3.5 Lightning locally on NVIDIA RTX PCs, RTX PRO workstations, DGX Spark, and DGX Station systems. The same command-line interface (CLI) and API are used for both local and cloud deployments. The system supports a tiered approach, where a local instance handles high-volume steps, while a larger hosted model manages more complex tasks.
