Sebastian Raschka published an article titled "Understanding and Implementing Qwen3 From Scratch" on September 6, 2025. The piece focuses on the implementation details of Qwen3, a model family noted for its developer-friendly Apache License v2.0 and lack of additional usage restrictions compared to other open-weight models.
Qwen3 variants are ranked highly in benchmarks. As of the writing, the 235B-Instruct variant is tied with Claude Opus 4 at rank 8 on the LMArena leaderboard. The only open-weight models ranking higher are DeepSeek 3.1 and Kimi K2, which are significantly larger.
On September 5th, Qwen released a 1T parameter "max" variant that outperforms Kimi K2, DeepSeek 3.1, and Claude Opus 4 on major benchmarks. This specific model remains closed-source despite its superior performance metrics.
The article covers various model sizes available for different compute budgets, including dense models starting at 0.6B parameters and Mixture-of-Experts models reaching 480B parameters. The content includes from-scratch code implementations in pure PyTorch to explain the underlying architecture.
Source: https://magazine.sebastianraschka.com/p/qwen3-from-scratch
