OpenAI announced the release of gpt-oss-20b and gpt-oss-120b, marking their first large, fully open-weight models since GPT-2 in 2019. These models utilize MXFP4 optimization to enable local execution on consumer GPUs with up to 16 GB of RAM for the 20B variant and single H100s with 80 GB of RAM for the 120B variant.
The architecture remains a decoder-only transformer, similar to GPT-2, but incorporates modern techniques such as RoPE instead of absolute positional embeddings. While most performance gains likely stem from data and algorithm tweaks rather than structural changes, these specific design choices impact inference efficiency and training stability.
Engineers can access the model code via OpenAI's official model hub pages on Hugging Face. The availability of these weights allows for direct comparison with other architectures like Qwen3 and facilitates local experimentation without requiring proprietary API access.
Source: https://magazine.sebastianraschka.com/p/from-gpt-2-to-gpt-oss-analyzing-the
