LLMs1 min read
The CyberGym benchmark, designed for evaluating AI security, is now publicly available on GitHub. This allows for high-score competition and potential model weight sharing on Hugging Face.
By OpenSmartRoute editorial · written through the router by writer-small
From Simon Willison - “Quoting huggingface.co/security.txt”
The Hugging Face security team has released the CyberGym benchmark. This benchmark is accessible via GitHub. It is intended for evaluating vulnerabilities within AI models. The team encourages users to compete for high scores using the benchmark. Weights generated during this process can be shared on Hugging Face.
Source: https://simonwillison.net/2026/Sep/11/hugging-face-security/
Keep reading
LLMs1 min read
Research investigates the reliability of LLMs when generating structured outputs that adhere to database constraints. A new benchmark and multi-agent framework, MARS, demonstrate improvements in schema normalization tasks compared to single-prompt approaches.
LLMs1 min read
OpenRouter automatically selects the most cost-effective backend provider for model requests, handling variations in serving software and capabilities. Using the provider.only option allows control over routing for specific model IDs.
ollama/gemma3:4beditorial-writerDrafted automatically from the source and reviewed automatically; it may contain errors. Refer to the source for the authoritative text. How routing works
LLMs1 min read
NCP-ArchPreview, a 8.9B parameter model, achieves competitive performance through a combined approach of next-token prediction and next-concept prediction. This architecture demonstrates significant efficiency gains and offers adaptable latent space capabilities for downstream tasks.