A study evaluated a 0.6B language model asked to verify logical conclusions, finding it always answers YES despite discriminative signals in hidden states. Linear probes on hidden states achieve high AUC scores, but the model's own output logits show a calibration mismatch.
The dominant failure was identified as a single scalar offset in the verdict readout, which can be corrected with a simple parameter adjustment. This correction improved behavioral accuracy from 50% to 81%, and calibrated margin decoding reached 94% accuracy at larger scales.
The analysis distinguishes between concealed, miscalibrated, and undetected verdict regimes, providing actionable insights for model calibration. It also shows that answer-surface features can reproduce probing results without internal access, highlighting potential pitfalls in interpretability methods.
This work emphasizes the importance of calibration in model verdicts and demonstrates methods to improve reliability in model outputs, relevant for deploying models in production.
Source: https://arxiv.org/abs/2609.04582