Nums AI released Causilo, a pretrained foundation model designed for classification and regression tasks. The model utilizes an in-context learning approach where calling fit does not update weights; instead, training rows are stored as context to predict query rows in a single forward pass.
The architecture consists of three phases: refinement, compression, and in-context learning. Features are grouped into sets of three, embedded with sine and cosine frequencies, and summarized by two column stages using 128 latent slots. A pooling block compresses rows into 512-dimensional vectors before a 12-layer prediction block processes queries.
On TabArena, Causilo achieved the highest Elo among single models for both classification (1794) and regression. While system ensembles place it third overall, it leads specific tasks like regression. ScoringBench confirmed its top ranking based on CRPS, R², and RMSE across 101 datasets.
