LLMs1 min read
NCP-ArchPreview: Latent Space Language Model with Next Concept Prediction
NCP-ArchPreview, a 8.9B parameter model, achieves competitive performance through a combined approach of next-token prediction and next-concept prediction. This architecture demonstrates significant efficiency gains and offers adaptable latent space capabilities for downstream tasks.
From arXiv cs.CL