Ministral 3 Family Uses Cascade Distillation Method

Mistral released the open-weights Ministral 3 family (14B, 8B, and 3B parameters), compressed from a larger model using a new pruning and distillation method called cascade distillation. Despite their smaller size, the vision-language models rival or beat similarly sized competitors on several benchmarks while requiring far less training data and compute. Learn more in The Batch: https://hubs.la/Q042TGnk0