#multimodal-models

#multimodal-models

The launch comes as Mistral, which develops open-weight language models and a Europe-focused AI chatbot Le Chat, has appeared to be playing catch up with some of Silicon Valley's closed source frontier models. The two-year-old startup, founded by former DeepMind and Meta researchers, has raised roughly $2.7 billion to date at a $13.7 billion valuation - peanuts compared to the numbers competitors like OpenAI ($57 billion raised at a $500 billion valuation) and Anthropic ($45 billion raised at a $350 billion valuation) are pulling.

Over the long term, it will commoditize - over the next couple of years,

These experiments led to two key discoveries, according to the paper. Tuning only the self-attention projection layers (SA Proj), the part of the model that helps it decide which input elements to focus on, allowed the models to learn new tasks with little or no measurable forgetting. Also, what initially appeared as forgotten knowledge often resurfaced when the model was later trained on another specialized task.

The way we see it, the real race for AI video hasn't begun. Our new identity, Mirage, reflects our expanded vision and commitment to redefining the video category, starting with short-form video, through frontier AI research and models, CEO Gaurav Misra told TechCrunch.

We delved into the five pretraining datasets of 34 multimodal vision-language models, analyzing the distribution and composition of concepts within, generating over 300GB of data artifacts that we publicly release.