#vision-language-models

#vision-language-models

Businesses are generating more video than ever. From years of broadcast archives to thousands of store cameras and countless hours of production footage, most of it just sits unused on servers, unwatched and unanalyzed. This is dark data: a massive, untapped resource that companies collect automatically but almost never use in a meaningful way. To tackle the problem, Aza Kai (CEO) and Hiraku Yanagita (COO), two former Googlers who spent nearly a decade working together at Google Japan, decided to build their own solution.

Since AlexNet5, deep learning has replaced heuristic hand-crafted features by unifying feature learning with deep neural networks. Later, Transformers6 and GPT-3 (ref. 1) further advanced sequence learning at scale, unifying structured tasks such as natural language processing. However, multimodal learning, spanning modalities such as images, video and text, has remained fragmented, relying on separate diffusion-based generation or compositional vision-language pipelines with many hand-crafted designs.