Doubleword focuses on inference, the process of running AI models. Clients often use multiple inference providers, including OpenAI, Mistral, and self-hosted fine-tuned models, which creates operational chaos. AI model gateways bring order to environments with many model providers by providing a unified access layer. Doubleword built an open-source AI model gateway after repeatedly seeing this problem. The gateway is not sold as a commercial product, but it is presented as important for organizations. The goal is to encourage every organization to use an AI model gateway, even at small deployment scales, to improve consistency and control over inference.
"We started Doubleword about four years ago, always focused on the problem of inference. The inference process being the process of actually running the models. What we kept seeing with our clients is we were providing inference and inference services to them, but we typically weren't the only inference provider they were using. They were probably using OpenAI and maybe Mistral and maybe some self-hosted fine-tuned models that they built themselves. We saw them getting into a bit of a chaos situation with all of these different providers. We ended up having to try and fix that for them."
"AI model gateways are a really easy way to bring order to a chaotic environment where you have a lot of different model providers. Because we kept seeing this problem over and over again, we actually built an open-source AI model gateway. We have experience building these things from the ground up. We don't sell it. It's not a commercial project of ours, but we think that AI model gateways are very important. We think everyone should be using them."
"Do you currently have an AI model gateway in your organization? Something like LiteLLM, OpenRouter. I'm going to try and convince you that every single person should have an AI model gateway, even if you're deploying it quite small scale. That is my goal of this session."
#ai-inference #model-gateways #multi-provider-orchestration #open-source-tooling #operational-control-layers
Read at InfoQ
Unable to calculate read time
Collection
[
|
...
]