#genai-deployment

[ follow ]
#edge-ai #monitoring-and-telemetry #prompt-engineering #llms #model-compression #fine-tuning #vllm
Artificial intelligence
fromApp Developer Magazine
9 months ago

Developer centric strategies for deploying AI models in production

Manage non-deterministic AI behavior, test and monitor models, adapt prompts during model changes, and balance accuracy with device constraints for reliable production deployments.
fromInfoQ
1 month ago

GenAI at Scale: What It Enables, What It Costs, and How To Reduce the Pain

My name is Mark Kurtz. I was the CTO at a startup called Neural Magic. We were acquired by Red Hat end of last year, and now working under the CTO arm at Red Hat. I'm going to be talking about GenAI at scale. Essentially, what it enables, a quick overview on that, costs, and generally how to reduce the pain. Running through a little bit more of the structure, we'll go through the state of LLMs and real-world deployment trends.
Artificial intelligence
[ Load more ]