Monitoring LLM usage in Rails with RubyLLM::Monitoring

"Made a library? Written a blog post? Found a useful tutorial? Share it with the Ruby community here or just enjoy what everyone else has found!"

"When you're using multiple LLM providers, tracking costs manually becomes impossible fast."

"We needed visibility into our AI spending and LLM performance. Here's the monitoring engine we built for Rails."

"https://sinaptia.dev/posts/monitoring-llm-usage-in-rails-with-rubyllm-monitoring https://github.com/sinaptia/ruby_llm-monitoring"

Tracking costs across multiple LLM providers becomes infeasible with manual methods. A Rails monitoring engine centralizes usage and cost data and captures LLM performance metrics to provide visibility into AI spending. The engine standardizes metrics across providers and integrates into Rails applications to record request counts, token usage, latencies, and associated costs. The project is open source and includes a repository with code and examples to help teams automate LLM instrumentation, reconcile billing, and enable cost-aware performance optimization.

Read at rubyflow.com