Lunary

Lunary is an observability and evaluation tool for AI applications built on large language models. It helps teams understand how their AI behaves in production by collecting key metrics, logs, and user behavior.

LLM and chatbot observability

Lunary tracks model requests and responses, errors, and latency so developers can see where quality changes over time and where users get stuck.

• Request/response logging for LLM calls
• Error and latency monitoring
• Chatbot-focused dashboards to spot gaps between user intent and model output

Prompt management and experiments

Store and version prompts, compare performance, and run A/B tests to iterate faster on prompt wording and model configurations.

• Prompt library with version control
• Prompt performance comparison
• A/B testing for prompts and configurations

Quality evaluation and product analytics

Combine automated and manual evaluations, label conversations, and analyze quality by scenario. Product metrics help connect LLM behavior to business outcomes.

• Automated and human review workflows
• Conversation labeling and scenario-based analysis
• Metrics tied to retention, conversion, and successful sessions