Tools & Platforms
The LLMOps landscape by category - filterable by what actually drives the decision: open source, self-hostable, OpenTelemetry, compliance and team fit. Editorial, not paid placements.
Attributes last verified June 2026. Attributes (especially SOC 2* and OpenTelemetry) are best-effort and change often - verify with the vendor before relying on them for procurement.
Logs, traces, latency and token usage for LLM applications.
Maps to the Observability layer of the stack.
Langfuse
Open-source tracing, evals and prompt management.
LangSmith
Tracing and evals from the LangChain team.
Helicone
Proxy-based logging, caching and cost tracking.
Arize Phoenix
Open-source LLM tracing and evaluation.
Fiddler AI
Model monitoring and observability platform.
Datasets, scoring and regression testing for prompts, RAG and agents.
Maps to the Evaluation layer of the stack.
Braintrust
Eval and experimentation platform for LLM apps.
LangSmith
Datasets, LLM-as-judge and offline evals.
OpenAI Evals
Open-source framework for model evaluations.
Giskard
Open-source testing for ML and LLM systems.
Versioning, change testing and rollback for prompts and configs.
Maps to the Prompt management layer of the stack.
PromptLayer
Prompt registry, versioning and analytics.
Humanloop
Prompt management, evals and collaboration.
Langfuse
Versioned prompts alongside traces and evals.
Vector storage and retrieval infrastructure for grounded answers.
Maps to the RAG operations layer of the stack.
Pinecone
Managed vector database at scale.
Weaviate
Open-source vector database with hybrid search.
Qdrant
Open-source vector search engine.
Redis
Vector search on top of in-memory data.
Input/output validation, PII, injection and safety controls.
Maps to the Security layer of the stack.
Serving, scaling and shipping LLM applications to production.
Maps to the Deployment layer of the stack.
BentoML
Open-source model serving and packaging.
Modal
Serverless compute for AI workloads.
Replicate
Run and deploy models via API.
Databricks
Data + AI platform with model serving.
Token accounting, caching and routing to keep spend in check.
Maps to the Cost control layer of the stack.
Helicone
Per-request cost tracking and caching.
Langfuse
Cost and token usage per trace and feature.
Portkey
AI gateway with routing, caching and budgets.
No tools match all selected filters.
Missing a tool, or an attribute out of date?
This directory is community-maintained and vendor-neutral. Suggest a tool, or flag an attribute we should correct.
Contribute or correct →