Langtrace – OpenTelemetry-Based LLM App Observability

Hacker News

Hey HN, Ola and Karthik here. We are working on Langtrace( https://github.com/Scale3-Labs/langtrace ), an open source, open telemetry based SDK and monitoring/evaluations client for LLM based applications. The SDK generates OTEL standard spans and traces for popular LLMs like OpenAI, Anthropic and Cohere, popular frameworks like Langchain and LlamaIndex and vectorDBs like ChromaDB and Pinecone. The LLM monitoring/evaluations space has seen a number of products off late, both open source and closed source ones. But, a couple of things we have observed are: lack of standard spans and traces that creates vendor lock in, different tools are optimized for solving different pain points - Evaluations, Prompt management, Datasets etc. We believe that adopting open telemetry(OTEL) standard tracing not only lets teams use our SDK without having to switch their observability client, but will also enable developers to develop tooling for any custom needs such as capturing datasets, prompts, evaluations etc. A quick note on what we have built so far: [1] We have a Python and a TypeScript SDK and we have broken down the support for the LLM layer into 3 groups, LLMs, Frameworks and VectorDBs. Our SDKs are open telemetry compatible, can be installed and used independently and we also provide an option to pass custom exporters to export the traces and spans to any observability tool of your choice. [2] An observability client that is hyper optimized for solving the unique pain points and challenges that come with LLM based apps like Evaluations, prompt iteration, datasets etc. We are SOC2 compliant and the client can also be self hosted if you have strict data privacy and protection requirements. [3] Both the SDK and the client are fully open source. We are leaning on the community to try it out and provide feedback. A note about Open Telemetry semantic conventions for LLMs - We would like to converge on standard names for trace attributes that follow the OTEL rules and are looking for feedback from experts here - https://github.com/Scale3-Labs/langtrace/discussions/71 We recognize that this project is early and there is a lot of room for improvement. Would love to hear your thoughts and feedback. Thanks! Links: [1] https://langtrace.ai/ [2] https://github.com/Scale3-Labs/langtrace [3] https://docs.langtrace.ai/introduction [4] https://langtrace.ai/blog/why-you-need-opentelemetry-based-o...

Visit product View on Hacker NewsApr 25, 2024

Share card

Actual performance

9points

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Indie HackersFits the IH revenue-focused audience · Strong signals: compatible · Missing: supports, reddit linkedin, podcasting

94%94% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Product HuntOn track for Day 1 leaderboard · Strong signals: apps, openai, open · Missing: mac, agents, macos

82%82% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: lua, open source, llama · Missing: https docs, excited, just released

67%67% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Strong signals: apps · Missing: mobile apps, ios, personal

36%36% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Strong signals: host · Missing: plus, platform, intuitive

31%31% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

11%11% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Langtrace – OpenTelemetry Based LLM App Observability63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Langtrace – OpenTelemetry Based LLM App Observability

Hacker News2

Langtrace – OpenTelemetry-Based Client & SDK for LLM App Observability51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Langtrace – OpenTelemetry-Based Client & SDK for LLM App Observability

Hacker News6

LLM Observability Platform65%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

LLM Observability Platform

Hacker News1

All you need is Prometheus and Jaeger for LLM Observability58%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

All you need is Prometheus and Jaeger for LLM Observability

Hacker News3

Promptive49%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

LLM observability for builders who can't afford surprises.

Indie Hackers1ai

OpenLLMetry – OpenTelemetry-based observability for LLMs67%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

OpenLLMetry – OpenTelemetry-based observability for LLMs

Hacker News154

MergN53%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

n8n’s observability, LLM’s flexibility

Indie Hackerscommitment-full-time

PDFAgent with Observability55%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

PDFAgent with Observability

Hacker News4

OpenLIT – Open-Source LLM Observability with OpenTelemetry69%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

OpenLIT – Open-Source LLM Observability with OpenTelemetry

Hacker News62

OpenLIT – Open-Source LLM Observability with OpenTelemetry

Hacker News1

Browse all products Analyze your launch