Course Format and Delivery

Delivery Method: LiveOnline
Schedule: Full day

All sessions are delivered live by an expert instructor in a fully interactive online environment.

*20% off for group bookings when booking 3 or more attendees from the same organization on the same course dates in the same transaction.

About this course

AI governance skill demand is up 150% year-over-year, yet most AI applications ship with zero observability. There's a documented 37% lab-vs-production performance gap and 50x cost variation for similar accuracy in agent deployments — problems that only surface with proper instrumentation.

This full-day workshop teaches participants to instrument, monitor, evaluate, and govern LLM applications using the converging observability stack: LangSmith for application tracing and evaluation (now with @decorator-style auto-trace, ~5 min setup), W&B Weave for full ML-lifecycle coverage, and RAGAS for automated RAG quality metrics.

Participants build a complete observability pipeline from auto-tracing through custom evaluation suites to automated quality gates.

This course is great for

For developers and ML engineers building LLM applications. Comfort with Python and basic LLM API experience required. Pairs naturally as a follow-on to the AI Agents, Multi-Agent, or Production RAG workshops.

Topics Covered

• Tracing architectures,

• LangSmith setup and analysis,

• W&B Weave integration,

• Custom evaluation metrics,

• Evaluation-driven development,

• RAGAS for RAG quality,

• Cost tracking,

• Quality gates for CI/CD,

• and red-teaming with promptfoo/garak.