Agent + Local Model + Evals — One‑Pager
🚀 Purpose If you’ve ever wondered how to compare a locally‑hosted LLM with a hosted OpenAI model side‑by‑side, while keeping the whole experiment observable from end‑to‑end, you’re in the right place. This one‑pager walks you through the starter repo that stitches together three distinct paths: Path Stack Observability A .NET 8 + Semantic Kernel + Ollama (local model) Azure Monitor (OTEL) B OpenAI Agents SDK (TypeScript/Node) OpenAI Logs → optional OTLP → Azure Monitor C Python Evals (OpenAI‑provided) Same prompts, same metrics The goal? A clean, reproducible harness that lets you run the same prompt against a local model and an OpenAI model, invoke tools, and then score the outputs with a single Python‑based eval suite. All of this is fully instrumented with OpenTelemetry, so you can follow the request flow from the client code right into Azure Application Insights (or any OTEL backend you prefer). ...
