Observability

A timeline titled Trust the model the least: my public trail before this weekend. Dated posts: Aug 10, 2025 a local model and a hosted model side by side, fully traced; Mar 28, 2026 evals: the agent says done, did it do anything; May 16 public audit-trail receipts, same receipt frontier or local; Jun 7 your agent said it did the work, I checked the disk; Aug 1 Receipts, Evals and the Loop on adotob.com/receipts; Aug 2 every task replayed on a local model, both graded; Oct 4 grading my own local AI by hand. Then Oct 10, 2026: Satya Nadella, Models as Insider Risks in the Super Intelligence Era.

Trust the Model the Least. I've Been Building That at Home for Months.

BLUF (bottom line up front): This weekend Satya Nadella published Models as Insider Risks in the Super Intelligence Era. His line: “The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least.” I agree, and I didn’t have to start this weekend. On my own hardware and my own time I’ve been running that architecture for months, and I’ve been writing it up in public as I went. ...

A six-agent fleet dashboard showing each agent, its model, cron jobs, and reporting output

Your Agent Said It Did the Work. I Checked the Disk.

Every morning one of my agents sends me a clean status report. Posts cross-posted. Messages delivered. Contacts processed. For a while I read those reports the way you read a receipt from a cashier you trust. Then the automation started giving me time back, so I sat down to run a retrospective and checked the reports against what was actually on disk. The trust did not survive contact with the evidence. ...

The per-iteration arc: four stages (Observe / Announce / Act / Verify) with four required artifacts under Observe. The artifacts are Sensor Roll Call, Conflict Receipt, Gap Map, Cross-Source Dedup.

Sense Before Act: Four Artifacts Every Agent Iteration Must Produce Before It Decides

By the end of this post you will know what every iteration of every production agent must produce BEFORE it decides anything: four small artifacts that catch the failures sharper prompts cannot prevent. You will know why “tell the model to be careful” never worked as a defense, and why the only durable fix is structural. And you will know what the discipline looks like in code, because we have been running it at MACONA for months and the receipts are public. ...

Five-box diagram of the agent infrastructure control points (Runtime, Identity, Data, Tool/Write, Observability) with the Agent Reliability Kit highlighted on the Observability box, plus a multilayer kill-switch strip across the bottom

Paying Down Supervision Debt: Why the Five Control Points That Decide Whether Your Agent Ships Have Nothing to Do With Your Model

By the end of this post you will know which of the five infrastructure control points your agent stack is shipping without, and you will have a concrete pattern for paying down the observability piece of that debt: a public, consumer-readable audit-trail receipt that satisfies both your security team and your finance team with the same document. The receipt was already in production when the broader practitioner conversation started naming the gap. ...

Side-by-side comparison of two audit-trail receipts — Claude Desktop on left, LM Studio with Qwen3.6 27B on right — both produced by the same MCP endpoint with identical six-check audit trail

One Agent Receipt, Two Buyers: Why Protocol-Neutral MCP Audit Trails Matter for Both Security AND Finance

Update, 2026-05-19: A2A agent-card now live at mcp.adotob.com/.well-known/agent.json, published 24 hours after Nate B Jones’s IO-2026 video named the agent-card primitive as the second of the four core agent-protocol layers. Three of the four layers of the open-protocol stack are now live in the storefront: MCP for tool access, A2A for agent discovery, and AG-UI manifested as the public receipt page. AP2/X402 is reserved for the MVP-2 paid Stripe flow. By the end of this post you’ll have two public URLs you can forward to your security team AND your finance team — and both audits get satisfied by the same document. That document is an agent receipt. The two URLs are real receipts produced by the same MCP endpoint, one driven by Anthropic-hosted Claude Desktop and the other driven by Qwen3.6 27B running fully offline on my MacBook. The point of this post is why that one-artifact-two-ledgers property matters more than the technology that produces it. ...

Agent + Local Model + Evals — One‑Pager

🚀 Purpose If you’ve ever wondered how to compare a locally‑hosted LLM with a hosted OpenAI model side‑by‑side, while keeping the whole experiment observable from end‑to‑end, you’re in the right place. This one‑pager walks you through the starter repo that stitches together three distinct paths: Path Stack Observability A .NET 8 + Semantic Kernel + Ollama (local model) Azure Monitor (OTEL) B OpenAI Agents SDK (TypeScript/Node) OpenAI Logs → optional OTLP → Azure Monitor C Python Evals (OpenAI‑provided) Same prompts, same metrics The goal? A clean, reproducible harness that lets you run the same prompt against a local model and an OpenAI model, invoke tools, and then score the outputs with a single Python‑based eval suite. All of this is fully instrumented with OpenTelemetry, so you can follow the request flow from the client code right into Azure Application Insights (or any OTEL backend you prefer). ...

Debugging GenAI with NLWeb and OpenTelemetry: A Real-Time Visibility Win

🧠 Debugging GenAI with NLWeb and OpenTelemetry: A Real-Time Visibility Win “If you can’t see it, you can’t debug it.” That’s been the challenge with GenAI and Copilot-style plugins — until now. 🎯 Goal I wanted to trace and troubleshoot my Copilot agent plugin activity using OpenTelemetry. The stretch goal? Pipe that into a natural-language search experience using NLWeb, so I could ask questions like: “What plugins were used?” “What calls failed?” “How long did token generation take?” ✅ As of today — we did it. Here’s how. ...