AI Engineering

Notes on making probabilistic systems shippable: evaluation harnesses, LLM judges, context engineering, agents, and the GCP estate underneath them. Everything here comes from production systems — method extracted from work, written to be reused.

2026-08-05 · evals · harness · agents

Calibrating an LLM Judge Until Its Verdict Can Block a Release

Everyone builds an LLM judge. Almost nobody calibrates it. Here is what calibration actually looked like on a production agent system — and the three failures that taught me what a judge should never be asked to score.

Read →

Related: the case study on evaluation gates for a 700,000-student agent system, the GCP dossier, and the working papers on agent harnesses and structural waste.