Evals

LLM customer with a hidden goal, 13-tool agent, reward gated on the database diff before a judge checks consent and honesty. Red-teaming caught a forged-consent hole in the judge.

Evaluator for an account-risk agent: 29 spec rules checked against corrected telemetry and source text. 259 of 629 numeric claims failed to reproduce.

Induced sycophancy in Qwen3-8B, then compared DPO, SimPO, IPO, GRPO and Constitutional AI. DPO fooled the judge while a probe still found the disposition; CAI preference pairs did best.