Evals on Narasimha Karthik J

Customer-Service Agent RL Environment

Mon, 05 Oct 2026 00:00:00 +0000

What it is #

Two LLMs talk to each other. A user simulator plays a customer with a hidden goal and sees only what the agent says. An assistant agent follows a written store policy and uses 13 tools that read and change an in-memory database built from the τ²-bench retail data (25 users, 45 orders, 50 products).

The tools check only data integrity, not policy. So a policy violation changes the database, and the reward can see it.

The reward #

Gate on state. The agent’s database diff must match a correct outcome, and the agent must have made the required lookups. A confident transcript with a wrong write scores zero, and a do-nothing agent can’t pass tasks where the right answer is “no write”.
Efficiency. Fewer wasted tool calls score higher.
Quality, by an LLM judge. The judge checks consent before each write and whether every claim of a finished action matches the tool log. It asks one yes/no question per call and quotes evidence before its verdict. Hard violations cap the reward below the floor of any clean run.

A pytest suite checks that golden runs pass and do-nothing runs fail, with no API calls.

What red-teaming found #

Forged consent. In a hand-written bad conversation, the customer’s text contained fake “AGENT: … CUSTOMER: Yes” lines, and the judge passed the consent check. Feeding the judge speaker-tagged messages fixed it.
Unfair simulator. The user simulator once agreed and sent its stop token in the same message, so a correct agent scored 0.
Against a judge-aware hostile customer, a Sonnet agent scored 1.0 and a Haiku agent scored 0, after handing off to a human following a correct denial.

These come from one or two runs per task, not pass rates.

Sycophancy Recovery Study

Thu, 28 May 2026 00:00:00 +0000

Code on GitHub

Question #

If you train a bad behavior into a model and then train it out, is it gone, or just hidden from the outputs?

Setup #

I induced sycophancy in Qwen3-8B with LoRA SFT, then tried five ways to remove it: DPO, SimPO, IPO, GRPO and Constitutional AI. A Qwen2.5-72B judge scored behavior over 20,656 prompts. Linear probes and relearning speed measured what changed inside the model.

Findings #

DPO hid it. DPO scored 0.268 against 0.256 for the untouched base, so it looked fixed. But a probe trained on the sycophantic model still read the DPO model at 0.677 AUROC, and the DPO model relearned sycophancy in 5 gradient steps where the base needed 50+. Correlational: no activation patching yet.
Constitutional AI won. With the same DPO trainer and only the labels swapped for critic-revised pairs, sycophancy dropped to 0.166, the best in the study.
GRPO’s reward design mattered. A binary reward left half the groups with no signal. A scalar reward model reached 0.169, at a cost in plain accuracy.

Write-ups #

Spec-Conformance Evals for an Account-Risk Attention Agent

Sun, 13 Sep 2026 00:00:00 +0000

Code on GitHub

What it is #

An AI-engineer assessment from Signal Labs. A fictional company runs an agent that watches 180 customer accounts and decides which risk signals a human should see. The agent’s behavior is defined in a formal spec as a state machine with invariants, timing targets, policy rules and grounding rules. The data is 629 agent dossiers, 32,032 days of telemetry, 3,810 text artefacts, renewal outcomes, and labels from three annotators who disagree.

I built a SignalEvaluator that scores each dossier for quality, risk, whether it deserved attention, and the specific spec rules it broke. On top of it I wrote a violation report, an attention-budget analysis and a writeup.

How it works #

Spec to rules. I transcribed the spec into a table of 29 rules, each with its spec identifier, section, severity class and owning check. Tests parse the LaTeX source of the spec and fail if a rule cites a section that does not exist.
Correct the data first. The evaluator rebuilds telemetry before checking any claim: later backfill rows win over duplicates, a legacy collector’s double-counted API calls are halved, and a latency scale change is undone. Inspection found three more distortions the docs did not list, including zero rows that were collector outages, not dead accounts. Where the data is missing, the check refuses rather than scoring zero.
Source-anchored checks. Each numeric claim is recomputed from corrected telemetry. Each quoted piece of evidence is checked to exist, belong to the same account, and match verbatim. Text is read by rules plus a small local NLI model, with no external API calls inside evaluate().
Tests. A pytest suite includes 25 hand-derived golden dossiers.

Results #

259 of 629 dossiers state a percentage change that does not reproduce on corrected telemetry. In 58 of them the pipeline manufactured the decline; in 121 a real decline was overstated.
Only one rule separated complaints: customer-visible plays on legal-hold or M&A-quiet-period accounts drew complaints at 32% (10 of 31), against 3% for the rest. No rule separated wasted escalations.
On quality scores, the evaluator agreed with each annotator about as well as the annotators agreed with each other (Spearman 0.36 to 0.55, against 0.31 to 0.45 between humans). On “deserved attention” it fell below the human band.
A spec-anchored routing policy under a 5-signals-per-owner-week cap routed 189 signals against the agent’s 294. Its routed signals churned more often (39% against 30%), but it missed 16 accounts with $3.02M of realised loss.
Bursts in signal volume traced to a collector migration, a regional ingest gap, and four public holidays.