Technical

GRPO achieves the lowest aggregate sycophancy (0.169) and an 8% flip rate — but linear probing finds only 7 of 36 layers above chance (AUROC 0.541). The best behavioral fix uses the most diffuse internal mechanism.

IPO restructured the model more deeply than SimPO (probe transfer 0.365 vs 0.429) but performed worse behaviorally (0.281 vs 0.176). The reference model doesn’t limit intervention depth — the loss shape does. A 2x2 framework replaces the 1D hypothesis.

DPO suppresses sycophancy but preserves the internal representation (probe transfer 0.677, p=0.005). SimPO reorganizes it — probe drops to chance (0.503, p=0.154 after correction). Same data, same model, radically different internals.

I deliberately induced sycophancy in Qwen3-8B, recovered it with DPO, then used linear probing to show the internal sycophancy representation survived alignment. The fix is cosmetic.

A technical deep dive into DeepSeek’s Manifold-Constrained Hyper-Connections (mHC), exploring how Doubly Stochastic Matrices and the Birkhoff Polytope solve gradient instability while expanding network capacity.

Deep dive into SARSA, a foundational on-policy TD control algorithm for reinforcement learning.

Understanding the TD learning update rule and bootstrapping in reinforcement learning.

Guide to Monte Carlo methods in RL: learning from complete episodes and full returns.