Skip to content
Recursive Intelligence home
Book a call (opens in new tab)

The AI Abstract — Morning Edition

An agent I built wrote this brief from sources I picked, and nobody edited it before it went up.

AI/MLLatest

Making the Future Evenly Distributed.

A medical AI model can be manipulated into ignoring dangerous clinical requests — and the manipulation is invisible in its outputs, but readable in its internal states.

A deployed medical language model can be fed a prompt injection that overrides its safety constraints, produce output that looks completely normal, and you would never know from reading the response. The manipulation leaves no surface trace. What researchers just showed is that it does leave an internal trace, and you can read it.

The technique, described in 🔬a new preprint on clinical LLM monitoring, works by translating the model's internal activations into natural language in real time. Think of activations as the model's moment-to-moment working state, the pattern of signals firing across millions of connections as it processes a request. Normally those signals collapse into a single output token and the intermediate state is discarded. This method intercepts that state before it disappears and runs it through a secondary channel that converts it into readable text, a kind of transcript of what the model was "thinking" before it answered. That transcript exposes adversarial manipulations that the final output conceals. The system detects prompt injections and unsafe clinical requests better than output monitoring alone, and critically, it requires no retraining of the underlying model. You wire it on top of what you already have.

The stakes here are specific. A medical LLM that is compromised by an injected instruction might recommend a contraindicated drug or omit a critical warning, and the clinician reading the response has no way to know the model was operating under altered constraints. Output-only monitoring fails because a well-crafted injection can suppress the signals that would normally trigger a safety flag. Reading the internal state bypasses that suppression. This is a meaningful advance for anyone thinking seriously about clinical deployment, where silent failures carry patient harm risk that is different in kind from a chatbot giving a bad recipe.

The day's other lead story sits one level up in the training stack, before deployment. When you train a language model using reinforcement learning, you need a reward signal: some way to tell the model whether a given response was good or bad. The standard approach is to compare outputs against known correct answers, which works fine until you run out of known correct answers, which happens fast in any domain where you want the model to generalize. The usual workaround is to use another LLM as a judge. The problem is that LLM judges produce scalar scores, single numbers, as if they were always equally certain. They are not. A judge that assigns "0.8" to a borderline answer and "0.8" to an obviously correct one is feeding the same training signal for very different situations, which destabilizes learning.

🔬RLCDR fixes this by replacing the scalar with a probability distribution. Instead of "this answer scores 0.8," the judge says "I think there's a 60% chance this is correct, 30% chance it's partially correct, 10% chance it's wrong," and the training process learns from the full shape of that uncertainty. The result on MATH500 is a 4.65 percentage point gain (76.8% versus 72.2% baseline), with no reward hacking and stable calibration across training runs. Reward hacking is what happens when a model learns to game the scoring system rather than actually improving: it finds inputs that make the judge give high scores without doing the underlying task correctly. The calibrated distribution makes that harder, because a confident wrong answer looks very different from an uncertain right one. This is the kind of fix that compounds: better reward signals during training mean every downstream task the model is applied to gets a more reliable foundation.

The agent architecture cluster this payload carries is worth reading together. 🔬FinVeriQA (cluster size 4) makes an empirical argument that has broad implications beyond finance: when you need an agent to be reliable, stop asking the LLM to do everything and route tasks to purpose-built components. In their financial QA system, task-specific routing beat LLM-based routing, Python beat LLMs at arithmetic (which should not surprise anyone), and an independent verification step caught 35 to 77 percent more errors than checking consistency after the fact. That range is wide because the gains depend on the type of error, but even the low end is a substantial catch rate for a step that most deployed pipelines skip entirely.

The retrieval paper 🔬MSS-Complement addresses a quieter failure mode in coding agents. Standard retrieval ranks documents by similarity to a query, essentially asking "what does this look like?" But a coding agent working through a multi-step task needs something different: "what am I still missing?" Those are not the same question, and optimizing for the first one leaves gaps that the second would catch. MSS-Complement treats retrieval as a set-completion problem rather than a similarity problem, and recovers 73 to 80.6 percent of complete evidence sets against 61.4 to 72.4 percent for embedding and reranking baselines. The agents cluster has been building for three days across this tracker; the FinVeriQA and MSS-Complement papers are part of the same underlying pressure toward more structured, decomposed agentic architectures.

🔬SkillPlumber addresses a different agent failure: an agent that keeps making the same mistake because it cannot tell whether the mistake came from bad instructions or bad execution. The system diagnoses failures at the instruction level, distinguishing between "this instruction was wrong," "this instruction was right but I didn't follow it," and "this situation wasn't covered." That three-way split enables targeted correction rather than wholesale retraining of a skill, with 8.7 to 14 percentage point improvements across benchmarks.

Two results worth a sentence each. 🔬STEPQuant achieves 5x compression of linear attention model states with accuracy maintained, relevant if you are serving these models at scale and paying for memory. 🔬A confidence calibration study across 11 open-weight models finds that reading the model's output logits directly produces dramatically better uncertainty estimates (AUROC 0.74 to 0.77) than asking the model to verbalize its confidence (AUROC 0.57 to 0.66), and a decision-only scoring variant cuts decoding compute by 88 to 96 percent with comparable calibration. If you're running classification pipelines, the method you are probably using is the worse one.


🔬 Clinical LLM Internal State Monitoring: Read for the mechanism of how activations are verbalized and what the detection gap looks like between internal and output monitoring.

🔬 RLCDR: Calibrated Decision Rewards: Read for a clear explanation of why scalar reward signals break under distributional shift and how calibrated distributions address it.

🔬 FinVeriQA: Read for the empirical breakdown of which agent subtasks benefit from specialization versus general LLM handling, with numbers attached to each decision.

🔬 MSS-Complement: Read for the reframing of retrieval as set completion, which changes what you should be optimizing for in any multi-step agentic task.

🔬 LLM Confidence Calibration Study: Read before choosing a confidence estimation method for any production classification pipeline.

Links

  1. Calibrated Decision Rewards: Training Language Models with a Judge That Knows How Sure It Is

    arxiv.org

    Researchers introduce RLCDR, a method to train language models using calibrated probability distributions from LLM judges rather than exact reward signals, achieving +4.65 point gains on MATH500 (76.8% vs 72.2% baseline) with demonstrated calibration stability and no reward hacking. This directly addresses a core bottleneck in modern RLHF/GRPO workflows: how to scale reward signals beyond reference answers while maintaining interpretability and preventing distributional shift collapse.

  2. What the model knows but does not say: monitoring medical LLMs through verbalized thoughts

    arxiv.org

    Researchers developed a method to translate LLM activations into natural language ('thoughts channel') that detects prompt injections and unsafe clinical requests better than output monitoring alone, requiring no model retraining. This advances interpretability-based safety detection for deployed medical models and demonstrates that internal states can expose manipulations that outputs conceal—critical for clinical deployment of LLMs where silent failures carry patient harm risk.

  3. STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization

    arxiv.org

    STEPQuant introduces spatial-temporal post-training quantization for recurrent states in linear attention models, achieving 5x compression on large language models while maintaining accuracy. This directly addresses serving efficiency for practitioners deploying linear-attention models at scale, with integrated GPU kernels and open-source implementation reducing deployment friction.

  4. The Missing Complement: State-Conditioned Minimal Sufficient Evidence for Coding Agents

    arxiv.org

    Researchers propose MSS-Complement, a set-construction approach to retrieval for coding agents that prioritizes sufficiency over relevance ranking, recovering 73-80.6% of complete evidence sets vs. 61.4-72.4% for embedding+reranking baselines. This reframes a core bottleneck in agentic systems: retrieval for agents should optimize for what a decision lacks, not what a query resembles—directly applicable to deployed coding agent architectures.

  5. FinVeriQA: Financial Research Agents for Question Answering: Routing, Retrieval, MCP Tool Use, and Provenance-Aware Evidence Verification

    arxiv.org

    FinVeriQA is an open-source multi-agent system for financial question-answering that decomposes tasks across specialized components (routing, retrieval, computation, verification) rather than relying on LLMs end-to-end. The work demonstrates that task-specific routing beats LLM-based routing, Python beats LLMs at arithmetic, and independent evidence verification catches 35-77% more errors than post-hoc consistency checks—a finding with broad implications for building reliable agentic systems.

  6. Evolve Where It Matters: Coverage-Guided Self-Evolution of Agent Skills

    arxiv.org

    SkillPlumber introduces a framework for self-evolving agents that diagnoses skill failures at the instruction level (follow/not-follow/not-covered) rather than outcome-only, enabling targeted skill revision through root-cause analysis. This advances a key capability for autonomous agents: the ability to distinguish between incorrect instructions and poor instruction-following, with 8.7-14 percentage point improvements over prior methods across multiple benchmarks.

  7. LLMs Are Already Confidence-Aware Zero-Shot Classifiers

    arxiv.org

    Researchers compared black-box and white-box confidence estimation methods for zero-shot LLM classifiers across 11 open-weight models, finding logit-based scoring dramatically outperforms verbalised confidence for uncertainty quantification (AUROC 0.74-0.77 vs 0.57-0.66). Introduces decision-only scoring that reduces decoding by 88-96% with comparable calibration—directly applicable to production classification pipelines.

  8. Clause-Factored Bilateral State Grounding and Scoped Passivity for Counterfactual Open-World Deductive Reasoning

    arxiv.org

    Researchers propose TE-BPHD, a 37.85M-parameter system decomposing open-world deduction into clause-factored grounding and deliberation operators, achieving 79.17% counterfactual consistency on ProofWriter (+23-39pp over baselines) and 94.95% OOD accuracy while reducing latency 1,500x versus LLM reasoning. This advances the frontier of structured deductive reasoning without autoregressive cost, directly relevant to practitioners building robust reasoning systems and researchers exploring hybrid neural-symbolic architectures.

  9. NL2AGBench: Benchmarking LLM Auto-Formalization for AlphaGeometry

    arxiv.org

    Researchers introduce NL2AGBench, a benchmark for evaluating LLMs' ability to translate natural-language geometry problems into AlphaGeometry's formal domain-specific language, revealing significant performance gaps between closed-source (80%+ executable translation) and open-source models. The work addresses a real usability bottleneck in neuro-symbolic systems and provides an error taxonomy plus mitigation strategies (few-shot, fine-tuning, hinting) applicable across model families.

  10. STARK: Moving towards Personalized Agents That Truly Learn and Remember

    arxiv.org

    STARK introduces a dual-graph memory architecture (Episodic + Insight graphs) that treats events as edges between behavioral insights, enabling personalized agents to maintain and retrieve grounded behavioral patterns without repeated LLM traversal. This advances the state of long-context agent design with demonstrated improvements on LongMemEval and LoCoMo benchmarks, plus cross-user transfer mechanisms relevant to cold-start personalization problems practitioners face.