AI/ML Reading List

Canada's banking regulator (OSFI) explicitly named Anthropic's Claude in an April email warning banks about AI risks and closing windows to remediate—unusual because regulators typically avoid naming vendors. This signals field-wide attention to frontier model governance and regulatory pressure on institutions deploying advanced AI systems.

A researcher built Research Radar, an open-source arXiv paper filtering tool that uses two-pass LLM scoring (cheap model for abstract screening, stronger model for deep PDF analysis) to surface papers relevant to specific research interests, with model-agnostic backends supporting Claude, OpenAI, and local Ollama. This matters because it directly addresses a pain point for active researchers and is immediately forkable/deployable—a concrete example of LLM-powered productivity infrastructure that's reproducible and generalizable across domains.

GPUHedge is an open-source tool that hedges requests across multiple serverless GPU providers using speculative execution, reducing p95 cold-start latency from 117s to 30s on a 17GB model benchmark while lowering per-request compute costs. This addresses a real operational constraint for ML practitioners deploying inference at scale and provides immediately usable code with clear performance deltas.

SPARK introduces a length-controlled hidden-state susceptibility method to diagnose and steer reasoning failures in LLMs at test time, separating input-scale effects from genuine reasoning activation deficits. Achieves 2.6pp and 3.2pp improvements on MATH-500 for Qwen3 models, advancing both mechanistic understanding and practical intervention for reasoning robustness—relevant to practitioners optimizing LLM performance and researchers studying latent reasoning dynamics.

GEIS introduces a declarative, modular skill-loop architecture for long-form article generation that decomposes the problem into inspectable, reusable components (writing, evaluation, improvement) and demonstrates 8-point gains over baselines and 4-point cumulative improvement through iterative skill patching. This matters because it reframes multi-agent workflows from opaque prompt-and-procedure chains into transparent, debuggable, and incrementally improvable systems—a key pattern for scaling agentic reliability in complex tasks.

A new arXiv paper reframes AI in high-stakes language testing as a validity problem rather than just an operational/security issue, introducing the concept of 'AI-mediated construct drift' and proposing 'bounded AI mediation' as a design principle. This matters because it signals how foundational assessment and credentialing systems must evolve as AI becomes ubiquitous in real-world language use—a field-wide signal about validity frameworks adapting to AI-augmented capability.

SWE-MERA introduces a dynamic benchmark for evaluating LLMs on software engineering tasks, replacing the contaminated SWE-bench (which suffers from 32.67% solution leakage and 31.08% inadequate test coverage). This matters because benchmark integrity is foundational to reliable capability claims—a cleaner evaluation standard will reshape how the field measures agentic coding performance and model rankings.

Tool-MCoT demonstrates how small language models can be fine-tuned on LLM-generated tool-augmented chain-of-thought data to perform content safety moderation with improved efficiency and accuracy. Relevant to practitioners deploying moderation at scale and researchers studying knowledge distillation and reasoning augmentation, though incremental rather than foundational.

Researchers evaluated 10 state-of-the-art LLMs on code understanding and found that while lexical recall is near-perfect, semantic recall degrades sharply when code is centrally positioned in long contexts. They introduce SemTrace, a new benchmark revealing 92.73% accuracy drops due to positional effects, suggesting current code understanding evaluations substantially underestimate LLM failures on semantic understanding tasks.

Researchers propose a learnable sparse cache using Dirichlet-process clustering that stores distinct items rather than all tokens, achieving full-attention recall with subquadratic memory. The work addresses a core constraint in modern sequence models—the tension between fixed-state compression and attention's quadratic scaling—with a principled middle ground validated on associative recall and real-world event streams.

Low-Rank Attention Residuals (LR-AttnRes) decouple routing from representation in LLM residual connections by using low-rank keys for depth-wise routing while maintaining full-dimensional residual values, improving validation loss with reduced computational overhead. This is a concrete architectural innovation with released code and models that practitioners and researchers building efficient LLMs can directly evaluate and integrate.

Researchers formalize and empirically test LLMs' ability to cooperate pragmatically in multi-party tasks with incomplete information, grounding analysis in Grice's cooperative principle. The work surfaces concrete failure modes relevant to agentic and collaborative AI pipelines, showing that current LLMs struggle with contextual reasoning when information is asymmetrically distributed—a critical signal for real-world deployment safety.

Researchers systematically quantify how preprocessing pipelines (speaker diarization) and measurement methods (LLM vs. keyword lexicon) introduce instability into stance analysis of interview data, finding cross-method disagreement larger than pipeline effects despite aggregate stability. This matters to practitioners building computational social science systems: the work provides a diagnostic framework to validate robustness across both preprocessing and annotation choices, addressing a blind spot in reproducibility of LLM-based content analysis.

Researchers introduce FATE, an 8B parameter model trained via knowledge distillation to automatically evaluate AI tutors across four pedagogical dimensions (mistake ID, location, guidance, actionability), achieving up to 22.63pp gains over baselines. This addresses a critical gap as LLMs proliferate in K-12/higher education but lack reliable evaluation frameworks—automating pedagogical quality assessment is essential infrastructure for responsible educational AI deployment.

Researchers identify and taxonomize silent failure modes in quantized LLMs—cases where post-training quantization (especially NF4) preserves task accuracy but fundamentally alters reasoning quality in ways invisible to standard metrics. This is directly actionable for practitioners deploying quantized models in production, as it reveals that accuracy alone is an insufficient proxy for model reliability in reasoning-critical applications.

Researchers developed an LLM-based system for merger-arbitrage forecasting that outperforms calibrated market probabilities and XGBoost on 400+ deals by combining long-context reasoning, expert-guided engineering, and hindsight-supervised finetuning. The work demonstrates practical value of LLMs in specialized high-stakes domains requiring document-scale reasoning, but represents application rather than architectural or capability breakthrough.

MUGEN introduces the first comprehensive benchmark for evaluating multi-audio understanding in large audio-language models, revealing that performance degrades sharply with concurrent inputs and proposing permutational self-consistency as a training-free improvement technique yielding 6.74% gains. This matters because audio-language multimodality is an emerging frontier for foundation models, and identifying systematic scaling bottlenecks and remediation strategies informs both model architecture and evaluation practices for practitioners building audio AI systems.

CRINN applies reinforcement learning to automatically optimize approximate nearest-neighbor search algorithms, achieving state-of-the-art results on multiple benchmark datasets while validating LLMs as tools for algorithmic optimization. This matters to practitioners building RAG and agent systems: faster ANNS directly reduces latency and cost in production retrieval pipelines, and the meta-insight (RL for algorithm synthesis) signals a broader capability for automating infrastructure-level optimization.

DR-Arena is a fully automated evaluation framework for Deep Research agents that constructs dynamic information trees from real-time web trends and uses an adaptive complexity escalation loop to test reasoning and coverage capabilities. This matters because reliable benchmarking of autonomous research agents is a critical gap as LLMs shift from static task completion to open-ended investigation, and the 0.94 correlation with human preferences validates automated evaluation as scalable alternative to manual assessment.

Researchers propose Abstention-Aware Reinforcement Learning (AWA-RL), which trains search-augmented LLMs to abstain rather than hallucinate when retrieval fails, improving precision by up to 10.3% while introducing RA-F1 as a capability-reliability metric. This directly addresses a known vulnerability in current LLM agent training paradigms and is immediately reproducible via open-source code, making it actionable for practitioners building search agents.

Researchers evaluated whether Speech Language Models exhibit sound symbolism—the human tendency to map speech sounds to perceptual qualities like roundness—using real audio recordings rather than text or images. They found SLMs fail to align with human perception on auditory judgments, missing acoustic cues like spectral tilt, suggesting current speech representations lack the perceptual grounding that humans use; this identifies a specific architectural weakness in how models encode speech and has implications for building more human-aligned multimodal systems.

A new arXiv paper proposes a human-centered AI framework for lexicography, arguing AI should augment rather than replace lexicographers while preserving professional agency and mitigating bias. Relevant primarily to NLP researchers and language-tech practitioners, but marginal for the broader AI/ML audience unless human-AI collaboration patterns in specialized domains are of strategic interest.

Researchers propose Harness-Native agentic routing—a step-level model-selection framework that conditions on full agent execution state rather than single queries, generating a self-improving data flywheel from execution traces. Matters to ML practitioners scaling multi-model agents: shifts routing from cost-optimization trick to core systems problem and training signal source.

ResearchQA is a new 6,211-example benchmark for evaluating whether LLMs answer questions about scientific papers with verifiable citations, spanning eight domains and four question types. The work demonstrates that citation-based metrics better discriminate model performance than LLM evaluators alone, and that open-weight models achieve competitive accuracy with substantially lower latency—directly relevant to practitioners building scientific AI systems and researchers designing trustworthy LLM evaluation frameworks.

Researchers are launching the first ChineseBabyLM challenge (2026 NLPCC) to train language models from scratch on 100M Chinese tokens, evaluated on NLU, cognitive alignment, and Hanzi knowledge. This extends the data-efficiency and cognitive plausibility research agenda to Chinese, providing a domain-specific signal about how model architectures and training data interact with non-Latin writing systems and cognition.

Researchers propose a framework combining acoustic and linguistic similarity metrics to select optimal source languages for cross-lingual transfer learning in ASR for Warlpiri, an extremely low-resource Australian Aboriginal language, showing that Assamese and Hindi outperform standard baselines. The work advances practical techniques for endangered language documentation and demonstrates that acoustic similarity is the strongest predictor of transfer effectiveness—relevant for practitioners scaling ASR to under-resourced languages globally.

Mistral AI released Robostral Navigate, an 8B vision-language model for embodied robot navigation using only RGB input, achieving 76.6% success on R2R-CE unseen environments and outperforming multi-sensor baselines. The work demonstrates practical advances in embodied AI and efficient training (22× token reduction via prefix-caching + online RL), relevant to practitioners building robotics systems and the broader trend toward single-modality, efficient embodied models.