All news

Evergreen checks claims in LLM-written summaries of tables by compiling each claim into a semantic verification query with citations. It keeps F1 at 0.94 while cutting cost 3.1x through early stopping, relevance sorting and prompt caching.

ThinkingBox turns database end-state checks into RL rewards for post-training agents, with training code released.

FrontierCS 2 benchmarks agents on frontier computer science research tasks.

EpiSEM-Bench tests whether tool-using agents notice missing safety evidence before acting.

A dermatologist found a fine-tuned 2015 Inception v3 CNN beat two frontier LLMs at diagnosing 200 biopsy-proven skin lesions, with the LLMs missing a melanoma.

Shows how to use cheap decision models as eval classifiers: tune the decision threshold instead of the default 0.50, compare cost and accuracy across models on 100 agent traces, and calibrate scores.

Characterizes how LLMs keep repeating instructions while ceasing to obey them in long dialogues.

Quant-Arena benchmarks LLM agents on quantitative trading research.

ProductWebBench grades frontend coding agents on whether the existing site keeps working.

Rewording questions changes tool-augmented model rankings.

Five Qwen3.8-27B models run through the GVS5H harness match Fable 5 on a benchmark, showing harness design matters as much as model size.

User simulators that stay on task and sound natural for multi-turn evaluation.

More metadata can mislead LLM museum descriptions.

Open LLMs on the Spanish MIR medical exam with uncertainty and image use.

AgentVQA benchmark for agentic visual understanding.