claude-news

Week of Monday, 26 October 2026

Making AI and data systems dependable: verifiable agents, disaggregated MoE serving, and the hidden costs inside Postgres

llm-agents+llm-evaluation

Agent evals shift from average accuracy to repeatable, state-checked success

What changed

This week several groups measured agents by what they leave behind and how often they get it right every time. ThinkingBox grades 507 tasks by the final database rows and found models with nearly equal pass@1 differ by 2x on tasks passed in all 20 runs. Reusing those checks as RL rewards lifted a 27B model from 38 to 107 always-correct tasks in 50 GRPO steps. VerifierBench found that agent-written verifiers score only 10.6 to 62.1 balanced accuracy across 19 models, and they reject valid solutions far more often than they accept bad ones.

Key techniques

  • Terminal-state grading: run the agent against a real backend, then check the final records with code, so any valid path passes and a confident but wrong answer fails.
  • Repeatability metrics: run each task many times and report the share passed on every attempt next to pass@1 and pass@20.
  • Policy-relative task selection: probe the base model once and train only on tasks it sometimes solves, since always-solved or never-solved tasks give GRPO no signal.
  • Verifier auditing: test generated verifiers against hidden valid alternatives and deliberate attacks, not just one reference solution.
  • State anchoring: restating fixed constraints in a structured block recovered up to 24.4 points when models remembered a rule but stopped applying it in long dialogues.

Open questions

  • RL makes agents more consistent but slightly narrower, as pass@20 fell. How much rare-case coverage is acceptable for production agents?
  • If agents write their own verifiers, who checks the verifiers, and can overly strict checks quietly bias training?
  • Most failures in long dialogues are drift, not forgetting. Will bigger context windows help at all, or is this an architecture problem?
llm-serving

vLLM matches DeepSeek own engine and makes MoE serving elastic

What changed

On one GB300 NVL72 rack, vLLM with prefill/decode disaggregation and wide expert parallelism served DeepSeek-V4.1-Flash at 7.9x its single-node throughput, with decode within 6% of DeepSeek own DeepInfer engine. For GLM-5.3 on 16 H200s, combining two kinds of context parallelism made 1M-token requests fit and gave 1.7 to 3.2x per-user speed versus plain data parallelism. Elastic expert parallelism cut the pause when growing from 16 to 20 GPUs from 197 seconds (restart) to 0.82 seconds.

Key techniques

  • Measure the model first: DeepSeek-V4.1-Flash needs only 890 bytes of KV per token and its decode is 62% MoE time, so KV capacity is not the limit and wide expert parallelism pays off.
  • Small prefill groups, wide decode groups: prefill runs at DEP2 to keep load balancing simple, while decode spreads experts across many GPUs and rebalances them with EPLB.
  • Prefill context parallelism (PCP): split one long prompt across GPUs by token, pairing early and late chunks so causal attention work is balanced, which raised sparse attention from about 26 to 191 FLOPs per byte.
  • Decode context parallelism (DCP): shard the KV cache by token round-robin so a 53.4 GiB 1M-token cache needs 6.7 GiB per GPU, at a 6 to 15% all-gather cost.
  • Prepare then commit: new workers load weights and warm up in the background while old ones keep serving, and a communicator that adds ranks in place keeps existing CUDA graphs valid.

Open questions

  • At NVL72 scale CPU overhead and scheduling become the bottleneck. How much of the next gain is software outside the GPU?
  • General engines now match model-specific ones on DeepSeek. Will that hold as model architectures keep changing every few months?
  • Elastic scaling is near zero downtime for experts. Can autoscaling policies react fast enough to agent traffic spikes to use it well?
postgresql

Postgres internals that bite at scale: prefetch, numeric, cascades and shard keys

What changed

A cluster of deep write-ups traced real production pain to design choices inside Postgres. recovery_prefetch turns large sequential reads on the primary into 8 KB random reads on replicas, up to 16x more IOPS, and with synchronous replication a short network blip after a write burst can stop all writes for 30 minutes or more. GROUP BY on 12 numeric columns ran almost 3x slower and used 9x more memory than the same query on double precision. Large ON DELETE CASCADE deletes can block migrations and write hundreds of GB of WAL in one transaction.

Key techniques

  • Disable or test recovery_prefetch on replicas with full page writes and limited IOPS, and reproduce the walreceiver restart scenario before trusting synchronous replication.
  • Understand numeric: values are variable length, carry their own scale and are passed by reference, unlike DuckDB or ClickHouse decimals stored as fixed integers with scale in the column.
  • Replace big cascades with batched application deletes, index every foreign key, and set a short lock_timeout on migrations so they retry instead of queueing writes.
  • Watch logical replication: subscribers skip cascades by default, so subscriber-only child tables can be left with orphan rows.
  • When sharding, pick the shard key from the dominant access pattern and add it to child tables, replicate small lookup tables, and use a global secondary index for lookups without the key.

Open questions

  • Will the community change recovery_prefetch defaults now that the outage is easy to reproduce?
  • Can Postgres add a fixed-width decimal for declared precision without breaking its exactness guarantees?
  • Cross-shard transactions in sharded Postgres products have no atomic commit. How will teams handle invariants that span tenants?