blogvLLM Blog·21 Oct 20264.5
Ant Group and vLLM open-sourced Humming, a quantized inference kernel library that covers NVIDIA GPUs from Turing on and other CUDA-compatible chips.
paperarXiv (systems, data, ML)·22 Oct 20264.2
Spherical lattice vector quantization-aware training for low-bit LLMs.
paperarXiv (systems, data, ML)·23 Oct 20264.1
Quantizes n-gram embedding tables in open LLMs to low bits.
paperarXiv (systems, data, ML)·22 Oct 20264.1
Compresses per-layer embeddings with cross-layer latent vector quantization.
paperHugging Face Daily Papers·21 Oct 20264.1
PACE encodes linear-attention recurrent states in four bits.
paperIEEE Xplore·1 Nov 20264.0
PrefixQuant adds prefixed tokens to absorb activation outliers, enabling simpler static LLM quantization.
paperIEEE Xplore·1 Nov 20263.8
BitNet v2 adds low-bit, sparsified activations to 1-bit LLMs to cut inference cost further.
paperHugging Face Daily Papers·22 Oct 20263.5
Timestep-aware quantization for diffusion LMs.
blogvLLM Blog·23 Oct 20263.2
vLLM supports MiniMax-M3.1-Flash on day 0, including its NVFP4 KV cache, NVFP4 MoE weights and a draft model for speculative decoding.
paperIEEE Xplore·1 Nov 20263.1
Survey of low-precision training for LLMs, including FP8 and lower formats.