All news

Part 1 of a guide to system-level profiling of PyTorch inference on AWS Trainium.

Cerebras explains how it deploys trillion-parameter models across wafer-scale chips that each hold only 44 GB of on-chip SRAM.

Google and Meta open-sourced TorchTPU, a native PyTorch backend for TPUs, and Meta plans to move larger LLM workloads onto TPUs; AWS released TorchNeuron for Trainium the same week.

vLLM TorchTPU is a new backend that runs existing vLLM model code on Google Cloud TPUs without changes.

TorchTPU is a native PyTorch backend for TPUs, so Hugging Face models run on TPU without torch_xla or JAX.

Stochastic-computing hardware accelerator for RoPE in LLMs.

Proposes a compatibility ratio metric to guide hardware and DNN co-design for embedded accelerators.