FairCache : Demystifying Cache-Induced Unfairness in Multi-Tenant Large Language Model Serving

FairCache shows prefix and context caching makes multi-tenant LLM serving unfair and proposes cache-aware scheduling to fix it.