As of Oct 26, MarkTechPost reports that Tsinghua and CUHK researchers open-sourced AKV, an Apache-2.0 SGLang-based serving system that lets agents drop and reposition KV cache instead of summarizing history, cutting average inference cost about 27.7% and raising average task score about 1.7 points across tested models.
- Rumor Historical snapshot
Did Tsinghua open-source AKV, a KV cache engineering server for LLM agents on SGLang?
Tsinghua and CUHK researchers open-sourced AKV, an Apache-2.0 SGLang-based serving system for agent KV cache engineering
Recorded snapshot
Did Tsinghua open-source AKV, a KV cache engineering server for LLM agents on SGLang?
Tsinghua and CUHK researchers open-sourced AKV, an Apache-2.0 SGLang-based serving system for agent KV cache engineering
Yes, per MarkTechPost. The outlet reports agents can drop and reposition KV cache instead of summarizing history, cutting average inference cost ~27.7% in tests.
Current rumor page →Source: Rumor record