AWS releases distributed KV cache skill, claiming 40% lower cost and 80% lower latency
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
Lower cost and time to first token for self-deployed models can now be achieved automatically: the agent itself checks whether a cache is needed and deploys it.
AWS has published a distributed KV caching skill in the Agent Toolkit for AWS. The coding agent itself checks whether a workload needs such a cache, sizes the cluster and deploys it on Amazon ElastiCache (Valkey 9.1) with LMCache and RDMA/EFA. AWS claims the distributed KV cache cuts the inference cost of self-deployed large language models by up to 40% and time to first token by up to 80%. According to AWS, the KVTrace tool assesses the benefit on a customer's own traffic for free before deployment, without recording token identifiers or prompt text. The skill is claimed to be compatible with Kiro, Claude Code, OpenAI Codex and other agents that support Agent Skills.
Sources 1