KVarN: Native vLLM backend for KV-cache quantization by Huawei

 (github.com)

99 points | by theanonymousone 6 hours ago

9 comments