minus-squarehummingbird@lemmy.worldtoLocalLLaMA@sh.itjust.works•llama.cpp in progress pull request for smart caching of MoE experts, 16% to 35% TPS boost for my RTX 2080linkfedilinkEnglisharrow-up1·3 days agoNeed to wait for AMD support. linkfedilink
Need to wait for AMD support.