Woth thé Qwen 3.6 3.8 something nice happened most US company LLM got beat by miles. NVIDIA nemotron 27b isn’t as good as qwen two version prior than the one just released.
US are beaten on Local LLMs already.
if you start excluding the 1000+ B param models …
using smaller models, would initially ease hardware demand by 60% .
OFC you cant… and probably shouldnt, ignore and disrespect SOTA flagship models
Even “big” open source models like DSV4 and Ling/Ring are very efficient. They’re big, but (seemingly) sparser than US models, so they’re cheap.
They run surprisingly well with hybrid CPU+GPU inference on desktops. And thats not even getting into the efficient attention mechanisms.
I can run DSV4 Flash, barely quantized, with ~1M context on my Ryzen desktop at ~11 tokens/s. If you told me that two years ago, I would not have believed you.
Woth thé Qwen 3.6 3.8 something nice happened most US company LLM got beat by miles. NVIDIA nemotron 27b isn’t as good as qwen two version prior than the one just released. US are beaten on Local LLMs already.
if you start excluding the 1000+ B param models …
using smaller models, would initially ease hardware demand by 60% .
OFC you cant… and probably shouldnt, ignore and disrespect SOTA flagship models
Even “big” open source models like DSV4 and Ling/Ring are very efficient. They’re big, but (seemingly) sparser than US models, so they’re cheap.
They run surprisingly well with hybrid CPU+GPU inference on desktops. And thats not even getting into the efficient attention mechanisms.
I can run DSV4 Flash, barely quantized, with ~1M context on my Ryzen desktop at ~11 tokens/s. If you told me that two years ago, I would not have believed you.