If I understand the nature of your hardware correctly, you should be able to run the MoE models like Gemma4 26B-A4B or Qwen3.6 35B-A3B at a high quantization fairly performantly.

You could try running some of the dense models (like today’s Qwen 3.8 27B) as well, but I expect they’ll be pretty slow (judging by my own experience with a unified RAM system that has a Strix Halo APU). Might still be useful for tasks that you can leave running on their own for a long time instead of for interactive chat style interaction though.

You’ve got enough RAM to load larger models, but there hasn’t been much released in between the “it fits on a 24GB or 32GB GPU that a gamer might own” and the “oh god you need HOW MUCH RAM!?” scales lately…

How much RAM do you have?

In my testing so far, it seems to lean more into “I don’t know” type answers, but I haven’t poked at it that much yet.

One of the queries that used to consistently trip up Qwen 3.5/3.6 was asking directly about a plausible sounding (but non-existent) work (e.g. Are you familiar with "A Valiant Effort (1989)"?); stock Qwen 3.8 will say it doesn’t know or isn’t sure and ask me for context.

You can also use something like llama-swap to seamlessly swap between them.

FYI: llama-server has built-in model swapping now so you don’t need a second piece of software (at least for the simple cases). I pass --model-presets and a path to an INI file with the per-model parameters when I run the server.

27b is much better though…

I haven’t really found that to be the case for my uses, but getting much faster decode is really useful for me… Have you tried running the MoE model at a higher quant? I usually use Q6_K.

qwen 3.6 27b runs on an AMD AI MAX 395+ fairly well.

Try Qwen 3.6 35B-A3B on your hardware instead; it’s much faster.

Everything runs well if one is patient enough 😂

I mean, if you’re really patient you can even run with CPU only as long as you have enough RAM… My ~decade old computer with DDR3 RAM gets 14 tok/s prompt processing and ~5 tok/s decode on MoE models (e.g. a Qwen 3.6 35B-A3B variant at Q6_K – my current default model). That pp tok/s is painfully slow… but it still works!

Same computer, same model but using a discrete GPU that can fit all the weights on the card – ~900 tok/s prompt processing, ~70 tok/s decode.

~2 days until 27B release apparently.

+1 for llmfan46’s heretic variants. I use one of his uncensored Qwen 3.6 35B-A3B variants as my default model.

rpDungeon’s Luchador models (gemma derived) are also quite interesting – they tend to have better prose quality for creative writing tasks.

• Luchador
• Luchador Rudo

I’ve been curious to try experimenting with using Rudo in particular for making more interesting NPC interactions in a text adventure for a while now, but haven’t gotten to it yet.