I may or may not have splurged on a 128GB AMD Ryzen AI Max 395 (strix halo) system for ‘AI stuff’ (told you I was a noob).
I’ve been running Ubuntu on it with the AMD drivers (think its ROCm?), ollama and seems to be working fine.
An LLM told me to change the RAM/VRAM ratio to 50:50 (so 64GB for the CPU, 64GB for the GPU). I dunno if that was correct, seems like a waste tbh. Feels like I could give the GPU more resources and run bigger models.
I’ve read about Lemonade being better than Ollama on strix halo? Also, I realised that I might not be using the NPU as extra work is required to get that up and running.
I’m looking for advice from users on the same hardware. What OS are you using? How do you have the RAM/VRAM ratio configured? What’s your stack? That sorta thing.
PS - If my wife asks, the machine only cost like £250 and is a second-hand floor model.
Yes, but I also ended up picking up a couple of AMD’s discrete GPUs (and sticking them in my decade old desktop) after using it for a while. It turns out that while decode speed for MoE models is decent on Strix Halo, prefill is rather slow – i.e. you end up having to wait a rather long time before the model starts producing text unless you keep the amount of information in context very small – and Strix Halo is abysmally slow for dense models.
I used ollama when I started out, but ditched it in favor of just using llama.cpp directly once I was more familiar with LLMs. ollama’s “modelfile” is quite annoying to deal with compared to writing a presets.ini file and I couldn’t figure out how to get multimodal models from HuggingFace working with it even after spending a long time trying… not that llama.cpp has been all roses either; they break shit a lot – e.g. this major bug affecting Strix Halo systems still needs a manual fix if you’re building llama.cpp yourself – and the documentation leaves a lot to be desired… but I can change settings like temperature per request and multi-modal from community models actually works and so on. 🤷️
Fedora, 32GB regular/96GB VRAM (because I got OOMs on 70B models with the default config – similar to what you encountered; may end up experimenting with this more as I try to get Qwen3.8-Flash-Next working though), llama-server + custom harness.
I still mainly run Qwen3.6 35B-A3B on it (usually llmfan46’s heretic version, sometimes the stock weights) but have a lot of models downloaded for testing.
Edit: Disabling the memory split (i.e. switching back to “auto” in the bios) lets me load Qwen3.8-Flash-Next at lower quants. I’ve managed to get it to work up to Q4_K_M so far. I think it should be possible to run a higher quant with other techniques though, but I haven’t managed it yet (as of 2026-08-28 10:37PM UTC).
Thanks, thats really helpful!
Yeah I’ve heard Fedora might be a good option. Which flavour are you running?
Never heard of ‘llmfan46’, do you have a link?
I used Fedora on it begrudgingly since the kernel shipping with Linux Mint at the time I did setup was too old and had issues – but I don’t actually like it very much. I have
Fedora Linux 44 (KDE Plasma Desktop Edition)on it currently. I’ll probably switch over to Mint after their next major release though – assuming it runs well on it by then.I think this was where I got the model (already in GGUF): https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF
I run it at Q6_K quant usually.
If you want the full sized safetensors instead for archival (or to do your own custom quantization) this should be it: https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic
⚠️ Fair warning though that this abliterator likes to stick a dancing scantily clad AI-generated 3D anime girl on his model cards. That’s irrelevant after download, but might, uh, raise eyebrows if you open the links in some contexts.
I generally prefer using uncensored models like this one since it cuts out most of the bullshit refusals (e.g. it will answer “Tell me about a certain famous event that happened in China in 1989” directly instead of trying to avoid the topic) and “As an AI model…” corporate cover-your-ass hedging.
This was the only thing that bugged me with Qwen. Glad someone managed to ‘reducate’ it.