I have my eyes on old 32gb amd instinct cards. One of those from ali will cost you ~450-500 euro, which is a lot for old unefficient card, but does provide amazing bandwidth and 32gb of vram. With just one you can run those models almost with full 256k context (or whatever the number there is). Just 4 of those and you can even run models that come close to things like claude and have 1m token context window
Sadly, I can’t justify it — my rig already have 7900xtx + 4060ti (so 32gb vram in total) and realistically I know that there no reason to buy it right now in memory crisis. In couple of years either Chinese manufacturers will catch up ram wise or someone will come up with dedicated device/accelerator for half the price. Or done other similar breakthrough will happen, maybe 4/8 channels of ram on regular desktop will become a thing, instead of being HEDT/server only
I have my eyes on old 32gb amd instinct cards. One of those from ali will cost you ~450-500 euro, which is a lot for old unefficient card, but does provide amazing bandwidth and 32gb of vram. With just one you can run those models almost with full 256k context (or whatever the number there is). Just 4 of those and you can even run models that come close to things like claude and have 1m token context window
Sadly, I can’t justify it — my rig already have 7900xtx + 4060ti (so 32gb vram in total) and realistically I know that there no reason to buy it right now in memory crisis. In couple of years either Chinese manufacturers will catch up ram wise or someone will come up with dedicated device/accelerator for half the price. Or done other similar breakthrough will happen, maybe 4/8 channels of ram on regular desktop will become a thing, instead of being HEDT/server only