This is my upgraded rig:
- Ryzen9 5950x with 64gb DDR4
- Dual NVIDIA RTX A4000 (16+16GB VRAM)
to whom read my previous posts, i jumped the gun and upgraded my server, it was worthwhile and somewhat cheap given i already had the two GPUs and the DDR4 RAM.
Anyway, i am currently running Qwen3.6-35B-A3B-UD-Q5_K_XL all in VRAM with 65536 context and pretty happy with speed (80-90t/s) and overall responses (mostly chat).
I would like to experiment with something beefier, with CPU offload, that i can run with my llama.cpp. Of course t/s is not a goal here, but precision and accuracy of responses is.
I tried to find a good model with claude and gemini, but always got short. Once the model suggested fully crashed my server (guess fill up RAM and ended up in a swap loop), more then once i ended up chasing non existent models. Pretty annoying.
Considering i would only use between 32 and 48GB or system RAM, can you suggest (preferably with links to HF) some models?
I like qwen3.6, but open to anything.
It’s funny; I was just reading about someone who went the other way
https://bitworking.org/news/2026/05/surprising-things-i-learned-putting-together-a-home-brain/
At a certain point, it becomes less about parameters and more about tools supporting those parameters. Something like Pithagoras, Understory, MCP tools etc.
https://github.com/thecodacus/pithagoras/
https://github.com/thecodacus/understory
https://www.youtube.com/watch?v=fpvF4n32lsE
https://www.youtube.com/watch?v=IwN-eK1s8og
This is very interesting, i have saved your comment, it feels too soon for my understanding of it all, but both understory and pithagoras feels less obscure than what would have been, to me, only a few weeks ago.