Hacker reverse engineers his Samsung TV and finds on-device AI listening for your dog and baby, watching what you watch, and borrowing your soundbar’s mic if the TV doesn’t have one. Then OpenAI banned him for reverse-engineering it.
Source:
Hacker reverse engineers his Samsung TV and finds on-device AI listening for your dog and baby, watching what you watch, and borrowing your soundbar’s mic if the TV doesn’t have one. Then OpenAI banned him for reverse-engineering it.
Source:
It’ll be slow but it’ll run.
Though even 128GB isn’t enough for the biggest models. 512 would open up more options and the very biggest need like 2-4TB, totally out of scope for personal use.
You could try a medium-big MoE model (Mixture of Experts) with less than 100b total parameters. Maybe start with something modest like Qwen3.6-35B-A3B at 8bit quants. That’ll leave you with plenty of memory left over for context size.
Hmm… I guess my board supports up to 256GB; would maxing out the RAM make a big difference? Also, I’ve got 2x16 core Opteron and an AMD Vega 56, if you have any advice related to those.
I don’t think you should go bigger because bigger = slower. It might already be too slow to use. Give a smaller model a try first and see how it runs.
try the Strata engine instead of llama.cpp
https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md#pick-by-memory
You should be able to run Qwen 3.8 Flash Next decently