leo@sh.itjust.works to LocalLLaMA@sh.itjust.worksEnglish · 11 days agoLocal LLM Inference Optimization: The Complete Guidecarteakey.devexternal-linkmessage-square2fedilinkarrow-up147arrow-down14file-text
arrow-up143arrow-down1external-linkLocal LLM Inference Optimization: The Complete Guidecarteakey.devleo@sh.itjust.works to LocalLLaMA@sh.itjust.worksEnglish · 11 days agomessage-square2fedilinkfile-text
minus-squaretheunknownmuncher@lemmy.worldlinkfedilinkEnglisharrow-up3·11 days ago Enabling XMP took my machine from roughly one-third speed back to normal. Huge red flag. XMP is not designed for use with memory-intense workloads like running LLMs.
minus-squareComradePenguin@lemmy.mllinkfedilinkEnglisharrow-up2·11 days agoI have a PC that has been crashing a lot recently, but only when running long RAM+CPU and RAM+GPU intensive tasks. It uses about 60%+ of ny RAM. I ran a memory test for 9 passes, no errors. No errors with the CPU only, and no errors with GPU only. So it can be the XMP that crashes it?
Huge red flag. XMP is not designed for use with memory-intense workloads like running LLMs.
I have a PC that has been crashing a lot recently, but only when running long RAM+CPU and RAM+GPU intensive tasks. It uses about 60%+ of ny RAM.
I ran a memory test for 9 passes, no errors. No errors with the CPU only, and no errors with GPU only.
So it can be the XMP that crashes it?