A practical guide to hardware, OS, and llama.cpp tuning, built from a year of experiments on a single consumer CUDA workstation.

Good overview IMHO.

  • theunknownmuncher@lemmy.worldEnglish
    3·
    11 days ago

    Enabling XMP took my machine from roughly one-third speed back to normal.

    Huge red flag. XMP is not designed for use with memory-intense workloads like running LLMs.

    • ComradePenguin@lemmy.mlEnglish
      2·
      11 days ago

      I have a PC that has been crashing a lot recently, but only when running long RAM+CPU and RAM+GPU intensive tasks. It uses about 60%+ of ny RAM.

      I ran a memory test for 9 passes, no errors. No errors with the CPU only, and no errors with GPU only.

      So it can be the XMP that crashes it?