Xylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-211 days agoMy 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:media1.tenor.comexternal-linkmessage-square13fedilinkarrow-up187arrow-down15
arrow-up182arrow-down1external-linkMy 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:media1.tenor.comXylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-211 days agomessage-square13fedilink
minus-squareffhein@lemmy.worldlinkfedilinkEnglisharrow-up1·3 hours agoAh, multiple GPUs? For some reason I thought you meant that with exllamav3 you had managed to load a model which was larger than your VRAM.
Ah, multiple GPUs? For some reason I thought you meant that with exllamav3 you had managed to load a model which was larger than your VRAM.