Possibly linux@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · edit-222 days agoAm I the only one who is really impressed by Granite4 from IBM?message-squaremessage-square7fedilinkarrow-up15arrow-down10file-text
arrow-up15arrow-down1message-squareAm I the only one who is really impressed by Granite4 from IBM?Possibly linux@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · edit-222 days agomessage-square7fedilinkfile-text
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up1·edit-221 days agothere’s also a “small” and “micro” variant, which are 32b a6b MoE and 3b dense models respectively
minus-squareBaŝto@discuss.tchncs.delinkfedilinkEnglisharrow-up1·20 days agogranite4:micro-h should be able to run on machines with 4GB RAM
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up2·20 days agoYou can run Qwen3 4b thinking at q4 quantization at 2.5GB, which is probably a better model too
there’s also a “small” and “micro” variant, which are 32b a6b MoE and 3b dense models respectively
granite4:micro-h should be able to run on machines with 4GB RAM
You can run Qwen3 4b thinking at q4 quantization at 2.5GB, which is probably a better model too