• Bread@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    3
    ·
    11 days ago

    Like all things llm, the quantisation makes the difference. Which, of course you need the vram for to have good results. Flash next works very well if you can run the higher quants.