Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • irmadlad@lemmy.worldEnglish
    21·
    7 hours ago

    I was thinking something along the lines of:

    • AMD Ryzen 9 9950X3D 4.3GHz
    • NVIDIA Tesla M10 Quad
    • 96GB DDR5
    • 4TB 990 EVO M.2

    Which, with all the other accoutrements like water cooling, etc, will put me right at the $4k mark.

    https://codeberg.org/BobbyLLM/picoGURU

    I’ll check it out.

      • irmadlad@lemmy.worldEnglish
        32·
        6 hours ago

        Well, I haven’t had any new equipment in 15 years. I always buy used or refurb’d. I’m getting old. I figure, I’ve worked hard enough, might as well enjoy the fruits of my labor.

    • SuspiciousCarrot78@aussie.zoneOPEnglish
      22·
      7 hours ago

      Nice bit of kit that. Very nice. Planning on serious AI shenanigans?

      I’ll check it out.

      Cool. It’s not ready any time soon but when it is, I’ll announce it and make sure it’s callable via SSH / terminal / OpenAI style chat end point.

      That way you don’t need anything fancier than a nice terminal to call it.

      • irmadlad@lemmy.worldEnglish
        22·
        6 hours ago

        Planning on serious AI shenanigans

        Absolutely. Like I said, if I’m going to do it, I want to do it up right. I don’t want to come back in 5+ minutes for a result. LOL