Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • curbstickle@anarchist.nexusM
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    1
    ·
    4 hours ago

    Just an m2 w/ 16gb I repurposed.

    Can’t really do a lot at once, and the context is limited, but it does the trick. I’d buy a few more if I saw them at the right price.

    • async_amuro@lemmy.zip
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      1
      ·
      3 hours ago

      Nice, I’ve got a Mac Studio M1 Max with 32GB of RAM that I use with Ollama and then I host OpenWebUI and OpenCode on my Arch Server. I use the Mac as a primary workstation, so it’s a little rough when I start running a model. I’m sure I could probably do and learn more about Ollama to improve my experience, but for now it works for certain tasks.

      • curbstickle@anarchist.nexusM
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        1
        ·
        2 hours ago

        I got mine a few years back for some iOS builds, don’t need to do them that often so it became the model host for me