Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • async_amuro@lemmy.zip
    link
    fedilink
    English
    arrow-up
    2
    arrow-down
    1
    ·
    3 hours ago

    Nice, I’ve got a Mac Studio M1 Max with 32GB of RAM that I use with Ollama and then I host OpenWebUI and OpenCode on my Arch Server. I use the Mac as a primary workstation, so it’s a little rough when I start running a model. I’m sure I could probably do and learn more about Ollama to improve my experience, but for now it works for certain tasks.

    • curbstickle@anarchist.nexusM
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      3 hours ago

      I got mine a few years back for some iOS builds, don’t need to do them that often so it became the model host for me