I’ve tried just about most of the small models. Tried NanoClaw. I just don’t have the equipment necessary to pull that off and make it a worthwile, in house tool rather than an in house oddity. I really, really want to tho. So much so that I have been looking at what it would take to accomplish that, which seems to be at the $4k to $5k USD range. The sweet spot for GPUs seems to be at the 32 gb level. It is pricey, but hell, at my age, I figure wtf…I should treat myself. Whats wrong with that? If I do pull the trigger, I want it to be a LTS type computer like I built 15 years ago and is still running like a champ today tho it’s probably worth less than a quarter of what I had invested. So, I’d probably overstock it to the max.
Dont waste 5k on an ai computer. Of you want new one, buy for 2k at most. Ai will get optimized more and more. Now we have MoE with which we can run things at home we couldn’t even dream about. The companies loose money fast, there will be massive optimization sooner or later.
You can get a P40 for much less than that, if your case can hold full height card. It’s an old card but its 24GB, 400GB/s.
Else yeah…$3-4,000 is about table stakes, which doesn’t amortise for just AI (not for my use cases anyway). I’d love a Strix but Santa is stingy.
Me - I have a fetish for tiny, low power computers. 1L lenovos, raspberry pis etc. That limits what I can run but with constraint comes inginuity. So I’m making an expert system for myself.
It’s not cooked yet (this is actually the first time I’m sharing it in public; it’s not in installable state and the repo is new) but once it’s done, I can have an always on local brain in a 2W envelope that runs fast. Might even port it to C64…I need an excuse to purchase the new Commodore ultimate.
Well, I haven’t had any new equipment in 15 years. I always buy used or refurb’d. I’m getting old. I figure, I’ve worked hard enough, might as well enjoy the fruits of my labor.
I’ve tried just about most of the small models. Tried NanoClaw. I just don’t have the equipment necessary to pull that off and make it a worthwile, in house tool rather than an in house oddity. I really, really want to tho. So much so that I have been looking at what it would take to accomplish that, which seems to be at the $4k to $5k USD range. The sweet spot for GPUs seems to be at the 32 gb level. It is pricey, but hell, at my age, I figure wtf…I should treat myself. Whats wrong with that? If I do pull the trigger, I want it to be a LTS type computer like I built 15 years ago and is still running like a champ today tho it’s probably worth less than a quarter of what I had invested. So, I’d probably overstock it to the max.
Dont waste 5k on an ai computer. Of you want new one, buy for 2k at most. Ai will get optimized more and more. Now we have MoE with which we can run things at home we couldn’t even dream about. The companies loose money fast, there will be massive optimization sooner or later.
Perhaps, but I probably won’t be around for that massive optimization. LOL
You can get a P40 for much less than that, if your case can hold full height card. It’s an old card but its 24GB, 400GB/s.
Else yeah…$3-4,000 is about table stakes, which doesn’t amortise for just AI (not for my use cases anyway). I’d love a Strix but Santa is stingy.
Me - I have a fetish for tiny, low power computers. 1L lenovos, raspberry pis etc. That limits what I can run but with constraint comes inginuity. So I’m making an expert system for myself.
https://codeberg.org/BobbyLLM/picoGURU
It’s not cooked yet (this is actually the first time I’m sharing it in public; it’s not in installable state and the repo is new) but once it’s done, I can have an always on local brain in a 2W envelope that runs fast. Might even port it to C64…I need an excuse to purchase the new Commodore ultimate.
I was thinking something along the lines of:
Which, with all the other accoutrements like water cooling, etc, will put me right at the $4k mark.
I’ll check it out.
Damn bro, you’re treating yourself for sure! The RAM alone is what, 2k? 🥴
Well, I haven’t had any new equipment in 15 years. I always buy used or refurb’d. I’m getting old. I figure, I’ve worked hard enough, might as well enjoy the fruits of my labor.
Nice bit of kit that. Very nice. Planning on serious AI shenanigans?
Cool. It’s not ready any time soon but when it is, I’ll announce it and make sure it’s callable via SSH / terminal / OpenAI style chat end point.
That way you don’t need anything fancier than a nice terminal to call it.
Absolutely. Like I said, if I’m going to do it, I want to do it up right. I don’t want to come back in 5+ minutes for a result. LOL