Programmers often discover solutions while explaining a problem to someone else, even to people with no programming knowledge. Describing the code, and comparing to what it actually does, exposes inconsistencies. Explaining a subject also forces the programmer to look at it from new perspectives and can provide a deeper understanding.
I hope NPU hardware like CIX Clawcore will become more common for Developers. They can run 30B models locally and these smaller models are getting better and more efficient as well (MoE etc).
The hardware still burns energy and resources and costs money but locally so you are responsible for it and don’t externalize everything into some data centers and investor / public money.
You can run a 30B model on a used 3090.
True but it needs more room, power and has less RAM. It’s probably faster though.
I admire your optimism.
You know what hardware is going to become common with developers? Dumb terminal tablets and cloud accounts, unless we stop the capitalists right away.
You’re not wrong. At work I have the bizarre situation that I have a laptop which is pretty powerful but I’m not allowed to develop on it and have no admin rights on it so I have to remote desktop into a less powerful VDI running in a different country with exactly the same access to company information but I have local admin rights on that machine. Also development is super cumbersome because it’s a windows VDI so I have to use WSL2 for a lot of the things I’m doing. Really weird, inefficient and expensive.