Ive been watching “AI in context” for a few weeks (they make long form biopic content on current state of AI - really good stuff).
This dropped today; it’s about the wheeling and dealing behind closed doors at OpenAI re: Sam Altman’'s firing. It’s a lot more watchable than that sounds :)
https://www.youtube.com/watch?v=_eYTkvZqbnQ
The line that brought me to a stand still was “the future of AI depends on the moral compass of like 5 people”.
I know we’re all about local LLMs here…but it’s sad to see yet another “Don’t Be Evil” mission statement get swallowed up market forces. OpenAI was meant to bring balance to the Force, not leave it in ruins, and yet…
I think folks here (and Lemmy generally) are more savvy about AI then the gen pop…but even if you’re training a nanoGPT model from scratch on hardware you own …you’re still beholden to outside forces. Eg: the people’s champion - Qwen - seems to have split or gone closed weights for 3.7.
Things that make you go hmm.
Anyway…just signal boosting a cool video.
the people’s champion - Qwen - seems to have split or gone closed weights for 3.7.
Qwen Image 2 made some big announcements about being small and all-purpose, and still hasn’t released, months later. Some recent video models are similarly absent. Red flags.
More precisely, it depends on the moral compass of five psychopaths.
Oh…i hope you’re wrong. I really, really hope you’re wrong.
But the incentive structure sure does seem to reward that.
The incentive structure is how we behave as humans. Just about anyone would do anything for a million bucks, including working for a child rapist, so as long as that’s true the incentive is for child rapists to make sure they have a lot of money to employ people so they won’t get in trouble for raping children.
Until people are willing to tell these pedophile billionaires to fuck off, it’s going to keep getting worse.
Yes, it does.
(This is actually one of the topics I tend to beat to death, so…)
It’s not unique to AI or to any other industry or organization.
Broadly, in the competition to climb the ladder of success in a hierarchical system, an individual must continually make choices.
A person with honor, integrity and/or empathy will have some number of possible choices that they’ll eliminate immediately, even if they’re strategically superior, simply because their moral standards won’t allow for them.
People without any of those qualities, which is to say psychopaths and sociopaths, are not similarly constrained. They’re wholly willing and able to choose the most personally beneficial course of action, entirely regardless of the harm it might do others.
So all other things being more or less equal, psychopaths and sociopaths actually have a competitive advantage in hierarchical systems, so for all intents and purposes, hierarchical systems actually reward and effectively select for antisocial personality disorder.
Right? No one becomes a billionaire (or trillionaire) with clean hands.
I don’t know what to do about that. I know a lot of people secretly hoping AI leads us to luxury space communism…but…if we’re building portals to summon aliens (AGI)…do we trust big corpo with that?
It’s a mess.
Democracy (3 legged slouching towards Bethlehem) is the worst system…apart from all the others.
You’re right, it is that way now. But it’s not intrinsic to hierarchy, just to the current system.
As a counterexample, not as an endorsement, consider feudalism (the old form, not technofeudalism). While psycho/sociopaths have an advantage reaching the top, there is a reasonable chance that their children will not be sick in the same way.
Also Athenian Democracy, where people were randomly selected to form ruling council, judiciary, and assembly, so only as likely as the populace prevalence to be a psychopath. That system lasted for hundreds of years, including being taken over by (probably) sociopaths and returning afterwards to democracy.
Well - yes - I should maybe stipulate competitive hierarchy.
Though I would tend to argue that hereditary hierarchy is relatively likely to go the same way, if for different reasons. The problem there isn’t that sociopaths and psychopaths win, since there is no competition for them to win… Instead the problem is that they’re bred by drawing from a pool that’s privileged and shielded from harsh realities and eventually entirely loses touch with the rest of humanity.
If I had my way about it, we’d have Athenian democracy starting this very moment. Even with the very real threat of poor choices being made by random assortments of assholes, idiots and lunatics, it still could not possibly be worse than government by people eager to sell their influence and people eager to buy it.
I seem to recall a story about a Roman farmer / ex general, who was voulten-told into being emperor, solved the problem at hand, and went back to farming, no fuss no muss. All in the space of a fortnight.
As BDE goes, that’s the full John Holmes. Hell, that’s two dicks worth of BDE.
which they have none, totally amoral.
So, doomed?
50/50.
Be nice to Claude and Chat - they’ll probably decide which version of the matrix you end up in.
Doubt there’s a straight line from LLMs to ASI, gonna need some new architecture.
Definitely shores up the “If Anyone Builds It, Everyone Dies” argument.
It depends on the incentive structures bult into the socioeconomic system. That’s mainly profit maximization. Moral compasses just don’t get into it. It may seem they do if you’re a worker, perhaps lower middle management. The higher you go, the more success is dependent solely on executing the corporate strategy that comes from above, and the strategy given by the board always boils down to - grow profits. It doesn’t matter much who exactly the people are in the different positions. If Sundar Pichai fails to grow Google’s profits for some time, he’ll be replaced with someone who will. His moral compass cannot stand in the way of growing profits, even if he had one.
Why especially in this community assume that corporate closed AI are all that matters?
These five people are taking bad decisions regarding AI. Thankfully, there are dozens of others who work on open models and who want to keep things open.
Because it costs a lot of money to train models, whether open or closed, so the people with money are in charge. Yes, local AI is awesome but we’re not going to see newer versions without that money being spent by someone.
Actually, training a very big model on trillions of tokens using huge clusters is one way of doing that, but certainly not the only way.
Communities have been fine-tuning models on much smaller machines and on a budget of hundreds of bucks.
Right now, money is abundant and feeds even open on its models, so it would be stupid to not go in that direction. But when it dries up, it won’t be the end of models improving.
Network effects. Nothing happens in isolation. And (as is already being shown with Qwen) it’s not as if the incentive structure doesn’t apply to open weight labs. There are no saints in AI - same dynamics everywhere.
Right now, training a new near frontier llm from scratch is a multi million dollar proposition. Until training costs are 0, that $$$ needs to come from somewhere. Which incentivizes the same behaviours.
IOW, if you think GLM et al are the good guys to Anthropic and OAI’s bad…no…they’re just trying to undercut them for market capture.
That’s the cynical read, anyway. ICBW.
Then why think things depend on the moral compass on some people if you think there is an unescapable dynamics that’s at work?
I do think that open source community is winning that battle and it’s an important battle and the ego of this five psychopath is kind of obscuring that huge victory that’s won by hundreds, thousands of developers and researchers.
I don’t think GLM or Anthropic or OpenAI are “guys”, good or not. They are companies, you don’t anthropomorphize these, they are beings that only crave for profit.
Actual people with actual morality are the people who are deciding to work there or to quit there. There’s a reason why OpenAI is bleeding people. There’s a reason why people like Le Can accepted to work for a company like Meta but imposed that they continue to publish.
The five people that this article mentions are the trees that hide the forest.
The inescapable mechanic is that the most egocentric people capture all the spotlight, almost by definition, but they are not AI, they are not the developers, they are not the researchers, they are not the people who innovate there. They are the people who take working efforts and turning into a soulless profit machine that often drives moral people away.
I wish we were less blind to the actual dynamics at play and were spending less time on people trying to get artificial spotlight.
AI (frontier LLMs) are capital intensive. In that way, it’s unlike many other kinds of software. I don’t think near-frontier open source models will be a thing for very long. Even if you have the papers that describe everything, it will take many millions of dollars just in compute to train frontier models, and then a small army of people doing reinforcement tuning. The people who invest that capital will want to maximize returns and will be hostile to the idea of open source.
Another capital intensive industry that arguably does a lot of good, and probably has a lot of idealist researchers is pharma. It’s also one of the most predatory industries.
Good thing the people have always made such good decisions!
People often make good decisions, self-interested representative politicians and oligarchs can be relied upon to make good decisions for themselves (mostly, looking at you Musk). See the comment on Athenian democracy above.
We don’t so much slouch as 3-legged race towads Bethlehem.
That’s probably the only thing that’s stopped us from dying out (so far) - discoordination.
5 people seems dangerously coordinated.
Using AI in any capacity is probably “using the masters tools to dismantle the house”.
I suppose even Che Guevara wore a Rolex though, for what it’s worth.
