I don't believe "true" agi/asi will happen, but I totally believe if a company theoretically invented they would immediately shut all API's and then go for software market dominance in every category.
While I've said in another comment on this thread that comparative advantage is flawed, I still think the opening moves, the very initial phase, would be as per comparative advantage, not everything all at once.
Until supply grows to saturate demand in whatever market, comparative advantage points to the fastest way to make more money, that money then gets used to construct more servers and data centres, which gets used to saturate supply for the next most profitable market, etc. until there's none left.
I wouldn't say "never" for AI that can economically substitute all human labour. I think that meaning of AGI/ASI is slightly easier than the "AI which learns from as few examples as humans do" meaning, which I also think is possible eventually but don't know how far away it is.
It's vogue on HN to sometimes criticize critically so I want to try and be as constructive as possible:
I work for one of these companies. I also have pay bills to pay. I'd like to understand what a real, good alternative is.
Frontier labs such as Google DeepMind are not just going to shutter their doors because 10% of the peons dropped their jobs. I believe, at least, that we should be demanding political accountability and safeguards for society. I only get to live once: if I am to spend it for social change, I best maximize by expected return.
And quitting a job in a capitalist society probably has negative return overall.
I also argue it depends on which 10%. 20% of all key talent may stall the project for years, so it would be a similar form of collective bargaining. But those at the top tend to be more complicit.
As is it, nothing is "real". But I'll offer options
1. Unionize
2. Work to form a general strike. Even an industry wide strike would be highly effective
3. Work your civic duties to push for stronger labor rights in the United States. It can include anywhere from 4 day work weeks to raising minimum wage to preventing abuse of the H1b Visa. There's dozens of initiatives to push for.
4. Similarly work to properly regulate Ai before the genie is out the bottle.
5. In the abscense of the ability to do 1-4 for whatever reason, research efforts out there and use capitalism to support them financially. Anonymously if need be.
Quittog your job without. Collective bargaining is a drop of water on a clear sunny day. You need to make it rain, and we can only do that together. Or have enough money to lobby he government to do your bidding. Too bad people at that level only seem to resort to authoritarian efforts.
> I believe, at least, that we should be demanding political accountability and safeguards for society.
You can't help but swing between "AGI is going to save us, praise the tech lords" and "AGI is going to kill us, tech lords have mercy" if you believe there is no counterpower to the tech lords.
"You, alone, leaving your job" is not a great way to counteract the tech lords (although at least it makes a point and show other people there is a problem.)
But there is the option to use your counterpowers (you know, legislators and all that ?) The tech lords are actively trying to avoid that (see the hilarious Musk vs Breton feud.)
It would be better if your system did not give money to power to choose lawmakers.
Maybe AI will make USA realize the definition of corruption and proper election funding laws.
But, if you don't want to join the underclass, maybe, just maybe, consider not picking tech lords as kings next time.
> I'd like to understand what a real, good alternative is.
The "real, good" part all depends on your expectations of life.
There's a real shortage in trades people and I'd love to see ChatGPT fix a leaky pipe, build a house or make a chair. So switching to the trades/manual labor, while financially tough at the beginning might be a good long term choice. But this requires much more physical work than most of us on HN are used to.
Moving away from capitalist society into a cheap tiny off grid house in a rural area and leading a much more basic life is also an option. You don't need 100k to survive, but you do need it in populated areas. (Also, I'm European and therefore not dependent on employment for health care, so I'm ignoring that part.)
There are many choices we can make that remove our dependence on big tech. But big tech is hella convenient and so is having expendable income, so it's a tough choice to make.
traveling more would inspire one to think positively of capitalism, rather than the reverse. to quote andrew carnegie roughly, the status quo has always been misery for everyone, and just recently have we begun to extricate ourselves from it. not to mention that it is sheer derangement of luxury to have plenty of funding for one's own family, and yet vocally dissuade others from taking the same steps, for some "end game theoretical" that certainly won't arrive in single digit generations
Wild that anybody thinks simply being in the office makes employees work. I have a colleague who sits within 2m of me who keeps their (personal) phone on their desk all day. They literally prop it up against the bottom of their computer monitor. They're not even subtle about it.
They get distracted every 2-3 minutes and spend upwards or 2-3 hours on it. It distracts me when it vibrates 100+ times per day.
Boss walks in, phone is down.
Boss walks out, phone in hand.
Is it not likely that people are more motivated to collaborate, talk about their work, plan together, feel a sense of excitement about work, etc. when they are communing in person? The ol watercooler mindset or whatever.
I mean - there’s this popular topic of the issue of loneliness lately. People are less motivated to do things that would maybe normally bring them social joy and get them out of their own homes and bring them together with others in the flesh. You’d expect people to be motivated to do that kind of thing, maybe? But it’s hard. And it’s harder every day when there’s a zeitgeist of growing isolationism.
I certainly don’t think the inflexibility of a 5 day in person work week with a hellish, uncompensated commute is the answer to the loneliness issue, nor the lack of motivation to do good work. But maybe there is some middle ground that would serve as a kick in the pants of sorts, without making us all miserable little ants going to and fro once again, that could help people get back out there in a way that helps.
I mean, at least, it doesn’t seem like the metaverse or whatever else is filling that gap as the techno-seers foresaw… but maybe future generations will prove that to be more realistic than bringing people back out together in meatspace. Or maybe we just stoop deeper into this new reclusiveness without any real stand ins for grabbing lunch together at all.
Maybe if the office was not a hellscape? Not just the commute, the offices themselves.
I didn't work in a properly colocated team since 2017, and that was mostly by accident. The norm is zoom/teams calls, often taken from the desk (which is 3-4 in a row with rows densely packed) because there's never enough space for meeting rooms so it becomes norm to not give a fuck that nobody can concentrate because someone is talking loudly on a work meeting.
And the watercooler is either office politicking or discussing how much the place sucks
Exactly this, its great that the person next to me can stand and talk to someone 2 desks down, over my shoulder while I'm on a teams call with someone from the other side of the floor, as there are only 3 conference rooms, and managers have priority. If you want people back in the office, redesign the whole space to small working areas where people can actually focus. Open office environments are the worst office experience possible, but i guess it makes the C-suite feel powerful or something having all these people sitting outside their office.
Well yeah, and that's actually the point: if you don't like it, you're free to leave! Headcount reduction without severance payment and getting rid of an unmotivated employee, win-win! At least for federal employees they had the decency of spelling it out clearly: https://traumaawareamerica.org/2025/04/28/deliberate-strateg... - the rest of us have to keep listening to the "it's all for your best" BS...
Sure, if you feel lonely and want the company of your co-workers, you're free to come to the office as often as you want. It's being forced to come to the office 3/5/whatever days that is actually decreasing motivation...
...and being forced back to the office for first three and then five days (as Elon Musk said years ago, you can work from home all you want, you just have to work 40 hours per week in the office) is not really going to improve your motivation.
There are concerns besides spying if you really don't trust the source of an open model. One is that the training incorporates a bias (added data or data omission) that might not be immediately apparent but can affect you in a critical situation. Another is vendor lock-in, if you end up depending on specifics of the model that make it harder to swap later.
It goes for all models though if you are looking at the values argument that original commenter made -- western values are probably more aligned than authoritarian governments - even if you do have your concerns about western companies. At least thats my read on the situation.
yeah, but try to convince a board or legal about it for a company that is not software first, for that they have to understand how it works. we have "chinese" AI blocked at work, even through i use self hosted models for myself at home hacking on my own stuff.
Good luck convincing others of this. I know it's true, you know it's true, but I've met plenty of otherwise reasonable people who just wouldn't listen to any arguments, they already knew better.
It's theoretically possible that your model will work OK except for code generation for security-relevant applications it will introduce subtle pre-designed bugs. Or if used for screening CVs it will prioritize PRC agents through some keyword in hobbies. Or it could promise a bribe to an office worker when asked about some critical infastructure :)
Sending data back could be as simple as responding with embedded image urls that reference external server.
You are totally right EU commissioner, Http://chinese.imgdb.com/password/to/eu/grid/is/swordfish/funnycat.png
Of course theoretically lots of things are possible with probabilistic systems. There is no difference with open source, openweight, chinese, french or american llms. You dont give unfettered web access to any models (locally served or otherwise) that can consume critical company data. The risk is unacceptable, even if the models are from trusted providers. If you use markdown to see formatted text that may contain critical data and your reader connects to the web, you have a serious security hole, unrelated to the risks of the LLM.
Of course, you want to limit that with training and proper procedures. But one of the obvious precautions is to use a service designed and controlled by a trusted partner.
Having the local LLM process sensitive data is a desirable usecase and more trustworthy than using a “trusted partner” [0]. As long as your LLM tooling does not exit your own premises, you can be technically safe. But yes, dont then click at random links. Maybe it is generally safer to not trust the origin of the local LLM, because it reduces the chance of mistakes of this type ;-)
[0] Trust is a complicated concept and I took poetic license to be brief. It is hard to verify the full tooling pipeline, and it would be great if indeed there existed mathematically verifiable “trusted partners”. A large company with enough paranoia can bring the expertise in house. A startup will rely on common public tooling and their own security reviews. I dont think it is wise to share the deepest darkest secrets with ourside entities, because the potential liability could destroy a company, whereas a local system, disconnected from the web, is technically within the circle of trust. Think of a finance company with a long term strategy that hasnt unfolded yet, a hardware company designing new chips, a pharma company and their lead molecules prior to patent submission, any company that has found the secret sauce to succeed where others failed—-none of these should be using trusted partners in favor of local LLM from untrusted origins IMHO. Perhaps the best of both worlds is to locally deploy models from trusted origins and have the ability to finetune their weights, but the practical processing gap between current chinese and non-chinese models is notable.
Maybe it can not spy on you but models can be totally (e.g. politically) biased depending on the country of origin. Try to ask european-, us- or china-trained models about "Tiananmen Massacre" and compare the answers. Or consider Trump's recent decisions to get rid of "woke" AI models.
Classic problem: "Who do you love more: mum or dad?" ;) Surely it's naive thinking but as the EU citizen I feel like I've got a little more influence on "European censorship" than on any other. I suppose that ASML feels the same way
Exactly. AR is still extremely early days, limited by hardware and software. I have no doubt that it has a future, there are just some impediments that have yet to be remedied (but I have no doubt that they will)
Wait, I absolutely would download a car if I could... or food... or clothing... I'd download the shit out of physical goods if the technology existed. Who wouldn't? You could solve scarcity. If we had Star Trek Replicators, we'd be living in a literal utopia.
> it was originally “You wouldn’t steal a car”, which I’d argue is true for most people.
Sure, but it's only true if you stretch the definition of what's occuring. If we stretched it in the other way, in that "stealing" a car in fact left the perfectly fine original right where you found it, the vast majority wouldnt think twice.
So long as you’ve paid for it before… maybe not. In many jurisdictions you are entitled to a backup. The fact that you have to pirate it… might be a gray area.
I could copy A New Hope once for every atom in the universe, and no money is lost and the original continues to exist.
Theft is moving stuff. You can't move software or digital assets, you can only copy them.
If I committed a burglary and instead of taking your TV I go to Walmart and buy a copy, then that's not burglary. You certainly wouldn't report me to the police.
I'm not concerned about my files being leaked because that's stealing. I'm concerned because they hold sensitive information that can be used for actual stealing, like for example with money.
Malware isn't bad because it's stealing. That's stupid. I know you know it's stupid, so I don't know why you said it.
The good ol' AGI and then ASI singularity everyone likes to talk about. To be fair, it is possible.
reply