I've realized my fear is not super-intelligent AI, but AI that is 20% worse than the typical knowledge worker and 90% cheaper. This could mean interacting with a ton of crappy AIs because companies will rush to save money with cheap "good enough" AI substitutes.
The economic basis of this fear seems solid, and it is pretty technically feasible already.
This is the relationship of ikea furniture to real furniture, of McDonald’s burgers to real burgers, of grocery store tomatoes to real tomatoes, of coach air travel to international first class.
Companies are already doing this. And they _suck_. And IMHO they are going to continue to suck more or less indefinitely, because the hallucination problem is inherent to the model. LLMs have no relationship to _truth_. They're truthiness machines -- they try to say stuff that sounds plausible, that _feels_ true, whether or not it _is_ true. When you're trying to get help reaching the _correct_ solution to a problem, that's a shitty model. You need a system that can be trained on _actual interactions with the product it's trying to troubleshoot_, such that its "solutions" can be tested against reality. At minimum you'd need to take a large corpus of historical cases, have it offer its diagnosis, and get reinforcement based on whether its diagnosis would have solved those cases.
Yeah. We can't rule it out. Given the pace at which AI is improving and the low variable costs, I would bet that we would move through any "crappy AI" phase pretty quickly. It may be hard to keep an LLM from hallucinating, but I don't think it's that hard to build a reliable system incorporating an LLM (e.g., have another LLM check it). Every AI system won't be ideal, but for most use cases it seems it'll be worth it to invest enough to have a good product.
Quite true. As Alice Kahn said, "For a list of all the ways technology has failed to improve the quality of life, please press three." But all of those ugly cases are situations where the ultimate customer would rather save a buck in the price of the product rather than get good service. E.g. first-class air travel now costs a bit less than coach did in 1977, and is at least as nice as coach was. But I choose to save the money and endure the smaller seats ... and the masses fly cattle-car coach rather than take the bus.
I will not go as far as Acemoglu does, but I share some of his views. My main take is that there are currently unrealistic expectations about what that technology can do, which may lead to its demise. Basically, we may reach a level where the productivity gains of implementing advanced algorithms are simply too low to justify the costs of maintaining and developing the technology.
I feel that software engineers are putting themselves unwillingly in a trap by shooting for the stars, rather than betting on improving productivity first and foremost, i.e. by creating tools that complement humans, rather than attempt to replace them. I also believe labelling advanced algorithms as "AI" is a very bad idea, as it reinforces those expectations among investors, who have far less of an understanding what the technology can and cannot do.
In essence, the current technology is very good in automating predictable tasks, but very poor in improvisation and quick adaption under unforeseen circumstances. It repeats the models we have seen in supercomputers, which can beat humans by sheer brute strength, not through original thought.
Thus, my expectation is that the technology will not be used appropriately, and as a result, it will lead to a lot of wasted money. This will be not because of an inherent flaw, but due to unrealistic expectations on the investor side. It doesn't mean the technology is bad, it just seems that many will use it in an entirely wrong away. I consider 3D cinema a good analogy, as it was expected to turn the way movies are shot on its head, and in the end, it turned out that the benefits are far less than expected initially.
Also, I think it is time we stop calling those advanced algorithms "AI" and then try to come up with some contrived terms for actual AI. There is a great term for what we currently have, coming from a popular sci-fi video game series - Mass Effect. There, they call advanced algorithms "Virtual Intelligence", or simply VI, which is a catchy enough term, by the way. Essentially, ChatGPT is nothing more than an Avina, and certainly not EDI (those who have played the games will get the reference).
> I think it is time we stop calling those advanced algorithms "AI" and then try to come up with some contrived terms for actual AI.
This is already what we have the term “AGI” for. I think it doesn’t help to pretend that there’s no intelligence in things like frogs and image classifiers and trees and language models just because their information processing is specialized and has limits that human intelligence does not.
Exactly. I dont find Noah's view that the "future" and the "past" as interconnected and thus a predictor of innovation and productivity growth as persuading. Im not sure Acemoglu is right either but only time will tell I guess.
I work for a company that has developed hardware and software that is pulled by a tractor to thin lettuce. The hardware takes photos of lettuce beds while rolling through a field at about 3 miles per hour. Then, the software uses machine learning to analyze these photos to distinguish crop plants from weeds and dirt, decides which lettuce plants to save and which to kill since farmers typically over-plant, and directs the hardware to spray a kill solution on areas containing the plants to remove, and optionally spray a secondary treatment on the plants to save.
The company operates a service that thins a significant portion of the lettuce acreage in the Salinas Valley. This service results in up to a 90% reduction of chemical use, which has environmental benefits as well as financial benefits to the farmer (some ag chemicals are $1000/gallon). There's also the public health risk reduction of having fewer people in the fields less often due to having the tractor substitute for thinning by hand, since pathogens aren't killed during typical meal preparation of lettuce since lettuce is typically consumed raw.
This use of AI is replacing human jobs, and adding fewer jobs. The benefits of that replacement are pretty substantial, reducing demand for the limited supply of farm laborers as well providing environmental and health benefits, cost reductions, and increased production. The environmental conditions in fields can be pretty hard on workers and equipment; we have to use cooling systems and dust filters for the computers running on the tractors and we replace sprayers pretty often, and our operators work in air-conditioned tractors unlike the workers performing hand thinning. In today's employment market I would think laborers are not going unemployed, especially since there is still a lot of acreage processed by hand, including for other crops. The income problem our operators have is often due to weather preventing them from working.
I'm always looking for evidence of bad decision-making in the machine learning when I am watching the thinners work on some lettuce field. Occasionally I see some questionable decisions; sometimes those we have found result from bugs that we can fix. The problems that occur are rare and minor enough so the farmers continue to find they are profiting from this approach.
I wish machine learning could provide the basis for every thin/save decision it makes (I wish the same thing for LLMs and their decisions and creations). I think testing would be easier since we could set up tests for the various decision trees and know we had good test coverage.
I run a bioinformatics startup. What Noah describes is very real. We are using agentic AI to extract insights in a way that was not previously possible. We aren't replacing anyone, in fact we are now hiring to expand on capabilities that never existed before.
Connectivity. There are a huge number of problems related to connecting disparate sources of information that are borderline impossible to hard code because the variables and context change so frequently. Agentic AI gives us the flexibility to create tools that I refused to work on as recently as two years ago.
Thanks! How much of the value is coming from having the agent summarize versus doing more, like transforming the data?
It seems to me like a core value driver of LLMs is "parsing data that is more voluminous than I would choose to review myself." I just needed to get up to speed on an area of FDA enforcement, dumped all the warning letters into Notebook LM and had a quick summary.
The other common value driver -- create something based on a brief text request -- seems like hit-and-miss at best and mostly a novelty at worst. The clearest example is code assistants where the value seems clear. Creating images for blogs also qualifies. But the value is inversely correlated with the precision required of the output. A blog is fine if the image isn't exactly what they wanted, but that may not be the rule.
LLM's don't touch our data. They don't understand it at all and I would never let them transform a perfectly good measurement. I don't use them for code completion either. I enthusiastically switched IDE's to try out Copilot. About a month later, I switched back. It wasn't just that the suggested code didn't work (it usually didn't), but the AI broke perfectly good functionality like autocomplete for typing directory names.
Imagine if you could hire an assistant with a photographic memory of everything your organization told him to believe. Further, give that assistant the ability to understand and navigate a fairly complicated software platform that is used by the rest of your organization. LLMs make it possible for every member of our company, and all of our customers, to have a personalized version of that assistant, providing language based access to more information than any of us could possibly store in our own heads. It's incredible.
2 good examples - LLM would be horrible for the real problem that Jonathon is solving (and I agree a ton on the "autocomplete" - after building software tools for 45 years - useful for a beginner and horrible for a pro).
The personalized assistant is also a good one - I'm more dubious than Jonathon about the accuracy of those LLMs and using them to access key info. BUT most people aren't great at how to search for key info (and perhaps it wasn't set up to be searched as well as the LLM is set up to do). Glad that Jonathon is having good luck with it - others have had more mixed results.
Call centers are an abomination. Automated ansering robots are designed to get you to hang up. They make it difficult to talk to a human who can actually understand your problem. I don’t how the current AI can do a better job.
For a Doctor or Nurse to have AI comment on the list of symptoms a patient is describing and print out a list of tests necessary to rule out evyer possible disease, virus or bacterial infection might be helpful. AI reading of mamograms or tumors will be helpful.
How about the military. In the movie War Games the WOPR looked at outcomes of various nuclear conflicts. it came to the conclusion that there is no winner. That said can it help design a remote controlled fighter jet? A new bomb? Wait and see.
As I wrote in your column on the Longshormen strike. Blacksmiths became machinist and mechanics.
"Call centers are an abomination. Automated ansering robots are designed to get you to hang up. They make it difficult to talk to a human who can actually understand your problem. I don’t how the current AI can do a better job."
It can, by rethinking this. Call centers work this way because human labor is expensive. FWIW, they are not designed to "get you to hang up", they are designed to force YOU to spend YOUR time (which is "free" to the company) to solve the problem, rather than to spend a human agent's time (which is expensive) solving the same problem. It's super annoying, yes, but that is because the entire system places no value on YOUR time, which means there is little incentive to maximize the efficiency of YOUR time.
If, however, you removed human labor entirely, the AI has no financial incentive to gatekeep. Cost simply scales linearly with compute, which is going to scale linearly with how long the call lasts. In this case, its incentives are simply to solve the customer's problem as efficiently as possible, because there simply is no cost savings involved in making the customer waste time jumping through hoops. That is, in theory, the AI's incentives and the customer's incentives align.
Now, whether the people employing AI actually recognize this at first is a good question. My prediction is "no", because people are still in the "do everything the same way we always did" mode, but that eventually someone realizes the problem, becomes the first to do it a different way, and customers flock to that company because of its superior customer experience. At which point everyone else says "Oooooh, yeah, right, of course, we're doing it wrong" and copies the pattern (which they will be able to do pretty easily, because, well... AI. It's basically just a manner of deploying a new LLM configuration).
I hope you are right - but I haven't seen anyone flock to a company that has superior customer experience in handling calls (maybe because most are so so awful). You might be right and I have definitely stopped using companies because the customer service was so awful - but often there aren't a lot choices to choose from.
But here is another question for you: Let's say you were offered the services of an army of people who were smart enough to pass the LSATs, but that occasionally made glaring mistakes, and that maybe 20% of the time they take some bad LSD and just trip out for a while and are useless until you intervene. .... BUT, their labor costs are zero.
Could you think of any uses for them?
I think the answer should be "hell yes", but it isn't "Hell yes, I will just let all my current employees go and replace them with my army full of smart, LSD-tripping dudes"
My guess is that human intervention will needed for quite sometime. There is a vast difference for example on knowing all knowledge but making a judgement about human existence.
Cory Doctorow writes about this as a centaur - a powerful horse body controlled by a human head - this is similar to you copilot example and, as a coder, I can agree that AI vastly increases my productivity. I have the vision of what needs to be done and build the prompt, the AI does the drudge work of making the actual code and then I touch it up and integrate it. Can do much more, especially if it's in a language I'm not really familiar with.
But don't discount another reason people want human replacing AI... to be Lords of their castle. They want to tell everyone around them what to do and have it be done, unquestioned and instantly. One of the worst part about humans if you have to treat them humanely, in this example, even the ones that don't actually warrant it, who don't understand the vision or the brilliance of the master. Humans have their own personal visions - for a fun weekend, or a bad relationship, or a better start up than this one - and no one has figured out how to "fix" those problems so that employees work like automatons. Starving the underclass gets you part of the way there, but you're risking an uprising.
And what if you're not really as smart as you think you are? AI won't take your job, or talk to the other AI's and point out your flaws.
I think there's a lot of psychology and sociology packed into the allure of AI for many tech/business leaders.
I like to think of AI like this. We have severe shortages of skilled labour in:
Programmers
Accountants
Electricians
Engineers
Psychologists
Project Managers
Healthcare Workers
Scientists
and many dozens or hundreds more areas of the economy.
If AI can help to supercharge the productivity of existing workers in these fields and others by taking away some of the easily automated work; improve supply of workers by speeding up the training process; lift the quality of work performed; or some combination of all of these - imagine how much more capacity our economies will have to solve future problems or transform our lives. I like to think that there are endless possibilities that we just haven't had the good fortune to stumble across yet.
This is embarrassing, but my use of IA has helped me advance my nerdiness and run better pen-and-paper RPG games for friends and family.
It has been hugely helpful there. I just use the free Co-pilot to help generate coherent plots and some other tools to generate supporting materials (like briefings). It has saved me a bunch of time and vastly improved the quality of my narratives and supporting material. I am not even tech savvy…I just go on to co-pilot with an idea, and we work through it together.
Last time unused it was to create a series of radio transcripts for a military base being overrun by something similar to the Mist in the Steven King movie of the same name. Took 10 minutes to generate some incredible transcripts that were way scarier than I could have.
I was doing this because l asked the co-pilot to help me outline a series of adventures with some parameters that defined the overall flow of the adventure. After getting the storyline set, I began working on the individual elements, again getting Co-pilot's help when I had writer's block or had a question (an example of a question might be finding a particular type of monster that had certain abilities or characteristics). I then used an image generator to help create some visuals. One downside was that filters would not let me generate images as scary as I had hoped, but that was a filter thing.
Long story short, I am a low-tech Gen X dungeon and dragon nerd, the kind of person who asks their children to help set up a new phone. I can still use this stuff to save time and improve my work quality on a stupid hobby. That said, I see fantastic potential for IA in entertainment. It would be revolutionary if choose-your-own-adventure books met VR in a Zoom world, where friends who are continents away could stay in touch and live out individualized movies.
This is what Noah means when he talks about rearranging the factory floor. The tools are all there….we just need to assemble them properly. I know some folks will joke because this is just a niche geek use case…. but so were games when I was growing up. Who would have thought the old Atari systems would evolve into something bigger than movies?
I suspect that when AI startup people say "AI will show value by replacing humans," what's really going on is "most companies will only buy AI today if it replaces humans and I need companies to buy my product to justify my valuation."
This is a function of LLMs not finding much product-market fit yet. When SaaS companies have product-market fit, they sell on value: the business improvement enables by their tool. When they don't, they are stuck selling on a pure cost-benefit basis.
Where the SaaS analogy breaks down is in the cost to start and scale a company. B2B SaaS companies exploded because AWS made it significantly cheaper to start a company. That meant they could live without revenues for long enough to figure out product market fit. AI is so expensive to build and run that patience isn't affordable. This is especially the case if it's hard to create sustainable differentiation. That's open to debate. But the fact that OpenAI and Google haven't turned their massive head start on development and spend into a material technological advantage versus companies like Anthropic, Mistral, and even Meta suggests that having the best model isn't sufficient.
Rather, venture capitalists are reluctant to pour money into a startup if it is projecting that the product will solve new problems, because there's no concrete proof what customers will pay to get those problems solved. So they pitch "we will replace humans doing task X (for which customer are already paying $N billions a year)". ... Remember that at this stage of the industry, the VCs are the customers, and the central question is what the VCs will buy.
Machine learning (I hate the term AI) will do for the cognitively weaker what steam did for the physically weaker.
Prior to the industrial revolution, economic success and physical strength were fairly correlated. Steam power changed that, opening opportunities for women and weaker (nerdy) men. In a simple example, a steam shovel can do the work of 10 men (which hurts the job prospects of those 10 men) but more importantly, the skills required to operate a steam shovel are different (and less physical) than those to dig a ditch.
AI is going to do that for cognitive tasks. We haven't seen much of it yet, but we will. Here's an early version of that:
Expect more of this. And expect it to go down the socio-economic ladder, as machine learning makes it possible for people who aren't that smart to compete with those who are. Considering we've spent the last 100+ years making our society more and more complex, and thus requiring higher cognitive abilities to successfully navigate it and make it to the middle class, I think this is a great thing. It's high time many of those people we been pushing to the margins for generations have a chance to get back into the game.
What steam power did for muscles, machine learning will do for minds.
"It’s quite possible that when it comes to AI, power and truthfulness just aren’t correlated. ... It seems quite possible that the AI we’ve created is a truly alien intelligence."
However, despite my optimism, these are rather terrifying statements, Noah,
> Prior to the industrial revolution, economic success and physical strength were fairly correlated
That’s really not true at all. The key to success was by and large to be born wealthy, and outside of that in many societies intelligence mattered for any kind of social climbing - maybe through a priesthood, apprenticeship or exams in the case of China.
Slaves and serfs were strong but not wealthy. You might be a slightly better off serf if you were strong, but intelligence mattered as much there too. It was running a business of sorts.
> And expect it to go down the socio-economic ladder, as machine learning makes it possible for people who aren't that smart to compete with those who are.
Maybe but I suspect it work like leverage - AI gains are amplifications or human intelligence, or they replace everybody. If it’s the former then the smart are paid even more, if it’s the latter then nobody earns money from “smart work”.
However because robots are not improving as fast as machine learning it’s likely that being a plumber will be even more lucrative relative to going to university than it is now.
The actual task used was classifying and age of subjects by pictures, so no realistic rom that perspective. But it hints that AI might be most beneficial for the least skilled among us (Yeah, I know, not exactly Earth-shattering, but research papers are known for stating the obvious in lots of words.)
Also, your comment about plumbers is spot on. A good plumber can pull in 6 figures already. It's amazing what suburban women will pay to not have to deal with poop.
I hope you're wrong about your leverage idea Peter, since rendering larger and larger percentages of your society economically disposable is a bad plan for a republic. I'm hopeful machine learning might be able to alleviate some of that. But certainly the book illustrator example I used is closer to your own theory than to mine. We shall see.
Well in reality the main divisions of income and wealth are really between the owners of capital and the rest of us. We all own, to a rounding error, about 0% of the wealth of billionaires. Making smarter workers poorer isn’t going to change that.
The net wealth of the 50th-90th percentile group (what most people would describe as middle-class+) is roughly equal to the wealth of the top 10%. That's total wealth not per-capita, so we've got about 4X less overall, but far from the rounding error. These are mostly college-educated, laptop-class workers or small business owners. My wife and I fall into this group; she's a teacher and we both have college degrees.
However, below that (the "working poor") things go downhill quickly. It is this high-school only, blue-colllar group that I'm most concerned about. They've gotten screwed by 50 years of uniparty neoliberalism. I'm hopeful that machine learning might be able to help some of them rise, and perhaps some of that bottom 20% to become economically useful again.
Perhaps I'm being optimistic, but the only alternative I see is UBI, and that's spiritual death -- an public acknowledgement that the bulk of our society is economically useless. I can't get Player Piano by Vonnegut out of my head.
> The net wealth of the 50th-90th percentile group (what most people would describe as middle-class+) is roughly equal to the wealth of the top 10%.
Thanks for that but it’s not what I’m talking about. I don’t even consider the top 10% to be rich in comparison to billionaires. You can’t use that kind of percentage distribution for Pareto distributions as unlike normal (ie bell shaped) distributions the top 10% are much closer to the bottom than the top.
A rounding error of 0.1% - which is low - applied to the lowest possible billionaire (ie a guy with $1B) would mean it’s reasonable to call somebody with $999M a billionaire. Which means he’s a billionaire +- $1M. Which means that we can say to a rounding error he’s got $1B more than a homeless guy and $1B more than a millionaire.
In reality most billionaires are multi billionaires and the rounding error is higher. I don’t think we would say a guy with $995M isn’t a billionaire. Musk is worth $269B.
I've seen this in my field - trying to work out how to use AI effectively is very tough - it requires a totally new way of thinking.
For example - AI is useless in automating monthly reports - traditional BI/coding is very efficient and doesn't hallucinate. You really need to think around the problem and try to identify new ways of solving problems, a much harder issue!
There's a nice insight here: "It’s just a heck of a lot easier to think of a task that people already do, than to think of a task that no one has ever done before". But I also think this piece makes an important (and very common) error: treating AI as a static thing rather than an evolving technology. When we ask whether AI will complement humans or replace them, do we mean the AI of 2024, 2030, or 2100?
You are talking about AIs that have inherent weaknesses, such as a tendency to hallucinate or an inability to carry out long-term creative thinking. That's a good description of AI today, and likely in the near to mid term as well. But it's not necessarily a good description of the AIs we'll have in, say, one to four decades. AI capabilities have grown dramatically, and will almost certainly continue to do so. For this reason, I think it's a fundamental error to talk about the impact of AI without specifying which level of AI capability we have in mind.
(I don't mean to single you out here; it's very common for anyone talking about AI to fail to specify a particular stage of AI, which leads to a lot of people talking past one another.)
One other note: you say that o1 "hallucinates much more than older models", but the link you cite doesn't say anything of the sort; it merely notes that there are several different places in which hallucinations can occur (in the hidden chain of thought, in the translation to the summarized chain of thought, or in the reported result). From everything I've seen, o1 is much more reliable (less likely to hallucinate) on a given task. In particular, it is able to carry out tasks requiring long reasoning chains, and it sometimes hallucinates on these tasks, but older models would be virtually *certain* to go awry on those same tasks.
I think hallucination is likely very deep and impossible to eliminate. It’s a byproduct of having a neural structure reconstruct sentences from an underlying knowledge trace that doesn’t have the same structure as language. I think “hallucination” was the right term when we were talking about image classifiers that force every image into one of their categories. But with language models, “confabulation” is a better term, and humans do it all the time too, like when Walz misremembers or misspeaks about having been in Hong Kong in June 1989, or when I misremember that time in 2002 when I got lost in San Francisco trying to find a friend’s house but realized I was carrying my laptop, and I was able to find an open WiFi signal, and I recall using gps and Google maps to find where I was going, but in 2002 it must have been Mapquest, and using the street signs to identify the intersection I was at.
Efficient neural memory doesn’t store every detail, but instead stores lots of gists, and lets details be filled in smoothly by background knowledge, even though it doesn’t always get things exactly right. I think “hallucination” will never be eliminated, but instead will just get under better and better control, but still will arise in edge cases at the frontiers of whatever the best capabilities of the system are.
Another thing I often think about is the difference between heuristics and algorithms (I mean the latter in the strict, original, sense of a "procedure for solving a problem", not the way everyone uses it now to describe any piece of software that makes an arbitrary decision).
Heuristics are often *much* faster than algorithms because they use a lot of shortcuts to jump to the (probably) correct conclusion. The problem is that, by definition, heuristics get things wrong because of bias.
So... now you've trained an LLM based on a huge body of content that was almost entirely created by... human beings using heuristics. Removing bias/hallucination from that LLM is... I won't say it is impossible, but it is definitely in the "holy grail" category of problems to solve, you aren't just going to iterate rapidly into the solution. And there are going to remain lots of problems where 99% (or 99.999%) accuracy isn't acceptable.
Some heuristics have bias. But others are just noisy. The heuristic of running away from anything you barely see in the shadows is biased one way, and the heuristic of moving towards things you barely see in the shadows is biased the other way. The heuristic of ignoring things you can't clearly see avoids either of those biases, but is noisy (though it's easy to implement).
Removing hallucination from AI would be like removing stereotypes from humans - you could only do it if you took away most of the neural methods that make our minds work usefully. But we can introduce features and training that minimize the particular types that are most likely to be harmful.
For what it's worth, I think the original definition of "algorithm" was always just a sequence of instructions to follow, and remained neutral on whether it was a perfect way to achieve one's goal, or was at least heuristically useful, or was irrelevant or even counterproductive.
The problem with removing hallucination by introducing features and training that minimize them is that you end up introducing a reverse bias (because we are human, our solutions also have bias). Then you end up with nonsense like Gemini refusing to paint a picture of George Washington in which he is caucasian, or you get LLMs that refuse to talk about the history of the German Nazi party, or the Jewish holocaust, because they "think" doing so is anti-semetic.
So far, the "safeguards" that try to counteract hallucination or bias have been very ineffectual or have very unwanted side effects. It remains an incredibly hard problem to solve, sort of like trying to teach "common sense" to an alien.
Yes. And similarly for humans, if you train people to avoid the errors that enable an interviewer to get them to recall the time they met Bugs Bunny at Disney World, they usually just end up being overly skeptical of their childhood memories generally.
Ethan Molic writes perceptively about the current uses of AI, but even he does not yet see the trend he demonstrates. AI works as guerrilla warfare.
Nowhere in science fiction has anyone written about how society's institutions will fare when they are attacked by an infinite number of bored teenagers with unlimited access to star powered AI resources. Yet schools are reeling from the impact on homework assignments. Molic points out that middle managers are using AI, not to enable the goals of their employer, but to advance their own powers in the firms.
In the internet hubbub we have forgotten that real entrepreneurship is simply finding the shortcuts to money.
Imagine the results if I make some subversive inquiries of my AI:
"Show me 50 ways around this patent."
"What lies can I tell the IRS in to order decrease my tax bill and what are the risks of each one."
"How can I pad my dissertation to make it look more important."
"What are the best ways to deceive my wife about my affair?
"Invent 50 useful lies that MAGAs are likely to believe."
Well, maybe not Ethan Molic. I can't seem to find the right citation. Help me out!
In the PC revolution we thought that we had empowered the little guy, but we enabled monsters like Facebook. Everyone is assuming that AI will enable Big Brother but current usage indicates that we are headed toward some serious chaos by enabling the little guy.
We're also failing to recognise the roles where human empathy is paramount. Even if an AI agent could _in theory_ replace the work of a human, in many cases, the end customer/user doesn't want that.
Whenever I speak to an AI agent or bot instead of a human being for a customer service issue, it's trust the brand is burning. Yet most continue to do so for the sake of "efficiency". Efficiently losing my trust!
I agree with your idea that it takes 20-30 years before people figure out a new technology. There are two other trends that tend to track with a major technological change. The new technology disrupts the balance of power between workers and owners. Nothing inherent in the technology drives job loss and lower wages, but when it empowers people whose goal is to reduce labor costs, that will happen until a new equilibrium is reached (with or with government intervention on the side of labor). The introduction of new technology is also fervent ground for marketing hype and financial bubbles. I’m convinced that that is the case with LLMs. There is great potential for AI to increase productivity once people figure out how to use it in new ways, but this is not going to come from LLMs. Training neural networks with petabytes of uncurated language does not produce a “brain” any more reliable than an average human. The AI revolution will come, but not before the LLM shakeout.
I've realized my fear is not super-intelligent AI, but AI that is 20% worse than the typical knowledge worker and 90% cheaper. This could mean interacting with a ton of crappy AIs because companies will rush to save money with cheap "good enough" AI substitutes.
The economic basis of this fear seems solid, and it is pretty technically feasible already.
This is the relationship of ikea furniture to real furniture, of McDonald’s burgers to real burgers, of grocery store tomatoes to real tomatoes, of coach air travel to international first class.
Yeah, Noah mentioned the idea of AI call centers.
Companies are already doing this. And they _suck_. And IMHO they are going to continue to suck more or less indefinitely, because the hallucination problem is inherent to the model. LLMs have no relationship to _truth_. They're truthiness machines -- they try to say stuff that sounds plausible, that _feels_ true, whether or not it _is_ true. When you're trying to get help reaching the _correct_ solution to a problem, that's a shitty model. You need a system that can be trained on _actual interactions with the product it's trying to troubleshoot_, such that its "solutions" can be tested against reality. At minimum you'd need to take a large corpus of historical cases, have it offer its diagnosis, and get reinforcement based on whether its diagnosis would have solved those cases.
Yeah. We can't rule it out. Given the pace at which AI is improving and the low variable costs, I would bet that we would move through any "crappy AI" phase pretty quickly. It may be hard to keep an LLM from hallucinating, but I don't think it's that hard to build a reliable system incorporating an LLM (e.g., have another LLM check it). Every AI system won't be ideal, but for most use cases it seems it'll be worth it to invest enough to have a good product.
Quite true. As Alice Kahn said, "For a list of all the ways technology has failed to improve the quality of life, please press three." But all of those ugly cases are situations where the ultimate customer would rather save a buck in the price of the product rather than get good service. E.g. first-class air travel now costs a bit less than coach did in 1977, and is at least as nice as coach was. But I choose to save the money and endure the smaller seats ... and the masses fly cattle-car coach rather than take the bus.
Good point.
I will not go as far as Acemoglu does, but I share some of his views. My main take is that there are currently unrealistic expectations about what that technology can do, which may lead to its demise. Basically, we may reach a level where the productivity gains of implementing advanced algorithms are simply too low to justify the costs of maintaining and developing the technology.
I feel that software engineers are putting themselves unwillingly in a trap by shooting for the stars, rather than betting on improving productivity first and foremost, i.e. by creating tools that complement humans, rather than attempt to replace them. I also believe labelling advanced algorithms as "AI" is a very bad idea, as it reinforces those expectations among investors, who have far less of an understanding what the technology can and cannot do.
In essence, the current technology is very good in automating predictable tasks, but very poor in improvisation and quick adaption under unforeseen circumstances. It repeats the models we have seen in supercomputers, which can beat humans by sheer brute strength, not through original thought.
Thus, my expectation is that the technology will not be used appropriately, and as a result, it will lead to a lot of wasted money. This will be not because of an inherent flaw, but due to unrealistic expectations on the investor side. It doesn't mean the technology is bad, it just seems that many will use it in an entirely wrong away. I consider 3D cinema a good analogy, as it was expected to turn the way movies are shot on its head, and in the end, it turned out that the benefits are far less than expected initially.
Also, I think it is time we stop calling those advanced algorithms "AI" and then try to come up with some contrived terms for actual AI. There is a great term for what we currently have, coming from a popular sci-fi video game series - Mass Effect. There, they call advanced algorithms "Virtual Intelligence", or simply VI, which is a catchy enough term, by the way. Essentially, ChatGPT is nothing more than an Avina, and certainly not EDI (those who have played the games will get the reference).
> I think it is time we stop calling those advanced algorithms "AI" and then try to come up with some contrived terms for actual AI.
This is already what we have the term “AGI” for. I think it doesn’t help to pretend that there’s no intelligence in things like frogs and image classifiers and trees and language models just because their information processing is specialized and has limits that human intelligence does not.
Terrific analysis!
And terric reader connenrs to noah's piece
Exactly. I dont find Noah's view that the "future" and the "past" as interconnected and thus a predictor of innovation and productivity growth as persuading. Im not sure Acemoglu is right either but only time will tell I guess.
I work for a company that has developed hardware and software that is pulled by a tractor to thin lettuce. The hardware takes photos of lettuce beds while rolling through a field at about 3 miles per hour. Then, the software uses machine learning to analyze these photos to distinguish crop plants from weeds and dirt, decides which lettuce plants to save and which to kill since farmers typically over-plant, and directs the hardware to spray a kill solution on areas containing the plants to remove, and optionally spray a secondary treatment on the plants to save.
The company operates a service that thins a significant portion of the lettuce acreage in the Salinas Valley. This service results in up to a 90% reduction of chemical use, which has environmental benefits as well as financial benefits to the farmer (some ag chemicals are $1000/gallon). There's also the public health risk reduction of having fewer people in the fields less often due to having the tractor substitute for thinning by hand, since pathogens aren't killed during typical meal preparation of lettuce since lettuce is typically consumed raw.
This use of AI is replacing human jobs, and adding fewer jobs. The benefits of that replacement are pretty substantial, reducing demand for the limited supply of farm laborers as well providing environmental and health benefits, cost reductions, and increased production. The environmental conditions in fields can be pretty hard on workers and equipment; we have to use cooling systems and dust filters for the computers running on the tractors and we replace sprayers pretty often, and our operators work in air-conditioned tractors unlike the workers performing hand thinning. In today's employment market I would think laborers are not going unemployed, especially since there is still a lot of acreage processed by hand, including for other crops. The income problem our operators have is often due to weather preventing them from working.
I'm always looking for evidence of bad decision-making in the machine learning when I am watching the thinners work on some lettuce field. Occasionally I see some questionable decisions; sometimes those we have found result from bugs that we can fix. The problems that occur are rare and minor enough so the farmers continue to find they are profiting from this approach.
I wish machine learning could provide the basis for every thin/save decision it makes (I wish the same thing for LLMs and their decisions and creations). I think testing would be easier since we could set up tests for the various decision trees and know we had good test coverage.
I run a bioinformatics startup. What Noah describes is very real. We are using agentic AI to extract insights in a way that was not previously possible. We aren't replacing anyone, in fact we are now hiring to expand on capabilities that never existed before.
What use cases is it proving helpful with?
Connectivity. There are a huge number of problems related to connecting disparate sources of information that are borderline impossible to hard code because the variables and context change so frequently. Agentic AI gives us the flexibility to create tools that I refused to work on as recently as two years ago.
Thanks! How much of the value is coming from having the agent summarize versus doing more, like transforming the data?
It seems to me like a core value driver of LLMs is "parsing data that is more voluminous than I would choose to review myself." I just needed to get up to speed on an area of FDA enforcement, dumped all the warning letters into Notebook LM and had a quick summary.
The other common value driver -- create something based on a brief text request -- seems like hit-and-miss at best and mostly a novelty at worst. The clearest example is code assistants where the value seems clear. Creating images for blogs also qualifies. But the value is inversely correlated with the precision required of the output. A blog is fine if the image isn't exactly what they wanted, but that may not be the rule.
LLM's don't touch our data. They don't understand it at all and I would never let them transform a perfectly good measurement. I don't use them for code completion either. I enthusiastically switched IDE's to try out Copilot. About a month later, I switched back. It wasn't just that the suggested code didn't work (it usually didn't), but the AI broke perfectly good functionality like autocomplete for typing directory names.
Imagine if you could hire an assistant with a photographic memory of everything your organization told him to believe. Further, give that assistant the ability to understand and navigate a fairly complicated software platform that is used by the rest of your organization. LLMs make it possible for every member of our company, and all of our customers, to have a personalized version of that assistant, providing language based access to more information than any of us could possibly store in our own heads. It's incredible.
2 good examples - LLM would be horrible for the real problem that Jonathon is solving (and I agree a ton on the "autocomplete" - after building software tools for 45 years - useful for a beginner and horrible for a pro).
The personalized assistant is also a good one - I'm more dubious than Jonathon about the accuracy of those LLMs and using them to access key info. BUT most people aren't great at how to search for key info (and perhaps it wasn't set up to be searched as well as the LLM is set up to do). Glad that Jonathon is having good luck with it - others have had more mixed results.
Call centers are an abomination. Automated ansering robots are designed to get you to hang up. They make it difficult to talk to a human who can actually understand your problem. I don’t how the current AI can do a better job.
For a Doctor or Nurse to have AI comment on the list of symptoms a patient is describing and print out a list of tests necessary to rule out evyer possible disease, virus or bacterial infection might be helpful. AI reading of mamograms or tumors will be helpful.
How about the military. In the movie War Games the WOPR looked at outcomes of various nuclear conflicts. it came to the conclusion that there is no winner. That said can it help design a remote controlled fighter jet? A new bomb? Wait and see.
As I wrote in your column on the Longshormen strike. Blacksmiths became machinist and mechanics.
"Call centers are an abomination. Automated ansering robots are designed to get you to hang up. They make it difficult to talk to a human who can actually understand your problem. I don’t how the current AI can do a better job."
It can, by rethinking this. Call centers work this way because human labor is expensive. FWIW, they are not designed to "get you to hang up", they are designed to force YOU to spend YOUR time (which is "free" to the company) to solve the problem, rather than to spend a human agent's time (which is expensive) solving the same problem. It's super annoying, yes, but that is because the entire system places no value on YOUR time, which means there is little incentive to maximize the efficiency of YOUR time.
If, however, you removed human labor entirely, the AI has no financial incentive to gatekeep. Cost simply scales linearly with compute, which is going to scale linearly with how long the call lasts. In this case, its incentives are simply to solve the customer's problem as efficiently as possible, because there simply is no cost savings involved in making the customer waste time jumping through hoops. That is, in theory, the AI's incentives and the customer's incentives align.
Now, whether the people employing AI actually recognize this at first is a good question. My prediction is "no", because people are still in the "do everything the same way we always did" mode, but that eventually someone realizes the problem, becomes the first to do it a different way, and customers flock to that company because of its superior customer experience. At which point everyone else says "Oooooh, yeah, right, of course, we're doing it wrong" and copies the pattern (which they will be able to do pretty easily, because, well... AI. It's basically just a manner of deploying a new LLM configuration).
I hope you are right - but I haven't seen anyone flock to a company that has superior customer experience in handling calls (maybe because most are so so awful). You might be right and I have definitely stopped using companies because the customer service was so awful - but often there aren't a lot choices to choose from.
LOL, what use then is AI if they cannot make it better?
I did not say that they cannot make it better.
But here is another question for you: Let's say you were offered the services of an army of people who were smart enough to pass the LSATs, but that occasionally made glaring mistakes, and that maybe 20% of the time they take some bad LSD and just trip out for a while and are useless until you intervene. .... BUT, their labor costs are zero.
Could you think of any uses for them?
I think the answer should be "hell yes", but it isn't "Hell yes, I will just let all my current employees go and replace them with my army full of smart, LSD-tripping dudes"
My guess is that human intervention will needed for quite sometime. There is a vast difference for example on knowing all knowledge but making a judgement about human existence.
Cory Doctorow writes about this as a centaur - a powerful horse body controlled by a human head - this is similar to you copilot example and, as a coder, I can agree that AI vastly increases my productivity. I have the vision of what needs to be done and build the prompt, the AI does the drudge work of making the actual code and then I touch it up and integrate it. Can do much more, especially if it's in a language I'm not really familiar with.
But don't discount another reason people want human replacing AI... to be Lords of their castle. They want to tell everyone around them what to do and have it be done, unquestioned and instantly. One of the worst part about humans if you have to treat them humanely, in this example, even the ones that don't actually warrant it, who don't understand the vision or the brilliance of the master. Humans have their own personal visions - for a fun weekend, or a bad relationship, or a better start up than this one - and no one has figured out how to "fix" those problems so that employees work like automatons. Starving the underclass gets you part of the way there, but you're risking an uprising.
And what if you're not really as smart as you think you are? AI won't take your job, or talk to the other AI's and point out your flaws.
I think there's a lot of psychology and sociology packed into the allure of AI for many tech/business leaders.
yes, yes, yes
I like to think of AI like this. We have severe shortages of skilled labour in:
Programmers
Accountants
Electricians
Engineers
Psychologists
Project Managers
Healthcare Workers
Scientists
and many dozens or hundreds more areas of the economy.
If AI can help to supercharge the productivity of existing workers in these fields and others by taking away some of the easily automated work; improve supply of workers by speeding up the training process; lift the quality of work performed; or some combination of all of these - imagine how much more capacity our economies will have to solve future problems or transform our lives. I like to think that there are endless possibilities that we just haven't had the good fortune to stumble across yet.
Have you been asleep since 2021? The only of these professions facing skills shortage are electricians and healthcare workers.
I think engineers on the physical side are in good shape as well.
This is not the first time I've seen AI compared to a Lovecraftian horror, but it is the first time I've seen that argued as a good thing :)
This is embarrassing, but my use of IA has helped me advance my nerdiness and run better pen-and-paper RPG games for friends and family.
It has been hugely helpful there. I just use the free Co-pilot to help generate coherent plots and some other tools to generate supporting materials (like briefings). It has saved me a bunch of time and vastly improved the quality of my narratives and supporting material. I am not even tech savvy…I just go on to co-pilot with an idea, and we work through it together.
Last time unused it was to create a series of radio transcripts for a military base being overrun by something similar to the Mist in the Steven King movie of the same name. Took 10 minutes to generate some incredible transcripts that were way scarier than I could have.
I was doing this because l asked the co-pilot to help me outline a series of adventures with some parameters that defined the overall flow of the adventure. After getting the storyline set, I began working on the individual elements, again getting Co-pilot's help when I had writer's block or had a question (an example of a question might be finding a particular type of monster that had certain abilities or characteristics). I then used an image generator to help create some visuals. One downside was that filters would not let me generate images as scary as I had hoped, but that was a filter thing.
Long story short, I am a low-tech Gen X dungeon and dragon nerd, the kind of person who asks their children to help set up a new phone. I can still use this stuff to save time and improve my work quality on a stupid hobby. That said, I see fantastic potential for IA in entertainment. It would be revolutionary if choose-your-own-adventure books met VR in a Zoom world, where friends who are continents away could stay in touch and live out individualized movies.
This is what Noah means when he talks about rearranging the factory floor. The tools are all there….we just need to assemble them properly. I know some folks will joke because this is just a niche geek use case…. but so were games when I was growing up. Who would have thought the old Atari systems would evolve into something bigger than movies?
This is a very interesting case - AI is serving to provide a social/artistic value which isn't very economically visible
I suspect that when AI startup people say "AI will show value by replacing humans," what's really going on is "most companies will only buy AI today if it replaces humans and I need companies to buy my product to justify my valuation."
This is a function of LLMs not finding much product-market fit yet. When SaaS companies have product-market fit, they sell on value: the business improvement enables by their tool. When they don't, they are stuck selling on a pure cost-benefit basis.
Where the SaaS analogy breaks down is in the cost to start and scale a company. B2B SaaS companies exploded because AWS made it significantly cheaper to start a company. That meant they could live without revenues for long enough to figure out product market fit. AI is so expensive to build and run that patience isn't affordable. This is especially the case if it's hard to create sustainable differentiation. That's open to debate. But the fact that OpenAI and Google haven't turned their massive head start on development and spend into a material technological advantage versus companies like Anthropic, Mistral, and even Meta suggests that having the best model isn't sufficient.
Rather, venture capitalists are reluctant to pour money into a startup if it is projecting that the product will solve new problems, because there's no concrete proof what customers will pay to get those problems solved. So they pitch "we will replace humans doing task X (for which customer are already paying $N billions a year)". ... Remember that at this stage of the industry, the VCs are the customers, and the central question is what the VCs will buy.
Machine learning (I hate the term AI) will do for the cognitively weaker what steam did for the physically weaker.
Prior to the industrial revolution, economic success and physical strength were fairly correlated. Steam power changed that, opening opportunities for women and weaker (nerdy) men. In a simple example, a steam shovel can do the work of 10 men (which hurts the job prospects of those 10 men) but more importantly, the skills required to operate a steam shovel are different (and less physical) than those to dig a ditch.
AI is going to do that for cognitive tasks. We haven't seen much of it yet, but we will. Here's an early version of that:
https://time.com/6240569/ai-childrens-book-alice-and-sparkle-artists-unhappy/
Expect more of this. And expect it to go down the socio-economic ladder, as machine learning makes it possible for people who aren't that smart to compete with those who are. Considering we've spent the last 100+ years making our society more and more complex, and thus requiring higher cognitive abilities to successfully navigate it and make it to the middle class, I think this is a great thing. It's high time many of those people we been pushing to the margins for generations have a chance to get back into the game.
What steam power did for muscles, machine learning will do for minds.
"It’s quite possible that when it comes to AI, power and truthfulness just aren’t correlated. ... It seems quite possible that the AI we’ve created is a truly alien intelligence."
However, despite my optimism, these are rather terrifying statements, Noah,
> Prior to the industrial revolution, economic success and physical strength were fairly correlated
That’s really not true at all. The key to success was by and large to be born wealthy, and outside of that in many societies intelligence mattered for any kind of social climbing - maybe through a priesthood, apprenticeship or exams in the case of China.
Slaves and serfs were strong but not wealthy. You might be a slightly better off serf if you were strong, but intelligence mattered as much there too. It was running a business of sorts.
> And expect it to go down the socio-economic ladder, as machine learning makes it possible for people who aren't that smart to compete with those who are.
Maybe but I suspect it work like leverage - AI gains are amplifications or human intelligence, or they replace everybody. If it’s the former then the smart are paid even more, if it’s the latter then nobody earns money from “smart work”.
However because robots are not improving as fast as machine learning it’s likely that being a plumber will be even more lucrative relative to going to university than it is now.
Peter, interesting article today from Tyler Cohen that is very germane to our discussion:
https://marginalrevolution.com/marginalrevolution/2024/10/who-benefits-from-working-with-ai.html
The actual task used was classifying and age of subjects by pictures, so no realistic rom that perspective. But it hints that AI might be most beneficial for the least skilled among us (Yeah, I know, not exactly Earth-shattering, but research papers are known for stating the obvious in lots of words.)
Also, your comment about plumbers is spot on. A good plumber can pull in 6 figures already. It's amazing what suburban women will pay to not have to deal with poop.
I hope you're wrong about your leverage idea Peter, since rendering larger and larger percentages of your society economically disposable is a bad plan for a republic. I'm hopeful machine learning might be able to alleviate some of that. But certainly the book illustrator example I used is closer to your own theory than to mine. We shall see.
Well in reality the main divisions of income and wealth are really between the owners of capital and the rest of us. We all own, to a rounding error, about 0% of the wealth of billionaires. Making smarter workers poorer isn’t going to change that.
Actually that's not quite true: https://fred.stlouisfed.org/release/tables?eid=813668&rid=453
The net wealth of the 50th-90th percentile group (what most people would describe as middle-class+) is roughly equal to the wealth of the top 10%. That's total wealth not per-capita, so we've got about 4X less overall, but far from the rounding error. These are mostly college-educated, laptop-class workers or small business owners. My wife and I fall into this group; she's a teacher and we both have college degrees.
However, below that (the "working poor") things go downhill quickly. It is this high-school only, blue-colllar group that I'm most concerned about. They've gotten screwed by 50 years of uniparty neoliberalism. I'm hopeful that machine learning might be able to help some of them rise, and perhaps some of that bottom 20% to become economically useful again.
Perhaps I'm being optimistic, but the only alternative I see is UBI, and that's spiritual death -- an public acknowledgement that the bulk of our society is economically useless. I can't get Player Piano by Vonnegut out of my head.
> The net wealth of the 50th-90th percentile group (what most people would describe as middle-class+) is roughly equal to the wealth of the top 10%.
Thanks for that but it’s not what I’m talking about. I don’t even consider the top 10% to be rich in comparison to billionaires. You can’t use that kind of percentage distribution for Pareto distributions as unlike normal (ie bell shaped) distributions the top 10% are much closer to the bottom than the top.
A rounding error of 0.1% - which is low - applied to the lowest possible billionaire (ie a guy with $1B) would mean it’s reasonable to call somebody with $999M a billionaire. Which means he’s a billionaire +- $1M. Which means that we can say to a rounding error he’s got $1B more than a homeless guy and $1B more than a millionaire.
In reality most billionaires are multi billionaires and the rounding error is higher. I don’t think we would say a guy with $995M isn’t a billionaire. Musk is worth $269B.
I've seen this in my field - trying to work out how to use AI effectively is very tough - it requires a totally new way of thinking.
For example - AI is useless in automating monthly reports - traditional BI/coding is very efficient and doesn't hallucinate. You really need to think around the problem and try to identify new ways of solving problems, a much harder issue!
There's a nice insight here: "It’s just a heck of a lot easier to think of a task that people already do, than to think of a task that no one has ever done before". But I also think this piece makes an important (and very common) error: treating AI as a static thing rather than an evolving technology. When we ask whether AI will complement humans or replace them, do we mean the AI of 2024, 2030, or 2100?
You are talking about AIs that have inherent weaknesses, such as a tendency to hallucinate or an inability to carry out long-term creative thinking. That's a good description of AI today, and likely in the near to mid term as well. But it's not necessarily a good description of the AIs we'll have in, say, one to four decades. AI capabilities have grown dramatically, and will almost certainly continue to do so. For this reason, I think it's a fundamental error to talk about the impact of AI without specifying which level of AI capability we have in mind.
(I don't mean to single you out here; it's very common for anyone talking about AI to fail to specify a particular stage of AI, which leads to a lot of people talking past one another.)
One other note: you say that o1 "hallucinates much more than older models", but the link you cite doesn't say anything of the sort; it merely notes that there are several different places in which hallucinations can occur (in the hidden chain of thought, in the translation to the summarized chain of thought, or in the reported result). From everything I've seen, o1 is much more reliable (less likely to hallucinate) on a given task. In particular, it is able to carry out tasks requiring long reasoning chains, and it sometimes hallucinates on these tasks, but older models would be virtually *certain* to go awry on those same tasks.
I think hallucination is likely very deep and impossible to eliminate. It’s a byproduct of having a neural structure reconstruct sentences from an underlying knowledge trace that doesn’t have the same structure as language. I think “hallucination” was the right term when we were talking about image classifiers that force every image into one of their categories. But with language models, “confabulation” is a better term, and humans do it all the time too, like when Walz misremembers or misspeaks about having been in Hong Kong in June 1989, or when I misremember that time in 2002 when I got lost in San Francisco trying to find a friend’s house but realized I was carrying my laptop, and I was able to find an open WiFi signal, and I recall using gps and Google maps to find where I was going, but in 2002 it must have been Mapquest, and using the street signs to identify the intersection I was at.
Efficient neural memory doesn’t store every detail, but instead stores lots of gists, and lets details be filled in smoothly by background knowledge, even though it doesn’t always get things exactly right. I think “hallucination” will never be eliminated, but instead will just get under better and better control, but still will arise in edge cases at the frontiers of whatever the best capabilities of the system are.
Agree with this 100%.
Another thing I often think about is the difference between heuristics and algorithms (I mean the latter in the strict, original, sense of a "procedure for solving a problem", not the way everyone uses it now to describe any piece of software that makes an arbitrary decision).
Heuristics are often *much* faster than algorithms because they use a lot of shortcuts to jump to the (probably) correct conclusion. The problem is that, by definition, heuristics get things wrong because of bias.
So... now you've trained an LLM based on a huge body of content that was almost entirely created by... human beings using heuristics. Removing bias/hallucination from that LLM is... I won't say it is impossible, but it is definitely in the "holy grail" category of problems to solve, you aren't just going to iterate rapidly into the solution. And there are going to remain lots of problems where 99% (or 99.999%) accuracy isn't acceptable.
Some heuristics have bias. But others are just noisy. The heuristic of running away from anything you barely see in the shadows is biased one way, and the heuristic of moving towards things you barely see in the shadows is biased the other way. The heuristic of ignoring things you can't clearly see avoids either of those biases, but is noisy (though it's easy to implement).
Removing hallucination from AI would be like removing stereotypes from humans - you could only do it if you took away most of the neural methods that make our minds work usefully. But we can introduce features and training that minimize the particular types that are most likely to be harmful.
For what it's worth, I think the original definition of "algorithm" was always just a sequence of instructions to follow, and remained neutral on whether it was a perfect way to achieve one's goal, or was at least heuristically useful, or was irrelevant or even counterproductive.
The problem with removing hallucination by introducing features and training that minimize them is that you end up introducing a reverse bias (because we are human, our solutions also have bias). Then you end up with nonsense like Gemini refusing to paint a picture of George Washington in which he is caucasian, or you get LLMs that refuse to talk about the history of the German Nazi party, or the Jewish holocaust, because they "think" doing so is anti-semetic.
So far, the "safeguards" that try to counteract hallucination or bias have been very ineffectual or have very unwanted side effects. It remains an incredibly hard problem to solve, sort of like trying to teach "common sense" to an alien.
Yes. And similarly for humans, if you train people to avoid the errors that enable an interviewer to get them to recall the time they met Bugs Bunny at Disney World, they usually just end up being overly skeptical of their childhood memories generally.
BLINDED BY MACRO THINKING, you are, Noah!
Ethan Molic writes perceptively about the current uses of AI, but even he does not yet see the trend he demonstrates. AI works as guerrilla warfare.
Nowhere in science fiction has anyone written about how society's institutions will fare when they are attacked by an infinite number of bored teenagers with unlimited access to star powered AI resources. Yet schools are reeling from the impact on homework assignments. Molic points out that middle managers are using AI, not to enable the goals of their employer, but to advance their own powers in the firms.
In the internet hubbub we have forgotten that real entrepreneurship is simply finding the shortcuts to money.
Imagine the results if I make some subversive inquiries of my AI:
"Show me 50 ways around this patent."
"What lies can I tell the IRS in to order decrease my tax bill and what are the risks of each one."
"How can I pad my dissertation to make it look more important."
"What are the best ways to deceive my wife about my affair?
"Invent 50 useful lies that MAGAs are likely to believe."
Well, maybe not Ethan Molic. I can't seem to find the right citation. Help me out!
In the PC revolution we thought that we had empowered the little guy, but we enabled monsters like Facebook. Everyone is assuming that AI will enable Big Brother but current usage indicates that we are headed toward some serious chaos by enabling the little guy.
We're also failing to recognise the roles where human empathy is paramount. Even if an AI agent could _in theory_ replace the work of a human, in many cases, the end customer/user doesn't want that.
Whenever I speak to an AI agent or bot instead of a human being for a customer service issue, it's trust the brand is burning. Yet most continue to do so for the sake of "efficiency". Efficiently losing my trust!
I wrote a post about how this is impacting recruiting, after seeing so many takes of "AI as a recruiter" online: https://hiringhumans.substack.com/p/ai-proof-your-hiring-tactics-to-humanise. Another example of what you're discussing in this post.
I agree with your idea that it takes 20-30 years before people figure out a new technology. There are two other trends that tend to track with a major technological change. The new technology disrupts the balance of power between workers and owners. Nothing inherent in the technology drives job loss and lower wages, but when it empowers people whose goal is to reduce labor costs, that will happen until a new equilibrium is reached (with or with government intervention on the side of labor). The introduction of new technology is also fervent ground for marketing hype and financial bubbles. I’m convinced that that is the case with LLMs. There is great potential for AI to increase productivity once people figure out how to use it in new ways, but this is not going to come from LLMs. Training neural networks with petabytes of uncurated language does not produce a “brain” any more reliable than an average human. The AI revolution will come, but not before the LLM shakeout.