"Historically, standing on each step of the ladder, the AI research community has been terrible at predicting how much farther you can go with the current paradigm, what the next step will be, when it will arrive, what new applications it will enable, and what the implications for safety are. That is a trend we think will continue."
One thing that's bothersome to me is that now many in the LLM promotion/biz are also conflating scaling of LLMs with the *way that humans learn*. e.g. this from Dario Amodei recently: https://youtu.be/xm6jNMSFT7g?t=750
it's as if not only do they believe in scaling leading to AGI (not true as per your article), they are now *selling* it to the public as the way that people learn, only better/faster/more expensive.
There are several interesting things that I appreciated from this issue, including the fascinating way of dealing with aspects related to Computer Science and explaining them in a clear and crystalline way while also going in depth. However, one of the insights that I take away from careful reading is certainly the intriguing way in which you have questioned some laws and concepts taken for granted. For example when you specified: "what do you mean by better?". This way of not stopping at the data, of going beyond paradigms and also appropriately questioning them in light of new data and possible advances is absolutely worth taking home and is a very important principle also in writing. Thanks for sharing.
This is your best article yet, and that’s saying something. I believe in a few years we’ll look back at the inability of LLMs to handle novelty as cringeworthy considering the current level of hype. LLMs are echo machines, repeating back to us remixes of data we fed into them.
I do believe AI systems will emerge that can effectively handle novelty, but they will be different from LLMs (though maybe LLMs could be a component of these systems). It will be interesting to see how this develops … any bets on what approach will make this breakthrough?
On emergence, I agree with your reasons for skepticism. I'd like to add two serious problems:
1. Independent researchers cannot query the training sets used for the latest LLMs. This renders all claims of emergence suspect, as we cannot assess "leakage". I would not be at all surprised if the "improved abilities" of newer models come from feeding them all that had been written about the limitations of their predecessors, either intentionally or unintentionally. I certainly wouldn't put it past any of the major players here to have done some targeted curating based on the kinds of metrics they knew would be used.
2. Tech companies and AGI "believers" (for lack of a better term) have a strong incentive to play up emergence as a mysterious phenomenon that no one can explain. This lets them credit the alleged mysteries to an ever-advancing form of intelligence. It's like how creationists play up alleged gaps in the evolutionary record and fill them in with divine intervention. Lacking any real scientific theory for how AGI is supposed to come about, particularly one for how machine learning could deliver it, they sell us on unexplained phenomena and let our imaginations do the rest. I don't mean to accuse them of bad faith (well, maybe OpenAI...) - this is part of a belief system, and I suppose they could turn this around and call skepticism of AGI a belief system that makes people like me unwilling to accept evidence for advancing intelligence. But knowing about the belief system does make it hard to take some kinds of claims at face value. People who see Jesus in their toast were looking for Him ahead of time, and people who see emergent phenomena arise mysteriously are the same ones anticipating the arrival of AGI.
I'm a fan, but I feel like this fell into a few of the same traps as the AGI Faithful, but from the other side:
1. Easy to do this without definitions. You did not define AGI or comment on how much of the forecasting is on a specific vague definition. I think you all are the perfect people to debunk to "high schooler intelligence" claims
2. I think you discounted synthetic data, where there are works on rewriting pre training data https://arxiv.org/abs/2401.16380v1 and more work under ways (hopefully I can share more soon)
3. I would like to see a closer explanation on the scaling plots meaning technically. I still hear them described wrong. The examples of clock speed and stuff are interesting at least.
4. the industry stopping making models bigger is a short term snip, long term not true if you look at capital expenditures.
May just be me narrowing my focus on what I should write about.
p.s. how do I preorder a signed copy of the book? Or, I may just preorder and track you two down for a signature in person
I appreciate the post, thanks to both. Could you provide a link to where a CEO has claimed that AGI would come in three years? I'm not aware of any such claim.
Altman, Amodei, and Musk have all made such claims; you can easily find them :) Another commenter posted a link to a recent Amodei interview for example. (Altman tends to be slightly more vague about his timeline but roughly in that ballpark.)
The interview you reference (https://youtu.be/xm6jNMSFT7g?t=750 for anyone else reading this) Amodei says he no longer thinks that AGI is this singular event in time.
That certainly supports your claim that CEOs are making more measured remarks about what AGI than they were before, but that's not what I'm asking about.
They might be hyping the technology as if they thought AGI was coming very soon, but that's a different thing than a falsifiable prediction, which, as far as I can tell, neither of the three have made.
But given Musk's track record (e.g. autonomous driving in a few years for 10 years now)...I think all of the AGI true believers would both stupid and disingenuous if they were to predict *anything* about AI within 3 years (positive or negative)...because it would be obvious that they were just stringing everyone along (i.e. hype).
Here's an interview with Dario Amodei from last August (so, almost a year ago): https://www.dwarkeshpatel.com/p/dario-amodei. In it, he speculates that human-level AI “could happen in two or three years”.
I may not be accurate here, but my experience tells me the following:
We cannot achieve something if we cannot define it. The first goal should be to define AGI.
Even if we go with a very basic definition of an intelligent system that can do most tasks (I am intentionally using most and not all) better than humans, I do not think that is achievable with hardware scaling and/or any amount of synthetic or non-synthetic data as our intelligence is more than just what is in the books and internet. Most of what makes someone an expert after we have built fundamental skills is tacit knowledge, and I believe that unless we find a way to codify it and incorporate it, current AI models will never achieve expert-level intelligence in most areas, if not almost all.
Although I think the idea of being clear about definitions (e.g. AGI) is generally a sound strategy, I don't think it works in this case...because: we don't really know how to define intelligence. Yes we know more than nothing about what intelligence is, based upon psychology, introspection, philosophy, observation, socialization, etc...i.e. our own experience as humans...but I don't think we can simply define intelligence as 'whatever we do'...which is basically the Turing test IMHO. Also, even *if* we precisely constrain the problem as: 'intelligence a certain mental skill' (like playing Go or chess or any game really well), it is almost comical to consider this a defining characteristic of 'intelligence'. That's because intelligence means a lot of things under a lot of different circumstances.
Thanks for a bit of healthy skepticism amid the hype. There’d be a lot more progress if LLM’s learned the way humans do—considering our brains come prewired for language and facial expressions; babies are not creating “models” from scratch. But Ai research appears to be mired in the old Tabula Rasa way of understanding people’s intellectual development.
CPU manufacturers did not "simply decide to stop competing" on clock speed. They ran into the limits of Dennard scaling. As transistors get smaller, they get leakier: they don't turn off as completely. Power consumption thus rises. Power also scales linearly with clock speed. In the early 2000s, it started to become difficult to increase clock speeds without making chips run unacceptably hot. Advances in transistor design have mitigated this problem sufficiently to allow transistors to continue to get smaller, but not enough to give us both smaller transistors and higher clock speeds.
"Historically, standing on each step of the ladder, the AI research community has been terrible at predicting how much farther you can go with the current paradigm, what the next step will be, when it will arrive, what new applications it will enable, and what the implications for safety are. That is a trend we think will continue."
Your best zinger this year :)
One thing that's bothersome to me is that now many in the LLM promotion/biz are also conflating scaling of LLMs with the *way that humans learn*. e.g. this from Dario Amodei recently: https://youtu.be/xm6jNMSFT7g?t=750
it's as if not only do they believe in scaling leading to AGI (not true as per your article), they are now *selling* it to the public as the way that people learn, only better/faster/more expensive.
What a breath of fresh air! Stellar piece from beginning to end.
Sam Altman is the very definition of a snake oil salesman. The term AGI is going to sound pretty laughable in a few years.
Venture Capital are betting the farm on generative AI and the losses are going to be staggering.
There are several interesting things that I appreciated from this issue, including the fascinating way of dealing with aspects related to Computer Science and explaining them in a clear and crystalline way while also going in depth. However, one of the insights that I take away from careful reading is certainly the intriguing way in which you have questioned some laws and concepts taken for granted. For example when you specified: "what do you mean by better?". This way of not stopping at the data, of going beyond paradigms and also appropriately questioning them in light of new data and possible advances is absolutely worth taking home and is a very important principle also in writing. Thanks for sharing.
This is your best article yet, and that’s saying something. I believe in a few years we’ll look back at the inability of LLMs to handle novelty as cringeworthy considering the current level of hype. LLMs are echo machines, repeating back to us remixes of data we fed into them.
I do believe AI systems will emerge that can effectively handle novelty, but they will be different from LLMs (though maybe LLMs could be a component of these systems). It will be interesting to see how this develops … any bets on what approach will make this breakthrough?
On emergence, I agree with your reasons for skepticism. I'd like to add two serious problems:
1. Independent researchers cannot query the training sets used for the latest LLMs. This renders all claims of emergence suspect, as we cannot assess "leakage". I would not be at all surprised if the "improved abilities" of newer models come from feeding them all that had been written about the limitations of their predecessors, either intentionally or unintentionally. I certainly wouldn't put it past any of the major players here to have done some targeted curating based on the kinds of metrics they knew would be used.
2. Tech companies and AGI "believers" (for lack of a better term) have a strong incentive to play up emergence as a mysterious phenomenon that no one can explain. This lets them credit the alleged mysteries to an ever-advancing form of intelligence. It's like how creationists play up alleged gaps in the evolutionary record and fill them in with divine intervention. Lacking any real scientific theory for how AGI is supposed to come about, particularly one for how machine learning could deliver it, they sell us on unexplained phenomena and let our imaginations do the rest. I don't mean to accuse them of bad faith (well, maybe OpenAI...) - this is part of a belief system, and I suppose they could turn this around and call skepticism of AGI a belief system that makes people like me unwilling to accept evidence for advancing intelligence. But knowing about the belief system does make it hard to take some kinds of claims at face value. People who see Jesus in their toast were looking for Him ahead of time, and people who see emergent phenomena arise mysteriously are the same ones anticipating the arrival of AGI.
I'm a fan, but I feel like this fell into a few of the same traps as the AGI Faithful, but from the other side:
1. Easy to do this without definitions. You did not define AGI or comment on how much of the forecasting is on a specific vague definition. I think you all are the perfect people to debunk to "high schooler intelligence" claims
2. I think you discounted synthetic data, where there are works on rewriting pre training data https://arxiv.org/abs/2401.16380v1 and more work under ways (hopefully I can share more soon)
3. I would like to see a closer explanation on the scaling plots meaning technically. I still hear them described wrong. The examples of clock speed and stuff are interesting at least.
4. the industry stopping making models bigger is a short term snip, long term not true if you look at capital expenditures.
May just be me narrowing my focus on what I should write about.
p.s. how do I preorder a signed copy of the book? Or, I may just preorder and track you two down for a signature in person
I appreciate the post, thanks to both. Could you provide a link to where a CEO has claimed that AGI would come in three years? I'm not aware of any such claim.
Altman, Amodei, and Musk have all made such claims; you can easily find them :) Another commenter posted a link to a recent Amodei interview for example. (Altman tends to be slightly more vague about his timeline but roughly in that ballpark.)
The interview you reference (https://youtu.be/xm6jNMSFT7g?t=750 for anyone else reading this) Amodei says he no longer thinks that AGI is this singular event in time.
That certainly supports your claim that CEOs are making more measured remarks about what AGI than they were before, but that's not what I'm asking about.
They might be hyping the technology as if they thought AGI was coming very soon, but that's a different thing than a falsifiable prediction, which, as far as I can tell, neither of the three have made.
Musk: AGI in 3 years: https://www.youtube.com/watch?v=pIJRFr26vXQ
But given Musk's track record (e.g. autonomous driving in a few years for 10 years now)...I think all of the AGI true believers would both stupid and disingenuous if they were to predict *anything* about AI within 3 years (positive or negative)...because it would be obvious that they were just stringing everyone along (i.e. hype).
Thanks Scott!
Here's an interview with Dario Amodei from last August (so, almost a year ago): https://www.dwarkeshpatel.com/p/dario-amodei. In it, he speculates that human-level AI “could happen in two or three years”.
This community might find some hope in this posting by Gary Marcus about Bill Gates' view of scaling: https://garymarcus.substack.com/p/marcus-goes-gaga-over-gates-clip
I may not be accurate here, but my experience tells me the following:
We cannot achieve something if we cannot define it. The first goal should be to define AGI.
Even if we go with a very basic definition of an intelligent system that can do most tasks (I am intentionally using most and not all) better than humans, I do not think that is achievable with hardware scaling and/or any amount of synthetic or non-synthetic data as our intelligence is more than just what is in the books and internet. Most of what makes someone an expert after we have built fundamental skills is tacit knowledge, and I believe that unless we find a way to codify it and incorporate it, current AI models will never achieve expert-level intelligence in most areas, if not almost all.
Although I think the idea of being clear about definitions (e.g. AGI) is generally a sound strategy, I don't think it works in this case...because: we don't really know how to define intelligence. Yes we know more than nothing about what intelligence is, based upon psychology, introspection, philosophy, observation, socialization, etc...i.e. our own experience as humans...but I don't think we can simply define intelligence as 'whatever we do'...which is basically the Turing test IMHO. Also, even *if* we precisely constrain the problem as: 'intelligence a certain mental skill' (like playing Go or chess or any game really well), it is almost comical to consider this a defining characteristic of 'intelligence'. That's because intelligence means a lot of things under a lot of different circumstances.
Thanks for a bit of healthy skepticism amid the hype. There’d be a lot more progress if LLM’s learned the way humans do—considering our brains come prewired for language and facial expressions; babies are not creating “models” from scratch. But Ai research appears to be mired in the old Tabula Rasa way of understanding people’s intellectual development.
"Emergence is not governed by any law-like behavior." "there is no empirical regularity."
Nah, there is; see "Broken Neural Scaling Laws" paper:
https://arxiv.org/abs/2210.14891
CPU manufacturers did not "simply decide to stop competing" on clock speed. They ran into the limits of Dennard scaling. As transistors get smaller, they get leakier: they don't turn off as completely. Power consumption thus rises. Power also scales linearly with clock speed. In the early 2000s, it started to become difficult to increase clock speeds without making chips run unacceptably hot. Advances in transistor design have mitigated this problem sufficiently to allow transistors to continue to get smaller, but not enough to give us both smaller transistors and higher clock speeds.