We don’t know that in the same way we don’t know the Kool-Aid Man won’t burst through the wall at any given moment and start singing Freebird. Technically the chances aren’t 0, but they may as well be.
I am a software engineer. I deal with LLMs and their implementation every day. I’m not a specialist, but I have a good enough idea of how they work. Their core functionality is fundamentally no different than the predictive text function on your phone. Even setting aside the fact that sentience is a complex and ill-defined concept, certainly you can agree that your autocorrect is nowhere in the realm of sentient.
Not the OP, but any software developer who has spent some time with the inner workings of LLMs can tell you that. Open weight models are a good opportunity to explore how LLMs work on the inside and to experiment with them a lot more than you can do with the so called Frontier models. The technology however is the same. These are just very fancy statistical prediction machines, and the reason why you are so impressed with them is that the statistical average of what sounds reasonable in a conversation is exactly what people expect to hear on average, given a specific context. It’s a neat little trick that works only only as long as the context fits (contentwise and spacewise), and once you realize that you’ll see LLMs as what they truly are. There is no magic, and the “intelligence” is the intelligence of the people who built those systems, the systems themselves are not intelligent in any commonly used sense of the word.
I am of the opinion that there’s no magic in sapience in the first place. Our brains are just pattern matching, statistical prediction machines that have had trillions of iterations of improvements over time. The intelligence we flatter ourselves with is only a small part of what we have evolved in our brains. For eons the problem space was all physical, but with AI, the starting point is the “intelligence” piece.
While that is partly true, it oversimplifies a few things. And these are the things that make LLMs a dead end for intelligence:
Evolution means constant learning and adaption. LLMs can’t be adapted on the go, they are stuck with their initial training data and retraining is a slow and externally controlled process. Apart from training, every additional information you give it while running must be loaded into the context, the space for context is limited and must be flushed before supplying different context. That’s a hard limit for evolution, at least with current technology.
The physical world is more important than you might think because everything hinges on this realm, including the electricity, cabling and hardware involved in running a model. The idea of an intelligent entity that exists purely as an ethereal cloud of logic unbound by physics is hard to justify, because the shape of logic itself is defined by the physical ground we are standing on. Therefore, an AI would need a first-hand access and understanding of the physical world to be successful. LLMs can’t have any of that, because they are purely language-bound.
I personally think Yann LeCun (a pioneer of machine learning and AI) is on the right track with his research and his opinion that LLMs are a dead end for artificial intelligence. The reasons are more on a technical than a philosophical side.
We don’t know that in the same way we don’t know the Kool-Aid Man won’t burst through the wall at any given moment and start singing Freebird. Technically the chances aren’t 0, but they may as well be.
What is your area of expertise that makes you so certain about how sentience can be attained?
I am a software engineer. I deal with LLMs and their implementation every day. I’m not a specialist, but I have a good enough idea of how they work. Their core functionality is fundamentally no different than the predictive text function on your phone. Even setting aside the fact that sentience is a complex and ill-defined concept, certainly you can agree that your autocorrect is nowhere in the realm of sentient.
Not the OP, but any software developer who has spent some time with the inner workings of LLMs can tell you that. Open weight models are a good opportunity to explore how LLMs work on the inside and to experiment with them a lot more than you can do with the so called Frontier models. The technology however is the same. These are just very fancy statistical prediction machines, and the reason why you are so impressed with them is that the statistical average of what sounds reasonable in a conversation is exactly what people expect to hear on average, given a specific context. It’s a neat little trick that works only only as long as the context fits (contentwise and spacewise), and once you realize that you’ll see LLMs as what they truly are. There is no magic, and the “intelligence” is the intelligence of the people who built those systems, the systems themselves are not intelligent in any commonly used sense of the word.
I am of the opinion that there’s no magic in sapience in the first place. Our brains are just pattern matching, statistical prediction machines that have had trillions of iterations of improvements over time. The intelligence we flatter ourselves with is only a small part of what we have evolved in our brains. For eons the problem space was all physical, but with AI, the starting point is the “intelligence” piece.
While that is partly true, it oversimplifies a few things. And these are the things that make LLMs a dead end for intelligence:
I personally think Yann LeCun (a pioneer of machine learning and AI) is on the right track with his research and his opinion that LLMs are a dead end for artificial intelligence. The reasons are more on a technical than a philosophical side.