LLMs Know Nothing
For Real
TL;DR
LLMs simply predict the next word in a sentence, which means they can never know the meaning behind the words.
An Old Graphics Card
A few weeks ago, on a nice, quiet summer day, my wife and I were relaxing after a few days of travel. My wife was playing some games on a computer I built for her using my old GeForce GTX 1080 graphics card. I was doing some programming, as per usual, when she came into my office to let me know that the graphics card wasn’t working. Being the talented software engineer I am, I figured it would be a quick and easy fix, but unfortunately it took me a while to discover the issue. I saw that there was an error code in Device Manager, and at that moment I thought to myself, “I should use ChatGPT!” I figured talking to ChatGPT using my phone would make things easier if I had to open my computer up or do any restarting. Here’s how it started:
Me: All right, my GeForce GTX 1080 is having some issues.
ChatGPT: Oh no, that can be frustrating! What kind of issues are you seeing—like driver errors, display glitches, or something else? I can help you troubleshoot!
Me: Okay, I have a device status that says Windows has stopped this device because it’s reported a problem. Code 43.
ChatGPT: A Code 43 typically means the system knows the device had a problem and stopped it. It could be a driver issue or hardware. I’d start by reinstalling or updating your GPU drivers. If that doesn’t help, checking connections or testing the card in another machine might be next. Let me know where you want to start!
I did what ChatGPT said and reinstalled the drivers and tried plugging the card into different ports. It had the same issue. During a restart, I noticed something wrong. The GPU fan was at 100%.
Me: …the fan is at 100% right now, even though it’s technically not working.
ChatGPT: That’s a useful clue. If the fan is stuck at full speed, it often points to a hardware fault—like the GPU not initializing properly. At this stage, it might be worth testing the card in another system if you can. If the issue persists, the card itself might need repair or replacement.
After some more back and forth with ChatGPT, it did not give me any more useful suggestions. I gave up and decided to call my dad who has a lot of experience doing computer repairs. I described the same problem to him, and he immediately said, “Your graphics card is overheating, blow out the dust and replace the thermal paste.” I did what my dad said, and immediately my graphics card started working again. Here, an AI that is trained on the knowledge of the whole internet couldn’t identify the problem with my graphics card, but my dad figured out the problem in a matter of seconds. That’s when I realized something: LLMs “know” nothing, at least in the way humans know things.
What Do You Mean LLMs Know Nothing?
I use LLM agents to write a majority of my code these days, and I’m sure you’re wondering how something that powerful cannot know anything. Well, it’s because they’re designed to be a better autocomplete. I’m sure we’re all sick of hearing about how LLMs simply predict the next word, but I don’t think everyone truly understands the implications of that fact. It means that LLMs cannot ever understand the meaning behind the words they “read”. When I, and I’m sure almost everyone else, consume some form of media, I internalize it. I think about what it means and how it impacts my current understanding of the world. I find abstractions and ways the knowledge could be used to create something new. Okay, maybe I don’t do that for all the media I read, but when I’m trying to learn something I do. Since LLMs cannot do anything beyond predicting the next word, they are highly dependent on the user saying the right series of words that will produce the correct answer. This means that it depends heavily on the user having the right knowledge to guide the LLMs to the right conclusion. If I had simply said to ChatGPT, “I see my fan is running at 100%. I think it’s overheating,” it probably would have told me to blow out the dust and replace the thermal paste. This means that there are people out there right now relying on LLMs to help them with their problems, but those problems aren’t getting fixed because they didn’t quite put the right words together.
Conclusion
Why did my dad figure out the problem with my graphics card so quickly? It’s because he knows how the hardware is supposed to work. He knows that the point of a fan is to cool the graphics card. He knows that the heat sensors check the temperature of the graphics card and dictate the speed of the fan. He knows that the sensors could be broken, or that there was too much insulation in the form of dust causing the fans to run at 100%. ChatGPT, on the other hand, knows nothing. ChatGPT predicted that the next words would be to reinstall the drivers and was not about to get out of that thought until it eventually predicted a paragraph, comically, that my graphics card needed to be repaired. The implication is that people who ask ChatGPT about things they have very little knowledge on may find themselves going down the wrong path because they put the wrong series of words together from the start. The lesson I learned is that I should talk to other human experts if I truly want to be confident I have exhausted my options when trying to solve a problem. I know dust can cause overheating, but since I’m not always fixing computers I didn’t think about it. I suppose humans are still better than LLMs at pointing out things other humans missed or forgot about. It’s funny to think about how I almost threw out my old graphics card because some other graphics cards in the cloud didn’t know how graphics cards work.