It’s possible to imagine other ways to train chatbots that wouldn’t favor bullshitting in the same way, though
How? The way they work is that they predict the next word (token). They don’t know anything. So, when someone says “what’s the book that features an X doing Y and has a Z that looks like a Q” the process is basically “if this were a forum post, what’s the next word that’s likely to appear, what’s the one after that, what’s the one after that”.
If the scene being described is something from Harry Potter that has been discussed endlessly on the web, it’s likely to predict next words that end up being helpful. If the scenario being described has never been described before, there’s no way for it to say “aha, this book doesn’t exist!” Instead it just invents a plausible conversation between two people about a fictional book.
Now, you could design a filter that sits in front of the LLM that makes things better. It could be designed to recognize quotes, and if it spots a quote, it can “search the web” (or really look through a big index) to see if those words in that sequence have actually been seen before. If not, it could reject the entire response as being a “hallucination”.
But, fundamentally, this isn’t a problem you can fix through training, because of how LLMs work.
You can’t fix it with a pure LLM, no. But very few of the major models are pure LLMs anymore. They’ve got LLMs built in to them as subsystems, but they’ve got control systems layered on top of those to adjust their behaviors. And those don’t have to be trained purely with next word prediction.
How? The way they work is that they predict the next word (token). They don’t know anything. So, when someone says “what’s the book that features an X doing Y and has a Z that looks like a Q” the process is basically “if this were a forum post, what’s the next word that’s likely to appear, what’s the one after that, what’s the one after that”.
If the scene being described is something from Harry Potter that has been discussed endlessly on the web, it’s likely to predict next words that end up being helpful. If the scenario being described has never been described before, there’s no way for it to say “aha, this book doesn’t exist!” Instead it just invents a plausible conversation between two people about a fictional book.
Now, you could design a filter that sits in front of the LLM that makes things better. It could be designed to recognize quotes, and if it spots a quote, it can “search the web” (or really look through a big index) to see if those words in that sequence have actually been seen before. If not, it could reject the entire response as being a “hallucination”.
But, fundamentally, this isn’t a problem you can fix through training, because of how LLMs work.
You can’t fix it with a pure LLM, no. But very few of the major models are pure LLMs anymore. They’ve got LLMs built in to them as subsystems, but they’ve got control systems layered on top of those to adjust their behaviors. And those don’t have to be trained purely with next word prediction.