Indeed, it’s not that LLM is totally uncorrelated to factual basis, it is correlated well enough for most educational fodder that is supremely well trodden and doesn’t seek surprises. It’s practice with backgrounds the teachers can actually already know and credibly provide feedback on. It’s so well known that any teacher knows, so it’s correlated with AI output that has been trained on verbatim prompts and essays that are on the exact same topic, and so the narrative correlated with the prompt is just very likely to also correlate to the facts of relevance.
Just like a pocket calculator can make short work of elementary school math. It’s not that we expect those kids to do some crazy novel stuff that calculators can’t do, it’s just that they need to operate in a context to actually illustrate they have the foundational understanding.
Indeed, it’s not that LLM is totally uncorrelated to factual basis, it is correlated well enough for most educational fodder that is supremely well trodden and doesn’t seek surprises. It’s practice with backgrounds the teachers can actually already know and credibly provide feedback on. It’s so well known that any teacher knows, so it’s correlated with AI output that has been trained on verbatim prompts and essays that are on the exact same topic, and so the narrative correlated with the prompt is just very likely to also correlate to the facts of relevance.
Just like a pocket calculator can make short work of elementary school math. It’s not that we expect those kids to do some crazy novel stuff that calculators can’t do, it’s just that they need to operate in a context to actually illustrate they have the foundational understanding.