Odd example but yes it gets uncertain very quickly. If we discover the mechanism of our own thought it’d be easy enough to apply those rules to the new subject and see if they match or at least look similar. A negative wouldn’t be conclusive, but it’d at least give us something to work with. As it stands right now we just guess, and that guess is essentially guided by bias or at best by the fact that the thing we’re looking at doesn’t appear even remotely similar to anything we’d call “thinking”. This is simply not the case with LLMs (they do a very good job of displaying something that looks like thinking) so our guess is imo even more unsound.
Odd example but yes it gets uncertain very quickly. If we discover the mechanism of our own thought it’d be easy enough to apply those rules to the new subject and see if they match or at least look similar. A negative wouldn’t be conclusive, but it’d at least give us something to work with. As it stands right now we just guess, and that guess is essentially guided by bias or at best by the fact that the thing we’re looking at doesn’t appear even remotely similar to anything we’d call “thinking”. This is simply not the case with LLMs (they do a very good job of displaying something that looks like thinking) so our guess is imo even more unsound.