This link is about reasoning system, not reasoning. Reasoning involves actually understanding the knowledge, not just having it. Testing or validating where knowledge is contradictionary.

LLM doesn’t understand the difference between hard and soft rules of the world. Everything is up to debate, everything is just text and words that can be ordered with some probabilities.

It cannot check if something is true, it just ‘knows’ that someone on the internet talked about something, sometimes with and often without or contradicting resolutions…

It is a gossip machine, that trys to ‘reason’ about whatever it has heard people say.

Is this some kind of touring test for chess players? Figure out if the person your are playing chess with is a human or robot? LLM’s aren’t yet successful doing that by chatting with them, and as a newbie I suppose that test could be easier, but what do I know…