Yes, such things can happen… I once asked an LLM a few questions about me (under my real name) that was publicly available on the Internet (i.e. should be in its training data). It answered a simple yes-or-no question wrongly. Then I asked it a followup question, which it answered more correctly, but the answer contradicted that wrong answer and it went “this seems to contradict my previous answer that…”.