cross-posted from: https://piefed.social/c/linux@programming.dev/p/2509642/linus-torvalds-posts-his-first-ai-assisted-patch-to-the-linux-kernel-synthetic-test-clo
cross-posted from: https://piefed.social/c/linux@programming.dev/p/2509642/linus-torvalds-posts-his-first-ai-assisted-patch-to-the-linux-kernel-synthetic-test-clo
AI does not have memories.
It has a comparatively small context window that makes things worse the more it fills up.
Eh I really think it does. You don’t news to load everything into context. You need to load where everything is located into context.
That isn’t equivalent to memories.
You are mistaken. Some AI models do indeed have memories.
They save some strings in a document to later read and “remember”. Calling them memories is a reach.
My point stands that they can learn from feedback. Your memories are only information in a virtual document, too. Just as susceptible to corruption, or misinterpretation.
Does an Alzheimer’s patient learn things just because they read the same note every morning explaining the reality to them?
No, their file system is corrupt. No reliable read or write access.
The AI is no different from the girl in “50 First Dates (2004)”. It has to go back and read the post-it notes every time you start a new session. It does not retain memory without writing shit down to read later.
You’re not wrong. It’s an inferior equivalent, but would you not say she learns with each note she makes, hampered though she is?
Yes I would say that she temporarily learns something while reading the notes. And she stores it in short-term memory.
The difference with AI though is that it has to re-read the same note again and again after each question you ask it. It does not keep things in “memory” from your previous conversations. It reads it again before answering you.
That is why context length matters and is important in LLMs. There’s a huge difference between talking to an AI with 10k in context length and 1mil in context length. You can try it yourself using Llama.cpp and running LLMs locally.
There is no concept of a filesystem in our brains. Weird hills, and such and so forth
Only because you don’t think of it as one. It’s just a different medium for the storage of information.