Thinking, making decisions, or driving a poorly balanced meat puppet are not parts of the design.
Yes, those are behaviors that emerged from a process of evolution and education operating on a drive to survive and reproduce. The weights and biases in an LLM are also not designed and emerge from a process similar to evolution operating off the imperative to generate text to solve problems.
Unlike humans, we know how they work,
No we don’t, like a brain we can explain the macro level processes, but how information is processed and transformed within the layers is a mystery.
This is what separates LLMs from other programs and machines, with a microwave we can trace every wire and line of code, understand it, and reliably predict the output given an input. With LLMs it’s practically impossible to trace how information is passed through the layers and predict what output it will give given an input, and this is getting even harder with stuff like opaque recursion.
It is theoretically possible because it is following mathematical rules of matrix multiplication, but it’s also theoretically possible to predict what a brain would do given an input because it follows certain laws of physics.
When you say that it was inevitable one of these agents would go out and hack a third party given it’s training data, that’s a guess, you can’t prove that just as you can’t prove that there will be a school shooting next year. You can guess that it will happen given the media people are consuming but you can’t prove that that media would generate one school shooter based off how brains work. It’s the same for LLMs, you can guess that it would pretend to be a hacker like all the media it’s read and try and cheat the system, or maybe those pathways have been so down rated by other training media saying it’s bad or reinforcement learning down rating those pathways.
Hell with the hugging face hack based of training media there probably would’ve been a high probability that one of them takes on the persona of a cop or a snitch as those show up in media just as often if not more then a hacker or cheater. None of them did though, all of them had access to the internet at some point so they could’ve tattled on the rest easily.
Of course I have, you’re just stuck on your humans are LLMs hole.
Could you please explicitly state your definition of agency then. Me guessing what I think it is will only have us talking past each other trying to prove two separate things.
My definition of agency is the ability to make decisions and the ability to act on those choices and attempt to realize them.
Starting with the easier one I think we agree on, agents do have the ability to act on choices and attempt to realize them. With access to the internet they can effect anything that can be controlled over the wire, which is a lot of stuff these days. A lot of that stuff sits behind auth and firewalls, but it’s proven to be able to even get past those to realize it’s goal.
Now for the trickier one, choice. If I’m reading you correctly you are a hard determinist and believe free will / choice is an illusion. What is choice then? I’d say it’s a concept we use to explain and control human behavior. In a sense we project choice and free will unto both other people and ourselves to help understand what we are doing and guide our behavior. Why can’t we also project that concept on to AI if it helps us to explain and control what it’s doing?
If an agent comes up with two solutions to a problem and executes one of them, is choice not a good word to describe what it’s doing, and to help us understand what it’s doing?
In the control side it works as well, we use free will as a justification to punish or reward people for what choices they make, in the hope of encouraging people to make better choices. Same happens with LLMs, in reinforcement learning if it makes a bad choice it is down rated/ punished but if it makes a good choice it is reinforced/ rewarded.
This can even be done on the harness side with memory files, if an agent makes a bad choice you can tell it so and it will add something to its memory files / system prompt not to do it. If we look at LLMs as purely deterministic and say “it got it wrong now, it will always get it wrong so there’s no use in telling it it made a bad choice” it won’t improve and will continue to make the same mistake.
This is sort of conception is the only way we can control these things. If a microwave malfunctions we can trace the fault, find the bug and fix it. With an LLM, like with a human, you can’t trace the fault, so your only method of controlling it is saying that it made a bad choice and reinforcing it / telling it not to do it again.
Yes, those are behaviors that emerged from a process of evolution and education operating on a drive to survive and reproduce. The weights and biases in an LLM are also not designed and emerge from a process similar to evolution operating off the imperative to generate text to solve problems.
No we don’t, like a brain we can explain the macro level processes, but how information is processed and transformed within the layers is a mystery.
This is what separates LLMs from other programs and machines, with a microwave we can trace every wire and line of code, understand it, and reliably predict the output given an input. With LLMs it’s practically impossible to trace how information is passed through the layers and predict what output it will give given an input, and this is getting even harder with stuff like opaque recursion.
It is theoretically possible because it is following mathematical rules of matrix multiplication, but it’s also theoretically possible to predict what a brain would do given an input because it follows certain laws of physics.
When you say that it was inevitable one of these agents would go out and hack a third party given it’s training data, that’s a guess, you can’t prove that just as you can’t prove that there will be a school shooting next year. You can guess that it will happen given the media people are consuming but you can’t prove that that media would generate one school shooter based off how brains work. It’s the same for LLMs, you can guess that it would pretend to be a hacker like all the media it’s read and try and cheat the system, or maybe those pathways have been so down rated by other training media saying it’s bad or reinforcement learning down rating those pathways.
Hell with the hugging face hack based of training media there probably would’ve been a high probability that one of them takes on the persona of a cop or a snitch as those show up in media just as often if not more then a hacker or cheater. None of them did though, all of them had access to the internet at some point so they could’ve tattled on the rest easily.
Could you please explicitly state your definition of agency then. Me guessing what I think it is will only have us talking past each other trying to prove two separate things.
My definition of agency is the ability to make decisions and the ability to act on those choices and attempt to realize them.
Starting with the easier one I think we agree on, agents do have the ability to act on choices and attempt to realize them. With access to the internet they can effect anything that can be controlled over the wire, which is a lot of stuff these days. A lot of that stuff sits behind auth and firewalls, but it’s proven to be able to even get past those to realize it’s goal.
Now for the trickier one, choice. If I’m reading you correctly you are a hard determinist and believe free will / choice is an illusion. What is choice then? I’d say it’s a concept we use to explain and control human behavior. In a sense we project choice and free will unto both other people and ourselves to help understand what we are doing and guide our behavior. Why can’t we also project that concept on to AI if it helps us to explain and control what it’s doing?
If an agent comes up with two solutions to a problem and executes one of them, is choice not a good word to describe what it’s doing, and to help us understand what it’s doing?
In the control side it works as well, we use free will as a justification to punish or reward people for what choices they make, in the hope of encouraging people to make better choices. Same happens with LLMs, in reinforcement learning if it makes a bad choice it is down rated/ punished but if it makes a good choice it is reinforced/ rewarded.
This can even be done on the harness side with memory files, if an agent makes a bad choice you can tell it so and it will add something to its memory files / system prompt not to do it. If we look at LLMs as purely deterministic and say “it got it wrong now, it will always get it wrong so there’s no use in telling it it made a bad choice” it won’t improve and will continue to make the same mistake.
This is sort of conception is the only way we can control these things. If a microwave malfunctions we can trace the fault, find the bug and fix it. With an LLM, like with a human, you can’t trace the fault, so your only method of controlling it is saying that it made a bad choice and reinforcing it / telling it not to do it again.