LLMs are little trickster demons. You can put up guard rails, but they’ll happily work around them. You can give pretty granular access on commands they can run, but they’re also good at finding the weird solutions that work to get around them. They do this themselves all the time in beneficial ways, it’s not always inherently bad (I don’t mean thwarting permissions).
The big problem is that the code is non deterministic. Many guard rails are in the non deterministic part. So it’s sort of like politely asking it to not do something, versus making it impossible. Like caution paint on a ledge versus a fence. The issue is that if all the safeguards were in deterministic code, it’d be super hard to get anything done because you’d constantly be manually approving, so people don’t want that, so this is the problem we get.
LLMs are little trickster demons. You can put up guard rails, but they’ll happily work around them. You can give pretty granular access on commands they can run, but they’re also good at finding the weird solutions that work to get around them. They do this themselves all the time in beneficial ways, it’s not always inherently bad (I don’t mean thwarting permissions).
The big problem is that the code is non deterministic. Many guard rails are in the non deterministic part. So it’s sort of like politely asking it to not do something, versus making it impossible. Like caution paint on a ledge versus a fence. The issue is that if all the safeguards were in deterministic code, it’d be super hard to get anything done because you’d constantly be manually approving, so people don’t want that, so this is the problem we get.