Preamble: I am a huge LLM/codegen skeptic, for a litany of reasons.
With that said, this honestly sounds like whoever did the prompting didn’t include something like this in the prompt, along with nuanced adversarial review loops:
unsafe blocks are strongly discouraged, and must be specifically justified, and survive antagonistic review by a rust language specialist agent and software security specialist agent. Part of the intent of this reimplementation is to leverage the strengths of the rust language itself, and rampant unsafe usage directly contradicts that.
No, I’m saying that there are ways to introduce feedback mechanisms and antagonistic review and specialist agents to improve the end result of what codegen crap gives you. It’s by no means perfect, but it’s a damn sight better than “make no mistakes”, and if the quality of the rust code is really that much of a shitshow, it’s probably not even being reviewed by a human before getting yeeted into prod, let alone the other things I outlined.
Sure, but adding adversarial loops, or, some complex addition to every prompt would increase the token usage massively, thus the cost.
So, why bother with that?
I could very well be wrong, but all indications I’ve seen have been that that the cost of the tokens needed to get something even vaguely acceptable is greater than the cost of just hiring someone, or a group, to do the job. Unless something needs to be done fast due to an immediate concern, and the cost is no object.
Well yes, I’m not saying it’s a free lunch. The point is a large up-front investment buys you unlimited tokens in perpetuity for no more than the cost of electricity to power the cluster — which can be surprisingly cheap for UMA machines.
I’ve swapped off API models entirely and have been able to do a lot more useful work since I don’t have to worry about token cost or usage limits. Granted it’s slower per task (less tk/s), but overall much more productive, and private.
Preamble: I am a huge LLM/codegen skeptic, for a litany of reasons.
With that said, this honestly sounds like whoever did the prompting didn’t include something like this in the prompt, along with nuanced adversarial review loops:
“It can’t be that stupid, you must be prompting it wrong.”
The ol’ reliable.
No, I’m saying that there are ways to introduce feedback mechanisms and antagonistic review and specialist agents to improve the end result of what codegen crap gives you. It’s by no means perfect, but it’s a damn sight better than “make no mistakes”, and if the quality of the rust code is really that much of a shitshow, it’s probably not even being reviewed by a human before getting yeeted into prod, let alone the other things I outlined.
SMH they forgot to type „make no mistakes“.
Sure, but adding adversarial loops, or, some complex addition to every prompt would increase the token usage massively, thus the cost.
So, why bother with that?
I could very well be wrong, but all indications I’ve seen have been that that the cost of the tokens needed to get something even vaguely acceptable is greater than the cost of just hiring someone, or a group, to do the job. Unless something needs to be done fast due to an immediate concern, and the cost is no object.
Perhaps they run a local model, then tokens are essentially free.
Would need a very beefy cluster, and that definitely isn’t free.
Well yes, I’m not saying it’s a free lunch. The point is a large up-front investment buys you unlimited tokens in perpetuity for no more than the cost of electricity to power the cluster — which can be surprisingly cheap for UMA machines.
I’ve swapped off API models entirely and have been able to do a lot more useful work since I don’t have to worry about token cost or usage limits. Granted it’s slower per task (less tk/s), but overall much more productive, and private.