I’ve maintained for years that my engineering job is, sarcastically, “to move entropy around.” This is no different, as we’ve squeezed the envelope at one end only to balloon it at the other.
I would add that once you’re all-in on codegen for a project that escapes human comprehension, you have to keep using more codegen to maintain it. It effectively hands off the whole game to the robots, and probably keeps growing in scale all the time. By way of your observation of increasing cost of human involvement, it resembles a massive tech-debt scheme to me. Which is to say that it’ll work impressively well in the short term until someone has to get their hands dirty with the gordian knot of code that has been quietly cooked by LLMs for years. At that point, the bill comes due.
I think this is the best argument for human-in-the-loop, or even my personal favorite “LLM as an advisor or intern” instead. I’ve also had traction with AI codegen for disposable/single-use programs, where I just need to know if something works.
I’ve pondered on the feasibility of using codegen to manage projects done by codegen and the problem I see with that is that regenerating a project just replaces known bugs with unknown bugs rather than actually fixing bugs, plus the user interface will also change even if the requirements specification is so complete and well done that it manages to have the same features, and user really dislike that plus there are actual costs in retraining people to use a whole new UI.
I agree. Although I’ve been thinking more along the lines of bug fixing, adding features, and incrementally changing said app. So like you would with a human team, disallow large-scale refactors as a rule. As far as I can tell, that relies on codegen to be well-skilled and well-behaved - a whole other area that requires labor with possibly diminishing returns.
To fix the bugs with targeted regeneration, you have first to find where there bugs are, meaning incurring the cost of understanding the generated code.
By the time you track down were the bug is you might as well just fix it by hand yourself.
I’ve maintained for years that my engineering job is, sarcastically, “to move entropy around.” This is no different, as we’ve squeezed the envelope at one end only to balloon it at the other.
I would add that once you’re all-in on codegen for a project that escapes human comprehension, you have to keep using more codegen to maintain it. It effectively hands off the whole game to the robots, and probably keeps growing in scale all the time. By way of your observation of increasing cost of human involvement, it resembles a massive tech-debt scheme to me. Which is to say that it’ll work impressively well in the short term until someone has to get their hands dirty with the gordian knot of code that has been quietly cooked by LLMs for years. At that point, the bill comes due.
I think this is the best argument for human-in-the-loop, or even my personal favorite “LLM as an advisor or intern” instead. I’ve also had traction with AI codegen for disposable/single-use programs, where I just need to know if something works.
I’ve pondered on the feasibility of using codegen to manage projects done by codegen and the problem I see with that is that regenerating a project just replaces known bugs with unknown bugs rather than actually fixing bugs, plus the user interface will also change even if the requirements specification is so complete and well done that it manages to have the same features, and user really dislike that plus there are actual costs in retraining people to use a whole new UI.
I agree. Although I’ve been thinking more along the lines of bug fixing, adding features, and incrementally changing said app. So like you would with a human team, disallow large-scale refactors as a rule. As far as I can tell, that relies on codegen to be well-skilled and well-behaved - a whole other area that requires labor with possibly diminishing returns.
To fix the bugs with targeted regeneration, you have first to find where there bugs are, meaning incurring the cost of understanding the generated code.
By the time you track down were the bug is you might as well just fix it by hand yourself.