Well, well, well…
The Linux Foundation published a new study today around the impact of generative AI on open-source software development. One key takeaway is that ~25% of new open-source software pull requests amount to AI slop.
The new Linux Foundation research study found that GenAI has the potential of cutting open-source coding time in half for increased productivity. They also found 59% of project maintainers found growing GenAI-based contributions. The 50% productivity boost was found to be across all types of open-source software development tasks. 71% of OSS maintainers agreed that GenAI reduces barriers to entry.
Very concerning though, their research found that 25% of incoming pull requests are estimated to be low-quality AI slop across open-source projects. Presumably that number will, sadly, continue to rise as more newcomers and users embrace AI/LLMs for working on patches on their behalf.
Those curious can find the Linux Foundation Research report in full via LinuxFoundation.org.
The whole productivity boost is such a non-argument. A lot of that code was written by people why don’t know what they are doing, so without the tool they wouldn’t have written any code at all. For the people that do know what they are doing, it helps them shit out code faster. But is that actually a gain in productivity? It would only be the case if the quality remains the same, which in most cases it doesn’t. And if the sloperator took the time to read and understand the code, even review it perhaps, the time gains would evaporate. But we all know good sloperators don’t read or think, they ship! And then there’s the additional workload for the people that have to actually review all of that code the AI pooped out. Especially with complex code this can be very hard and with the increased amount of code submitted, their workload has grown. If we take that extra (harder) work into account, AI is actually making us as a whole less productive.
If the gains were as significant as the slop glazers claim, the results would speak for themselves. In reality we aren’t sure, people THINK they are faster, they might ship shit out faster, but at expense of quality or extra work for others. Software as a whole would be transformed and we would all be in awe, in reality we see a bunch of shitty slop projects on Github, shit out and forgotten within a month. We see more issues than ever before, users are complaining about how bad everything is. Best case scenario you get something that kinda sorta works, good enough, while looking and feeling like icky slop.
It’s been years now, where are the results?
That’s a very good point. The gains / results are less significant apparently. Recently heard about a software supplier actually asking for MORE money than contracted, to cover for the expenses on tokens for the software work they provide. That means, the cost for the customer actually went up. That feels completely backward.
Thats less than I would think.
After all, Sturgeon’s Law
It’s eternal september
Productivity gains are substantial but vary by task complexity: GenAI accelerates OSS development in more ways than simply writing code faster. Respondents estimate average time reductions across all six OSS development activities measured, including proposal discussions (19%) and code authoring (52%). For high-complexity tasks, however, 21% of respondents report taking more time when using GenAI.
You lose some you also win some though.
Everything still goes through human review though regardless.
It just shifts the time spent from writing code and other things to rewiting code and reviews most likely
If they’ve never received a patch potentially from someone too, it’s going to be more likely to be bad code too that can be reviewed fairly quickly
This is just a survey though, people are notoriously bad at estimating their own speed.
Several MIT studies have shown that while people feel more productive they are basically actually equal because of overhead. They have to spend more time managing the LLMs and designing prompts and reviewing output and it basically is a wash.
We could argue everything here is wrong then 😂
