“Reading through this paper AI adoption is shifting…”

S@solidangle.bsky.social8 Oct 03:25 UTC

Reading through this paper AI adoption is shifting programmer labor from writing code to code reviews, which sounds like a good way to burn out a lot of programmers (and limit the pool of future reviewers), for "statistically insignificant" gains in final software output

View full-size image

F@fadeinpro.com8 Oct 05:13 UTC

This is some real million-monkeys scaling. (This paper is detailed and methodical and lengthy. It's worth looking at, though, for the not-so-surprising conclusions as well as everything you ever wanted to know about software development.)

A@athomashemlock.bsky.social8 Oct 03:34 UTC

I liked the "AI is Radium" metaphor for its impact on people's cognition, but the most apt metaphor for AI's impact on cyber security is essentially "We have turned every single middle-to-large enterprise on the planet into the Triangle Shirtwaist Factory," and we are about to see so, so many fires.

D@davidcrespo.bsky.social8 Oct 01:41 UTC

interesting paper! they find increases in coding productivity (indicated by more PRs and more LoC) but find that these do not translate to more work getting done (indicated by number of issues closed) due to bottlenecks in review. they argue through various tests that issue scope did not increase

T@thomasfuchs.at8 Oct 03:13 UTC

Writing code hasn’t been the bottleneck in software production since the 1980s; RAD, IDEs, OOP, and off-shoring made code (in corporations) a relatively cheap quasi-commodity long ago. The bottleneck has always been making software that’s actually useful.

P@psychic.surgery8 Oct 02:36 UTC

When you write code, you understand it. When you understand it, you find the gaps in the specification. When you specify those gaps, you build understanding among the stakeholders. You can’t shortcut any of this unless you want a big blob of liability.

R@r.v.cx8 Oct 02:29 UTC

I haven't read this study yet, but having read a LOT of "quantitative software engineering" research...studies almost *never* identify productivity increases. Despite vast, vast increases in engineering productivity over the decades. So maybe temper your enthusiasm for an endorsement of your biases.

T@taulby.bsky.social8 Oct 02:11 UTC

AI doesn't have an imagination. I have seen that AI bots have made connections not yet anticipated by researchers, but all of the preceding information was derived from human work. I think that is impressive, and can serve some fields, but it's also why AI doesn't break any ground without theft.

J@jman4747.bsky.social8 Oct 05:04 UTC

People won't tend to use LLMs the "right" way. They'll tend use them in whatever way accords with the path of least resistance. Any future where these are widely used is one where the harms continue to outweigh the benefits.

L@leftoutside.bsky.social8 Oct 08:18 UTC

Data ends in March 2026, so about 8 months into Claude Code being available and three months before Fable. The bottlenecks in code review have since reduced because AIs can do it, but that won't stop anti-AI people using the paper to say AI doesn't work. There'll always be bottlenecks.

D@drew-lewis.com8 Oct 04:25 UTC

This reminded me of a post I saw today from a mathematician-turned-AI-bro noting (without an ounce of self awareness) the exponential increase in submissions to arXiv and math journals

O@olivia.science8 Oct 06:51 UTC

She called it in 1983! > Bainbridge argues that new, severe problems are caused by automating most of the work, while the human operator is responsible for tasks that cannot be automated. en.wikipedia.org/wiki/Ironies...

B@balinares.bsky.social8 Oct 09:35 UTC

A thought: the hard part of software engineering always was and remains determining what's the right problem to solve. Solving the wrong problem can easily be a net-negative and if you do it faster, the cost accrues faster.

S@shirosiri.us8 Oct 07:20 UTC

I’ve been thinking about The Rats of NIMH a lot recently

S@selene.euyis.me8 Oct 05:55 UTC

not to comment on the merits of the text but i just really don't like the framing of "harvard just dropped a study" giving it some sort of undue presumed quality, credibility and the assumption of it having made it past peer review instead of "some people affiliated with harvard uploaded a pdf"

G@grogsgamut.bsky.social8 Oct 05:53 UTC

I'm going to go out on a limb and say that if AI can't improve productivity of coding, then all other claims about productivity improvements are essentially horseshit. Coding was the one thing everyone was like "well yeah of course it will be able to do that faster"

D@dawnxianamoon.bsky.social8 Oct 05:05 UTC

This is a pretty substantial study - 718 firms. AI allows engineers to write crappy code faster - but devs end up having to spend more time reviewing & revising code so in the end there's no actual productivity gain.

L@leonardpierce.bsky.social8 Oct 03:21 UTC

AI is not intended to 'work' or 'increase productivity' or 'streamline coding' or whatever the fuck. It is intended to discipline, deskill, and devaluate labor first, and a way to generate more financialized capital and keep it out of the tax system

W@wvslasthandloader.bsky.social8 Oct 03:45 UTC

Between this and the coffee/health study it's been a really big day for studies that vindicate my personal life choices

A@atticusgf.bsky.social8 Oct 06:00 UTC

I think this is interesting but: 1) time scale of this kind of makes me shrug at the results 2) I think we're in a post code-review world (at least in the traditional sense) and we don't really realize it yet.

F@foxes.ceo8 Oct 07:43 UTC

The irony of implementing LLM coding requirements by software companies who also gutted their QA Department shouldn’t be lost on anybody

P@pengzell.bsky.social8 Oct 04:49 UTC

If an AI codebase falls in a forest and no one is around to hear it, does it make a sound?

C@chimerror.blacksky.app8 Oct 07:24 UTC

The sad thing is that I know exactly how this will be spun: yes, but wait until the models improve; this is just growing pains. (Ignore all the externalities and other high costs, which aren't likely to come down.)

K@klaskeymusic.bsky.social8 Oct 02:20 UTC

When I advised the orchestrations for a college musical theater group, if a student submitted a bad chart, it was faster for me to rewrite from scratch than to revise it.

P@planetoffinks.bsky.social8 Oct 03:54 UTC

We're now at two objective studies for the best use case anyone has for LLMs and both found it useless even for that. It's a scam. It has no use

M@mewtation.bsky.social8 Oct 03:42 UTC

just today we got pressure from management to give a "casual approval" to multiple thousand(s)-line PRs from a junior because doing it the right way would have taken too long. exasperating

G@graay.bsky.social8 Oct 05:40 UTC

It's kind of starting to look like the true AI bottleneck is accountability.

S@stephencoxauthor.bsky.social8 Oct 07:52 UTC

Sit down. Take a deep breath. Harvard says AI does not increase productivity. Who knew? WHo could possibly have guessed?

P@pespizero.bsky.social8 Oct 04:00 UTC

set aside some time in your prayers tonight for the legion of c-tier app devs on here who believe that vibe coding is mankinds single greatest technological achievement 🙏

S@stevensantos.bsky.social8 Oct 01:55 UTC

Getting the feeling more that those impressed by Claude are not very smart or competent at their jobs.

K@kenlowery.bsky.social8 Oct 01:45 UTC

I will simply say I am seeing exactly this situation play out right now in a different field. Work that would have taken about 3 weeks start to finish via traditional methods is now into week 6 and still in what amounts to first draft territory

O@oisinmcgann.bsky.social8 Oct 07:21 UTC

So yes, it enables you to code faster, but then all the extra finding and fixing mistakes cancels out that advantage.

L@liammci.bsky.social8 Oct 01:59 UTC

Unless the study didn’t account for this, I’m a bit surprised it’s not less than zero.

C@chartreusewahoo.bsky.social8 Oct 03:48 UTC

But I’m sure it’s soooooo close to replacing us all just another trillion dollars and it’ll definitely be smart enough then to take all our jobs you’ll see

L@lalegault.bsky.social8 Oct 02:55 UTC

AI’s primary use was always meant to be surveillance, it’s not good at much else.

A@aggiebrett.bsky.social8 Oct 01:50 UTC

The research continues to show that AI is fools gold. The emperors new clothes. A steaming load of horseshit.

A@astro-kraken.bsky.social8 Oct 04:18 UTC

Im glad im infrastructure and not software. AI could never ask someone else what lights are on as well as I do.

S@sriku.org8 Oct 05:40 UTC

The lesson is clear then. Just stop reviewing generated code already. (.... and watch your company go down 😉)

J@jakekvz.bsky.social8 Oct 06:06 UTC

Now imagine how inefficient it is to be instructed to use CoP*lot in the workplace.

K@ketanjoshi.co8 Oct 07:23 UTC

This is the application of generative AI that is meant to be 'the good one'

P@patigallardo.bsky.social8 Oct 03:38 UTC

Computer make much code. Human have to read and check much code. Everything same fast.

M@maggiemcneill.bsky.social8 Oct 07:16 UTC

Another study from the Institute of Obvious Results

D@dicefunctional.bsky.social8 Oct 03:52 UTC

Oh my god who could have seen it coming.

T@tomboyrespect.bsky.social8 Oct 01:48 UTC

Very based take by my man you gotta take slytherin out of your name please

H@hampus.bsky.social8 Oct 09:14 UTC

Special boys who can use this software CORRECTLY and are seeing REAL GAINS are asking ChatGPT for the best retort

M@mikethomas.bsky.social8 Oct 01:58 UTC

Seems worth $3 trillion or whatever

P@phatcontroller.bsky.social8 Oct 07:33 UTC

Claude AI increases productivity 0% study shows

A@aimoratoriumnyc.bsky.social8 Oct 01:49 UTC

But I was told this would take all of our jobs.

B@bumblefudge.com8 Oct 06:28 UTC

AI is really revolutionizing bullshit jobs

M@mendinso.bsky.social8 Oct 07:37 UTC

I'm SHOCKED. SHOCKED I tell you.