This one experiment gets recited over and over and over again. I'd like to see some more recent data on this rather than seeing the 8th article telling me about the same unreproduced experiment from 2024
2025 is such old news that this just isn't relevant.
METR already redid the study at a later date and now finds a likely 18% speedup
"For the subset of the original developers who participated in the later study, we now estimate a speedup of -18% with a confidence interval between -38% and +9%" (note their use of - and + here could be slightly confusing but they do mean 18% faster per the post)
But the link you shared explains that few of the participants filtered the tasks and chose which to do and which to avoid, based on AI availability, so, as per their review, the data is probably not entirely useful.
4 Comments
PortugalSpaceMoon@infosec.pub · 15 pts · 67d
This one experiment gets recited over and over and over again. I'd like to see some more recent data on this rather than seeing the 8th article telling me about the same unreproduced experiment from 2024
vivalapivo@lemmy.today · 0 pts · 67d
This data will hurt your feelings
Eyekaytee@aussie.zone · 11 pts · 67d
top comment: https://news.ycombinator.com/item?id=48757440
2025 is such old news that this just isn't relevant.
METR already redid the study at a later date and now finds a likely 18% speedup
"For the subset of the original developers who participated in the later study, we now estimate a speedup of -18% with a confidence interval between -38% and +9%" (note their use of - and + here could be slightly confusing but they do mean 18% faster per the post)
https://metr.org/blog/2026-02-24-uplift-update/
must be heart breaking for Lemmy ðŸ˜
massive_bereavement@fedia.io · 3 pts · 67d
But the link you shared explains that few of the participants filtered the tasks and chose which to do and which to avoid, based on AI availability, so, as per their review, the data is probably not entirely useful.