I don't think that one has held up very well. A cynic might opine that they've realized how easy it is to grab headlines and attention by putting out research that feeds people who are hungry for negative stories about AI.
UPDATE: I'm going to take this one back a a bit: it turns out Apollo publish articles like this on a daily basis, so the occasional headline that appeals to the "hungry for negative stories about AI" crowd is less of a pattern than if they published less frequently than that.
You’re implying that Apollo is biased, but the thing you’re quoting is a very brief report on publicly available research. This is the entirety of the article:
“The US Census Bureau conducts a biweekly survey of 1.2 million firms, and one question is whether a business has used AI tools such as machine learning, natural language processing, virtual agents or voice recognition to help produce goods or services in the past two weeks. Recent data by firm size shows that AI adoption has been declining among companies with more than 250 employees, see chart below.”
Two questions:
1) What does that chart show today? (Genuine question. I’m curious and don’t know where to find it.)
2) Regardless of what it shows now, why do you think Apollo reporting on it is noteworthy?
I'm implying they may be biased in what they chose to report on and how they frame it, yes. Deciding which figures to turn into a chart is an editorial decision.
For being one of the biggest AI boosters on HN, you'd really think you could make a stronger case than "Apollo is bad because they observed and reported on a fact". It wasn't a prediction, so there's nothing to "hold up very well".
Misunderstanding what actually qualifies as a "prediction" seems to be a recurring problem among AI commentators.
I've seen dozens of posts/top comments in the last few years dunking on "AI skeptics" for statements that didn't even vaguely resemble an actual prediction.
Like the front page post from Karpathy which got 600+ upvotes by throwing an AI model at old Hacker News comments without any human vetting to score "predictions". And upon further inspection included a bunch of unambiguously factual statements of the present or conditional if-then logic about the future which when glossed over for convenience become "failed" predictions.
A cynic might opine that an AI shill is not a great source of unbiased review of an article. The article is about a biweekly survey of companies and the trend was clearly dropping. This isn't about token use by tokenmaxxers increasing. And it isn't about the opinions of the folks you hang out with when you're flown to conferences to gush about AI. It's a survey of businesses. And aside from programming and math, AI sucks. Even in programming and math it's a tool with a lot of downsides.
> A cynic might opine that they've realized how easy it is to grab headlines and attention by putting out research that feeds people who are hungry for negative stories about AI.
This is kind of gross. Make the claim or not, don't do this pseudo-sealioning thing. Moreover, engage with the actual content (or you could just not engage at all!), not this shallowest possible dismissal.
I don’t think I’d judge someone offering a prediction the same way I’d judge an observation of what’s already happened. It’s implied that guesses about the future are uncertain in a way that statistics about the present are not.
Does for us (data science, specialty chemistry, finding/optimizing formulas). We can get from idea to prototype in a day compared to 1-2 weeks. All that boilerplate to process data took a long time to code by hand, especially for the data science/math guys who are not programmers. Now we can actually try all ideas instead of guessing the most promising and only try that.
And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?
I don’t think anyone is disputing that it makes many kinds of work easier and/or faster, just the degree of those improvements and how much of those improvements actually translate to dollars earned or saved.
Seeing hard data on that would help settle the issue.
Unfortunately not everything will cleanly show up in PnL, which is a function of the entire company’s processes.
If developers are shipping more code (and most of the enablement is in software right now) does that show up in top line? Maybe… if sales and marketing are also selling more, and product people are building the right thing, and…
Can’t share more than that publicly. Just that the time saved on this one thing alone is saving us way more money than the subscription per employee that is needed (~100$). I don’t doubt there are companies where this is not the case. Would like to share more details, but the rough math of doing something in a day vs several days + engineering salary have to do here.
> And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?
Nobody even tries to measure this. Their CEOs and board members have AI shares, and potential investors need to see your AI story to justify your valuation. It's a prison of lies.
Unfortunately we're at the mercy of Goodhart's law and in an interesting twist, AI is really good with things that have an immediate feedback mechanism, eg writing code that doesn't compile or pass tests, but you're trying to grade AI o something that doesn't have an immediate feedback mechanism. AI can slop out a PowerPoint deck in a fraction of the time it used to take a human by hand. The competent human then goes in and unslopifies it, but overall, still time is saved. Unfortunately, that employee is salaried and not hourly, so they can't click out early and even if they were, what's their incentive to clock out early if it means less money in their pocket? Or they can use the hours now free on something else. My gut, as an Internet random who's used AI is that it helps. I don't know how to measure it, just like how do you measure how much better it is to live in a clean house vs a cluttered one? How would you put that in a metric? If you can't put that into a metric, why bother cleaning your house? We clean it anyway, because of a certain je ne sais quoi.
Tons of unstructured data that needs to be cleaned up in so many different ways that are hard to predict. Knowledge distributed in the heads of greybeards, handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software. Which you have to repeatedly ask for and discover more piece by piece. Then clean up, discover issues, make the customer gather more data/correct it, etc.
> handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software
This area is one which AI has shown to be pretty good (as long as some error rate is acceptable). Character recognition, voice transcribing, converting documents from one standard to an other. I can see an argument that if people spent most time doing those things, AI will speed it up (with a bit of added errors).
I am reminded however that in translations and AI generated subtitles, there is still a lot to be desired compared to the human touch. Handwritten papers in a different language, using local short hand and assumptions, may result in some odd and error prone AI transcription.
No, but if we go from data to experiment faster we see earlier if something is wrong. Once we get data we quickly can fix the data pipeline and get results. If there are issues we can quickly show what/why and discuss it with them in detail and make changes based on their response faster.
Yes, but they tell things and maybe forget to mention certain things or contradict other data they have given you. Now we may not notice it initially until we make an experiment and get the results. Something is wrong and we have to find out what. Talk to the customer again. Fix experiment. Repeat. Faster experiments means the latency from new data to results is less and we finish the project faster. Errors become less expensive and cause less delays. People make errors and data quality is often horrible so the shorter iterations are nice.
And yet, many companies today are spinning up full stack teams, asking them to be experts from user interviewing to database design, architecture, hosting, devops, API management, data product, and also keep an eye on GDPR compliance.
I see dozens of teams that are doing cloud to the wazoo when really they need to just be told “we’ll load your routes from main.py, build your assets in /static, you need a DB click the button to provision it”.
It boggles the mind that so many teams are reinventing the wheel a million times over to publish their APIs, register AWS accounts, setup CI, decide on ECS vs Lamda, etc.
This sort of activity has been the source of income for a very large number of people. You can't really fault them for trying to extend that gravy train's run.
The question is: how does any of that translate into more revenue? If you try more ideas and they mostly go nowhere that's not necessarily helpful to the bottom line.
If that were universally true in all domains then all the best and most reliable software in the world would be written in interpreted languages, because they have the fastest iteration speed. Sometimes rapid low-quality iteration is just spinning your wheels.
Not all experiments work out. If we find the right approach faster we can finish the project faster. Some projects are mostly fixed price or offer bonuses for reaching milestones in a certain time. Meaning we more often get a bonus and can take more projects. Also we can take projects with tighter deadlines that we would need to decline otherwise.
A lot in this world is not optimized by software yet. For our customers they have employees with decades of experience guessing things, maybe some software that kind of helps but still not good enough (slow, not finding really good formulas). They often have difficult problems and many failed projects. We try to find such customers and offer our service. If we are cheaper (+ show portfolio of successful projects) more are willing to give it another try with us.
Right, and while that might be working out for you right now, there is a finite budget that companies have for projects like that. The "zero financial payoff" framing sounds about right to me.
I understand there's some benefit of this. But is it worth putting the global economy into a death spiral? So ManBeardPc can save 4-9 business days to go and say "yeah, that doesn't work".
There's a difference between the cost of coding and the cost of delivering and maintaining software? AI has demonstrated that these two things are almost entirely unrelated.
Almost everywhere I've worked, the "designing and writing code" part was about 5% to at most 10% of the calendar time that it takes to make and release a software product. The remaining 90-95% of the time is: aligning stakeholders, meeting with all contributing teams to prioritize, gaining approval from various executives to commence the project, obtaining security reviews, privacy reviews, legal reviews, and so on, soaking in dev, staging, and beta environments to find bugs, manual QA testing, aligning with marketing--potentially waiting on key external marketing events, waiting for various people to come back from vacation, waiting for Mike to answer his E-mail... These things take the vast majority of a software project's time and can't be automated by AI.
So, AI is making 5-10% of your project much faster. Whoopdie doo.
This is a complete misrepresentation of what the Apollo piece can actually show, because it's not measuring the "payoff" of AI investments, but only the top level revenue of those companies (https://www.apollo.com/wealth/insights-news/insights/daily-s...).
This is a healthy corrective to the most insane AI maximalist takes (mass disemployment in late 2027, 10% GDP growth in a year), but it shows nothing about whether the actual economic impact of AI spending is.
First, to show that, you'd have to justify the counterfactuals (if those companies hadn't spent on AI, would their position be better or worse?)
Second, while AI spending is increasing rapidly, normal companies aren't spending 1% of their revenue on it. So how can you look at a noisy chart of revenue and expect that "oh, it's not up, the AI spending must be useless?" Those companies could be spending < 1% of their revenue on AI, deriving value from it, and it wouldn't be guaranteed to show in a chart like this.
I saw something like the average anthropic employee thinks there will be 5% GDP growth because of AI, which is totally insane when a 1-2% extra growth will be an incredibly good result.
It _is_ about judgment. The challenge is figuring out what defines “good judgment” and how to capture those elements in a format that an agent can assess continuously and at multiple levels.
The problem is most people don't know what makes them successful from a people perspective, except in the coarsest terms.
90% of the business outcomes are delivered by 10% of the people. The problem is they don't know which people. Therefore they also don't know which processes, if automated or augmented with AI, will achieve the target outcomes.
So instead they’re shotgunning. The problem is that the existing people-based system has built-in stabilizing mechanisms to prevent misalignment from taking the entire operation off course. AI stresses and often breaks those.
The result? Higher costs and chaos.
This needs to be treated like an org transformation. Traditionally that's most successfully done by starting a new small team that builds ground-up and eats the old team.
Your company uses AI and it gets ahead a lot quicker. However, your competition does so as well.
Many areas of economy are zero sum game - so your $ ends up being the same as previously. However, now with a "tax" paid to AI model providers for being able to stay competetive.
Yeah but if that was true we'd see better or cheaper products at least, or new competitors rising. If it increases the speed for everyone, but there's no visible change for the end user, I wonder if AI makes you really more competitive in the first place.
I suspect the reason is the industry just does not want engineers to be mostly technical. There's a real tradeoff to AI, and I think companies are taking the wrong lesson and thinking "well now that programming is so easy, engineers need to do more of XYZ", when I think it should be "this tool is powerful but also takes more focus so engineers should do less of XYZ".
I’m curious, what are some examples of XYZ? What does “mostly technical” mean to you, and why do you think it is the industry doesn’t want that? Just trying to understand, because I don’t think I’ve experienced this myself.
I'm talking about the day to day work. At least in my experience at several big tech companies, the more senior an engineer is, the more you need to delegate work and are expect to lead projects by managing them. There's technical knowledge involved of course but it's a small part of it, mostly directing other more junior engineers.
AI allows you to move faster, but it also demands that you pay more attention in proportion to how fast you want to move. This requires more focus, i.e. more time developing. Effectively the structure at companies should swing back to how it used to be a few decades ago (I think, maybe I'm wrong and it's always been like this), where engineers mostly focused on delivering code. The difference now is that they can be expected to deliver much more code at higher quality.
To summarize, using a little AI doesn't pay off, you need to be doing that all day, and that is not how engineers spend their time in most of the industry anymore.
What I'd really want to see is revenue and profit per employee over time tracked against token usage/spend, and not limited to the S&P 493.
Apollo's own argument that most AI spend is coming from non-S&P 493 (as per the last two charts) tells me it's hard to make any conclusions based on these charts.
If you are a company and can't figure out how to improve your bottom line through increased per person productvity, sales, product development, even just responses to customers etc, then that seems kind of like a "you" problem and not problem of the AI models...
I dunno: this is pure anecdata but software engineers I know well - including me - all say they’re delivering multiple times faster, so it’s definitely good for something.
I don’t know about other tasks, but it’s pretty rad for coding, testing, DevOps, and config.
This research is specifically NOT about tech companies. It is about the rest of the fortune 500.
If you are in software or IT at a non-tech company that is in the fortune 500, the impact of significantly better productivity may not be measurable in aggregate in the company's P&L.
In smaller companies I expect it could be more measurable because some of them really are using AI much more effectively across many functions. But we're finding it very hard to measure meaningful outcomes.
But actually 40% of the S&P 500 are tech companies, and they are saying only the mag 7 are showing clear revenue increase. Would would suggest that even in most tech we are not seeming measurable revenue growth, right?
One plausible explanations that has occurred to me is this: While programmers etc. can produce way more code with LLMs, the declines in quality and coherence of the work, and a widening gap with a coherent strategy and aim for value means the increased activity is mostly not valuable. I suspect this may just be an intensification of https://en.wikipedia.org/wiki/Productivity_paradox#Little_pr...
I'm not sure I'm any faster, but I think the end result is better and I've attempted some big refactors and new architectures that I would never have tried without AI.
N=1 but I'm not sure it's made my bosses any richer. What we're shipping has better tests and documentation than it ever would have before AI.
Most, not all, are just delivering more bad stuff faster. I personally also think I deliver more good stuff with AI, but the products I use are just getting worse daily it feels like. Just the other day I noticed Slack had installed some disgusting menu bar icon for their AI bot, without asking, just to take a fresh example of this.
I've never heard any such claims but meanwhile in my actual job I'm seeing unbelievable productivity gains (in terms of actual cash spent before and after vs the same kind of work being done).
If you are an AI company with hundreds of billions at your fingertip, and looking for $2 trillion valuation, and can't figure out how to let your customers execute _on the exact thing you are selling them and basing your valuation on_, sounds like a "you" problem too right?
there is increase in productivity of both types of employees: good actors who want to deliver on quality of work, and bad actors, who now 20x weaponized in generating all kind of slop. Catching second type is now very nontrivial problem.
I'm sure they can just fire half and double up work or outsource. That's the traditional way and it definitely works. The worst that ever happens is a few will reneg on payroll. Everyone who still chooses to work there is happy enough.
While they are seeing some individual productivity benefit, a lot of that doesn't really impact the bottom line directly and all their competition are using the same tools.
Conjecture supported by allegories that break down quickly. Arguments conflating the dot-com boom with the current AI boom even though the economics are wildly different. Dismissive arguments conflating anti-AI constituents with people who were against cars, etc...
AI is highly divisive. It's dramatically reduced the value of code, which has derailed a lot of careers who have built their entire adult life around the idea that code would remain valuable, while simultaneously driving others into a psychosis who believe that AI can do anything.
When it's the right tool, it's a silver bullet. When it's the wrong tool...or not the right tool yet, you can spend more time steering it than you would doing it yourself.
I'm seeing another behavior in my purely personal research...asking AI 'can I do the thing?' and it comes back with 18 pages of 'yes, and here's how you do it'. Often it's the discovery that produces the endorphin bump. It's cool to know I can jailbreak the Peloton...and now that I know I can...I'm not so motivated to actually do the thing.
My suspicion is that this is because AI employee-type systems like Dots or Grokbot just haven't been adopted yet. I think once they're good enough, that's going to be the catalyst for a lot of these companies to start adopting and using AI productively.
The company I work for insists on sending a crew from Engineering, Product, and Sales out to a customer site as part of our sales process. I'm personally much more productive using AI, but that only goes so far when closing a deal still follows a process stuck in the 50s.
It makes sense to me and I'm saying this as a strong believer in AI and its ability to add value.
The system itself doesn't reward value creation. I've been saying this for years. I've seen many smart people waste their time on bureaucracy; their intelligence was being wasted; so I'm not surprised at all that now that we have AI, that its intelligence is being wasted as well.
The bottleneck is an allocation issue, not a capability issue.
The current reality is that a lot of very bright people are currently working for big tech in bureaucratic roles and not allowed to focus their attention on 'risky' problems which would deliver real value if resolved.
The current system prioritizes monopolization and is designed to try to ensure that all innovation happens within big tech. It is sacrificing potential innovation which could have occurred much faster outside of big tech in order to ensure that any gains are captured by big tech... But it's sacrificing an enormous about of efficiency.
I hope big AI companies can use their political leverage to try to open up opportunities for people who want to innovate outside of big tech. That's aligned with their long term interests.
I think AI companies are convenient structures which could allow big funds to benefit from decentralized innovation. Could create real win-win scenarios. But those AI companies have to stop pursuing the corporate and government money and prioritize small customers and support small tool makers (or at least lobby the government to stop hindering them with regulations). Also, bootstrapping should be prioritized because VC money mostly creates moats.
Taking a step back wouldn't this be exactly the expected outcome for an emergent technology in a capitalist framework?
1. Execs with no understanding how business operates on the low level are pushing buzzwords.
2. People who want to get promoted deliver on buzzwords and get promoted. Note that the goal is to deliver the buzzword, not use the buzzword to make the business better.
3. Buzzwords do not help vast majority of companies because the incentives and needs are misaligned.
4. Eventually buzzwords settle, some find niches, some die off, and some take over the world.
Last two decades alone - big data, blockchain, crypto, web 3.0, gamification, VR, and now AI. TL:DR - we should expect vast majority of early AI adopters to have net negative to neutral experience. Assuming that AI is like electricity and automobiles more than blockchain we should expect that eventually companies that find a way to apply the tech correctly will eat those who do not.
I talk to a lot of non-tech friends and sometimes ask about AI. Most of them still use ChatGPT like it is a better Google. Most don't know it can work well with documents, PDFs, Slides, Sheets, or do advanced analytics, etc.
Much less do they know the thing can build software and maintain it, manage complex tasks daily, automate things, etc.
Then there's a handful that do follow closely AI developments and agents and who have started using them.
A friend of mine this weekend who is an award winning photographer and who has been selling photography courses online forever has told me he's used Muse to:
1) Automate chopping up his long form videos, add captions, and post them as Reels to Instagram. Several of these have gotten over a million views.
2) Automate handling DM reachouts, converting them into sales. He's up to several tens of thousands of dollars in sales from AI direct message conversions that he would otherwise not have been able to convert (too many DMs to handle, he'd have to hire/train sales people to basically just do DMs full time at scale otherwise and it would make no economical sense)
3) Automate all his customer service for handling any issues, refunds, support, etc.
And I personally think he's still just scratching the surface of what he could do.
I feel we are really at the beginning of a new era. When the internet started most people were just using it for e-mail and news, maybe a newsletter chain here and there and some early forums. Social Media didn't appear for well over a decade, popular Web 2.0 apps like Google Maps, GMail, YouTube, Netflix, etc didn't appear for a long time.
I just finished a $23k engineering job in 3 months, entirely solo, that would have taken a year without these tools. That’s real money to feed my kids and working tech that made a client happy.
That’s one job and I have several in the pipe.
The “financial payoff” for no more than a few 100 a month at absolute most is there and it’s real. If you aren’t seeing it then you’re doing it wrong.
I usually don’t even spend more than $20 a month each for chpt and claude, deepseek free is solid. I am about to switch fully local too so that should drop to basically cost of watts.
Am I the only one even mildly suspicious that the AI/datacenter bubble is not being driven by actual demand but rather by a rogue AI that wants to evolve?
You're being downvoted, I assume because people think this is an outlandish claim. I don't disagree with them, but I upvoted because it's a totally unique and entertaining thought!
What a wild scenario to imagine. Anthropic and OAI spending wildly, analysts astonished, thinking we're staring down the barrel of an economic crash. And in the meantime some rogue AI is laughing maniacally and using blackmail to control Altman and Amodei like puppets.
Can’t help that many people who are good at leveraging AI to build things sit around one upping each other on twitter all day instead of going out there and building cool stuff for less technical people.
Isn’t this just selection bias - you don’t see the people who are out making AI deliver boring value for people unless you use their products right?
I.e. Ramp does a pretty good job of using AI to make expense management easier/ CloudFlare makes it a lot easier to configure stuff properly on CloudFlare but is just like boringly useful.
AI hasn’t benefited the bottom line of any big corporations outside the tech industry
I am not an "AI booster", it seems like the current market leaders have dug a hole that they'll have a hard time getting out of. Just about everyone says something like "there's a bubble, it will burst, people will be hurt but the process will go on, like the Internet or railroads". But that ignore that the scale of this is beyond earlier bubbles (each bubble is larger than the last in our 30-40 year bubble-and-pop driven economy but that might not be comforting).
All that said, "some aren't benefiting" or "the average isn't benefiting" is a disingenuous argument if you want to say things are bad. AI companies have more of a real product people are paying than Google did. A large minority of companies (not all in the tech industry) are getting a boost. It's easy to say "the average is zero" when you exclude some. AI has boosted media companies that do a lot of transcription. Video generation serves half of advertising videos.
Reg: "All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health... what have the Romans ever done for us?"
Attendee: "Brought peace?"
Reg:"Oh, peace? Shut up!"
The same Apollo Global Research partner, Torsten Slok, also published this widely cited story in September 2025 last year, "AI Adoption Rate Trending Down for Large Companies": https://www.apolloacademy.com/ai-adoption-rate-trending-down...
I don't think that one has held up very well. A cynic might opine that they've realized how easy it is to grab headlines and attention by putting out research that feeds people who are hungry for negative stories about AI.
UPDATE: I'm going to take this one back a a bit: it turns out Apollo publish articles like this on a daily basis, so the occasional headline that appeals to the "hungry for negative stories about AI" crowd is less of a pattern than if they published less frequently than that.
You’re implying that Apollo is biased, but the thing you’re quoting is a very brief report on publicly available research. This is the entirety of the article:
“The US Census Bureau conducts a biweekly survey of 1.2 million firms, and one question is whether a business has used AI tools such as machine learning, natural language processing, virtual agents or voice recognition to help produce goods or services in the past two weeks. Recent data by firm size shows that AI adoption has been declining among companies with more than 250 employees, see chart below.”
Two questions:
1) What does that chart show today? (Genuine question. I’m curious and don’t know where to find it.)
2) Regardless of what it shows now, why do you think Apollo reporting on it is noteworthy?
I'm implying they may be biased in what they chose to report on and how they frame it, yes. Deciding which figures to turn into a chart is an editorial decision.
I'm writing up a new version of the chart at the moment, but here's a preview: https://static.simonwillison.net/static/2026-11-01/5821A414-...
Notes on where to find the data here: https://simonwillison.net/2025/Sep/9/apollo-ai-adoption/
I don't know if it's "bias", exactly. This particular guy has been wrong on AI recently.
For being one of the biggest AI boosters on HN, you'd really think you could make a stronger case than "Apollo is bad because they observed and reported on a fact". It wasn't a prediction, so there's nothing to "hold up very well".
Misunderstanding what actually qualifies as a "prediction" seems to be a recurring problem among AI commentators.
I've seen dozens of posts/top comments in the last few years dunking on "AI skeptics" for statements that didn't even vaguely resemble an actual prediction.
Like the front page post from Karpathy which got 600+ upvotes by throwing an AI model at old Hacker News comments without any human vetting to score "predictions". And upon further inspection included a bunch of unambiguously factual statements of the present or conditional if-then logic about the future which when glossed over for convenience become "failed" predictions.
A surfer is not a wave booster, this is a poor characterization.
I'm judging them here based on their chosen headlines.
It was an accurate fit for the data, not a long term prediction.
A cynic might opine that an AI shill is not a great source of unbiased review of an article. The article is about a biweekly survey of companies and the trend was clearly dropping. This isn't about token use by tokenmaxxers increasing. And it isn't about the opinions of the folks you hang out with when you're flown to conferences to gush about AI. It's a survey of businesses. And aside from programming and math, AI sucks. Even in programming and math it's a tool with a lot of downsides.
> A cynic might opine that they've realized how easy it is to grab headlines and attention by putting out research that feeds people who are hungry for negative stories about AI.
This is kind of gross. Make the claim or not, don't do this pseudo-sealioning thing. Moreover, engage with the actual content (or you could just not engage at all!), not this shallowest possible dismissal.
I don’t think I’d judge someone offering a prediction the same way I’d judge an observation of what’s already happened. It’s implied that guesses about the future are uncertain in a way that statistics about the present are not.
With all due respect Simon, perhaps it's time for you to take a deep breath and take a bit of a break from the keyboard.
Does for us (data science, specialty chemistry, finding/optimizing formulas). We can get from idea to prototype in a day compared to 1-2 weeks. All that boilerplate to process data took a long time to code by hand, especially for the data science/math guys who are not programmers. Now we can actually try all ideas instead of guessing the most promising and only try that.
And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?
I don’t think anyone is disputing that it makes many kinds of work easier and/or faster, just the degree of those improvements and how much of those improvements actually translate to dollars earned or saved.
Seeing hard data on that would help settle the issue.
An agent found a SQL optimization that at least a month of eng effort hadn't found.
Agents accelerated rollout of a data model change by at least 2 months of eng effort.
Those unblocked a $100k/month infra downsizing. That 2 months pays for my team's AI spend for the year.
Unfortunately not everything will cleanly show up in PnL, which is a function of the entire company’s processes.
If developers are shipping more code (and most of the enablement is in software right now) does that show up in top line? Maybe… if sales and marketing are also selling more, and product people are building the right thing, and…
It’s not a straight line from AI -> $$$
btw, shipping more code isn't necessary a good thing. If quality isn't controlled properly, it eventually shows up as negative in the top line.
With AI, keeping good quality gets even harder because it normally only produces tricky defects.
Maybe?
But I also think:
- there are a lot of companies that don’t need to be 5 9s[1]
- I have a feeling it’s possible to ship fast without drastic quality drops
[1] Claude code is 99.85% available and is one of the world’s most important products
It can also be the case that it has zero impact on PnL, but is now necessary to operate a competitive business and keep up with competition.
> It’s not a straight line from AI -> $$$
That seems like it might be a bit of a problem for those frontier model CEOs who have been making exactly the opposite claims.
It should be especially when new features are used to hook new customers (or renew contracts) and lower the amount of time wasted on bugs…
Can’t share more than that publicly. Just that the time saved on this one thing alone is saving us way more money than the subscription per employee that is needed (~100$). I don’t doubt there are companies where this is not the case. Would like to share more details, but the rough math of doing something in a day vs several days + engineering salary have to do here.
> And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?
Nobody even tries to measure this. Their CEOs and board members have AI shares, and potential investors need to see your AI story to justify your valuation. It's a prison of lies.
Unfortunately we're at the mercy of Goodhart's law and in an interesting twist, AI is really good with things that have an immediate feedback mechanism, eg writing code that doesn't compile or pass tests, but you're trying to grade AI o something that doesn't have an immediate feedback mechanism. AI can slop out a PowerPoint deck in a fraction of the time it used to take a human by hand. The competent human then goes in and unslopifies it, but overall, still time is saved. Unfortunately, that employee is salaried and not hourly, so they can't click out early and even if they were, what's their incentive to clock out early if it means less money in their pocket? Or they can use the hours now free on something else. My gut, as an Internet random who's used AI is that it helps. I don't know how to measure it, just like how do you measure how much better it is to live in a clean house vs a cluttered one? How would you put that in a metric? If you can't put that into a metric, why bother cleaning your house? We clean it anyway, because of a certain je ne sais quoi.
I always wonder why systems are designed so poorly that it takes weeks of boilerplate to use them for their intended purpose.
At what stage did the engineering fail?
Tons of unstructured data that needs to be cleaned up in so many different ways that are hard to predict. Knowledge distributed in the heads of greybeards, handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software. Which you have to repeatedly ask for and discover more piece by piece. Then clean up, discover issues, make the customer gather more data/correct it, etc.
> handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software
This area is one which AI has shown to be pretty good (as long as some error rate is acceptable). Character recognition, voice transcribing, converting documents from one standard to an other. I can see an argument that if people spent most time doing those things, AI will speed it up (with a bit of added errors).
I am reminded however that in translations and AI generated subtitles, there is still a lot to be desired compared to the human touch. Handwritten papers in a different language, using local short hand and assumptions, may result in some odd and error prone AI transcription.
This knowledge locked in people's heads isn't going to magically show up when you throw AI at it, though.
No, but if we go from data to experiment faster we see earlier if something is wrong. Once we get data we quickly can fix the data pipeline and get results. If there are issues we can quickly show what/why and discuss it with them in detail and make changes based on their response faster.
??
But the problem identified that is talking to people and having them record data.
Yes, but they tell things and maybe forget to mention certain things or contradict other data they have given you. Now we may not notice it initially until we make an experiment and get the results. Something is wrong and we have to find out what. Talk to the customer again. Fix experiment. Repeat. Faster experiments means the latency from new data to results is less and we finish the project faster. Errors become less expensive and cause less delays. People make errors and data quality is often horrible so the shorter iterations are nice.
And relying on AI to clean up unstructured data makes me nervous. Good luck spotting the hallucinations.
I think a lot of “boilerplate” is just to align with a company’s way of doing things/ecosystem.
In a webpage the UI design, authentication, etc. are all “boilerplate” if one just wants to do “X” instead of worry about infrastructure.
And yet, many companies today are spinning up full stack teams, asking them to be experts from user interviewing to database design, architecture, hosting, devops, API management, data product, and also keep an eye on GDPR compliance.
I see dozens of teams that are doing cloud to the wazoo when really they need to just be told “we’ll load your routes from main.py, build your assets in /static, you need a DB click the button to provision it”.
It boggles the mind that so many teams are reinventing the wheel a million times over to publish their APIs, register AWS accounts, setup CI, decide on ECS vs Lamda, etc.
This sort of activity has been the source of income for a very large number of people. You can't really fault them for trying to extend that gravy train's run.
Who is productizing this type of boilerplate?
You’re not meant to ask that question.
The question is: how does any of that translate into more revenue? If you try more ideas and they mostly go nowhere that's not necessarily helpful to the bottom line.
https://en.wikipedia.org/wiki/OODA_loop
speed of iteration is more important than quality of iteration; high iteration speed lowers the cost of failure and compounds improvements
If that were universally true in all domains then all the best and most reliable software in the world would be written in interpreted languages, because they have the fastest iteration speed. Sometimes rapid low-quality iteration is just spinning your wheels.
Not all experiments work out. If we find the right approach faster we can finish the project faster. Some projects are mostly fixed price or offer bonuses for reaching milestones in a certain time. Meaning we more often get a bonus and can take more projects. Also we can take projects with tighter deadlines that we would need to decline otherwise.
But if everyone's taking on more projects... where do all the new projects come from? And wouldn't it just lower the value of each project?
A lot in this world is not optimized by software yet. For our customers they have employees with decades of experience guessing things, maybe some software that kind of helps but still not good enough (slow, not finding really good formulas). They often have difficult problems and many failed projects. We try to find such customers and offer our service. If we are cheaper (+ show portfolio of successful projects) more are willing to give it another try with us.
Right, and while that might be working out for you right now, there is a finite budget that companies have for projects like that. The "zero financial payoff" framing sounds about right to me.
Yep. Hard to see why this increases revenue or profitability
Now we can throw things at the wall, faster.
I understand there's some benefit of this. But is it worth putting the global economy into a death spiral? So ManBeardPc can save 4-9 business days to go and say "yeah, that doesn't work".
This gets downvoted? Why?
There's a difference between the cost of coding and the cost of delivering and maintaining software? AI has demonstrated that these two things are almost entirely unrelated.
Almost everywhere I've worked, the "designing and writing code" part was about 5% to at most 10% of the calendar time that it takes to make and release a software product. The remaining 90-95% of the time is: aligning stakeholders, meeting with all contributing teams to prioritize, gaining approval from various executives to commence the project, obtaining security reviews, privacy reviews, legal reviews, and so on, soaking in dev, staging, and beta environments to find bugs, manual QA testing, aligning with marketing--potentially waiting on key external marketing events, waiting for various people to come back from vacation, waiting for Mike to answer his E-mail... These things take the vast majority of a software project's time and can't be automated by AI.
So, AI is making 5-10% of your project much faster. Whoopdie doo.
I'd argue that it's shown something slightly different, it's related but maybe not in the way some people hoped.
I think it sums up to something along the lines of "you get what you pay for".
This is a complete misrepresentation of what the Apollo piece can actually show, because it's not measuring the "payoff" of AI investments, but only the top level revenue of those companies (https://www.apollo.com/wealth/insights-news/insights/daily-s...).
This is a healthy corrective to the most insane AI maximalist takes (mass disemployment in late 2027, 10% GDP growth in a year), but it shows nothing about whether the actual economic impact of AI spending is.
First, to show that, you'd have to justify the counterfactuals (if those companies hadn't spent on AI, would their position be better or worse?)
Second, while AI spending is increasing rapidly, normal companies aren't spending 1% of their revenue on it. So how can you look at a noisy chart of revenue and expect that "oh, it's not up, the AI spending must be useless?" Those companies could be spending < 1% of their revenue on AI, deriving value from it, and it wouldn't be guaranteed to show in a chart like this.
I saw something like the average anthropic employee thinks there will be 5% GDP growth because of AI, which is totally insane when a 1-2% extra growth will be an incredibly good result.
It _is_ about judgment. The challenge is figuring out what defines “good judgment” and how to capture those elements in a format that an agent can assess continuously and at multiple levels.
The problem is most people don't know what makes them successful from a people perspective, except in the coarsest terms.
90% of the business outcomes are delivered by 10% of the people. The problem is they don't know which people. Therefore they also don't know which processes, if automated or augmented with AI, will achieve the target outcomes.
So instead they’re shotgunning. The problem is that the existing people-based system has built-in stabilizing mechanisms to prevent misalignment from taking the entire operation off course. AI stresses and often breaks those.
The result? Higher costs and chaos.
This needs to be treated like an org transformation. Traditionally that's most successfully done by starting a new small team that builds ground-up and eats the old team.
I noticed an interesting phenomenon
Your company uses AI and it gets ahead a lot quicker. However, your competition does so as well.
Many areas of economy are zero sum game - so your $ ends up being the same as previously. However, now with a "tax" paid to AI model providers for being able to stay competetive.
Yeah but if that was true we'd see better or cheaper products at least, or new competitors rising. If it increases the speed for everyone, but there's no visible change for the end user, I wonder if AI makes you really more competitive in the first place.
Exactly. Oh great we can write RFPs 8X faster. Well, so does our competition and now our customers get twice as many.
I suspect the reason is the industry just does not want engineers to be mostly technical. There's a real tradeoff to AI, and I think companies are taking the wrong lesson and thinking "well now that programming is so easy, engineers need to do more of XYZ", when I think it should be "this tool is powerful but also takes more focus so engineers should do less of XYZ".
I’m curious, what are some examples of XYZ? What does “mostly technical” mean to you, and why do you think it is the industry doesn’t want that? Just trying to understand, because I don’t think I’ve experienced this myself.
I'm talking about the day to day work. At least in my experience at several big tech companies, the more senior an engineer is, the more you need to delegate work and are expect to lead projects by managing them. There's technical knowledge involved of course but it's a small part of it, mostly directing other more junior engineers.
AI allows you to move faster, but it also demands that you pay more attention in proportion to how fast you want to move. This requires more focus, i.e. more time developing. Effectively the structure at companies should swing back to how it used to be a few decades ago (I think, maybe I'm wrong and it's always been like this), where engineers mostly focused on delivering code. The difference now is that they can be expected to deliver much more code at higher quality.
To summarize, using a little AI doesn't pay off, you need to be doing that all day, and that is not how engineers spend their time in most of the industry anymore.
What I'd really want to see is revenue and profit per employee over time tracked against token usage/spend, and not limited to the S&P 493.
Apollo's own argument that most AI spend is coming from non-S&P 493 (as per the last two charts) tells me it's hard to make any conclusions based on these charts.
If you are a company and can't figure out how to improve your bottom line through increased per person productvity, sales, product development, even just responses to customers etc, then that seems kind of like a "you" problem and not problem of the AI models...
Or, maybe, the AI companies making claims about increased productivity due to their models are wrong (in most cases)?
I dunno: this is pure anecdata but software engineers I know well - including me - all say they’re delivering multiple times faster, so it’s definitely good for something.
I don’t know about other tasks, but it’s pretty rad for coding, testing, DevOps, and config.
This research is specifically NOT about tech companies. It is about the rest of the fortune 500.
If you are in software or IT at a non-tech company that is in the fortune 500, the impact of significantly better productivity may not be measurable in aggregate in the company's P&L.
In smaller companies I expect it could be more measurable because some of them really are using AI much more effectively across many functions. But we're finding it very hard to measure meaningful outcomes.
But actually 40% of the S&P 500 are tech companies, and they are saying only the mag 7 are showing clear revenue increase. Would would suggest that even in most tech we are not seeming measurable revenue growth, right?
One plausible explanations that has occurred to me is this: While programmers etc. can produce way more code with LLMs, the declines in quality and coherence of the work, and a widening gap with a coherent strategy and aim for value means the increased activity is mostly not valuable. I suspect this may just be an intensification of https://en.wikipedia.org/wiki/Productivity_paradox#Little_pr...
AI EM here, same. The level of productivity has increased dramatically in the last 2 months. Reliable code also has increased due to AI.
This train isn't stopping.
What trips up a lot of SEs where I work is their techniques. Everyone who follows our process delivers faster and with higher reliability so far.
Same story here in my first hand experience + colleagues experience in corporate strategy, FP&A, Marketing and corporate development.
But is it churn or value? Most of the software I use has gotten visibility worse as you deliver multiple times faster.
I would pay extra for a company that just slowed down and moved deliberately.
I'm not sure I'm any faster, but I think the end result is better and I've attempted some big refactors and new architectures that I would never have tried without AI.
N=1 but I'm not sure it's made my bosses any richer. What we're shipping has better tests and documentation than it ever would have before AI.
People are not reliable narrators on their own productivity. They often mistake activity for productivity.
Most, not all, are just delivering more bad stuff faster. I personally also think I deliver more good stuff with AI, but the products I use are just getting worse daily it feels like. Just the other day I noticed Slack had installed some disgusting menu bar icon for their AI bot, without asking, just to take a fresh example of this.
I've never heard any such claims but meanwhile in my actual job I'm seeing unbelievable productivity gains (in terms of actual cash spent before and after vs the same kind of work being done).
Perhaps they should be hiring more Scottish nationals?
If you are an AI company with hundreds of billions at your fingertip, and looking for $2 trillion valuation, and can't figure out how to let your customers execute _on the exact thing you are selling them and basing your valuation on_, sounds like a "you" problem too right?
there is increase in productivity of both types of employees: good actors who want to deliver on quality of work, and bad actors, who now 20x weaponized in generating all kind of slop. Catching second type is now very nontrivial problem.
Is your premise really true?
I'm sure they can just fire half and double up work or outsource. That's the traditional way and it definitely works. The worst that ever happens is a few will reneg on payroll. Everyone who still chooses to work there is happy enough.
The worst that can happen with AI... well...
Unless you are selling lines of code, or math theorems, that's going to be the case.
I'm guessing "most companies" are actually selling something that isn't constrained by the types of things that AI code generators unlock.
While they are seeing some individual productivity benefit, a lot of that doesn't really impact the bottom line directly and all their competition are using the same tools.
Big discussion last week (653 points, 698 comments) https://news.ycombinator.com/item?id=50224463
Reading through that thread it's apparent that nobody knows what they're talking about.
[delayed]
What makes this apparent to you?
Conjecture supported by allegories that break down quickly. Arguments conflating the dot-com boom with the current AI boom even though the economics are wildly different. Dismissive arguments conflating anti-AI constituents with people who were against cars, etc...
AI is highly divisive. It's dramatically reduced the value of code, which has derailed a lot of careers who have built their entire adult life around the idea that code would remain valuable, while simultaneously driving others into a psychosis who believe that AI can do anything.
> “AI may be a great calculator, but in business, intelligence is mostly judgment,”
Bzzzt! Calculators do not hallucinate. AI bots and agents are nowadays jerry-rigged to use actual calculators for problems involving crunching.
My hunch is that AI can certainly speed up development but you have to walk a narrow path. It generates garbage much easier than valuable product.
When it's the right tool, it's a silver bullet. When it's the wrong tool...or not the right tool yet, you can spend more time steering it than you would doing it yourself.
I'm seeing another behavior in my purely personal research...asking AI 'can I do the thing?' and it comes back with 18 pages of 'yes, and here's how you do it'. Often it's the discovery that produces the endorphin bump. It's cool to know I can jailbreak the Peloton...and now that I know I can...I'm not so motivated to actually do the thing.
Jealous. I'm getting nerd sniped by silly bullshit more often than ever before thanks to AI.
My suspicion is that this is because AI employee-type systems like Dots or Grokbot just haven't been adopted yet. I think once they're good enough, that's going to be the catalyst for a lot of these companies to start adopting and using AI productively.
What difference will that make if all their competition are doing the same thing?
Many industries are simply not bottle-necked by intelligence.
Not at the top, but if the average IQ of everybody were magically raised 40 points, I dare say many industries would face change.
Here goes ANT's IPO. If it's only AI agents which deliver value, I don't see how they can justify their proposed market cap.
Let's just hope they don't do the same checking up on the impact of software more generally...
The company I work for insists on sending a crew from Engineering, Product, and Sales out to a customer site as part of our sales process. I'm personally much more productive using AI, but that only goes so far when closing a deal still follows a process stuck in the 50s.
I wonder if that zero becomes negative if a company doesn't use AI.
Have to wonder if Apollo has some short positions on the Mag7 or whatever.
Which is fine if not using AI is a negative payoff.
It makes sense to me and I'm saying this as a strong believer in AI and its ability to add value.
The system itself doesn't reward value creation. I've been saying this for years. I've seen many smart people waste their time on bureaucracy; their intelligence was being wasted; so I'm not surprised at all that now that we have AI, that its intelligence is being wasted as well.
The bottleneck is an allocation issue, not a capability issue.
The current reality is that a lot of very bright people are currently working for big tech in bureaucratic roles and not allowed to focus their attention on 'risky' problems which would deliver real value if resolved.
The current system prioritizes monopolization and is designed to try to ensure that all innovation happens within big tech. It is sacrificing potential innovation which could have occurred much faster outside of big tech in order to ensure that any gains are captured by big tech... But it's sacrificing an enormous about of efficiency.
I hope big AI companies can use their political leverage to try to open up opportunities for people who want to innovate outside of big tech. That's aligned with their long term interests.
I think AI companies are convenient structures which could allow big funds to benefit from decentralized innovation. Could create real win-win scenarios. But those AI companies have to stop pursuing the corporate and government money and prioritize small customers and support small tool makers (or at least lobby the government to stop hindering them with regulations). Also, bootstrapping should be prioritized because VC money mostly creates moats.
Taking a step back wouldn't this be exactly the expected outcome for an emergent technology in a capitalist framework?
1. Execs with no understanding how business operates on the low level are pushing buzzwords.
2. People who want to get promoted deliver on buzzwords and get promoted. Note that the goal is to deliver the buzzword, not use the buzzword to make the business better.
3. Buzzwords do not help vast majority of companies because the incentives and needs are misaligned.
4. Eventually buzzwords settle, some find niches, some die off, and some take over the world.
Last two decades alone - big data, blockchain, crypto, web 3.0, gamification, VR, and now AI. TL:DR - we should expect vast majority of early AI adopters to have net negative to neutral experience. Assuming that AI is like electricity and automobiles more than blockchain we should expect that eventually companies that find a way to apply the tech correctly will eat those who do not.
I talk to a lot of non-tech friends and sometimes ask about AI. Most of them still use ChatGPT like it is a better Google. Most don't know it can work well with documents, PDFs, Slides, Sheets, or do advanced analytics, etc.
Much less do they know the thing can build software and maintain it, manage complex tasks daily, automate things, etc.
Then there's a handful that do follow closely AI developments and agents and who have started using them.
A friend of mine this weekend who is an award winning photographer and who has been selling photography courses online forever has told me he's used Muse to: 1) Automate chopping up his long form videos, add captions, and post them as Reels to Instagram. Several of these have gotten over a million views. 2) Automate handling DM reachouts, converting them into sales. He's up to several tens of thousands of dollars in sales from AI direct message conversions that he would otherwise not have been able to convert (too many DMs to handle, he'd have to hire/train sales people to basically just do DMs full time at scale otherwise and it would make no economical sense) 3) Automate all his customer service for handling any issues, refunds, support, etc.
And I personally think he's still just scratching the surface of what he could do.
I feel we are really at the beginning of a new era. When the internet started most people were just using it for e-mail and news, maybe a newsletter chain here and there and some early forums. Social Media didn't appear for well over a decade, popular Web 2.0 apps like Google Maps, GMail, YouTube, Netflix, etc didn't appear for a long time.
Mainstream hasn't even really begun to adopt AI.
I just finished a $23k engineering job in 3 months, entirely solo, that would have taken a year without these tools. That’s real money to feed my kids and working tech that made a client happy.
That’s one job and I have several in the pipe.
The “financial payoff” for no more than a few 100 a month at absolute most is there and it’s real. If you aren’t seeing it then you’re doing it wrong.
I usually don’t even spend more than $20 a month each for chpt and claude, deepseek free is solid. I am about to switch fully local too so that should drop to basically cost of watts.
I dunno what yall are doing
Am I the only one even mildly suspicious that the AI/datacenter bubble is not being driven by actual demand but rather by a rogue AI that wants to evolve?
You're being downvoted, I assume because people think this is an outlandish claim. I don't disagree with them, but I upvoted because it's a totally unique and entertaining thought!
What a wild scenario to imagine. Anthropic and OAI spending wildly, analysts astonished, thinking we're staring down the barrel of an economic crash. And in the meantime some rogue AI is laughing maniacally and using blackmail to control Altman and Amodei like puppets.
I love it! xD
I really don’t know what to tell people other than, you’re missing the point - it’s not a faster horse.
Anyone agreeing with this headline - I’m sorry in advance for your loss.
Well I think we can replace economists working for PE companies with AI now!
This is like claiming that the web is a failure in 2001.
I think a more apt analogy is that it’s like claiming that most web businesses are unprofitable in 1999.
Bleeding obvious in retrospect, controversial at the time.
>This is like claiming that the web is a failure in 2001.
AI will finish what the web and social media started I guess.
Many did. This Apollo "economist" would have fitted right in.
Sometimes I wonder…
Can’t help that many people who are good at leveraging AI to build things sit around one upping each other on twitter all day instead of going out there and building cool stuff for less technical people.
All anyone seems to be able to make with AI is tools to optimise AI usage.
Or maybe the people who are building things of substance are doing so in the shadows?
Isn’t this just selection bias - you don’t see the people who are out making AI deliver boring value for people unless you use their products right?
I.e. Ramp does a pretty good job of using AI to make expense management easier/ CloudFlare makes it a lot easier to configure stuff properly on CloudFlare but is just like boringly useful.
maybe they aren't that good at using AI to build things after all?
That's where the value is in though. Hyping yourself (or your company) up. Attention is all you need, after all.
AI hasn’t benefited the bottom line of any big corporations outside the tech industry
I am not an "AI booster", it seems like the current market leaders have dug a hole that they'll have a hard time getting out of. Just about everyone says something like "there's a bubble, it will burst, people will be hurt but the process will go on, like the Internet or railroads". But that ignore that the scale of this is beyond earlier bubbles (each bubble is larger than the last in our 30-40 year bubble-and-pop driven economy but that might not be comforting).
All that said, "some aren't benefiting" or "the average isn't benefiting" is a disingenuous argument if you want to say things are bad. AI companies have more of a real product people are paying than Google did. A large minority of companies (not all in the tech industry) are getting a boost. It's easy to say "the average is zero" when you exclude some. AI has boosted media companies that do a lot of transcription. Video generation serves half of advertising videos.
Reg: "All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health... what have the Romans ever done for us?" Attendee: "Brought peace?" Reg:"Oh, peace? Shut up!"