We’re lost, but we’re making good time.
Yogi Berra
At an AI workshop that I recently attended, the moderator polled the audience asking what the primary goal around AI was for each of us at our respective organizations. The number one answer was embedding AI into as many systems and processes as possible. Delivering business value came in second. Think about that for a moment. In a room full of technology leaders, most of them believed that getting more AI into the organization was more important than delivering business value. This illustrates a key misjudgment that many of us make as humans. We often focus on how fast we’re going without making sure that we’re headed in the right direction.
This is exactly what is happening with AI right now. AI giants are talking about all of the capacity that they are building. The models are getting faster. The quality of the responses is getting better. Technology leaders keep talking about how if you’re not deploying agents or building with AI you’re running behind. But where are we going? What’s the destination? We’re lost, but we’re making good time.
Last summer, a preliminary study out of MIT’s Media Lab found that about 95% of enterprise AI pilots showed no measurable impact on the P&L. When it was first released, many used this as an example of how AI wasn’t working. I’ll be honest, I was one of them. When all the AI hype started really ramping up a couple of years back, I was a curmudgeon. I was skeptical of people’s predictions about where it would go, how it would take everyone’s jobs, or even end up creating the robot apocalypse. On the surface, this study seemed to support my skepticism. But if you look deeper, that’s not what it’s saying. The failures weren’t about the quality of the AI models, they were about how organizations couldn’t connect the technology to how the work actually gets done.
After the study’s release, people started to poke holes in this number. It was a small sample size. The ROI window of six months was too short. The study wasn’t peer reviewed. But when one reporter went looking to see if companies were scaling back their AI spending in response to the report, he couldn’t find any willing to say that they had.
The problem with AI is the same problem that I’ve seen with technology over the last 25 years. People see it as a solution to a problem that they haven’t defined. I wrote about this back in 2021 in my article, “Are You Ready to Implement an ERP?.” The article was based on my personal experiences with implementing an Enterprise Resource Planning (ERP) system for Goodwill of Central and Northern Arizona. I was charged with finding a replacement HR, Finance, and Payroll platform to consolidate over a dozen legacy systems. While I was researching options, I read article after article about ERP or other large software implementation failures. Projects that cost millions of dollars that ended up either going over budget, getting scrapped after years of work, or both. And the common denominator for almost all of these failures was a lack of clarity at the beginning of what the software implementation was supposed to achieve. What pain points needed to be addressed to enable the organization to continue to grow? What was the return on investment for moving to a new system? How was this project going to help us to achieve our long-term objectives? These were the questions that we needed to answer before we even started looking at systems.
It’s too soon to know which AI companies will come out on top or whether the technology will ultimately prove too expensive to provide the kind of value being promised by the likes of Anthropic, OpenAI, and others. I have no doubt that this technology will survive in some form once the hype has settled down, and it will continue to change the way that we as humans work in the future. But what hasn’t changed, and won’t ever change, is the need to get clear about what you’re trying to accomplish before getting started.
Before continuing to spend on AI, ask yourself, what are the pain points you are trying to solve? What return are you getting on your investment? How does the AI initiative you’re leading tie to the organization’s long-term objectives?
Your AI implementation is an ERP implementation in disguise.
AI isn’t your problem. Clarity is.
