Friday 15 Podcast

Measuring the ROI of AI: A Framework for B2B Leaders

Brian Beck, Andy Hoar, and guest Howard Blumenthal on why most AI projects fail to deliver returns, the four questions to ask before starting any AI initiative, and the hot categories where B2B companies are seeing measurable wins.

Friday 15 Podcast · Guest: Howard Blumenthal, eCommerce advisor and former digital leader at Motion Industries and Advanced Auto Parts

Key takeaways

  • According to sources cited in the episode, AI projects are two times more likely to fail than traditional IT projects, with an estimated 80% failure rate and only 5% of enterprise AI pilots delivering measurable impact.
  • Gartner reportedly predicts that 60% of AI projects fail because data is not ready, and that 40% of agentic AI projects will be cancelled by 2027 due to escalating costs and unclear returns.
  • Howard Blumenthal recommends asking four questions before any AI project: Are we targeting the right workflows (high volume, repetitive tasks)? Are we measuring outcomes rather than outputs? Is the data clean for the specific use case? Do we have a governance plan that balances top-down structure with bottom-up experimentation?
  • The four categories where B2B companies are seeing AI success are search (semantic and vector search for product discovery), personalization (focused on categories and reorders rather than behavioral signals), pricing (with reported 2% to 5% margin improvements), and operations (forecasting, demand planning, and order exceptions).
  • When polled, 56% of the Master B2B community said both experimentation and ROI are important for AI investments, but among those who chose one, experimentation was three times more popular than narrow ROI, suggesting the market is still in an exploratory phase.

Amazon passes Walmart as the largest US company by revenue

The hosts opened with news that Amazon’s 2025 revenue reached $716 billion, surpassing Walmart to become the largest company in the United States by revenue. Amazon also announced $200 billion in AI investments, but the stock declined following the announcement. Brian attributed this to short-term investor skepticism about AI spending, while Andy recalled predicting in 1998 that Walmart should acquire Amazon.

Genuine Parts separating Motion and Napa

The breaking news segment also covered Genuine Parts’ decision to separate Motion Industries and Napa Auto Parts into two independent public companies, with combined revenue of $24 billion. Andy attributed the move to activist investors who believe the whole is worth less than the sum of its parts. He noted that software and AI have reduced the operational efficiency gains that once justified conglomerate structures, shifting the calculus toward speed over size when companies lack channel synergies.

The brutal reality of AI project failure rates

Brian presented statistics on AI project performance. According to sources cited in the episode, AI projects are two times more likely to fail than traditional IT projects, with an estimated 80% failure rate. Only 5% of enterprise AI pilots reportedly deliver measurable impact, and only about one in ten proofs of concept reach full-scale production.

The reasons cited included data unreadiness (Gartner reportedly estimates 60% of projects fail because data is not ready), hallucinations and trust issues (51% of organizations have reportedly stalled deployments), and the hype trap (30% of generative AI projects are reportedly abandoned post proof-of-concept due to escalating costs). Gartner reportedly predicts that 40% of agentic AI projects will be cancelled by 2027.

E-commerce uncovers all the sins of a company. AI does that on steroids.

Howard Blumenthal, eCommerce advisor

The steam engine analogy: why process change matters

Andy drew a parallel to the industrial revolution. When companies replaced steam engines with electric motors, productivity did not improve for 30 years because factories continued to operate the same way. Only when companies redesigned their processes around electric power did they unlock the assembly line and transform manufacturing.

The same dynamic applies to AI. Companies that try to apply AI to existing workflows often find it does not help because the process was designed for human execution. The real gains come from rethinking processes around AI capabilities, but that requires change management, which remains the hardest part of B2B transformation.

Four questions to ask before starting any AI project

Howard Blumenthal joined the conversation as guest and presented a framework for evaluating AI initiatives. He recommended asking four questions before any project:

First, are you targeting the right workflows? AI works best on high-volume, repetitive tasks. If the workflow is low volume or highly variable, AI may not be the right tool.

Second, are you measuring the right things? Many companies measure outputs (emails produced, graphics generated) rather than outcomes (revenue impact, cost savings, customer satisfaction). Establish outcome baselines before starting.

Third, is the data ready for this specific use case? Trying to clean all data is a trap. Focus on the data required for the specific workflow you are targeting.

Fourth, do you have a governance plan? Howard recommended a hybrid model with top-down structure (core rules, frameworks, and guidelines) combined with bottom-up flexibility (allowing departments to experiment within those boundaries).

The AI scale traps

Howard identified several traps that prevent AI projects from scaling beyond pilots:

First, can the tool handle exceptions? B2B operates in a world of exceptions, and AI tools that work on clean cases often fail when edge cases appear. Understanding how the system handles exceptions (or escalates to humans) is critical.

Second, are you sharing learning across the organization? Many companies have multiple departments running similar tests with different tools and data, with no mechanism for identifying what works.

Third, do you have a tool review plan? AI systems experience drift over time as data becomes stale or models diverge from their intended purpose. Like mobile apps, AI tools require ongoing monitoring and refinement.

Fourth, are you avoiding AI creep? This is when employees use AI to expand into tasks outside their expertise. Content writers try to do graphic design; graphic designers try to write code. MIT research cited in the episode suggests the time spent on these tangential tasks can consume the efficiency gains from core applications.

With great power comes great responsibility. That gets to the governance, the rules, following these frameworks.

Howard Blumenthal, eCommerce advisor

The four hot categories for AI success in B2B

Howard identified four areas where B2B companies are seeing measurable AI results:

Search: Semantic and vector search can improve product discovery by pulling from databases, PDFs, and content pages simultaneously, rather than relying on keyword matching alone.

Personalization: In B2B, personalization should focus on categories, reorders, cross-sells, and quotes rather than the behavioral signals that dominate B2C. The use case is different, and companies that apply consumer personalization logic often miss the mark.

Pricing: AI-enhanced pricing tools are reportedly delivering 2% to 5% margin improvements by layering intelligence on top of existing business intelligence systems.

Operations: Supply chain tasks like forecasting, demand planning, and order exception handling are ideal for AI because they are high-frequency and repetitive, which is where AI performs best.

Experimentation versus ROI: what the community thinks

The hosts polled their community on whether experimentation or ROI is more important for AI investments in the next 6 to 12 months. The results: 56% said both are important. Among those who chose one, experimentation was three times more popular (33%) than narrow ROI (11%).

Howard agreed with the both answer. Companies need to experiment to understand how AI changes their processes and organizational structures, but they cannot experiment indefinitely. The key is timing: demanding ROI too early stunts learning, but failing to circle back to measurable outcomes eventually wastes resources.

The parallels to early eCommerce are striking. Initial eCommerce investments often could not show immediate ROI because infrastructure had to be built first. The companies that succeeded were those that balanced exploration with discipline, learning what was possible before optimizing for returns.

Frequently asked questions

Why do most AI projects fail?

According to sources cited in the episode, most AI projects fail for several reasons: data unreadiness (Gartner reportedly estimates 60% fail because data is not AI-ready), hallucinations and inaccurate outputs (51% of organizations have reportedly stalled deployments), and the hype trap (30% of generative AI projects are abandoned post proof-of-concept due to escalating costs). The hosts also emphasized that change management is a major barrier, as companies often try to apply AI to existing processes rather than rethinking how work is done.

What are the four questions to ask before starting an AI project?

Howard Blumenthal recommends asking: First, are we targeting the right workflows? AI works best on high-volume, repetitive tasks. Second, are we measuring the right things? Focus on outcomes rather than outputs like emails produced or graphics created. Third, is our data ready? Clean the data for the specific use case rather than trying to clean everything at once. Fourth, do we have a governance plan? The most successful approach combines top-down structure with bottom-up flexibility for experimentation.

What is AI drift and AI creep?

AI drift refers to AI systems that degrade over time as data becomes stale or the model diverges from its intended purpose. Companies need monitoring systems to detect when AI outputs are no longer accurate. AI creep is when employees use AI to take on tasks outside their expertise, such as content writers trying to do graphic design or graphic designers trying to write code. MIT research cited in the episode suggests that time spent on these tangential tasks can consume the efficiency gains from core AI applications.

Where are B2B companies seeing AI success?

According to Howard Blumenthal, the four hot categories for AI success in B2B are: search (using semantic and vector search to improve product discovery across databases, PDFs, and content pages), personalization (focused on categories, reorders, cross-sells, and quotes rather than behavioral signals), pricing (with companies reporting 2% to 5% margin improvements from AI-enhanced pricing tools), and operations (forecasting, demand planning, and order exception handling, which benefit from high frequency and repetition).

How does AI adoption compare to eCommerce adoption?

The hosts and guest drew parallels between AI and early eCommerce adoption. Both required experimentation before ROI could be measured. Both started with companies trying to replicate existing processes before discovering that the real value came from rethinking how work was done. And both faced skeptics who demanded immediate returns while others advocated for exploration. Howard Blumenthal compared it to the steam engine to electric motor transition, which took 30 years before companies realized they needed to redesign factories, not just swap power sources.

Should B2B companies focus on AI experimentation or ROI?

When polled, 56% of the Master B2B community said both are important. Among those who chose one priority, 33% said experimentation and 11% said ROI. Howard Blumenthal agreed with the both answer, emphasizing that companies need to experiment to understand how AI changes their processes and structure, but they cannot experiment indefinitely. The timing matters: demanding ROI too early stunts learning, but failing to circle back to measurable outcomes eventually wastes resources.

Sources & methodology

  1. Rand Corporation, on AI project failure rates, cited in the episode
  2. Gartner, on data readiness and agentic AI project cancellations, cited in the episode
  3. MIT research on AI creep and productivity, cited in the episode
  4. Amazon earnings announcement, February 2026
  5. MDM reporting on Genuine Parts separation, February 2026
  6. Friday 15 Podcast, Master B2B
Andy Hoar Andy Hoar
Co-Founder, Master B2B

Andy is a Co-Founder of Master B2B, founder of Paradigm B2B and author of the book Bot2Bot: The New Future of B2B Commerce. Andy is one of the leading global authorities on B2B commerce strategy.

Brian Beck Brian Beck
Co-Founder, Master B2B

Brian is a co-founder of Master B2B, Managing Partner of Amazon agency Enceiba, and author of the book "Billion Dollar B2B Ecommerce." Brian has also been C-level digital commerce executive with two decades of experience.

New: B2B Exchange at Shoptalk Presented by Master B2B

X