How to Fix Your Reporting When CRO Results Decay Over Time

You got a 31% lift from that A/B test? Cool.Now ask yourself: “will that number still hold in 6 months?"

If you’re in marketing or growth, you’ve probably seen this happen:

- You run a successful experiment- You see a conversion lift, let’s say 25-40%- You slap that % on top of your average revenue and forecast the annual impact- Victory slides in the QBR

...except there's one major problem:That lift probably won’t last.

In this episode of the Revenue Funnel Optimization Podcast, Arun Sivashankaran and David Janczyn dig into one of the biggest lies in CRO:

- Most ROI projections are inflated.- Linear extrapolations of test wins are misleading.- And the majority of uplift comes from low-intent visitors who are least likely to stick.

Let’s break it down.

The Problem with Linear ROI Projections

The way most CRO programs calculate ROI is dead simple: "We got a 31% lift from this test. Let’s apply that across all future traffic for the next 12 months and show a $$$ win."

This sounds great on paper. But in the real world, it’s fundamentally flawed.

Here’s why:

• Results decay over time, especially when you're optimizing for low-intent traffic (more on that below)
• Not all tests have the same lasting impact, friction removal ≠ motivational copy tweak
• Stakeholders get misled by inflated wins → misallocate budget → question CRO credibility later

Enter: The Conservation of Intent Framework

At the core of FunnelEnvy’s new model is a powerful concept: The Conservation of Intent.

It means:

• You can’t manufacture intent out of thin air
• Your website visitors fall into 2 buckets:
  • High-intent: ready to convert regardless of tweaks
  • Low-intent: need persuasion, UX help, motivation
• Most A/B test lifts come from that group (low-intent)
• But those low-intent gains fade faster over time

What This Means for Your Experiments

Let’s say you run a test and get a 31% lift. Here's what happens next:

• The high-intent users you win will likely keep converting
• The low-intent users are more volatile, some convert due to urgency or friction removal, but many won’t stick long term
• Over the next few months, your results decay, eventually hitting a stable point

Instead of pretending that your lift holds steady, FunnelEnvy now uses a decay-based model that factors in:

1. Initial lift
2. % of high vs. low intent visitors
3. Decay rate by experiment type (e.g. friction reduction vs. motivation boost)

You get a much more credible, defensible ROI number, even if it's smaller.

Why Smaller ROI Is Actually a Win

We get it. Telling clients or execs that your 31% win is actually worth 10% long-term isn’t sexy.

But here’s the truth:

- It builds trust and credibility- It lets you prioritize smarter- It sets realistic expectations for testing ROI- And it avoids misallocating budget based on inflated forecasts

As Arun puts it in the episode: “Do you want ROI that looks good in the QBR or ROI that’s actually real?”

Want to Report Your Test Results More Accurately?

We are building simple models you can plug your own data into, but in the meantime, here’s how to level up your reporting:

• Segment your test impact by intent (use GA4 or session replay to get insight)
• Estimate decay based on test type (we’ll cover this in part 2 of the episode)
• Stop using annual linear projections (it’s a credibility killer)
• Educate your stakeholders on why smaller wins = long-term trust

Watch the Full Episode Now

This is one of the most insightful, behind-the-curtain episodes we’ve released to date. If you’re in CRO, RevOps, or growth strategy, you need to watch it.

Watch Episode 5 Part 1 on YouTube »

And don’t forget to subscribe, Part 2 goes even deeper into test types, decay rates, and the actual model we use.

Bonus Resource: Website Optimization Quick Wins

If you're still optimizing with gut feel or best guesses, this free guide will change how you approach form UX, friction, and full-funnel optimization.

Get the Website Optimization Quick Wins guide here.