How to Calculate True ROI from A/B Tests (Without Lying to Your CFO)

Let’s get real about A/B testing.

You ran a test. Got a 28% lift. Extrapolated it over 12 months. Share the number with your VP.

Congrats. You just lied to your CFO.

It’s not your fault. The industry has clung to linear ROI projections for years because they’re simple and look great in decks.

But they’re also wildly inaccurate.

In this episode of the FunnelEnvy podcast, Arun Sivashankaran and David Janczyn go deep into a smarter, more truthful model: Intent Decay.

Why Linear ROI Models Fail

When you apply a constant uplift over time based on your test results, you're making two big assumptions:

1. That your test lift is permanent
2. That all site visitors behave the same way

Neither of these is true.

Most A/B tests, especially those that rely on urgency, gimmicks, or motivational tweaks, start to decay in performance within weeks.

The deeper truth? You're mostly capturing low-intent traffic that wasn't going to convert anyway. That behavior is temporary.

Introducing Intent Decay: A Smarter ROI Model

Arun, David and the team at FunnelEnvy built an approach that accounts for visitor intent and test type to calculate long-term ROI more accurately.

Here’s how it works:

There are two types of visitors:

• High intent: Already motivated, further down the funnel
• Low intent: Uncertain, exploratory, need persuasion

Most CRO experiments lift low intent visitors. But those conversions decay faster.

By identifying the type of experiment you ran (e.g. friction reduction vs. urgency play), you can apply a decay multiplier to estimate more realistic ROI.

Test Types That Decay (and Those That Don’t)

Not all experiments are created equal.Here’s how test types break down in terms of decay risk:Test TypeDecay Rate   WhyFriction Reduction (e.g. form simplification)          Low   Captures both high + low intent usersMotivation/Urgency (e.g. limited time offers)        High   Triggers short-term emotion onlyTrust/Social Proof     Medium   Builds credibility but can fadePersonalization     Variable   Depends on depth & targetingOffer/Positioning Changes   Low-Medium      Impacts core value perceptionThis framework lets you prioritize high-leverage experiments that will stick.

How to Apply This to Your CRO Program

David lays out a simple approach:

1. Estimate Intent Split (use 70% high intent / 30% low intent as baseline)
2. Identify Test Type and corresponding decay factor
3. Apply Monthly Decay to initial lift
4. Model ROI Over Time, not just Week 1

"Your ROI projections should reflect real-world behavior, not vanity metrics."

Bonus: Use This Model to Prioritize Experiments

Most teams use ICE (Impact, Confidence, Ease) for prioritization. But it ignores long-term decay.

By layering in a decay factor, you stop prioritizing flashy, fast-fading tests, and start investing in sustainable wins.

This is how you:

• Defend your roadmap
• Earn budget credibility
• Build a testing culture based on truth, not just temporary spikes

Final Takeaways

• Linear ROI is easy. But it’s wrong.
• Intent Decay gives you a credible, repeatable model
• Use it for reporting, prioritization, and strategic alignment

You don’t need to overhaul your stack. Just start using better math.

Watch episode 5 (Part 2) of the FunnelEnvy Podcast to go deeper.

Want to see how this applies to your funnel? Book a Full Funnel Conversion Audit.

‍