About James Niehaus

James ran his first A/B test back in 1999. Since then he has optimized websites, emails, products, support and sales operations, technology, people, teams, strategy, and anything else that can be measured and improved. His speciality is optimizing the intersection of strategy, marketing, product, technology, and data. Today James runs the strategy and services team for FunnelEnvy, a leading platform and agency in the CRO and personalization space. Prior to FunnelEnvy, he has run marketing, product, and analytics programs for a variety of companies, including Ensighten, Symantec, Salesforce, and E-LOAN.

Multi-Step Interactive Experiences (Going Beyond the Form)


Hi everyone. My name is James Niehaus from FunnelEnvy. And today I want to talk to you about multi-step interactive experiences and going beyond the form. So this is the third video in a series that we focus specifically on a technique we call multi-step interactive experiences. In the first two videos we talked about how to use this on B2B forms to really drive massive improvements in conversion rates. And today I want to talk about how to use the same technique on other parts of your site.

We Will Cover

  • A quick recap on multi-step interactive experiences
  • Why they work
  • How they can work on other parts of your site
  • 5 popular ways to leverage interactive experiences on your site

Full Recap of Multi-Step Interactive Forms

If you have not watched the introduction and multi-step advance use cases videos, watch these in these given links.

Examples of Multi-Step Form Flows

Here are just some examples of past client interactive experiences. So this should be self-explanatory, but as you see here, all we’re doing is taking your typical static B2B form, breaking it up into interactive steps, and leading with intent questions that make it easier for the user to raise their hand, provide you information, commit to the process and convert at higher rates. And we say convert at higher rates, we do mean higher rates. So here are just some examples of recent conversion lifts we’ve seen on client forms just by moving the static forms to multi-step experiences. So as you see here, it’s definitely pretty powerful and effective. And this is also a reason why we kind of then explored using this technique on other parts of the site.

multistep form examples

Some Key Takeaways from previous videos

With a multi-step form experience, you usually want to ideally lead with intent questions that’s more about what’s in it for them. What features do they care about? How big is their company? What are they looking to do? What’s their role? But the whole idea here is that asking them easy to answer intent questions, helps guide them down the path towards finding a better outcome or helping you better guide them down to the right solution. So that’s the first takeaway.

The second takeaway is also ideally asking them a couple of those questions first because what you want to do is make it easy for them to get started, continue, and create that momentum and commitment to completing that process. And that’s how we get the higher conversion rates.

And then the nice part about this technique is that you then layer on additional strategies, and this is where it gets even more powerful. So on the left, you see an example where we actually start personalizing step two and step three of the form experiences based on what they provided as an answer in step one. So this is where it can get a little more targeted and personalized. Or on the right is a good example where you can use their information to then potentially route them to a different funnel or flow. An example there, they’re using company name, based on the email address, to decide whether they should send them to a scheduler as you see there, or to just give them the rest of the regular form. So this is where you can potentially provide custom experiences based on company size or target account or their role or what they’re looking to do. But this allows you to then go from a one size fits all static experience, to providing smarter, routed experiences that align with your personalization and ABM strategies.

More Powerful w/ ABM + Personalization

Personalize the rest of the form by their inputs

Multi-Step Works Beyond Forms

Multi-Step isn’t just for forms. They can work on most parts of your website. I mainly want to focus on five key areas of the site, or techniques.

Step #1 Homepage: funnel them to a multi-step form flow on the entry

So starting this all off, starting the funnel on the home page. So for most B2B sites, the homepage is typically a static billboard that tries to communicate one message to a broad and diverse set of visitors. So recognizing that you’re probably not going to align exactly to what that user’s intent is, the idea here is that you actually use the homepage real estate to engage the visitor. And in this case, engagement means trying to get them to raise their hand, express intent, and get started with you, and go down a certain conversion funnel right at the homepage. So whether you start interacting then with them, with engaging questions on the left, you see examples there. Or, if they click on a certain action, present out that type of experience. But the whole idea here is that, rather than guess at what their intent is, or how they want to get started, give them some options to make it easy for them to kind of start exploring. And ideally, without even knowing, go start kicking off that conversion funnel from the home page.

Step #2 Homepage: guide and route users to the right content by intent

Use the homepage strategy, and instead of taking them to a form flow, guide and route those users based on their intent, to a better page or a better flow.

So that here is we’re trying to help them by skipping steps and helping them land on the right page so you can cut down on the cycles to get them to where they want to go. So this is more about routing and navigation. But again, the whole benefit of these techniques is that we’re engaging them from the get-go. We’re not forcing them to kind of decide themselves, try to find the right information, and leave it up to chance that they find the right place to go. And this same strategy makes a lot of sense on product pages.

This is an example of one of our clients, where on the left you see their traditional static page. As you see there, it’s your typical solution type page where you provide a lot of content. And give them their choice but through a lot of text and diagrams.

Step #3 Product Page: help them find the right solution

On the right, it’s taking that same content and packaging it up into interactive questions to make it easier for them to kind of, again, find what they’re looking for, raise their hand, express their intent or interest, and you help navigate them to the option that makes the most sense. So we saw some very positive engagement and conversion metrics when we did this technique on product pages like in this example here. But the whole idea here is you’re trying to provide that educated, guided hand that helps them find better what they’re looking for based on what they told you. So in this case you’re being helpful, helping them complete their job better, and in a more timely fashion. And of course, this makes sense on your product and pricing pages. So when you have packages and plans, most B2B sites have your typical good, better, best, here are three or four plans, whether it be based on company size or number of seats needed, or whether it be based on certain features, that’s your typical layout that we’re all used to.

On the product pages, a nice technique is to really use the strategy to help them find the right solution.

Step #4 Plans/Pricing Page: help them find the right package

The plans and pricing page is also another great place to use it to help them find the right pricing package or combination.

So we’re not saying you have to move away from that, but in these examples here they still show those tables of plans, but in both examples here they give options where, if you want to specify your interests or what you’re looking to do, or maybe talk through a questionnaire about who you are and what you’re looking for, they can then narrow down the plan packages to fit what you provided them as far as intent or profile. So the whole idea here is that, when you have a little more complex set of packages and pricing plans, rather than have them guess or maybe choose incorrectly, or maybe just waste their time combing through all of this, you give them an easier path of which they just simply specify what is their key intent, profile questions, and use that information to help narrow and guide them down to the right package or pricing plan that works for them.

Again, you know your product more than anyone else. You’ve seen the success of your products and packages on a variety of clients across your industries. This is all about using that intelligence to better guide new visitors, who first come to your site, come to your pricing page, and help them find a solution that fits their needs and best matches their profile. Because you want them to be successful, and this is your chance to guide them down that path in a more direct way. And both sides will win in the end.

Step #5 Quizzes/Calculators: help them get insights

    And lastly, we shouldn’t forget the fact that, for most B2B sites, they do have interactive experiences typically in things like your ROI calculators, your organizational assessments, your company benchmark, and your various quizzes, right? So by all means, these are great options and techniques. We encourage you to keep using them. So in the end we’re just fans of anything that provides that interactivity. We’ve seen, from years and years of testing, that whenever you can give the visitor a chance to interact, raise their hand, you make it easier for them to get started, you reduce the complexity of that visit by allowing them to be guided down the right path. And you actually end up having more control over where they go through their journey, which is what we really want to do. Ideally, if we had a choice, we’d want to guide each visitor on the right journey for them.

    Since we can’t really predict who they are and what they want, we think interactive experience are the best way to meet in the middle and provide them a set of choices that allow them to kind of really narrow down to their best options based on what their profile is and based on what their intent is. So it’s really a win-win for both sides.

    Key Takeaways

    1. Multi-Step experiences, especially on the forms, work great as a great conversion tactic
    2. It works even better when combined with your ABM/Personalization strategies. So this is a great example where your conversion techniques and your strategic techniques should really work well together because they really compliment each other very well.
    3. They really work well in a couple of key places, like your homepage, your product pages, your pricing and packaging pages, as well as your traditional quizzes and calculators.
    4. Here are 5 ways to get started:
      1. Homepage: Start them in a conversion funnel
      2. Homepage: Route them to the right page by intent
      3. Product Pages: Find the right solution(s)
      4. Pricing/Plan: Find the right package
      5. Quizzes/Calculators: Provide custom insights

    So really want to say here, we want to advocate for it, try it out, explore this on your site, you really won’t regret it. This is, again, better experiences for all.

    And then lastly, as I said, if you haven’t had a chance yet, go visit our site. We have a couple of other videos that talk a little more in-depth about the strategy, especially as it relates to forms and personalization, and ABM. So check it out if you haven’t had the chance yet.

    1. Introduction to Multi-step Interactive Forms
    2. Multi-step Interactive Forms (Advanced Use Cases)

    If you have questions, just drop me an email. And if you want to see our own interactive quiz, you can hit our website. And that quiz will actually help you evaluate whether you’re the right fit for working with us. So check it out and hopefully we can talk soon. With that, take care.

    Maximize Site Revenue with Multi-Step Forms


    Hi, everyone. My name is James Niehaus from FunnelEnvy. Today I want to talk to you about multi-step forms, why our clients love them, and how they can maximize your side revenues.

    So why our clients love them because.

    • They work. We typically can see 20, 30 – 50% improvement in form conversions, so meaningful results.
    • We can often enhance and compliment your existing ABM and personalization programs. So it really adds to the value.
    • It works great on form but also works pretty well across most of your site. We’ll talk about some examples of that in this presentation.

    But how do you make such user-friendly forms?

    We all hate forums. Whether it’s a mortgage form, a tax form, or a B2B lead form, right? So the industry recognizes this. And some industries have actually adapted and evolved.

    So, leading the way, mortgage. So, now we complete a mortgage application online. It’s going to be an interactive multi-step experience. Same thing with taxes, an interactive multi-step experience, to guide you down the path.

    Multi-step forms for B2B

    B2C loves multi-step, whether it’s to guide people down a better path, or especially on lead gen, to help convert at higher rates. But B2B is slow to adopt. So we still typically see static forms on B2B, typically because of operational hurdles. It’s just easier to embed a form. But, if you’re willing to make the effort and like our clients, past and present, and see examples of their multi-step experiences, you can see firsthand that all we’re doing here is taking their simple static forms, breaking them up into multiple steps, making it easier to digest, asking easy and 10 questions at the beginning to get them started. And in the end, it improves engagement and significantly improves conversion rates.

    Key Results

    So things we’ve seen from some past and current clients are significant lifts. These are not small lifts. These are significant lifts that can really maximize and change your funnels for maximizing revenue.

    multi step form results

    And some key things to keep in mind with multi-step forms, we want to make it easy. So if you think about doing it for yourself, think about in your funnel, what are easy and 10 questions to ask to get them started? Also, you want to make sure you ask a couple of questions, so you want to get the ball rolling, create that momentum, and get them committed to completing that conversion process with and completing the rest of the form. So here you don’t want to ask for first name, last name, email, to get started, you typically want to ask for, how large is your team? What is your role? What is your product interest? But things that are more intent-focused so they can get started without hitting hurdles.

    And again, these were great with your advanced programs like personalization and ABM. So on the left, you see an example where we personalizing the rest of the form experience based on the answers on step one. Or on the right, think about maybe we skip the form, based on their being part of a target account. So based on their company name on the email, you can show them, say, a scheduler instead of say, of the rest of the form, or maybe attempt to a Drift Bot. But the idea here is based on their inputs or based on their industry company size, or even if they’re part of your target account, you can personalize from experience that step experience, or even give them different routes and experiences to better convert them.

    And like I mentioned, it works great on forms, it also works really well across the site. So we’ve run this on home pages, on product pages and solution pages, and pricing pages. And typically what we see, is significant improvement engagement, and also uptakes and conversion rate.

    So we definitely recommend you explore this, try this out on your site, and check, see whatever it works for you on your forms and beyond.

    3 Takeaways with Multi-Step Forms

    1. They work. And can often generate a 30-50% lift in form conversions.
    2. They can enhance your ABM and personalization programs
    3. The technique is effective across the site, not just on forms.

    If you are as excited as we are about getting started with multi-step forms, visit our blog funnelenvy.com/blog, jump into our quiz and see if you’re a good fit to work with us today.

    Up Next

    Learn Muti-step interactive experiences (going beyond the form)

    Multi-Step Interactive Forms Advanced Use Cases(ABM/Personalization)

    Hey everyone. My name is James Niehaus, and today I’m going to walk through some advanced use cases for multi-step interactive forms. Specifically, where it can help you with ABM and personalization on your website. And this is actually part two to an initial video I did around introducing everyone to interactive forms, why we love them so much, why they’re pretty effective for our clients, and how you can get started with them as well. So this is kind of part two of that series. All right, let’s jump right into it.

    So we’ll cover here the following things.

    • A quick recap on multi-step interactive forms
    • Why they work
    • Why they are ideal for your ABM and Personalization programs
    • 5 ways multi-step forms can enhance your ABM and Personalization programs

    All we’re doing here is taking your longer static forms on your website, breaking them up into steps, making them interactive, and leading with intent questions that get them to raise their hand and express who they are and what they want to do. So this has converted really well for our clients. As you see here, these are some examples of actual lifts we’ve seen with form conversions on forms that get started and talk to sales and get demos.

    Full Recap of Multi-Step Interactive Forms

    Before we jump into those, just a quick recap of why we think it works well, and what some best practices are.

    So always lead within 10 questions if possible. So what’s in it for them, who they are, what they’re looking for. As opposed to starting with, “What’s your first and last name, your email, or your phone number?” Right? We want to make it easy for them to get started and engage. Secondly, we want to ask a couple of those 10 questions initially, before we show them the rest of the form, because we wanted them to, one, commit, get some easy answers out of the way, and get momentum towards completing their task. So we found that this is definitely a sweet spot to kind of maximize conversions on a multi-step experience. And then lastly, just really set proper expectations. How many steps are involved, and what happens after you submit the form? Just so they right context about time and what’s going to happen next. All makes sense hopefully? Good.

    Examples of Multi-Step Form Flows

    multistep form examples

    Some best practices of multi-step interactive forms

    Results we’ve seen from multi-step interactive forms

    multi step form results

    So let’s jump right into why multi-step forms are ideal for targeting and segmentation. Hopefully, it’s pretty obvious, but in these experiences, and in those first couple of questions where we ask for either profile or intent questions, we’re getting valuable information that they want to share to better customize their experience or get better information from us.  So we want to use that information to provide, one, them a better experience, but also ideally personalize based on who they are and their company. So that’s what ABM and personalization are all about. Personalizing based on who they are and the company they’re from. Based on what you know about them and based on what you hopefully want to achieve with them in partnership.

    1. Use their answers to assign them to a segment

    Use their answers to better assign them segments for analytics and campaigns. That seems pretty obvious. More importantly, use those answers to personalize the rest of that form experience. So as they provide you information about them, provide feedback that you can support their needs, you can provide specific information about their product interest. But use that moment to actually provide reinforcement for what they’re looking for. Also, you can actually potentially use that information to route them to a different funnel or experience. So the whole idea is that not everyone should be treated the same. Your higher value prospects may be given a shortcut to talk with sales. Maybe the less valuable users may be given more of a self-service route. But use their answers and their profile to route them in the most appropriate place to maximize your limited resources, but also provide them the more appropriate experience.

    What’s even more compelling and potentially more exciting is the idea that we actually change the experiences based on who they are and what company they’re from. Now, rather than simply have one form for everyone, the idea here is that we take their answers or their inputs, and based on our business strategy we may serve them a different next step. So an example on the right, you see here based on the email and company domain, based on whether that matches a target account or not, they either can skip the form and go right to the scheduler. Or if they’re maybe not our targeted account, they would get a regular traditional form.

    Here are some examples of getting their answers and then assigning them to a segment

    2. Use their answers to personalize the rest of the form

    Customize the remaining questions, messaging, and visuals to reinforce the benefits of their selection. By using these answers to personalize the rest of the actual form experience. So if they express certain product interest or indicate they’re from a smaller enterprise, you want to reinforce that you are the right business for them, that here are the benefits of that product, or maybe here are the reasons why you’re great for SMB or enterprise, or for this industry, or for this type of role, technical or maybe marketing focused.

    You see an example, a very simple example where, on a contact us form, we just simply asked for their product interest in step one, and we then personalize the rest of the form based on that answer. So this is where you can use that information at the moment to reinforce the benefits of your business, your offering, your expertise, in a way that’s going to reinforce based on who they are and what their intent is. We’ve done this with a couple of clients and we’ve seen pretty nice positive upticks in conversion rates from this simple concept.

    3. Use their answers to route them to a different funnel

    If you identify a top target you can:

    • Skip form
    • Shorten form
    • Change questions
    • Trigger Drift/Chat

    4. Target and personalize in future sessions/other channels 

    We can target and personalize not just at that moment, but also in future sessions, in other channels. We don’t have to stop at the form. So that’s by the information we know about them, use it wherever and whenever you see them again. And then lastly, target. Over time you actually can create different questions based on their profile. So if you have certain key segments that come to your site, you can potentially target them ahead of time and actually serve them with a different actual form experience. A little more advanced, but again, as you’re committing to this strategy, you’ll see more and more ways to use it to your advantage.

      5. Target intent questions based on their profile

      The idea here is, don’t let go of the information after they complete that form. What you want to do is repeat that message, reinforced that information, and follow up interactions, whether it be a session to the site, whether it be another channel like email or display, retargeting perhaps. But the whole idea here is you’re getting valuable information at the moment from an engaged user. Use it for your advantage. Okay. So that would mean if they express certain product interests. When they come back, maybe that homepage here will change to show that product, sort of that category affinity type technique. Or based on their role, reinforce the use cases for their role. Or based on their industry, show again, case studies for their industry, for their size, or any other ideas. An example on the right you see here, where we’re targeting the homepage here all based on their different stages they’re in that we identify in the process.

      And then lastly, the idea here is … and over time, as you refine the strategy, this will become its own strategy. So much as you have with, say, your Marketo email programs, you have different nurture sequences. Or in Drift, you may have different Drift playbooks. The idea here is, over time, you recognize that you have certain key target accounts, segments, and ICPs that you’ll want to route through different form experiences. So rather than having one interactive form for everyone, you may eventually end up where you have different forms for your top segments and groups. So I don’t recommend doing this day one, but as you evolve the strategy, actually see what works, what doesn’t work, you’re going to see natural segments that perform better, or might need more guidance or handholding. This is where the strategy, once it starts getting those double-digit improvements, these are natural ways to further enhance and refine the program. And again, this is going to only make your personalization and ABM programs more targeted, because you’re targeting based on those same attributes that you care about and you want to personalize for.

      So the quick takeaways here.

      Multi-step forms. great conversion tactic. And we recommend you do this on your site today. But we think it works even better, as you’ve hopefully seen here when you combine it with your ABM personalization strategies.

      5 ways to get started:

      1. Segment by their answers they provide you
      2. Customize a form based on those answers for a better conversion rate and better experiences.
      3. Route them by their answers
      4. Target them in return sessions/other channels by their answers
      5. As you get more advanced and more mature in this strategy, start building different forms for your top use cases and your top segments.

      Learn More on Multi-Step Experiences

      Go to our website, funnelenvy.com/blog, and you’ll be able to check out the content and hopefully enjoy this and other content that’s similar. And with that said, thank you for your time. If you have questions, just drop me an email. And if you want to see our own interactive quiz, you can hit our website. And that quiz will actually help you evaluate whether you’re the right fit for working with us. So check it out and hopefully we can talk soon. With that, take care.

      Introduction to Multi-Step Interactive Forms


      Hey everyone. My name is James Niehaus. And today in this video, I’m going to walk you through what are multi-step interactive forms. So at FunnelEnvy, we use this all the time with our clients, and we’ve seen some great wins. And we thought today we would actually share with you what we’ve learned from those efforts, and hopefully encourage you to do the same on your side.

      1. What are multi-step interactive forms?
      2. Results we’ve been seeing with our clients.
      3. Some best practices we’ve learned from doing this with our clients.
      4. Some techniques and lessons learned that you can use and adapt to make it easier for you to get started.
        Keeptruckin Multi-Step

      We all hate forms. Right? So whether it’s a mortgage application or whether it’s a tax form, it’s something we prefer not to do. And the providers know this, and that’s why it’s very hard for you today to complete a mortgage application online without going through a multi-step experience as we see here. Or as we all know with taxes, TurboTax and others have made that the standard experience, step-by-step guided questions.

      That is Why We All Prefer This…

      multi step forms

      Why? It’s common sense.

      • It’s less intimidating, easier to get started. Even though we know there are more fields,
      • it’s an easier way to start and get the process going.
      • We feel less perceived commitment.
      • And lastly and most importantly, ideally we do it right, as an end-user we think we’re being guided down a better path, which will, as we know, save us time and give us better results.

      So there are all positives and why we’ve seen this kind of being the prominent way, at least in B2C, where complex forms are being presented. So they’ve been doing this for like 15, 20 years. LendingTree is one of the pioneers of this. They found out early on that, in a competitive space, this gave them an advantage. Can provide you a better user experience, make it easier for you to fill out a form, and become a lead. And now, fast forward to today, you really can’t complete a mortgage lead or auto insurance quote without going through some similar experience. So it is now the standard for B2C.

      But for B2B, unfortunately, it still seems to be the 1990s, where it’s still no static forms, no interactivity, and it’s pretty much the standard for most of the industry. This is unfortunate, but it’s slowly changing. So if you see here, there are some examples out there.

      So like Drift and Intercom and others like them who provide us chatbots. That’s helping. So if you can get [inaudible 00:03:02] one of their bots, you can usually get a nice playbook experience of decision tree experiences. So it’s a good start, but as we all know, the majority of website visitors still want to interact with your website, not a chatbot.

      interactive form experience of the industry

      And we’ve seen some people provide multi-step forms, like Salesforce here with their trials. It’s a good start, but there’s obviously more that can be done. But for B2B, there are some additional hurdles that are typical of most of our clients. So for example, if you’re in the B2B space, typically you’re going to have one of these four vendors.

      form embedded vendors

      And they’re providing your email, your forms, your landing pages, and your workflow. So they make it really easy to by things like let’s copy and paste our embedded form, put it on your website, or we’ll host your landing page, and you’re done. That’s pretty easy. But of course, the default experience is a single, static form.

      A second hurdle we often see is that you do need web developer resources to do this. Most DemandGen teams are resource-constrained. And don’t have access to a developer for the 2-3 days required to implement

      And the third hurdle is, it’s not a priority. Typically the website is not the focus, it’s about getting traffic to the website. So they’re chasing the latest AdTech and Martech to make it easier to target, intent, ABM, and get more people to the site. So I figured this is what people are most interested in hearing about, which is: what are the actual results?

      So these numbers are actually real numbers we’ve had with recent clients in the past year running multi-step interactive forms on their site. So again, these are meaningful lifts on forms like demos and pricing and get started forms. So your core sales forms. So that’s a huge win for many of our clients. And for the most part, for the majority of our clients we tried this on, it’s worked. Typically double-digits or higher. I think, and only one example I can recall, we ran this where it didn’t necessarily win across the board, but it’s still won for the returning segment for around 20 plus percent. So even if we don’t get a clear win across the board, we still see important lifts for key segments. So this is why we want to stress that we think this is probably the most viable tech tactic you can apply to your website to really maximize your revenue and your pipeline for the website.

      multi step form results

      And here’s some just examples of the actual client experiences.

      So as you see here, again, simple, straightforward experiences that make it easier for their visitors to start and complete a form experience.

      And with that said, let’s talk about some of the best practices we’ve seen working with our clients in this area. Now, the first thing you want to consider is, if at all possible, you want to lead this experience, that first question or two, with intent questions. Right? So you want to ask them things that they care about first before you ask them for their name and their email. Right? You want to get them excited and want them to get started and make it easy for them to get started. So intent questions, how large is your organization? What industry are you in? What features do you care about? What’s your role? What are some of your integrations?

      The second learning is you want to create momentum. So we want to ideally provide two or three questions that get that intent ball rolling. So here’s an example from one of our clients. But again, the whole goal here is you want to make it easier for them to get started, you want to create that momentum so that they feel committed to the process. So therefore it’s easier for them to convert and complete that final piece of the form that’s going to be the personal information that we need to collect to run the business.

      Now the third piece I’m going to call out also is it’s good to also set the proper expectations. So when you provide that form, that multi-step experience, call out the number of steps involved. And when possible, call out what you plan to do with that information. So you can help them make a better choice. Is this going to be passed by your sales team to make a more effective quote or a demo or discovery session? But let them know that this information is not going to be dropped to the floor, it’s going to be used to help provide them a better outcome or experience, which is what they want. That guided treatment. Now, those are some simple best practices. And hopefully, they make sense.

      Lessons learned along the way from doing this with our clients.

      • Pick the right form. Ideally, where the outcome could vary based on their inputs:
        -Best working forms -pricing, get started, talk to sales, choosing the right demo or trial, ROI calculators.
      • Don’t be afraid to ask for more information if the question(s) aligns with user intent. like about your team size, about your feature of interest. It will be easier for them to get started.
      • It will work for more than just forms. Now that’s a broader topic, so I’m going to save that for another video, that you can access on our site shortly after this one. But those are some lessons learned

      How to Get Started

      • Identify the right form
      • Figure out the design first
      • Work with a web development resource to build the interactive form (typically a few days to a week effort)
      • If you are adding fields work with your marketing automation team to capture the new fields
      • Run the experience as a test and remember to look at performance by your top segments
      • Run 1-2 iterations over time to fine-tune the question sequence, visuals, or layout
      • Expand to other forms on your site.

      Don’t overlook this, but you want to expand this to other forms on your site in other areas. So again, I’ll have more about this topic in another video about where else can this work for you on your website. But that’s hopefully enough to have you get started. And then with that said, as I mentioned, come back to the website, I’ll be posting a couple of other videos about, one, how this can be used in other places beyond the form.

      Learn More on Multi-Step Experiences

      Here are the two more articles to learn about multi-step interactive forms

      Go to our website, funnelenvy.com/blog, and you’ll be able to check out the content and hopefully enjoy this and other content that’s similar. And with that said, thank you for your time. If you have questions, just drop me an email. And if you want to see our own interactive quiz, you can hit our website. And that quiz will actually help you evaluate whether you’re the right fit for working with us. So check it out and hopefully we can talk soon. With that, take care.

      Attributing Campaigns to Sales Opportunities in Marketo

      I’m writing this because it’s the year 2020 and we’re still having trouble attributing onsite campaigns/testing to Marketo MQLs/Opportunities/Revenue.

      It is meant primarily for technical practitioners as a quick-start guide, and is so simple you’ll probably be able to get it hooked up today.

      As a result, this guide is very tactical in nature, with the end goal of helping you answer the question ‘What impact is my onsite campaigns/tests having on revenue?”.

      What solution is for you

      If you happen to already be using FunnelEnvy, then enabling our Marketo Integration is all you need to do. This not only enables attribution, but unlocks additional functionality, including the ability to target onsite users based on their Marketo classifications. It’s a paradigm shift in how you think about marketing automation and I encourage you to read our recent article to find out more.

      If on the other hand you’re still using a web-based testing tool like Optimizely or Adobe Target to run your onsite campaigns then read on, as this guide is primarily for you. But first, a cautionary tale:

      Always optimize for downfunnel outcomes, not onsite vanity conversions

      Recently, FunnelEnvy ran a very visible experiment via Adobe Target for a well known SaaS company. Early results were trending downward and web analytic data showed a sharp downtrend in incremental conversions/leads. Talks were had about ending the test early as this was potential a huge economic impact.

      Luckily, the attribution module you are about to see had already been installed, allowing us to query Marketo directly, telling a different story. We were able to calculate that the test was actually responsible for a significant increase in annual recurring revenue! Without this additional layer of information, we likely would have moved on, but.

      But back to our discussion:

      The Solution

      We’re going to break this guide up into two main parts:

      Part 1: Create/maintain a running list of campaigns/tests a user has seen

      Part 2: Pipe this list into Marketo

      Create/maintain a running list of campaigns a user has seen.

      In order to automate this as much as possible, we’re going to create a centralized module that fires on every page. This allows us to just install it once (via GTM, Launch etc.) and not have to make any edits when new campaigns/tests are launched.

      You’ll need to take advantage of Adobe’s Response Tokens or Optimizely’s client side object if you want to go this route. Users of other platforms like Google Optimize will need to follow a more manual approach, adding the module to every new campaign/test. The principles will stay the same, it just requires a bit more to upkeep.

      The Code
      (function () {
      // Callback for Adobe Target response tokens
      document.addEventListener(adobe.target.event.REQUEST_SUCCEEDED, function(e) {
      var tokens = e.detail.responseTokens;

      if (isEmpty(tokens)) {

      var uniqueTokens = distinct(tokens);
      } catch(err) {
      The start of our module – An Adobe Response Token listener
      [javascript] //Cycles through each token
      uniqueTokens.forEach(function(token) {

      var cookieName = token["activity.name"] + ‘ ‘ + token["experience.name"];

      // Slugify the cookie name.
      cookieName = cookieName.toLowerCase().replace(/\\((evar.*?)\\)|\\[(.*?)\\]/g, ”).trim().replace(/[^a-z0-9]+/g, ‘-‘);
      Next we cycle through each response (there is 1 per active campaign) slugifying the campaign/variation name.
      Find the existing cookie if it exists. Adds new campaign values to the front of the cookie
      var existingCookie = _satellite.cookie.get(‘marketoCookie’) || ”;

      if (existingCookie.indexOf(cookieName) === -1) {
      var newCookie = cookieName + ‘|’ + existingCookie;

      _satellite.cookie.set(‘marketoCookie’, newCookie, {expires: 30});
      Since this module runs on every page load, we also need handle duplicates in the cookie name
      [javascript]var checkLength = function checkLength() {
      if (newCookie.length > 2000) {
      newCookie = newCookie.split(‘|’);
      newCookie = newCookie.join(‘|’);

      Finally, Marketo inputs (you’ll set this up soon) have a character limit of 2000 characters, so we truncate older campaigns.

      Here’s the reusable function in it’s entirety:
      (function () {
      // Callback for Adobe Target response tokens
      document.addEventListener(adobe.target.event.REQUEST_SUCCEEDED, function(e) {
      var tokens = e.detail.responseTokens;

      if (isEmpty(tokens)) {

      var uniqueTokens = distinct(tokens);

      //Cycle through each token
      uniqueTokens.forEach(function(token) {

      var cookieName = token["activity.name"] + ‘ ‘ + token["experience.name"];

      // Slugify the cookie name.
      cookieName = cookieName.toLowerCase().replace(/\\((evar.*?)\\)|\\[(.*?)\\]/g, ”).trim().replace(/[^a-z0-9]+/g, ‘-‘);

      Find the existing cookie if it exists.
      Adds new campaign values to the front of the cookie
      var existingCookie = _satellite.cookie.get(‘marketoCookie’) || ”;

      if (existingCookie.indexOf(cookieName) === -1) {
      var newCookie = cookieName + ‘|’ + existingCookie;
      If above the 2000 Marketo input character limit,
      truncate old campaign values
      var checkLength = function checkLength() {
      if (newCookie.length > 2000) {
      newCookie = newCookie.split(‘|’);
      newCookie = newCookie.join(‘|’);

      _satellite.cookie.set(‘marketoCookie’, newCookie, {expires: 30});


      function isEmpty(val) {
      return (val === undefined || val == null || val.length <= 0) ? true : false;

      function key(obj) {
      return Object.keys(obj)
      .map(function(k) { return k + "" + obj[k]; })

      function distinct(arr) {
      var result = arr.reduce(function(acc, e) {
      acc[key(e)] = e;
      return acc;
      }, {});

      return Object.keys(result)
      .map(function(k) { return result[k]; });
      } catch(err) {
      console.log(‘Error in Target Marketo Cookie’);
      A simple solution – copy and pasteable.

      Create a hidden input field on all Marketo forms

      Lastly, we need a way to get the running list into Marketo.

      Marketo has out of the box functionality to create an input to ingest a cookies value. All you need to do is add this input to your forms, specify the cookie name (in our case, marketoCookie) and the rest happens by default.

      You’ll now have a historical list of campaigns/tests an individual user saw whenever they submit a Marketo form.

      Easy peasy.

      By |2020-03-22T19:05:26-07:00March 22nd, 2020|Uncategorized|0 Comments

      Real-Time Personalization with Segment and FunnelEnvy

      Segment is a very popular CDP (customer data platform) that specializes in unifying data sources in real-time across your digital touchpoints and with your ever growing marketing technology stack.

      Segment is an especially popular choice with product and engineering teams because it can serve as the organization’s centralized hub for data collection, integration, and syndication. And what really makes Segment so powerful is its 300+ API integrations with top marketing technology platforms.

      This has resulted in Segment becoming the foundational data platform for thousands of organizations that now use this unified data to power a variety of business use cases throughout their entire marketing stack and across their digital touchpoints.

      Today we are going to do a deep dive on delivering site personalization using Segment data. This can be accomplished through the FunnelEnvy’s Segment integration which makes it easy for Segment customers to test, target, and personalize user journeys using all of that rich user and event data in real-time.

      Activating Segment Data with FunnelEnvy

      Segment is one of the out of the box integrations that come with the FunnelEnvy Platform.

      The first step to activate the integration is to select Segment as a Data Source within the FunnelEnvy Platform.

      Next, copy the API Key displayed in the integration details, check the “Active” checkbox and save the integration.

      The Segment integration will be listed as active and FunnelEnvy will now start tracking identify calls and associated traits from Segment.

      The last step is to configure FunnelEnvy as a new Destination within the Segment Platform. In Segment search for and add FunnelEnvy as a destination.

      In the settings you’ll need to add the API key that you copied from the FunnelEnvy platform when you activated Segment integration.

      Save the destination and you’re done. Identify calls and track events and associated traits/properties will start flowing from Segment to FunnelEnvy.

      That’s it! The FunnelEnvy and Segment integration is now ready to go. If you want to also set up FunnelEnvy as a data source for Segment (Segment as a FunnelEnvy destination) you can follow the steps outlined at the bottom of this help guide. This step is useful for syncing FunnelEnvy audiences with Segment and tracking which FunnelEnvy campaigns and experiences a user is associated with.

      Now that we are up and running and sending our Segment data to FunnelEnvy, let’s explore a few popular personalization use cases.

      Popular Personalization Use Cases With Segment Data

      The most exciting use cases with using Segment data for personalization is the ability to deliver targeted 1:1 experiences based on the user’s relationship with you and serving the most relevant offers for them.

      The exciting promise of CDP’s like Segment is they contain your best data view of your users and customers. Now by integrating Segment with FunnelEnvy we can target your users in a more personalized way and present them the best offers and experiences to drive better business outcomes.

      Let’s look at how to set up a few of these personalization use cases in FunnelEnvy.

      Personalizing for Existing Customers

      The majority of websites are static and prioritize the site experience for that new visitor or prospect. This is the right strategy for growth but it results in a suboptimal user experience for our most valuable audience, existing users and customers.

      Common sense tells you that if you can, you should personalize the site experience for your existing customers that may be coming back to your site frequently to login and use your products and services.

      In this client example, we have a SaaS business where users must login to the website to access their services. Because this client tracks active logins in Segment we can easily personalize the site for this user audience.

      Once the Segment integration is activated you can create new conditions, audiences, or goals based on any combination of Segment data. To begin, just create a new condition and select Segment as your data source.

      In the below screenshot we will create a new condition in FunnelEnvy based on the user being an active user based on Segment data. In the drop down I can pick from any and all Segment data being passed to the FunnelEnvy Platform. In this case, I select the “Is Active User?” attribute and specify it must be true.

      Once created, this new condition can be used to build targeted audiences that I can then use to personalize content to existing customers. My newly created Active User audience can now be easily applied to any of our A/B, predictive, and rules based campaigns, like in this example below:

      I can now target active users with personalized customer content the next time they return to my website to login.

      We can create simple audiences like in the example above or we can create more advanced audiences that combine multiple attributes from Segment and other data sources. In this example below, I am creating an audience based on the user being an active user AND has viewed our resource content section based on their onsite browsing behavior.

      And the best part is that I can set the audience to automatically sync with Segment as one of FunnelEnvy’s data source destinations.

      Use Next Best Offers to Convert More Prospects into Customers

      Another great use case with personalization is the idea of serving your prospects the next best offer based on where they are in their buyer journey and based on what offers they have already activated.

      While this is a very straightforward campaign in your email nurture campaigns, it’s often hard to execute on your website in real-time. Because of this challenge, most sites continue to show the same offer over and over again to prospects even if they already signed up for that offer. The end result is another missed opportunity to provide a more engaging experience to another high value audience, active prospects.

      Luckily, this challenge is easily solved, and is just another straightforward site personalization use case when you pair Segment data with the FunnelEnvy Platform.

      For this client, a product demo is the most popular entry offer for new prospects, but once they watch a demo there are a number of other offers that can help move the deal forward. They have webinars, calculators, and case studies that all work well as offers in the middle and bottom of the buyer journey.

      In this example below we want to target active prospects who have completed a demo. Our client tracks in Segment which users complete a demo based on a Registered for Demo attribute. That allows us to easily create conditions and audiences based on the user being in the demo stage, like in the condition screen below.

      With a few clicks we have created an audience of all site visitors that have completed a demo. We can now target our homepage banner to run a personalization campaign that will predict the next best offer for that active prospect from a collection of their top offers.

      This is another example where instead of presenting an irrelevant offer, we can provide a more relevant experience and personalize the next best offer for our valuable active prospect audience.

      Make Better Real-time Personalization and Attribution Decisions

      The first two personalization use cases I shared focused on creating very specific audiences based on user stage data coming from Segment. Our last use case is focused on leveraging all of that rich Segment data to make better personalization decisions and to measure the attribution impact on revenue.

      If your Segment implementation is typical of most of our clients, you likely have dozens if not hundreds of data attributes about your users coming from your customer databases, transaction systems, and various other marketing platforms. Typically, Segment has the richest and most accurate real-time data profile for your users, especially when it comes to revenue data.

      For example, did that trial user successfully convert into a paid user 30 days later and generate revenue? How much revenue has that user generated over their customer lifetime?

      Going back to our previous SaaS client example, they have a service that you typically start as a trial, and they can generate revenue from that user from both a monthly subscription fee and from specific actions they take in their account. In this case, the client tracks all revenue activities for a user with a Segment event named Generated Revenue.

      Because all Generated Revenue for a user is tracked within Segment we can easily build a goal in the FunnelEnvy platform that will update whenever that Segment event fires and populate with the actual revenue generated from that event.

      Since Segment track events become FunnelEnvy events, they can be used for goal tracking like any other event in our platform.

      In the example below, we are setting up a new goal in FunnelEnvy that fires each time the Segment track event named Generated Revenue fires.

      In real-time, every time that Segment track event fires we are now tracking that revenue and associating it back to the user and personalized experience they saw.

      With this data flowing back to use in real-time we can now unlock two powerful capabilities.

      Predicting Better Experiences to Maximize Revenue

      Because we can track the total revenue a user has generated we can now start using that data to predict which personalized experience generates the most revenue for the business, not just the most trials or initial orders.

      The FunnelEnvy personalization platform uses machine learning to predict which experience will perform best for an individual based on that user’s profile and on the history of how similar users converted on those same experiences.

      This allows us to take advantage of all the data we have as the FunnelEnvy platform will predict and promote those experiences that maximize generated revenue for each user.

      So in this case Segment data helps us convert more visitors to higher revenue generating experiences, all without the manual hassle of pre-defining segments or personalization rules. We let machine learning crunch all the data and predict the best experience for each user, all in real-time.

      Better Attribute Incremental Revenue to Personalization

      While generating more revenue is always priority one, as marketers we also need to demonstrate that our programs and campaigns are working and providing strong ROI.

      In our example with the Generated Revenue event, in addition to making better predictions, we can also clearly demonstrate the impact on the revenue we are creating through our personalization initiatives.

      With our integration with Segment, we are able to credit and report on how much revenue our personalization activities are generating and which personalization experiences covert best. We can then show incremental revenue by running a controlled A/B test that serves half the audience the control experience and the other half the predicted experiences. We then measure the difference between the two groups and report on uplift in conversion and value per visitor and calculate statistical significance.

      With the Segment integration, it is now much easier to measure and demonstrate the incremental value of personalization. Not just on the initial order, lead, or trial but on customer lifetime value (CLTV).

      This can all be done in real-time right out of the FunnelEnvy Reporting UI without any ongoing support from your IT or data team and without having to wait for delayed manual reporting.

      As a marketer, you can finally focus on optimizing your personalization strategy without all the data and operational headaches. Wouldn’t that be nice.

      Getting Started

      As you can see, integrating Segment data into your personalization program is very straightforward and can unlock some very valuable capabilities and use cases.

      If you’re not yet using FunnelEnvy but are interested in personalizing your website using Segment data we’d love to hear from you! You can contact us here: https://www.funnelenvy.com/contact/

      10 Tips for Running Effective Predictive Personalization Campaigns

      When it comes to personalization there is a growing trend with using machine learning to predict and optimize for the user experience at a 1:1 level.

      We call the collective use of these machine learning techniques as predictive personalization campaigns.

      Predictive personalization refers to a type of campaign where a machine learning model is used to predict what is the best experience to serve a visitor based on current/historical performance and the user’s individual data profile (contextual bandit). Decisions are made in real-time and at a 1:1 level and the model makes use of all the data available about that user and also takes in the context about the location, content, and other factors that go into that experience.

      The predictive campaign will send the majority of traffic to the experiences that the model predicts will perform best, exploiting those insights in real-time, but continue to hold out some traffic to continue learning and exploring performance trends for the other experiences.

      Just like with an A/B test, you can test predicted campaigns vs. a control to determine if a statistically significant uplift is achieved. But unlike an A/B test, the expectation with predictive campaigns is that it is “always on” and that it’s constantly adjusting traffic to the right experience at a 1:1 level.

      The adoption of predictive personalization campaigns is still in its early days for our industry. For many programs, experience and maturity with these techniques are still low but there is a growing interest, especially as the solutions on the market continue to grow.

      In this article we discuss best practices we have acquired from years of running predictive campaigns across a range of mid-market and enterprise clients using the FunnelEnvy platform.

      What Makes For More Effective Predictive Personalization Campaigns

      Below is a list of our observations after running hundreds of personalization campaigns over the years on the optimal conditions that contribute to successful predictive personalization campaigns.

      1. A variety of user intent. With a predictive campaign the more varied in intent and behavior that exists with users the more valuable predictive models will be in detecting and predicting better user experiences. The more varied the signal the more value predictive decisions can play a role in deciding and predicting the best outcome for a range of user intentions.

      A good example of this is with predictive campaigns on the homepage, where user intent typically varies across the board.

      1. A variety of goals to predict for. A variety of possible user outcomes to predict for is another valuable ingredient for a successful predictive campaign. Like with user intent, we want to capitalize on the strengths of predictive models, and the more varied outcomes to predict for usually results in better business outcomes.

      Good example of this is with a campaign that rotates a variety of goals or user journeys as the primary offer and call to action (CTA). Like in B2B, where you may have 4-5 different offers like free trial, request demo, request pricing, book a Drift meeting, download whitepaper, or register for a webinar. Having multiple offers to present or predict usually results in better performance of the predictive models.

      1. A variety of goal values to predict for. Multiple goals are a good thing, and having multiple goals with different values is even more helpful to your predictive campaigns. Once again we are seeking out scenarios where we have multiple goal options, all with a range of perceived value. This becomes important because we want to give the model more detailed feedback on what is working and what is more valuable to the business. If like in the previous example, you had 5-6 B2B goals but all of them were valued the same, (say $100 per goal completion), all goals would be perceived as equal, providing less signal back to the model. However, if your goals vary widely in range from $50 to $1000 per lead, then the predictive model has far more data and data points to work with.

      An example of goal variety we typically see with SaaS clients is that goal values will fall into 2 tiers, lower value content engagement goals (content download, webinars, video views), and higher value sales intent goals (request demo, request pricing, contact sales, free trial).

      1. Goals are aligned to business revenue. While goals can vary widely, it is important that your goals are either revenue goals or events that correlate closely with revenue. Ultimately, predictive campaigns do best when they predict which experience is the most valuable.Therefore it’s important that you set up your campaigns to predict high value outcomes like purchases, trial to paid conversions, MQL/SQL, and closed/won deals. Where possible avoid micro-conversion and vanity metrics as your primary KPIs whenever possible. Examples of vanity metrics would be clicks, page views, video plays, etc. Whenever your KPIs are not tied to revenue/business value the harder it will be to have predictive campaigns be effective for you.
      2. There is some version of a revenue journey that you can track and optimize for. The more diverse and complex your buyer journey is the more you will benefit from using a predictive campaign that can take in all the data and outcomes and predict for better outcomes for your users.

      If anyone has the same exact journey, there is less to predict for. However, if your a typical SaaS business and you have a variety of SMB to enterprise offerings, and you offer self service and enterprise sales scenarios, or you have a longer sales funnel that includes MQLs/SQLs/Opportunity Stages, then identifying the valuable trends across multiple goals in that revenue journey is a specific challenge that machine learning models do better at predicting for.

      1. Testing of high impact placements on the page. This recommendation is not unique to predictive campaigns and holds true for all types of campaigns, including A/B tests. If you are going to run a predictive campaign, you need to run it in a high impact location for it to be most effective.So examples of good placements would be primary offer/CTA on page sections above the fold. We have seen predictive campaigns run on small content strips or on content sections below the fold, and while they may contribute value, it’s just not going to generate a significant impact compared to continually optimizing for the high impact locations.
      2. Higher volume of traffic and conversions. Another no brainer but worth calling out. You want to run campaigns on highly trafficked pages that correlate well to conversions and revenue (think product and pricing pages vs. community or blog pages).

      You should be prioritizing opportunities where you can move the needle and truly impact revenue for your site.

      1. More available data attributes for predictions. Predictive models need data to provide relevant signals. In many of the above recommendations I call out that the more variety of intent and goal outcomes, the more data points the model has to work with for its predictions. The objective here is to generate more contextually relevant data attributes that can feed into the model. The strategy here is to instrument more meaningful data events and attributes to feed the predictive model, often referred to as feature engineering.

      Examples here would be setting up more targeted audience segments (flagging prospects and customers for example), or integrating a new data source like a firmographics API from Demandbase or Clearbit, or integrating with your CDP, CRM or Marketing automation platform, or defining content and product affinities based on user behavior and pages visited. In each of these examples we are feeding additional relevant data around user intent, behavior, or data we have about our users. All of which can help a model identify additional experience/user matches that correlate better to revenue.

      1. Stakeholders understand and appreciate the differences between A/B testing and predictive campaigns. This is an organizational consideration, but when you run predictive campaigns, there are certain differences that you and the larger organization needs to be aware of and comfortable with. Unlike an A/B test, your experiences may not be sticky to a user over the life of the campaign (optional). Secondly, predictive campaigns are designed to maximize revenue, you may not have a clear winner or a straightforward outcome or winner after running the campaign. In predictive campaigns there is often no one winner. You find that different experiences perform better for different user segments. Some organizations are very comfortable with this reality, others struggle and prefer the simpler situations and outcomes that come with traditional A/B tests where you have a clear learning and you can full scale a single winner for the entire population.This is where you need to educate your organization on the pros and cons of running predictive campaigns.
      2. Stakeholders value and are responsible for revenue. While everyone in the company will say they value revenue, not everyone is responsible for generating revenue on behalf of your business. And that is ok. But, when it comes to predictive campaigns the best outcomes are usually achieved when the primary stakeholder is aligned to revenue targets and takes the responsibility to grow those revenues as best they can with proactive initiatives like site personalization. This is where we often see the most success with our clients. Being motivated by revenue vs. just learnings, will result in a more focused approach to revenue optimization, and in many cases a predictive campaign is more effective than an A/B test in maximizing revenue for your site and user experience.

      Please keep in mind that while these are 10 preferred conditions we believe improve the likelihood of your predictive campaigns being successful, this full list should not serve a prerequisite or requirements before you launch your first or next predictive campaign. Typically, if you can align on 2-3 of these conditions you are often in good shape to see success.

      The goal of this list was to share with you lessons learned and what to look out for as you continue to grow and expand your personalization programs and are ideally incorporating predictive campaigns more often into your portfolio of marketing and personalization initiatives

      Getting Started

      Now that you know what to look out for when designing and prioritizing your predictive campaigns, let us know how we can help you maximize your personalization initiatives.

      We’d love to hear from you! You can contact us here: https://www.funnelenvy.com/contact/

      When to A/B Test and When to Use Predictive Bandits

      For the majority of marketers, when you talk about CRO and optimization programs, people immediately jump to A/B testing and assume that is the primary tactic and is the only tactic to demonstrate success for their programs and initiatives.

      While A/B testing is a critical tool in your optimization program it shouldn’t be the only option on the table. While A/B testing is a valuable solution it is not a one size fits all solution for all your business challenges.

      With advances in new data capabilities, marketing technologies, and real-time computing power in recent years, there are now more ways to solve for the same optimization problem.

      For this discussion, we want to focus on two different techniques.

      A/B test – A controlled experiment where traffic is split across 2 or more experiences and visitors are randomly assigned an experience and they stay in that experience for the duration of the test. At the end of the test, you determine a single winner based on which experience generated the best outcome for your primary KPI based on a predefined sample size/test duration and ideally reaching a specific statistically significant threshold. You then stop the test and full-scale the single winner for all traffic either directly in the testing platform or hard-coded into your CMS/platform.

      Predictive Bandit – An experiment where traffic splits are not equal and where visitors are not randomly assigned. In these campaigns, a machine learning model predicts what is the best experience to serve a user either based on current + historical performance (multi-arm bandit) or based on current/historical performance + the user’s profile (contextual bandit). Like an A/B test, you can run the predicted experience vs. control to determine if a statistically significant uplift is achieved. But unlike an A/B test, the expectation with bandits is that it can run ongoing.

      Now that we better understand the two techniques let’s explore reasons why both have a place in your optimization strategy.

      Pros and Cons of A/B Tests

      A/B testing is embraced in the analytics community as the smart and scientific way to measure the impact of change. Controlled experiments are used in testing the effectiveness of new medicines, academic/scientific studies, and of course in marketing. A/B testing is not new to marketing. It’s been done for decades in the direct mail/direct response world. And has become the recommended way to test changes on your website and mobile app assuming you can convince your stakeholders that testing is needed within your organization (it’s 2020 but still some organizations resist).


      1. Quantify the impact of changes on your site. Don’t leave change to chance, measure and quantify the positive and negative impact of changes in experience with confidence.
      2. A/B tests are well understood in our industry. And for the most part well understood across an organization. Statistics may not be, but in general most folks understand the approach. In the end, if everything goes according to plan you have a clear outcome; either control won, or one of the challengers is declared the winner.
      3. Clear learnings. Related to the 2nd benefit of being well understood, benefits of an A/B testing is not just impact on business results, it’s shared learnings of what may or may not work for your site/business. With A/B tests you ideally gain a better understanding of your visitors and customers.
      4. At scale, you can create an organizational culture of experimentation. This change in culture leads to more creativity, risk taking, and better data driven decision making. Typically organizations that test more, make better and more informed decisions for their organization.


      1. Success rates will vary, but are often low. Industry averages place the average success rate of A/B tests at 30-40%. So you have to expect that the majority of the time you are not going to end up with a new winner. Even within the winners, even fewer are high impact wins. The one hidden benefit to this, is that it makes the argument of why A/B testing is so important. If we didn’t test, the majority of the time those great ideas we think will perform better, actually perform worse or have no impact.
      2. One size fits all and the winner takes all. When you analyze your typical A/B test you will often see at a segment level, different groups convert very differently across the variants. At the end, you pick the variant that was best on average across all your traffic and full scale that one. But in doing so you do leave some money on the table, as the winner will not be the winner of all segments.
      3. Results can and do change over time. If you look at performance trends over the life of the test, it’s often the case where test results trend up and down over time. Guess what, that data reality continues to occur even after you hard code and full scale the winner. You can lock in the experience but you can’t lock in the results going forward. Results will continue to change, and a decent percentage of the time, as tests continue over time you experience a regression to the mean, where results start to flatten out. After all, if you run the same test twice, you will seldom get the same results. This is why, for many programs when you full scale that winner you usually don’t experience that lift ongoing. It still is a better way to make decisions, but as we know audiences and behaviors change. Seasonality can also be a factor.
      4. You sacrifice business value for concrete learnings. By design, an A/B test is designed first and foremost to generate a solid learning. Usually you are willing to sacrifice short term uplift if a variant does better than the rest, and willing to suffer through some short term downside if a variant clearly underperforms. You are willing to accept a sub optimal impact on revenue during the duration of the test because you are prioritizing a clear test result for short term business benefits.
      5. You need traffic. Not every organization can run A/B tests. Sufficient traffic and conversions are required to reach a statistically significant outcome. Not all sites have that, especially in B2B for example.
      6. Operational costs can run high. From selecting a testing platform, bringing on web developers, additional creative, and data analysis, the tool and people costs can be meaningful. Plus, there are the operational costs of introducing more time and resources to launch something, and the occasional negative costs from broken experiences or flawed tests. All marketing teams and programs incur people, tools, and operational costs, and testing is no different and often carries more cost overhead.

      While I go into more detail on the cons with A/B tests, I do some because some of the cons are less understood. Still, when done right, A/B test is worth the effort and the benefits will far outweigh the cons.

      But as I hopefully outlined above, A/B tests do have different strengths and weaknesses.

      Pros and Cons of Predictive Bandits

      Bandits have gained more popularity in marketing in recent years as computing power has advanced to the point where real-time machine learning predictions can be applied in more and more marketing technologies and for more use cases.

      We have seen the trend emerge in digital advertising where bid and creative recommendations are often driven by machine learning decisions.

      However, when it comes to site optimization the adoption and acceptance of bandits as a proven technique is still in the early days. For many programs, experience and maturity with these techniques are still low. Even though some of the most popular testing platforms have included those capabilities for a number of years. Let’s discuss why.


      1. Bandits are by design biased toward business outcomes. Unlike A/B tests which are designed to maximize time to a clear learning. Bandits are typically designed to maximize the business outcomes at the expense of clear and precise learnings. The algorithms typically send more traffic to experiences that perform best, and route traffic away from experiences that underperform as a whole or for a specific audience segment.
      2. Bandits use all the data to your business advantage. In A/B tests you may use data to inform and drive your test hypothesis, but when it comes time to setup your test, you typically set it a controlled randomized experiment where traffic is split evenly and your visitors are randomly assigned to one of the variants during the duration of the test. Again, this is absolutely the right way to run a controlled experiment to generate your best chance at a concrete learning. However, this also means you are ignoring the data and trends that live within a test and the micro trends of which traits and segments perform best for each variant. With Bandits, machine learning is consuming and using all available data on the variant performance and the user to determine the best possible user experience for that user to drive the optimal business outcome. It’s far from perfect, but you are not leaving the outcome to chance. You are using all the data you know about the user and situation to make the experience decision to maximize your business outcomes.
      3. Bandits adjust to changing trends and behaviors. With Bandits, the intention is for it to be always on and constantly adjusting to the latest performance trends of the campaign. Unlike an A/B test where you pick one winner and lock it in, a Bandit can adjust as results change over time, and minimize any loss in performance and capitalize on any shifts in winning experiences.
      4. Bandits can work with less traffic. Because you are optimizing for revenue instead of learnings, you can still benefit from bandits even though you have less than optimal traffic to run a clean test.
      5. Bandits work well in time sensitive situations. Popular content and Black Friday sales are good examples. By the time an A/B test gives you the right answer the opportunity may have passed you by. With Bandits, it reacts in real-time to the trends and that allows you to take advantage of short term and seasonable situations.


      1. Bandits are hard to interpret, understand and communicate. While A/B tests are well understood, bandits are not. The fact that decisions are controlled in real-time by a machine learning model vs. set A/B splits, means users do not have certainty about why an experience was shown or control over the traffic rotation.
      2. Bandits offer limited learnings. A/B tests by their design as controlled experiments are designed to produce learnings. Bandits will typically sacrifice clarity of learnings when it comes to which experience works best. You can often be informed about what segment or what feature influenced the bandit model decision, but it’s not as clear cut on what is the final absolute winner. Often you are A/B testing the technique, do bandits outperform control or an A/B test for this page/site. But with bandits it’s harder to generate clear learnings on winning experiences. In the end you are optimizing for revenue and other business outcomes at the expense of a clear isolated learning. As an organization you need to be aware and accept this reality and your stakeholders need to accept this reality too.
      3. Bandits can provide an inconsistent user experience. As we called out earlier, to maximize business outcomes, bandits by design do not make experiences sticky to the user, and will often serve a different experience to the same user if the data suggests it will result in a better outcome. While this maximizes revenue, it can lead to experiences being more dynamic and changing for a given user. While dynamic websites are generally considered a positive, because it is not very common in 2020 (surprisingly), this dynamic approach to site experience can be a drawback for some users.
      4. Bandits require more thoughtfulness on experience design. While you can A/B test most things from layout, to copy, to color, to offers, the same isn’t always true for predictive bandits. Bandits work best where you have varied user intent and ideally varied offers and outcomes. Bandits are more effective if they can predict for more outcomes for a larger range of user intent. If you are running CTA color and size changes you are better off running a traditional A/B test. There is likely less signal in the data in terms of user preference of a button color/design and the results will likely hold true on average for most users. That is why bandits work better when intent varies across the visitors and potential offers displayed can also vary.

      When to Use A/B Tests and When to Use Predictive Bandits

      Now that you know the PROs and Cons for A/B tests and predictive bandits let’s talk about some practical applications of each and when you should one over the other.

      When A/B Tests are Recommended

      1. When you need a clear learning and more certainty on the final decision. Examples here would be a pricing or homepage test.
      2. When you want to lock in a specific design or experience for all. Examples here are things like a new homepage design, a new form layout, or say our site wide CTA treatments. Here there is more operational value in locking in that specific winner and then optimizing further on that.
      3. When intent and offer options are narrow in scope. When intent for all users is similar the offer is the same for all, then A/B testing usually works better than predictive bandits. An example would be optimizing for the final cart checkout page. Everyone there is ordering (or not) and the offer is a checkout (or not).

      When Predictive Bandits are Recommended

      1. When you want to maximize revenue. If you have aggressive revenue goals then bandits are a better tactic to get you there as you are using all the data available to make the optimal revenue decision. A good example here is presenting the right offer on a homepage hero or promoting a specific price/package on the pricing page. In those situations, the general site is the same but you are predicting the best offer and experience to spotlight to maximize revenue.
      2. When intent and offer options vary broadly. Predictions work better when there’s a wider range of users and intent and a wider range of offers and outcomes to present. This is where the value of machine learning and crunching dozens and hundreds of data attributes in real-time is helpful. Good examples here can be homepage, brand landing pages, and pricing/plan pages. In these scenarios intent and options/offers presented can vary.

      Now that you know the strengths and benefits of both tactics I think you will agree that both tactics should be part of your optimization and personalization toolkit.

      Running A/B Testing and Predictive Campaigns in FunnelEnvy

      With the FunnelEnvy platform we give you the ability to run both A/B and predictive campaigns and apply the right tool for the job. Unlike other platforms, both tactics are available as part of our standard license.

      To get started you just create a new campaign and then select the preferred template option between Predictive and A/B testing.

      If you selected an A/B testing then the “Campaign Decision” section will default the settings typical of an A/B test.

      As you can see A/B testing is the decision type defaulted, and the “Persistence Variations Decisions” feature is checked so the same experience is served to the user across sessions. Lastly, you can adjust the traffic allocation to determine what percentage of traffic enters into the test. By default and typically it’s set to 100%.

      If you were to select a Predictive campaign template, the decision type would instead default to Predictive as seen in the screen below.

      In addition, the “Persist Variation Decisions” feature is not selected by default. As mentioned earlier, we typically see improved revenue performance when the campaign can rotate different offers over time to the same user. That makes sense, as a user may not respond to an initial offer, but maybe convert when presented another. But we do recognize their legitimate reasons to also make predictive decisions sticky, especially when running in locations like your pricing and plans page. And like with the A/B template, you also can set traffic allocation from 0-100%.

      What is specific to Predictive campaigns is the predictive experiment options. Here you have two choices to make. The first choice is what percentage of traffic should be included in the predicted group vs holdback. The experiment section is where you can test the incremental value of a predictive campaign by holding back a portion of your traffic as the control.

      In the screen below we have set the holdback to 50%. This will result in 50% of the traffic being assigned to the predictive experience and the remaining 50% gets served a control. This allows you to test the incremental value of running a predictive campaign on your site.

      Typically, we recommend the client start a new campaign as a 50/50 campaign and as they see the predictive campaign outperform the control we then recommend full scaling the predictive campaign to 100%.

      The second option you have with a predictive campaign is to determine the composition of your holdback group. You can choose between assigning the variation a specific variation, like baseline control, or you can set holdback to random in which case the holdback will run as an A/B test for that traffic across all the available variants. The “Random” option is useful if you want to determine the incremental uplift in running a predictive campaign vs. an A/B test.

      As you can see, our predictive campaign setup still allows you to isolate and measure the incremental impact of a predictive bandit approach by running it as a controlled experiment.

      Our typical client will often run both types of campaigns simultaneously on different areas of their site and targeted for different audiences. Rather than having to choose one tactic or another, FunnelEnvy clients have the best of both worlds.

      Getting Started

      As you can see, A/B testing and predictive bandits are both valuable tactics in your optimization/personalization programs. Like any tactic you need to pick the right tool for the job.

      If you’re not yet using FunnelEnvy but are interested in personalizing your website using a combination of A/B tests and predictive campaigns we’d love to hear from you! You can contact us here: https://www.funnelenvy.com/contact/


      Real-Time Personalization with Marketo and FunnelEnvy

      For many organizations, Marketo serves as the real-time customer database for marketing. Unfortunately, for most organizations today this rich intelligence living in Marketo is not being leveraged to drive personalized user experiences across your site which is one of the most valuable opportunities with this data.

      The good news is that when it comes to personalizing with Marketo, you don’t have to be limited to just personalizing your emails and Marketo forms. You can actually use all that valuable customer centric Marketo data to drive your website personalization programs.

      Why might you want to do this? Instead of showing everyone the same lead capture experience, you could show prospects who have already filled it out more product content. Or show existing customers opportunities to expand. Maybe even segment your experiences and customer journey by company size or industry.

      With FunnelEnvy’s Marketo integration you can use your rich Marketo data in real-time to deliver personalized experiences across your site.

      Setting up the Marketo Integration in FunnelEnvy

      Within the FunnelEnvy user interface you can activate and configure the Marketo integration. FunnelEnvy fetches Smart Lists periodically from Marketo and automatically keeps these updated with Marketo. Configuring the integration also lets you setup offsite goals triggered by Marketo webhooks such as Marketing Qualified Leads (MQLs).

      The Data Filtering interface lets you choose which fields to import, and exclude PII or other data based on your compliance policies.

      Typically these four steps are done by the Marketing Ops team that manages the Marketo instance:

      1. Activate the Marketo data source.
      2. Authorize FunnelEnvy to access Marketo
      3. (Optional) Configuring Data Filtering
      4. Selecting Smart Lists to Import

      Step 1: Find and activate Marketo under the Integrations settings. You should see it as an activated Data Source.


      Step 2: Authorize FunnelEnvy to access the Marketo REST API with API keys.

      Step 3: Optionally configure data filtering rules. When fetching lists FunnelEnvy will only import lead attributes that are selected.

      Step 4: Select Smart Lists for Import. Assuming your API credentials in Step 2 were correct, you should see a list of Smart Lists available for import. Note that it may take up to an hour for this list to reflect any recently added Smart Lists.

      Once you’ve configured the Smart Lists for import you’re done! FunnelEnvy will refresh the lists every few hours, retrieving leads and refreshing the local copy of Marketo data, which is then available immediately for audiences, predictive campaigns and offline Marketo-triggered goals.

      More details on setting up the integration can be found in our knowledge base article.

      Using Marketo for Site Personalization in FunnelEnvy 

      Once you’ve configured the Marketo Data source you open up a number of valuable personalization use cases. Below are three ways you can use FunnelEnvy and Marketo together to better target, personalize, and measure your personalization initiatives.

      Target Experiences and Offers using Lead Attributes and  Smart Lists

      Stop serving a static one size fits all website experience to all your visitors. Want to personalize your site experience only for prospects, or to specific accounts, or members of specific campaigns? 

      With FunnelEnvy you can create very rich audiences that can be built off Marketo data and that can also be used as part of more advanced audience segments that combine Marketo data with firmographics and/or real-time user behavior as well.

      In the condition builder interface you have access to all of the Marketo lead fields that were imported, and can define logical conditions based on them.

      These conditions can also be combined with other data sources. In the audience screenshot below we’re combining a Marketo condition with a user’s behavior (but this could also be Demandbase, Clearbit or any of the sources we support). 

      And just like any of the FunnelEnvy audiences, these can be used for targeting within predictive campaigns or A/B Tests:

      This flexibility allows you to setup a dynamic “always on” personalization strategy that targets the right user segments in real-time based on that visitor’s stage and their relationship with you.

      Personalize Experiences at a 1:1 Level with Marketo Data

      While targeting is a powerful first step in executing your personalization strategy, the more powerful opportunity is to use all that rich user data to predict the best experience to serve each visitor. 

      Choosing in real-time which experience to serve each user based on their full user profile truly allows for 1:1 marketing. That is where the personalization magic really happens.

      FunnelEnvy uses machine learning to predict which experience will likely convert best based on all the data we see for that user, including their Marketo data and based on the history of how similar users converted over time.

      And unlike A/B tests where a specific experience is randomly assigned, or rules based personalization where you fix a specific experience to an audience, FunnelEnvy allows you to take advantage of all the data you have on that user and serve the experience mostly likely to convert for that user.

      This allows you to avoid the manual analytics effort of trying to identify and capitalize on all the possible experience and segment combinations that perform best. As a marketer you can stay focused on the message and offer and allow the algorithms to optimize the segment/experience matches.

      As the report below shows, we are scoring/weighing the effectiveness of every attribute we see for every user by experience.

      Here, Marketo audience data along with all the other behavioral and firmographics data is used to predict the best possible outcome for each and every user and experience combination.

      This allows us to use all the data to our advantage and serve the right experience that will most likely result in revenue. 

      The best part is that there’s no additional setup required here. Once we have the Marketo data within our profiles we’ll use it as long as the decision mode on your campaign is set to “Predictive”.

      Measure and Attribute Personalization Campaigns by Revenue (not Form Fills)

      With personalization, one of the bigger challenges is being able to measure the program’s contribution to revenue and business outcomes. 

      It can be done, but often requires integrating data sets or pulling reports from multiple systems and generating manual reports after the fact.

      WIth FunnelEnvy, once you set up your important online, MQL, and any other revenue goals you then start tracking and attributing success to each personalized experience. Below is an example where we created a MQL goal based on a Marketo List and assigned a specific MQL value to it.

      To setup this, ensure that the Marketo Data Source is activated and configured and create a new individual goal. Under “API Triggering” you’ll should see an option for Marketo. Once selected, this is the URL that your Marketo instance will hit via a webhook to trigger the goal conversion. More details on setting up these webhooks is available in our knowledge base article.

      Once that’s done the Marketo goal will shows in real-time in our campaign reporting dashboards.

      It now becomes much easier to tell the story of how specific tests or personalized experiences are driving down funnel goals like MQLs, SQLs, opportunities, and deals won in addition to top level goals like trial signups, demo requests, or engagement.

      This makes it much easier to attribute the positive impact personalization has on the organization’s revenue outcomes. Now instead of talking about form completes you can talk the language of sales which is revenue.

      Getting Started

      As you can see, integrating Marketo into your personalization program is very straightforward and can unlock some very valuable use cases and capabilities. The best part with this approach is that there is no custom development or IT involvement to get this up and running. You can setup the integration and be live with your first campaign on the same day.

      If you’re not yet using FunnelEnvy but are interested in personalizing your website to Marketo Leads and Contacts we’d love to hear from you! You can contact us here: https://www.funnelenvy.com/contact/

      Personalizing the Revenue Journey with Segment Data

      Accelerate your customers journey to revenue with FunnelEnvy, now powered with Segment.

      Segment helps their customers instrument, store and unify data about their visitors and the actions they take all the way to revenue. Now with the FunnelEnvy Segment integration you can deliver personalized, 1:1 website experiences and optimize for revenue using all of that rich customer data that you’re already collecting in Segment.

      What does this mean? Segment customers will be able to run more effective campaigns using better data with less custom code required.

      Check out our integration on Segment


      Go to Top