Artificial Intelligence and CRO

William Rodil

There’s a great deal of talk and speculation about artificial intelligence these days. Some say it’s the next best thing since sliced bread, others say that it’s the first step towards the annihilation of the human race. These views are incredibly extreme and whilst I see how both scenarios could be considered correct, I think it’s time we look at what impact artificial intelligence and machine learning will have on the discipline of Conversion Rate Optimisation.

Before we delve into AI, I want to talk about the HIPPO for a moment. We all know what this means and we have talked about experimentation as a means of negating the HIPPO effect within our organisations. But, by doing so could we as CRO practitioners have turned into unsuspecting HIPPOs? Take for example the reactions that I have often witnessed from fellow CRO practitioners, when asked if they would like to use technology that automates the process of declaring a winning variation of an experiment, which can nudge more traffic towards a particular variation based on machine learning: “Oh no, we couldn’t possibly use a machine to make such decisions, they can’t be trusted”. I for one, beg to differ.

Artificial intelligence isn’t about to take our jobs nor is it about to take over the world anytime soon. The technology isn’t built to replace the creative element of our jobs it’s here to supplement the existing approach to CRO in order to improve its effectiveness. Sure, AI can’t write convincing emails, poetry, songs or anything that is designed to invoke an emotional reaction form a human being. That job belongs to people and that is the way it should remain. However, a machine can tell you with far greater accuracy which variation of the design does invoke the desired reaction and do so much faster and much more accurately than a human being could ever do – even a marketer! It can also determine if the wining variation has fallen out of favour with certain individuals that have been exposed to the variant previously and adjust accordingly.

Given the right amount of time and data an algorithm can be used to determine what the optimum variation of a given experiment is, what the optimum price point is as well as other things such as optimum CPC (Cost Per Click) that’ll drive the lowest CPA (Cost Per Acquisition) and highest ROAS (Return On Ad Spend) when it comes to acquisition marketing or what is the optimum time to send emails for CRM/loyalty management. However, a marketer or a product lead still needs to build the variants, put together the content, create the experience and define the parameters, thresholds and goals that the machine will strive towards.

The best thing about handing over control to a machine is that a computer can process vast amounts of data in an instant, provide near real-time feedback and lots of it for that matter.

Take the concept of website personalisation. No, it’s not a dark art nor does it need to be complicated, it can be simple and very effective. Let me explain using a simplistic analogy. Take our democracy for example, the candidate that has the highest number of votes wins and is elected into office. In essence, the majority are represented and their needs are often put above everybody else’s. Wouldn’t it be great if all the candidates could win and get together to form a coalition that aims to combine and address the needs of the people? Well fortunately, when it comes to your website this idealistic concept is possible.

Rather than declaring A as a winner in an experiment just because 60% of those who converted were exposed to A and only 40% were exposed to B. You can use technology to split the traffic so that users who exhibit similar characteristics are placed into one segment and exposed to one variant, whilst other users can be exposed to a different variation that is in keeping with their needs. This approach works if you can look back at your data to determine your segments and variations against these, but what about users who are brand new to your site where you have no prior knowledge about their preferences?

This is where AI comes in to place. My former employer, Adobe has developed a feature in its testing platform called Automated Personalisation and many other suppliers offer something similar. A use case that I often talked about with customers and still do is as follows. If you knew what your users looked like on paper i.e. you had persona definitions you could start creating experiences that you know would resonate with each persona group and run the algorithm to target the correct variation to the right segment.

Imagine for a moment that your persona definitions relate to the following:

  • High Value customer, loyal (affluent young professionals)
  • Ad-Hoc customers, purchase as and when (average income, young professionals)
  • Bargain Hunters, always looking for a good deal

Rather than having a rotating carousel as your home page hero you can now make effective use of this real-estate by inserting one of 3 variations designed to appeal to would be users of each of the prior definitions, let’s call them A, B & C as follows:

  • A – Contains an offer/content aimed at high value customers
  • B – Contains an offer/content aimed at ad-hoc customers
  • C – Contains an offer/content aimed at bargain hunters

To keep things simple, let’s also imagine that the goal here is to increase site conversion rate. Once these variations have been configured, the machine could take over control of the distribution of each of these variations to determine who to serve what to. The algorithm looks for patterns within the data between users to make an informed decision in real-time as to which variation to serve. What happens behind the scenes is it looks at all the users who converted that were exposed to each variant and produces a report that exposes the top 5 predictive variables, that influenced the machine’s decision to serve each variation, applying a digital footprint to each paper based persona definition. Armed with this insight, a marketer can further personalise the journey with greater effect. See below for a graphical illustration:

Whilst this is a rudimentary depiction of how AI can be used when it comes to CRO its potency is not without question. Customers that I have worked with who use this technology (or its equivalent) have enjoyed significant increases in conversion rates and would all agree that its power has delivered significant returns to their business.

It goes without saying, that AI is not the answer to everything, nor is it here to replace each of us. From the diagram above, a highly skilled human being still needs to quantify each of the customer profiles outlined above, design and build each variation for the experiment and in turn make an informed business decision as to how to proceed once in receipt of the information provided by the experiment.

Rest assured, our jobs and our very existence is safe for now…


Let's talk!


61 Queens Street,
London, England, EC4R 1AE

+44 (0)20 7682 2820