Helping MyVoucherCodes to establish a highly successful Experimentation programme

The Challenge

With over 2,500 brands, MyVoucherCodes (MVC) is the UK’s #1 online destination for free vouchers, discounts and deals. Their platform helps millions of members and users to find the best deal.

MVC understood the huge potential of optimisation and continual innovation through experimentation and had already invested in the Optimizely platform. However, due to the size of the team and limited resources, they needed the right consultancy to help them fulfil their ambitions.

After a thorough selection process, they chose Creative CX due to their experience in experimentation and considerable technical expertise.

MVC set Creative CX the challenge of:

1. Supporting the team with building a high performing experimentation programme.
2. Delivering excellent results for the business to prove the value of experimentation.

What we did

1. We started the process with technical onboarding where we validated and improved the set-up of the experimentation platform and integrations with analytics, to ensure that there were minimal data discrepancies.
2. We then worked very closely with the MVC team on the strategic framework, the experimentation processes, and building the initial roadmap drawing upon all of the team’s knowledge of the brand and their customers.
3. We then began the process of delivering a fully managed experimentation programme including ongoing strategy, ideation, design and development.
4. Alongside the fully managed experimentation programme, we helped the team to get going with their own parallel stream of experimentation. Supporting the team with the more complex builds, metrics tracking, and providing advice and training on strategy and development to ensure that they were continuing to learn and grow. We also provided training in the more advanced side of experimentation development so that they could build tests from start to finish and ultimately be self-sufficient.
5. Finally after 12 months of working with MVC the internal team were fully trained and enabled so that they could continue on their own without our fully managed support.

Three reasons why the programme was successful

1. Good understanding of user behaviour

One of the most important factors was understanding the unique behaviour and mindsets of users arriving on the Myvouchercodes website. Understanding where they were coming from and what their particular goals were was critical for the success of the programme.

The MVC team was already conducting remote usability testing using the Usertesting.com platform and this insight along with our analysis of Google Analytics and Hotjar gave us the understanding that we needed to generate powerful evidence-based A/B/n test ideas.

2. A willingness to be creative and take risks

The level of creativity and risk that the MVC team was open to taking was a key factor in the success of the programme. The average number of variations tested against the control per experiment was 3.77. This allowed us to maximise the learning and also include wildcard variations that again enabled us to take bigger risks and explore different directions.

Both this level of insight into user behaviours (i.e. the number of evidence-based tests) and the risk/creativity (i.e. the number of variations) were two key factors in how we were able to achieve a 39% win rate on the business metrics.

3. Strong focus on proving the business value

As experimentation was still fairly new to the business, proving the business value was incredibly important. Therefore in order to be extra confident in the incremental uplift that we had already calculated from the process of rolling out winning variations, we went one step further.

We created a large validation experiment where we combined several of the previous winning variations into a single multi-page A/B/n test to test against a rolled-back version of the website. The result of this test was conclusive and actually even more positive than we had first predicted from the initial individual experiment results.

The Results

39% win rate & 12% site-wide uplift in business metrics

Due to the level of insight into the customer problems we were addressing, the great collaboration with the MVC team and a lot of creative and inventive thinking, we were able to achieve a win rate of 39% (on business metrics) and an overall site-wide uplift to the most important metric to the business of 12%.

The MVC team fully self-sufficient in experimentation

After 12 months of working with MVC we had done what we set out to do which was to prove the value of experimentation and make the MVC team self-sufficient when it comes to experimentation.

Testimonial

“Working with the CCX team expanded my team and I’s horizons and imaginations when it came to experimentation. We learnt that ideas are cheap and plentiful, but observations and data are the best guiding principles when it comes to helping your users to engage with your product to gain maximum value.”

“The CCX team worked with us to research our industry, our product offering and our users and then worked with us to build everything from simple, to highly complex experiments, some which gave us amazing results and others incredible and surprising learnings.”

“All in all, CCX equipped us with the skills, knowledge and mindset so that we could become fully autonomous in CRO/Experimentation.”

Richard Shnieder, Senior Product Manager