horizontal experimentation across product teams diagramhorizontal experimentation across product teams diagram

| Tech & Development

The future of product experimentation: How we create highly scalable experimentation architectures

Since our inception, Creative CX have made it our business to partner with a broad range of the best technologies in our field. Optimizely has been one of our strongest partners during that time, and our collaboration is rarely more impactful than when delivering large-scale experimentations in a multi-touch-point customer journey.

This May, I will be one of a handful of industry experts, invited by Optimizely to discuss our most exciting new service to date. One which we believe represents the future for large scale experimentation architectures.

It is undeniable that for most large organisations their customers operate horizontally across multiple parts of the business, and with that, multiple departments within your organisation. It has become the norm to begin a transaction on one device, and finish it elsewhere at your convenience. 

Whilst businesses understand this, most product teams are aligned vertically, and therefore their capacity for experimentation is aligned vertically too. 

Our process for scaling with Agent allows your business to experiment not only vertically, but horizontally too, through the lens of the customer. That’s why we believe this new implementation could represent the future for experimentation at scale.

Moving between experimentation silos across the customer journey

As Creative CX clients grow and reap the rewards of successful experimentation, their ambitions often grow too. Maturing companies push the boundaries of experimentation, taking more risks.

Experimentation SDKs present a detailed but siloed approach to experimentation at this point, which is limiting for businesses that want to run end-to-end experimentation at scale.

It is here where features such as Optimizely Agent comes in. By moving experimentation up the agenda into the build phase and by taking advantage of the powerful capabilities of Optimizely, alongside correct governance and best practices delivered by us, our clients can run experiments which pass through the borders of one customer touch-point to another.

To put it simply, this is the implementation for businesses who interact with their customers through multiple touch-points and are ready to run experimentation across all of them.

When is a single SDK not enough?

For many organisations, mainly single-platform businesses or those focused on high-volume experimentation and optimisation of single applications the use of an SDK can be a fantastic method to get the job done.

However, if your business interactions with your customer move beyond one platform into mobile apps, separate pricing systems or detailed search functionality, it’s time to move on from SDKs or risk creating severe headaches.

As illustrated below, the logical conclusion as you add more touch-points, can be to add more SDKs too. Whilst this approach will allow you to drill down deeply into each element of the journey, the reality is that none of these SDKs will be able to talk to one another, which means you’re creating experimentation silos within each element of your journey. Not only that but with the added engineering resource required to run each one, as well as other overheads, things will get expensive quickly.

experimentation within siloed product teams diagram
Diagram 1: Siloed experimentation, restricted by local SDKs deployed at Product Team level

The up-front investment your business will require

Horizontal experimentation requires considerations at 7 different levels in order to be done as effectively as possible, and to scale for the long term.

  1. Discovery
  2. Business Requirements
  3. Technical Design
  4. Technical Governance 
  5. Technical Implementation 
  6. Enablement at Scale
  7. Technical Centre of Excellence Hub

These areas are the pillars which Creative CX have used to form our Agent Experimentation Package, at the core of cross-touch-point experimentation. We have found that by taking the time to map business requirements to best-in-class governance and technical support, we can create a ‘Technical Centre of Excellence’ package that supports sustainable growth and action from experimentation over time.

Breaking down barriers and looking across the journey

Our new approach, pioneered by the Creative CX team, brings experimentation in at the structural level, which not only removes many of the issues of maintenance cost, but destroys the siloed effect of SDKs and eliminates resource black holes.

Deliver experimentation on your customers’ terms

The customer journey is often nuanced and can rely on many stages. What’s more, your customers won’t see beyond their own experience interacting with your business (and nor should they). Our industry-leading approach to vertical & horizontal experimentation views the world through the eyes of your customer rather than your Engineering department.

We believe this approach will lead to insights and delivery that can take your business’ experimentation to new heights.

horizontal experimentation across product teams diagram
Diagram 2: Global service enabling horizontal experimentation across Product Teams

Nelson Sousa – CTO & Experimentation at scale expert

Nelson is CTO at Creative CX, and a part of our founding team. During his 5 years at the business he has pioneered some of the most exciting customer projects in the experimentation space. In his experience of more than a decade, he has been at the heart of change in the world of experimentation. Our projects for scaling experimentation across your organisation are his brainchild, and his inclusion in Optimizely’s expert panel of speakers is evidence of his accomplishment in the space.

Interested in working with us?

Book a 30 min call