This was my first time at ConversionBoost and also my first time in Copenhagen. Safe to say the conference lived up to the hype, and the city was beautiful too.
The day before things officially kicked off, I got the chance to see a bit of Copenhagen with Lucia van den Brink and Ruben de Boer as our impromptu tour guide.
It was lovely to have a wander through the city with someone who has lived there and who knows his stuff, not just about experimentation, but also where to find a decent coffee! Lucia and I are both on the board of Women in Experimentation, but usually talk from behind a screen, so catching up in person and seeing her deliver her talk was a highlight.
This blog article pulls together the main takeaways from each of the talks and a great case study from Elkjøp. The ideas, the surprises, and the things that made me rethink how we’re doing experimentation in 2025.
There was plenty of talk about AI (as expected), but also some brilliant sessions on team culture, testing velocity, and the value of involving more people (and more types of people) in the process.
If you missed it or want a refresher, here’s what stood out:
Ruben de Boer – The Experimentation Mindset: How We Excel at Websites but Overlook Our Organisation
Ruben de Boer on stage at Cboost, sharing insights on applying the experimentation mindset
Ruben de Boer kicked things off with a high-energy call to arms for anyone who’s ever thought, “Why does no one else in my company care about experimentation as much as I do?” (Spoiler: it’s not just you.)
This wasn’t your typical “top 10 CRO tips” talk. Instead, Ruben turned the lens inward. We’re brilliant at optimising websites using research, data and iterative testing, but when it comes to applying that same approach inside our organisations? Not so much!
From Free Lunches to Dead Silence
Ruben shared his early efforts at “I Amsterdam,” where he tried the classics: update emails, lunch & learns, and even a “Win a bottle of wine” test idea competition. Initial engagement was promising because who doesn’t like a free lunch or free wine? But it quickly fizzled. Why? His colleagues didn’t see the point in changing their ways, and Ruben wasn’t yet using his own experimentation mindset internally.
Stakeholders Are Users Too
Stakeholders are just another kind of user and deserve the same curiosity and rigour we bring to digital UX. He introduced two frameworks for mapping your organisation:
Formal landscape: The org chart colour-coded into supporters, neutrals to win over, and those actively against experimentation (Ruben used the other speakers to demonstrate this, which was amusing as I hope they are all strong supporters!).
Informal landscape: A “Force Field Analysis” where you map people, relationships, and influence. Then, empathise like a UX researcher: What motivates them? What blocks them? What’s their perception of CRO?
Bonus tip: if someone’s blocking your efforts, don’t get mad, get strategic. Find shared ground (like a mutual love of Copenhagen), use allies, and be curious, not combative.
Avoiding the ‘Frustration Cycle’
We often see a CRO specialist who becomes increasingly pushy because “the data proves it”, while colleagues dig in their heels. Ruben calls this the Frustration Cycle: a death spiral of facts, resistance, and internal shouting.
Instead, he proposes the Collaboration Cycle:
Conduct “colleague research” the same way you’d do user research
Personalise your updates and insights so they’re useful to your audience
Test Internally Like You Test Online
Ruben wrapped up with a powerful reminder: experiment your way to organisational change. That means:
Prioritising key stakeholders (but not always starting at the top)
Testing different communication approaches
Gathering feedback and adjusting
Learning what works in your context, not blindly following blog posts or best practice lists
He shared charming examples, like placing a live GA dashboard by the office entrance (so even the execs got curious) and helping a marketer hit their goals via CRO to gain internal allies.
Favourite Quote:
“Experiment in your organisation just like you experiment on your digital products.”
Lucia van den Brink – The Unexpected Experimentation Bonus
Lucia van den Brink on the stage at Cboost to explore how to support creativity and ideation
Lucia’s talk wasn’t just about CRO wins, it was about the humbling and occasionally upside-down process of making them happen. Quite literally, in fact: she opened with a YouTube story about why 5–10% of videos were mysteriously uploaded the wrong way up.
Turns out, the answer came from a left-handed user and that moment perfectly captured the crux of her talk: the best ideas often come from unexpected places.
Lucia’s session focused on what she calls the “Experimentation Bonus.” This is the measurable value of involving more people, with diverse perspectives, in your experimentation programme.
At CCX we have been talking about the benefits of wider collaboration for some time, Georgiana Hunter-Cozens spoke at the TLC Mentorship Summit last year on the subject of “Recruiting Outside of The Bubble” – you can watch her talk [here]
The Data Doesn’t Lie (Unless You’re the Only One Looking at It)
Lucia drew from her own in-house and agency-side work to share case studies where expanding the circle of idea contributors led to significantly better win rates.
One programme saw her own win rate hover around 22%, while the collective team, which included everyone from brand managers to SEO to paid ads, managed 28%.
In another B2B case, her personal win rate was 9.3%, but when a few extra people joined the ideation process, it ticked up to 10%.
Lucia wanted to show the audience that involving more people isn’t just good for team spirit, it’s statistically better for business.
But Not Always Better
Of course, not every programme benefits from adding more cooks.
One programme started with Lucia’s 45% win rate, and dropped when others joined in – this is likely due to maturity. Early-stage programmes have lots of low-hanging fruit; bring in an expert and they’ll pick up those ideas fast. But once the easy wins dry up, collaboration becomes key to uncovering the next big ideas.
Lucia distilled the conditions required to unlock the Experimentation Bonus into four rules:
The task must be complex.
You need psychological safety.
Diversity only helps when people share their unique perspectives.
It works in mature experimentation environments.
The Power of Diverse Brains
Creativity, Lucia reminds us, is about generating novel and useful ideas and that happens more often when your team isn’t made up of carbon copies.
Through visual examples and the classic “brick test,” she showed how different professional backgrounds (developer vs designer vs intern) result in more unique ideas when combined, especially when compared to a team of like-minded creatives who tend to overlap in their thinking.
She even shared stats from Harvard Business Review: companies with diverse teams were 70% more likely to capture new markets. Not too shabby for what’s often seen as a “soft” initiative.
“Brainstorms Are Bad”
Yep, she said it. Lucia critiqued the classic group brainstorm format where the loudest or most agreeable ideas float to the top, while the quiet genius from dev keeps the gold to themselves.
She instead proposed a structured approach called 1-2-4-All, where individuals generate ideas solo, then share them in pairs, then small groups, before presenting to the full room. It’s basically a funnel for surfacing hidden brilliance.
She also stressed the importance of measuring collaboration not just in terms of outcomes, but participation:
Number of people suggesting ideas
Number of departments involved
% of winning experiments implemented
A Quote That Stuck:
“98% of our thoughts are the same as yesterday.” (You might need other people’s brains more than you think.)
Craig Sullivan – Experimentation AI: A New Kind of Intelligence
Even from afar, Craig Sullivan brought the energy to Cboost
Craig couldn’t be there in person, so his talk was played through the speakers as VOC (this time meaning voice of Craig rather than customer). Craig’s talk was a look at how AI is changing everything in experimentation and how we’re wildly under-using it.
Craig wanted to let the audience know that AI is not here to take your job unless you ignore it completely. It’s not replacing us; it’s rewiring the way we work.
History Repeats If You Don’t Adapt
From typewriters to the IBM PC to ecommerce and mobile, Craig reminded us that every big tech shift felt like a job killer until it wasn’t. AI is just the latest evolution. The difference is it’s moving a lot faster.
“Most of your jobs didn’t exist before 1995. This is just another reinvention.”
If you’re a junior with AI skills, you’ll outperform seniors who refuse to learn. If your company upskills and experiments, you’ll beat competitors who just install a chatbot and call it transformation.
AI Isn’t Magic, It’s a Co-Pilot
Craig has spent the last year running AI workshops with over 900 people and testing LLMs across experimentation tasks from research to idea generation to test analysis. AI doesn’t need to be perfect. It just needs to get you started faster.
Some big surprises from his work:
Creativity: AI is great at divergent thinking — it’ll give you more ideas than a human ideation session. Not all good, but many are worth stealing or building upon.
Critical thinking: It challenges weak hypotheses and helps connect research > problem > experiment > metric. Less “why did we test that?” later.
Skills lift: Junior folks benefit the most, like 200% improvements!
Velocity: Faster flywheel = more tests = more wins. AI helps you scale without needing three more heads and an extra sprint.
AI in the Wild: Real Use Cases
Craig shared some real examples of companies already using AI like a Swiss Army knife:
Generating product imagery for 200,000 SKUs without studio shoots
Translating e-commerce sites with open-source LLMs for better tone & ROI
Running tone-of-voice A/B tests across markets
Using Slackbots to evaluate backlog ideas (and poke people to add a hypothesis!)
Creating agentic AI workflows (yes, really) in 12 weeks
Bonus: One client even tested whether having plants in bathroom images boosted conversions. It did.
What AI Can’t Do (Yet)
Craig’s not a proponent of the “AI will replace us” doom. He’s clear about the limitations:
AI doesn’t know your boss, your brand, or your business context
It needs your judgment, nuance, and real-world experience
The best outcomes come when you do the convergence work, after AI gives you a buffet of options
Think of it like wearing a super-powered exosuit. You’re still steering, you’re just doing it faster, smarter, and with less grunt work.
Favourite Quotes:
“The best ideas come from a mix of human and AI input. You’re not competing with it: you’re working with it.” “Where we’re going, we don’t need prompts.”
Final Thoughts
Craig’s key takeaway is simple: play with the tools. Build your fairy castles out of digital LEGO bricks. Don’t wait for your company to mandate AI strategy; start making your own.
Because in the end, it’s not about replacing humans. It’s about amplifying what we’re already great at and letting AI handle the boring bits so we can do the brilliant ones.
You can download a playbook that Craig, Iqbal Ali, Johann van Tonder and myself.
Erin Weigel – Design for Impact: The Conversion Design Process
Erin Weigel on stage at Cboost, breaking down the conversion design process with clarity and insight
Erin Weigel took the stage like your favourite brutally honest friend, the one who’ll tell you your hypothesis is trash, but also teach you how to make it better.
Her session wasn’t your usual “tweak and pray” kind of design talk. Instead, Erin introduced us to Conversion Design, a way of working that blends design, science, and business into something that’s built for impact, not just aesthetics or stakeholder appeasement.
Also: Uncle Roger, a debate about the origin of a meme, and a whole lot of brutal wisdom about how most of our test backlogs are, frankly, rubbish.
Conversion ≠ Just Revenue
Erin opened with a reframe: conversion is not just about revenue. The word literally comes from convertere: to change. So at its core, conversion design is about intentional change. Not just launching stuff for the sake of it, but improving things on purpose.
And when you do it right? That sweet spot in the middle is where the magic happens.
Use Your “Whole-Ass Brain”
One of Erin’s core arguments is that we need to move from linear thinking (the old “discover-design-develop-launch” model) to systems thinking, something that recognises complexity, feedback loops, and the fact that work doesn’t flow neatly in a straight line.
Conversion Design embraces that mess. It’s a seven-step cycle:
Understand (qual + quant research)
Hypothesise (framed by business goals)
Prioritise
Create (design & dev)
Test (RCEs like A/B tests)
Analyse
Decide
You can move fluidly between these by sometimes looping back, skipping ahead. It’s not a flowchart; it’s a system.
The Hierarchy of Evidence
A key takeaway: not all data is created equal. Erin talked about the “Hierarchy of Evidence” a scale that runs from expert opinions (biased, often untested) up to randomised controlled experiments (the gold standard), and finally to systematic reviews where human judgment weighs all forms of evidence to inform decisions.
Don’t just ship something because it “went green” in your testing tool.
Ask what else it’s affecting. Consider the qualitative data. Think systemically, not just statistically.
Backlogs Are Full of Bad Ideas
One of the most refreshing moments? Erin saying out loud: most of our ideas suck. And that’s okay because with good testing, we can systematically throw out the bad ones and keep what works.
She illustrated how testing can shift a typical “feature factory” from shipping a chaotic mix of good and bad ideas to shipping less, but better, avoiding the drag of negative-impact launches, and leaning into the compound effect of repeated small wins.
Favourite quotes:
“Use your whole-ass brain.” “There’s a difference between making a change… and making something better.”
Simon Willer – How “Always Be Testing” Makes Peak Season Actually Work
Simon Willer from Elkjøp (that’s Elgiganten or Gigantti, depending on your Nordics postcode. Or Curry’s if you happen to be in the UK) gave us a behind-the-scenes look at how his team uses testing to deliver results when it matters most: Black Friday to Boxing Day.
Meet Jan: The Peak Shopper in a Hurry
Simon built his talk around a fictional customer named Jan, he knows what he wants, wants it fast, and is allergic to fluff.
Jan just wants his new telly, not a sales pitch. But Elkjøp wants him to buy accessories, protection plans, and maybe even enjoy the checkout experience.
How do you win over a high-intent, low-patience shopper like Jan?
Nudges That Actually Nudge
First up, urgency messaging on PDPs. Simon admitted he didn’t think it would work on Nordic shoppers but data proved otherwise:
“Low in stock”, “Many purchases”, “Popular right now” = solid boost in conversions
Only used on high-traffic, high-stock-velocity products (no fake scarcity!)
Checkout: A Thousand Problems, One Jan
Elkjøp’s checkout flow had plenty of friction, especially for “big box” items like TVs and appliances. Using customer feedback (30,000+ open text responses!) and a bit of AI-powered categorisation, they prioritised key pain points and got to work.
Some key changes:
Cart Page: Moved from a side cart to a full page view for better review and upselling opportunities
Delivery Options: Simplified choices from three to two (“fastest” vs “cheapest”)
Time Picker: Switched from dropdowns to a calendar view (with 6 months’ worth of dates!) for clearer scheduling
Payment Step: Replaced a swipeable carousel with a standard list for easier comparison
Result: +8% conversion rate on big box items Over 100 million NOK in additional sales +8 point NPS uplift
Not bad for what was essentially a series of “small” changes. As Simon put it: “None of it was rocket science. But it worked.”
Simon Willer shares a case study from Elkjøp
Final Message: Don’t Get Precious, Get Testing
Simon’s parting message was a crowd-pleaser:
“There are always 1001 problems in e-comm. So figure out which matter most and test your way through them.”
Juliana Jackson – Insights, Intent, Impact: Rethinking Marketing Analytics in 2025
Juliana Jackson on the stage at Cboost
If you’ve ever seen Juliana Jackson speak, you’ll know she doesn’t do bland.
Her energy is relentless, her delivery is sharp, and she doesn’t waste time sugar-coating hard truths.
She came in to close the day, calling out bad metrics, broken attribution models, and half-baked AI hype, while showing exactly how to do things better.
Juliana said that most of us are optimising for the wrong things, ignoring the data that matters, and measuring customer journeys like it’s still 2010.
We’re Optimising Checkouts. Customers Are On Reddit.
Juliana’s main point? We obsess over micro-conversions while real customer perception is shaped elsewhere on TikTok, Reddit, Quora, App Store reviews, and WhatsApp groups. If you’re only watching your site analytics, you’re missing the story.
Your Data Stack Is Fancy. You’re Still Using It Wrong.
Juliana walked through the gap between what marketing teams think they’re doing (data-driven, AI-ready, omnichannel masters) vs. what’s actually happening (shiny tools, shallow insights, fragmented journeys).
Resulting in misaligned content, missed opportunities, and frustrated customers. To avoid this Juliana encouraged the audience to start looking at unstructured data. The messy, human stuff like app reviews, social comments, videos, and memes because that’s where the sentiment about your brand lives.
Starbucks (and the Power of Small Language Models)
Juliana shared a case study: Starbucks had 500,000+ app reviews and zero clarity on what customers really thought. Her team used machine learning (not ChatGPT, but task-specific small language models) to:
Analyse sentiment at scale
Identify high-impact user issues by market
Distinguish needs by platform (iOS users were more focused on the look and feel and Android users were more concerned with functionality and performance)
Shift Starbucks’ roadmap from “feature frenzy” to “core experience first”
Taking this approach with Starbucks led to smarter prioritisation, less noise, and crucially testing that aligned with real user problems.
Beyond Search: What Your Customers Really Want
Juliana also tackled brand discoverability, pointing out that in 2025, people aren’t searching for your brand; they’re searching for answers. And your FAQ page, help docs, and landing pages might just be “cannibalising” your conversions if they hoard attention but offer no next step.
Using tools like AlsoAsked, she mapped emotional and informational search patterns and paired them with “page intent” – the purpose a page should serve based on what users are looking for, rather than what the brand wants to push.
She showed how mapping search patterns to content, especially emotional and informational queries, helps ensure pages reflect what users are genuinely looking for, not just what brands want to say.
Final Takeaways:
“You’re not competing on product. You’re competing on perception.”
As a first-timer at ConversionBoost I wasn’t sure what to expect. But between the genuinely insightful talks, the warm welcome from the Danish locals, and the conversations I had with fellow attendees (who were very varied in their roles), it was a great event.
What stood out most was the openness of the community. Whether it was seasoned practitioners sharing practical frameworks or people swapping stories over coffee, the conference was refreshingly collaborative with a shared enthusiasm for learning and pushing experimentation forward.
The talks themselves didn’t shy away from the big topics. AI came up a lot (as expected), but so did the messy, human side of experimentation, from cross-functional team dynamics to navigating organisational blockers. Speakers weren’t just presenting polished case studies; they were honest about what worked, what didn’t, and what still needed figuring out.
As someone working agency-side, it was great to hear from in-house teams tackling experimentation at scale and to see how agencies are playing a supporting role in that. The case studies from GANNI, Husqvarna and Elkjøp were a reminder that when internal teams are trusted to test, learn and iterate alongside the agencies they work with, then the results speak for themselves.
ConversionBoost was packed with ideas, but also grounded in the realities of doing the work. I left with plenty to think about and even more to try. Oh, and I got to see the first live performance of Ole Gregerson’s original “Experimentation Heroes” song (his kids tried to convince him to stick to his day job)!
Ole Gregerson, host and organiser of Cboost, wrapping up the event
