Skip to main content

Reporting on Custom Campaigns

Written by Charley Bader
Updated over a week ago

This article covers reporting for Custom Campaigns. For information around Agentic Campaign reporting, look here.


What you're measuring

Custom Campaign reporting shows you how well your campaign performed against your chosen goal, across two lenses: segment-level and global.

  • Segment-level is the primary view. It tells you whether the experience changed behaviour for the specific audience it was designed for.

  • Global shows the campaign's contribution to site-wide performance.

Both views are important, but start with segment-level. That's where you'll find the most meaningful signal.

When you look at Campaign Reporting, you can see how your campaigns perform against your chosen Campaign Goal and Intent Metrics.


Campaign Goals

By default, reporting tracks transactions and revenue. If your campaign is designed to influence a different behaviour β€” email sign-ups, live chat engagement, clicks on a key CTA β€” you can set a Custom Goal instead.

Learn how to set up Custom Goals β†’


How bucketing works

Understanding bucketing is key to reading your results correctly.

When a visitor lands on your site and meets your campaign's global rules, they are immediately assigned to either the variant (or experience) group or the control group - before anything happens on screen. Nothing changes visually at this point.

The experience only triggers later, if and when that visitor matches your segment conditions.

This means your control group reflects real visitors who would have been eligible for the experience, not just everyone who visited the site. Making the comparison meaningful.


Reporting on A/B Tests

A/B tests compare one or more variants against a control (the baseline). Results are available at three levels:

Segment Breakdown: performance by segment, comparing variant to control within each segment. Start here.

Global Reporting: all users who met the campaign's global rules, across both groups. Use this to understand site-wide contribution.

Real-Time Overview: live figures for today (from 00:00 UTC), updated in real time, in both table and graph format.

In the Segment Breakdown and Global Reporting views, you'll see:

  • Users in variant and control

  • Selected Intent Metric and change vs control

  • Total conversions, conversion rate, and uplift vs control

  • Total revenue and average order value

  • Probability to Beat Baseline (when looking at experiments at a Global Reporting level. This baseline can be the control, or another variant and is set in the drop downs)

Note: Segment Breakdown and Global Reporting data runs from campaign launch to the previous day. Real-Time Overview shows today's data only.

Holdback group: All campaigns include a holdback group to measure the value the campaign is driving. In A/B tests, the holdback is included within the control.

Reporting on Experiences

If your campaign isn't running an A/B test, it will have one or more experiences and a holdback group. The same three reporting levels apply:

  • Segment Breakdown; experience vs holdback, by segment

  • Global Reporting; all eligible users, including holdback

  • Real-Time Overview; live figures for today

The same metrics are available as in A/B test reporting, with the holdback group as the comparison baseline.

Holdback group data appears as a separate line within reporting.


Intent Metrics

Reporting also shows how users progressed through intent levels during the campaign period:

  • Low Intent - percentage of users who reached Low Intent

  • Building Intent - percentage of users who reached Building Intent

  • High Intent - percentage of users who reached High Intent


Understanding your results

What does "Probability to Be Best" mean?

This metric estimates how likely it is that a given variant or experience is the top performer among all options being tested, including the control. Rather than asking "is this statistically significant?", it asks: "How confident are we that this is the best option right now?"

It's calculated using Bayesian simulation. For each variant, we model many possible outcomes based on observed performance, then measure how often each one comes out ahead. A result of 75% means that variant outperformed all others in 75% of simulations.

These probabilities update as more data is collected, giving you an honest, evolving picture of confidence rather than a binary pass/fail.

What does "Probability to Beat Baseline" mean?

This shows how likely a specific variant or experience is to outperform the selected baseline (either the control, holdback group, or another variant). You set the baseline using the dropdown in the reporting view.

Why is my segment uplift strong but global impact modest?

This is expected and normal. Intent-based segments are often a fraction of total traffic. A meaningful improvement for the right audience won't always move the overall site number dramatically, and that's fine. Strong segment uplift with modest global impact is a sign you're reaching the right people at the right moment. Use segment reporting to validate whether the experience works; use global reporting to understand its contribution to the business.

Did this answer your question?