Skip to main content

Reporting on Agentic Campaigns

Written by Charley Bader
Updated this week

The Agentic Report is your live view of how the campaign is performing, how the agent is distributing traffic, and where performance is being created across audiences and content.

Use it to answer three questions:

  • Is it working?

  • How reliable is the performance signal?

  • Where should traffic be focused next?

For Agentic Campaigns, you are also able to see our standard campaign reporting, at a global level and in real-time.


The Two Tabs

Overall Performance

Impact, trends, traffic allocations, and variant outcomes. This is your go-to view for understanding results.

Agent Activity

A log of every optimisation run, with transparent reasoning behind each decision the agent made.


Recommended Reading Order

  1. Start with the Agent Campaign Summary — phase, performance rating, strategy, and goal

  2. Check the Performance Snapshot for top-line impact versus control

  3. Use Trends to validate direction over time

  4. Use Allocations to see where traffic is shifting

  5. Use Experience Breakdown to find where lift is concentrated


Key Terms

Control: Baseline visitors who didn’t receive a personalised experience.

Treatment: Visitors who were shown a personalised experience.

Contextual Allocation: Routing traffic to the options most likely to perform well for each visitor context.

Probability to Beat Control: The likelihood that the observed uplift is real, not just noise.

Statistical reliability: Data maturity indicators (like power and certainty) that tell you how trustworthy the current signals are.


Detailed Reporting Pages

Performance Snapshot

What is the Performance Snapshot?

The Performance Snapshot is the headline readout of your campaign’s impact against control for the active reporting window. It combines outcome, confidence, and data maturity into one summary view. It should be the first place to look when you open the Agentic Report.


Core signals

Goal metric vs control — Direct outcome comparison between variant and control visitors.

Probability to Beat Control — How likely it is that variant is truly outperforming, not just by chance.

Statistical reliability — Data maturity indicators like certainty and power, plus sample depth. These tell you how much to trust the numbers.

Uplift — Absolute and relative impact above the control.


Optimisation Phase and Performance Rating

Exploration — The agent is still learning broadly. Traffic is spread across experiences to gather data.

Optimisation — The agent is increasingly confident. Traffic is shifting toward stronger performers.

Performance Rating — Great, Good, Promising, Uncertain, or Unlikely.


Uplift and projections

  • Total uplift — The incremental value seen in the time the campaign has been live

  • Annual projection — A projection of what uplift you would see annually. If we have been collecting data across previous months, this will be based on your traffic

  • Monthly projections — Projected uplift by month, with an optional cumulative view for a running total

Important: Strong-looking uplift with weak reliability should be treated as an early signal, not a final conclusion. Always check confidence before acting on headline numbers.

Traffic Allocations

What are Allocations?

Allocations are the proportion of eligible treatment traffic directed to each experience. Unlike fixed split testing, these percentages adjust continuously as the agent gathers evidence.

Seeing allocations shift is a normal, healthy sign that the agent is learning.


Why allocation percentages change

  • The agent increases share to experiences showing stronger performance

  • It keeps some exploration active so alternatives can still be tested

  • Daily shifts are expected as new evidence is incorporated


Allocations by Day chart

This stacked chart shows each experience’s daily traffic share across the reporting window.

  • Growing share usually indicates increasing confidence in that experience

  • Shrinking share indicates the agent is deprioritising it


Content mode vs Timing mode

Content mode — Shows which content type is currently being favoured.

Timing mode — Shows which trigger moments are currently being favoured. Only available when timing optimisation is enabled.


Biggest Movers table

Compares current allocation to a prior period and highlights the biggest increases and decreases. Use it to quickly spot where the agent has recently shifted confidence.


How to interpret shifts

We recommend you focus on trajectory, not isolated single-day movements. Validate major shifts against performance and reliability indicators. Expect more fluctuation during Exploration, and more stability as the campaign matures into Optimisation.

Experience Breakdown

What is Experience Breakdown?

Experience Breakdown explains performance at audience-content level — so you can see which combinations are driving lift, and which aren’t. It’s where you go when you want to understand where results are coming from.


Experiences vs Variants

Experience — A piece of content applied to an audience context.

Variant — A specific audience × content combination that is receiving traffic and being evaluated against control.


Overview metrics

  • Active Variants — Combinations currently receiving traffic

  • Low Traffic Variants — Combinations without enough evidence yet

  • Beat Control / Lost to Control — Win/loss counts versus baseline

  • Total Experience Users / Total Control Users — Scale of treatment and baseline

  • Active Segments / Active Content — How broadly personalisation is covering your audience


Detailed view

Drill into results at three levels:

  • Content level — Performance aggregated by experience

  • Segment level — Performance aggregated by visitor group

  • Variant level — The most granular view; each specific audience × content row with full metrics


Variant-level metrics

Users — How many visitors were exposed to this combination.

Goal value (e.g. Revenue Per User) — Outcome per exposed visitor.

Relative Uplift — Percentage difference versus control.

Probability to Beat Control — Confidence that the uplift is real.

Traffic Allocation — Current share of eligible traffic being sent to this variant.


How to use this section

Use Experience Breakdown to identify scalable winners, combinations that need more data, and persistent underperformers worth iterating on or removing.

Did this answer your question?