Reading the experiments’ page

There is lots of useful information in your experiments’ page. Here’s a brief guide to help you find your way around.

The experiments’ page is designed to give you a visual representation of the results of the different layout experiments being tested by ezoic.  The results are organized by device (desktop, tablet and mobile) and can be customized to show particular date ranges.

Understanding the Overview Chart 
Near the top of the page you will find a multi-colored chart that shows the different experiments that have been shown to your visitors.  The vertical axis represents the % of visitors who see a particular layout and the horizontal axis is the date the experiments were shown.
The different colors in the body of the chart represent the different layouts and there is a key to show which color represents which layout.  For example, on the screengrab below the layout experiment called Pollack, shown in red, was shown to 91.61%of the traffic on 10 Aug 14.

Understanding the data table
The experiments are ranked based on the optimization goals you set for your site (you can change these in “Settings>Optimization Goals>Manage.”).   The data metrics, Page Views per Visit (PV/Visit), Average Time on Site (AVG TOS), Bounce Rate (Bounce %) and Earnings Per 1000 Visitor (EPMV) are weighted according to your aims.  You can see from this picture that experiment ‘Pollack’ is ranked at #1.  It beats #2,  Penguine, for PV/V, but is 1 second less on AVG TOS and has a slightly higher bounce rate and a lower revenue. But Penguine is showing an EPMV of $17.80 against Pollack’s $12.17.   The system is promoting Pollack above Penguine based on the combination of the weighted metric scores.  So Pollack’s scores from PV/Visit, AVG TOS and Bounce % were deemed to be worth more towards achieving the optimization goals set by the publisher than the extra revenue from Penguine.
If the publisher changed the optimization goal is to a revenue focus, Penguine would likely increase its ranking, and Pollack would drop.  The same weighting is keeping Penguine above Plaice, even though Plaice is scoring slightly better for PV/Visit and AVG TOS.


How accurate are the results?
It takes a certain amount of visits to any particular layout experiment to achieve statistical confidence in the results.   Those experiments that have reached statistical confidence are awarded a green bar under the Confidence tab – the data on these experiments is very accurate.  Any data in experiments that have not yet gained the green bar status should not be deemed reliable — so Skate is a way off yet!

Projection of Results?
Lots of data is important but just as important is ‘what does this data mean to me?’  The data in the table shows past results only.  The projection tool, which you can access by pushing the blue ‘View’ button, shows what difference sending 100% of traffic to one layout for a year would make, based on current performance.  Penguine, the highest performer for revenue, is projected to give an increase of $12,408.74, while reducing Bounce % by 0.12%.  Others will give different projections – try it for yourself – it’s fun!

So there are lots of data that will help you understand the performance of your experiment – and all in glorious color!  Have a play around – it will really help you understand the way the system is evaluating your experiments based on your optimization goals.

As ever, your Ezoic representative will be happy to explain in more detail.