Interpreting Test Results – Performance

This article is designed to ensure you understand the data found in the ‘Performance’ tab  of an Optimize test report:

First up you’ll notice a bar chart which gives a graphical representation of the views and conversions for each experiment. You can toggle the graph on or off by clicking on the Graph icon on the right-hand side, just above the graph itself.

Scroll down to view the tabular data where the ‘meat’ of the report is located.

You’ll notice that each conversion point has it’s own row of results. Pay close attention to the three columns on the far right-hand side as these help you determine how the test is performing for each conversion point:

 

In the above example, the first column “Lift Over Control” shows a 1.7% increase for the first variation. The next conversion point shows a -0.08% decrease for the first variation and the final conversion point shows -0.63% decrease.  This may be a little difficult to understand where the positives really lie.

In the above example the first lift percentage (-94.30%) is negative. It’s highlighted in a dark red colour as its ‘chance to beat control‘ percentage is in the strongly negative range. The next one down is highlighted in dark green as its ‘chance to beat control’ percentage is also in the strongly positive range.

If you’re not getting the highlighted colours you may need click on the toggle button on the top right corner:

The ‘Significant?’ column enables you to see if you have reached significance for a given conversion point. Webtrends Optimize currently uses a 95% confidence rate to calculate significance.