Fundraising 2010/Report/Draft Test 1
Landing Page Test: "Where Your Donation Goes Info" Info Box
[edit]For additional documentation on the testing methodology please see the following pages:
Landing Pages
[edit]
Result
[edit]Test Time: 2010-12-30 20:56:00 UTC - 2010-12-30 21:54:00 UTC
Sampling Interval = 2 minutes
Testing Interval = 8 minutes
Total Number of Samples per Class = 29
Donations / View*: "Info box" landing page won by 13.71%.
Amount50 / View**: "Info box" landing page won by 40.12%.
WINNER: "Info box" Landing Page
(*) The rate of donations per banner view over a fixed time interval.
(**) The rate of amount50 per banner view over a fixed time interval. Amount50 is the dollar amount raised from donations initiated under a given banner where all donations of more than $50 are recorded as $50 donations. This counters the skewing effect of outlier donations.
Data Analysis
[edit]This section analyzes and interprets the results of the tests. It should be noted that the data is analyzed over a period at least as large as the full testing period and that the testing period was chosen based on the period of time where significant hits and donations were observed.
Data Consistency and Cleaning
[edit]
Analyzing the above plots the donation and impression data appear to be quite regular over the interval 2010-12-30 20:56:00 UTC - 2010-12-30 21:54:00 UTC. Therefore, two minute intervals will be used for sampling over this period as a source for the paired t-test to assess confidence in the winner.
Modelling and Hypothesis Testing
[edit]The plots below depict the mean and standard deviation of each banner over four ten minute testing intervals. Samples were taken over two minute periods. The hypothesis testing was performed over the average mean and variance of the test intervals. The landing page featuring a "Info box" won in each case for donations/view and amount50/view with increases of 13.71% and 40.12% respectively. The student's t-test was used to assess confidence over each metric and the confidence in the winner for donations/view and amount50/view is at least 97.5% in each case. It is notable that the increase in amount50/view is more than donations/view. There are two potential causes here, (1) the variability of amounts tends to be much greater than donations (since donations are binary and amounts are values subject to a relatively wide range), (2) the winner actually had an affect on engaging higher amount donors.
TOTAL DONATIONS "Where your money goes (Info box)": 349 TOTAL DONATIONS "Donate button (No box)": 320 TOTAL AMOUNT50* RAISED "Where your money goes (Info box)": $6491.59 TOTAL AMOUNT50* RAISED "Donate button (No box)": $4749.00 * AMOUNT50 indicates the total amount raised where all donations greater than $50 are taken to be a donation of $50.
DONATIONS PER VIEW: Between 97.5% and 99.0% confident about the winner. Pie Box VS No pie box -- 2010-12-30 20:56:00 - 2010-12-30 21:54:00 item 1 = No box item 2 = Pie box The winner Pie box had a 13.71% increase. interval mean1 mean2 stddev1 stddev2 0 0.02306 0.05517 0.01493 0.01316 1 0.05379 0.07330 0.04721 0.01289 2 0.07057 0.06291 0.00842 0.02517 3 0.04835 0.06040 0.03297 0.02013 4 0.04772 0.05847 0.00494 0.03185 5 0.07303 0.07110 0.03519 0.02404 6 0.07154 0.05990 0.01473 0.03591 Overall Parameters: mean1 mean2 stddev1 stddev2 0.05544 0.06303 0.02696 0.02467
AMOUNT50 PER VIEW: Between 97.5% and 99.0% confident about the winner. Pie Box VS No pie box -- 2010-12-30 20:56:00 - 2010-12-30 21:54:00 item 1 = No box item 2 = Pie box The winner Pie box had a 40.12% increase. interval mean1 mean2 stddev1 stddev2 0 0.28990 0.86432 0.24925 0.23407 1 0.98852 1.56268 0.95563 0.69020 2 0.96921 1.06585 0.03701 0.68849 3 0.64897 1.31208 0.49512 0.26510 4 0.78157 1.08586 0.04731 0.43027 5 1.11967 1.23437 0.66021 0.29319 6 1.03137 1.04250 0.17909 1.25092 Overall Parameters: mean1 mean2 stddev1 stddev2 0.83274 1.16681 0.49165 0.64491
Endnotes
[edit]- Campaign = "20101230JA096_US"
- "Info box" landing page = "WMFJA073"
- "No box" landing page = "WMFJA074"