Template A/B testing/ImageTaggingBot Analyses
Overview
[edit]
Templates invlolved in the experiment: z131 - z142 [1]
Timeframe: 2011-12-21 - 2012-02-21
Analyses Results
[edit]Counts: (Z-template - count)
131 - 107
132 - 107
133 - 114
134 - 207
135 - 188
136 - 219
137 - 42
138 - 41
139 - 43
140 - 18
141 - 23
142 - 24
Total Control: 374
Total Test: 759
Edit Activity
[edit]Case: z131 vs. z133
[edit]Below is the measurement of all users that went on to make at least one edit in namespace 6 after receiving the ImageTagging template given that they were not blocked after seeing the template.
Test response: 9.649%
Control response: 18.69%
Modelling Analysis, Registered Users, z133 (test) vs z131 (control) - R Output
|
|---|
Call:
glm(formula = template ~ metric, family = binomial(link = "logit"),
data = temp_df)
Deviance Residuals:
Min 1Q Median 3Q Max
-1.250 -1.250 1.107 1.107 1.440
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.1688 0.1456 1.159 0.2463
metric -0.7667 0.4026 -1.904 0.0569 .
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 306.15 on 220 degrees of freedom
Residual deviance: 302.37 on 219 degrees of freedom
AIC: 306.37
Number of Fisher Scoring iterations: 4
|
Test Users that edited namespace 6 after the posting
[edit] [1] "Bolidist" "Chrisyasay" "Dborase"
[4] "Dj Ran" "Dpsthota11" "Einie101"
[7] "Erapo" "Fuse809" "HahnMahlay"
[10] "Hlakungpui" "Iambonusboy" "Ilovethe2000s"
[13] "Imhungry4444" "Iviscera" "Jazzbeard"
[16] "JLara-cl" "Jrobin08" "Kennethcooke609"
[19] "Kimmeleshkolot" "Kracoukas" "Maslusar"
[22] "MasterSmcd" "Mishrasanjeew" "NEWBRAIN"
[25] "NickSCFC" "Peppage" "Pg 6475"
[28] "Purificationwiki" "QwerpoiXX" "RTBoughner"
[31] "Salilpawar1" "Smallzc" "Swaggie73"
[34] "The 5th Silver Beatle" "TheBigJagielka" "Timepricer"
[37] "Wadamz" "Warriorpoet23" "WISEPHAROHPSN"
[40] "Zachary.burrows"
Control Users that edited namespace 6 after the posting
[edit] [1] "1000 Volts" "Ansonjae" "Balbon32"
[4] "Bloodios" "Bubertov" "D20120101"
[7] "Dborase" "Dertogada" "Diango"
[10] "Duncan3dc" "Emix.G.A" "Foxonline"
[13] "Fuse809" "Galwaybuck" "Hafeezullah2k"
[16] "Ilovethe2000s" "Iviscera" "Jazzbeard"
[19] "JLara-cl" "K.yakovleva" "Kennethcooke609"
[22] "KingFredrick VI" "Klr8" "Kracoukas"
[25] "KristineValdheima" "Malekalameh" "MasterSmcd"
[28] "MelanieBrown" "MikelZap" "Mohdwasi"
[31] "Monica982" "Mthen" "Muzammil786"
[34] "Naveen Ramanathan" "NEWBRAIN" "Nick Rokk"
[37] "Pg 6475" "Pilch 84" "Polo phil"
[40] "Prz4587ill" "Purificationwiki" "Rcoyy"
[43] "Rsamahamed" "RTBoughner" "Scribblerman"
[46] "Smallzc" "Sorayanwayne" "SpongePappy"
[49] "Sumitkumarjha75" "Swaggie73" "The 5th Silver Beatle"
[52] "TheDarkPyrano100" "Victorianny" "Voravitjessica"
[55] "Wadamz" "Willtoco" "Zavila"
[58] "Zywxn"
Blocking
[edit]Case: z131 vs. z132 (test) -- This result is no longer valid. For documentation purposes only.
[edit]Number of blocks after the posting was a marginally significant (p-value = .145) predictor of the template. The control template had a lower rate.
Test average blocks / user after posting: 0.06854839
Control average blocks / user after posting: 0.03212851
Modelling Analysis, Registered Users, z132 (test) vs z131 (control) - R Output
|
|---|
Call:
glm(formula = template ~ blocks_after, family = binomial(link = "logit"),
data = all_data)
Deviance Residuals:
Min 1Q Median 3Q Max
-1.651 -1.165 -1.165 1.190 1.190
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.03018 0.09136 -0.330 0.741
blocks_after 0.54926 0.37683 1.458 0.145
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 688.99 on 496 degrees of freedom
Residual deviance: 686.60 on 495 degrees of freedom
AIC: 690.6
Number of Fisher Scoring iterations: 4
|
Case: z134 vs. z135 (test) -- This result is no longer valid. For documentation purposes only.
[edit]Number of blocks after the posting was a marginally significant (p-value = .120) predictor of the template. The test template had a lower rate.
Test average blocks / user after posting: 0.002645503
Control average blocks / user after posting: 0.01703163
Modelling Analysis, Registered Users, z132 (test) vs z131 (control) - R Output
|
|---|
Call:
glm(formula = template ~ blocks_after, family = binomial(link = "logit"),
data = all_data)
Deviance Residuals:
Min 1Q Median 3Q Max
-1.147 -1.147 -1.147 1.208 1.931
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -0.07146 0.07156 -0.999 0.318
blocks_after -1.62419 1.04421 -1.555 0.120
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 1092.4 on 788 degrees of freedom
Residual deviance: 1088.6 on 787 degrees of freedom
AIC: 1092.6
Number of Fisher Scoring iterations: 4
|
Case: z137 vs. z139 (test) -- This result is no longer valid. For documentation purposes only.
[edit]Number of blocks after the posting was a marginally significant (p-value = .19) predictor of the template. The test template had a lower rate.
Test average blocks / user after posting: 0.03947368
Control average blocks / user after posting: 0.109589
Modelling Analysis, Registered Users, z132 (test) vs z131 (control) - R Output
|
|---|
Call:
glm(formula = template ~ blocks_after, family = binomial(link = "logit"),
data = all_data)
Deviance Residuals:
Min 1Q Median 3Q Max
-1.218 -1.218 1.137 1.137 1.494
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.09591 0.16882 0.568 0.57
blocks_after -0.81601 0.62203 -1.312 0.19
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 206.50 on 148 degrees of freedom
Residual deviance: 204.48 on 147 degrees of freedom
AIC: 208.48
Number of Fisher Scoring iterations: 4
|
Overall Warnings
[edit]Average warnings / user before 0.8744589 and afte 0.8556999.
Fraction of users warned before: 0.4300144
Fraction of users warned after: 0.3347763
Th latter is a significant difference.