Research talk:Anonymous editor acquisition/Signup CTA experiment/Work log/2014-06-16
Add topicMonday, June 16th[edit]
Picking up where I left off on Friday, I should have a dataset with a complete set of reverted & archived flags.
--Halfak (WMF) (talk) 14:40, 16 June 2014 (UTC)
![The geometric mean number of article revisions saved is plotted by experimental condition.](http://upload.wikimedia.org/wikipedia/commons/thumb/d/d1/Signup.experiment.main_revisions.geo_mean.by_bucket.svg/600px-Signup.experiment.main_revisions.geo_mean.by_bucket.svg.png)
![The geometric mean number of productive edits is plotted by experimental condition.](http://upload.wikimedia.org/wikipedia/commons/thumb/7/78/Signup.experiment.productive_edits.geo_mean.by_bucket.svg/600px-Signup.experiment.productive_edits.geo_mean.by_bucket.svg.png)
![The geometric mean number of productive editors is plotted by experimental condition](http://upload.wikimedia.org/wikipedia/commons/thumb/4/4a/Signup.experiment.productive_prop.by_bucket.svg/600px-Signup.experiment.productive_prop.by_bucket.svg.png)
It looks like -- no matter how I measure it -- overall productivity went down in the pre-edit condition. --Halfak (WMF) (talk) 14:48, 16 June 2014 (UTC)
Now it's time to get some basic stats on burden. Since I just measured the revert rate, I can figure out how much burden relevant anons inflicted on Wikipedians.
![The geometric mean reverted article revisions per user/token is plotted by experimental condition. Note that this method for measuring revert activity does not control for decreased article activity.](http://upload.wikimedia.org/wikipedia/commons/thumb/4/46/Signup.experiment.reverted_main_revisions.geo_mean.by_bucket.svg/600px-Signup.experiment.reverted_main_revisions.geo_mean.by_bucket.svg.png)
OK, so I measured this two naive ways and a robust way. #Reverted article revisions plots the expected number of reverted article revisions per user/token. This measurement is affected by the decreased rate of article editing. #Reverted revisions prop plots the proportion of reverted article revisions for the whole group. This measurement is affected most strongly by prolific editors and is therefor, not robust to outliers. For #Reverted sampled revision prop, I randomly sampled 1 article revision from each user/token who made at least one and included that in the proportion measurement. This method controls for both concerns.
So, it looks like editors in the pre-edit condition are consistently less burden than the control.
Here's the data:
wiki | bucket | n | main_revisions.sum | main_revisions.geo_mean | main_revisions.geo_se.upper | main_revisions.geo_se.lower | productive_edits.geo_mean | productive_edits.geo_se.upper | productive_edits.geo_se.lower | reverted_main_revisions.sum | reverted_main_revisions.geo_mean | reverted_main_revisions.geo_se.upper | reverted_main_revisions.geo_se.lower | productive.k | main_editor.k | sample_reverted.k | productive.prop | productive.se | reverted_main_revision.prop | reverted_main_revision.se | sample_reverted.prop | sample_reverted.se |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
dewiki | control | 42441 | 12314 | 0.08973791 | 0.09150446 | 0.08797422 | 0.06175928 | 0.06329375 | 0.06022702 | 2123 | 0.02960919 | 0.03042239 | 0.02879663 | 2383 | 3636 | 1329 | 0.05614854 | 0.00111745 | 0.1724054 | 0.003403962 | 0.3655116 | 0.007986389 |
dewiki | post-edit | 38593 | 8075 | 0.08627629 | 0.08798654 | 0.08456873 | 0.05689025 | 0.05833854 | 0.05544394 | 1942 | 0.0303633 | 0.03121313 | 0.02951417 | 2127 | 3356 | 1309 | 0.05511362 | 0.001161622 | 0.2404954 | 0.00475606 | 0.3900477 | 0.008419684 |
dewiki | pre-edit | 39331 | 10166 | 0.07274083 | 0.07439115 | 0.07109304 | 0.05510174 | 0.05657932 | 0.05362622 | 1319 | 0.0192839 | 0.01997635 | 0.01859193 | 2014 | 2722 | 776 | 0.05120643 | 0.001111426 | 0.1297462 | 0.003332693 | 0.2850845 | 0.008653068 |
enwiki | pre-edit | 198936 | 61737 | 0.107385 | 0.1083025 | 0.1064682 | 0.08511951 | 0.08593776 | 0.08430188 | 10209 | 0.02552958 | 0.02591357 | 0.02514574 | 14982 | 18763 | 4380 | 0.07531065 | 0.0005916562 | 0.1653627 | 0.001495185 | 0.2334381 | 0.003088221 |
enwiki | control | 208205 | 77814 | 0.1321544 | 0.1331415 | 0.1311681 | 0.09897115 | 0.09983017 | 0.09811281 | 14828 | 0.03657449 | 0.03701676 | 0.03613241 | 18566 | 24619 | 6873 | 0.08917173 | 0.0006245774 | 0.190557 | 0.001407916 | 0.2791746 | 0.002859024 |
enwiki | post-edit | 189530 | 65349 | 0.1311697 | 0.1321865 | 0.1301538 | 0.09805692 | 0.09894065 | 0.09717389 | 13201 | 0.0361366 | 0.03659432 | 0.03567909 | 16891 | 22510 | 6338 | 0.08912046 | 0.0006544556 | 0.2020077 | 0.001570594 | 0.2815637 | 0.002997745 |
frwiki | pre-edit | 30115 | 9673 | 0.1023034 | 0.1046157 | 0.0999959 | 0.08316826 | 0.08527892 | 0.08106169 | 1191 | 0.02137573 | 0.02223767 | 0.02051451 | 2228 | 2764 | 605 | 0.07398306 | 0.001508288 | 0.1231262 | 0.003340898 | 0.2188857 | 0.007864968 |
frwiki | post-edit | 28649 | 9874 | 0.1238606 | 0.126376 | 0.1213509 | 0.09654348 | 0.09880157 | 0.09429004 | 1435 | 0.02865029 | 0.02963839 | 0.02766313 | 2568 | 3324 | 833 | 0.08963664 | 0.001687701 | 0.1453312 | 0.00354676 | 0.2506017 | 0.007516538 |
frwiki | control | 31714 | 10589 | 0.1230304 | 0.1254209 | 0.120645 | 0.09200022 | 0.09410328 | 0.0899012 | 1894 | 0.03245121 | 0.03347216 | 0.03143127 | 2691 | 3631 | 1034 | 0.08485212 | 0.001564773 | 0.1788649 | 0.00372428 | 0.28477 | 0.00748957 |
itwiki | pre-edit | 20310 | 8328 | 0.117881 | 0.1210721 | 0.1146989 | 0.1004393 | 0.1034149 | 0.09747166 | 800 | 0.02071121 | 0.02176015 | 0.01966334 | 1669 | 1984 | 368 | 0.08217627 | 0.001927072 | 0.09606148 | 0.003229042 | 0.1854839 | 0.008726339 |
itwiki | control | 20297 | 9856 | 0.1431397 | 0.146633 | 0.139657 | 0.1156532 | 0.1188154 | 0.1124999 | 1341 | 0.03125235 | 0.03259601 | 0.02991044 | 1982 | 2462 | 549 | 0.0976499 | 0.002083567 | 0.1360593 | 0.00345347 | 0.2229894 | 0.008389027 |
itwiki | post-edit | 18359 | 9604 | 0.1442438 | 0.1479507 | 0.1405488 | 0.118409 | 0.1218074 | 0.1150208 | 1047 | 0.02996144 | 0.03128555 | 0.02863902 | 1808 | 2237 | 490 | 0.09848031 | 0.002199064 | 0.1090171 | 0.003180211 | 0.2190434 | 0.008744712 |