Jump to content

User:Renklauf/Archive 2

From Meta, a Wikimedia project coordination wiki

Thursday, March 1st

[edit]
  • Regenerating postings for z60 and z62
  • much of this was taken up by debugging postings.py since all of these template postings were being missed
  • chat with Dario and Karyn (separately) regarding next steps with Huggle analysis


Friday, March 2nd

[edit]
  • Huggle 3 analysis on specific template postings - (60,62,66,76 VS. 61,63,67,77)


Data Munging / Filtering: 

Only tracking edits in the first three days after posting
Blocks after = 0 (no blocks after seeing template), 
namespace = 0, 
first_warning = TRUE

> Non-registered

3 <= deleted edits before <= Inf 
test dataponts = 214
test dataponts = 177

> Registered:

5 <= deleted edits before <= Inf 
test dataponts = 30
test dataponts = 30


Findings:

For non-registered the mean decrease in test edits exceeded the control 83.83% and 75.02% respectively.  The result is 94.59% confident.
For registered the mean decrease in control edits exceeded the test 83.20% and 70.58% respectively.  The result is 84.00% confident.

The result of the effect is swapped between registered and non-registered users.


Modelling Analysis, Non-Registered Users - R Output

Call:
glm(formula = template ~ edits_decrease, family = binomial(link = "logit"), 
    data = all_data)

Deviance Residuals: 
   Min      1Q  Median      3Q     Max  
-1.319  -1.319   1.043   1.043   1.596  

Coefficients:
               Estimate Std. Error z value Pr(>|z|)  
(Intercept)     -0.1510     0.2243  -0.673   0.5007  
edits_decrease   0.4769     0.2476   1.926   0.0541 .
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 527.28  on 383  degrees of freedom
Residual deviance: 523.36  on 382  degrees of freedom
AIC: 527.36

Number of Fisher Scoring iterations: 4




Percentage decrease in deleted edits Test:

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
-1.6670  0.8260  1.0000  0.8384  1.0000  1.0000 

Percentage decrease in deleted edits Control:

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
-1.6670  0.6818  1.0000  0.7502  1.0000  1.0000


Modelling Analysis, Registered Users - R Output

Call:
glm(formula = template ~ edits_decrease, family = binomial(link = "logit"), 
    data = all_data)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.5728  -1.0761  -0.1728   1.2366   1.2894  

Coefficients:
               Estimate Std. Error z value Pr(>|z|)
(Intercept)      0.8939     0.6968   1.283    0.200
edits_decrease  -1.1533     0.8205  -1.406    0.160

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 83.178  on 59  degrees of freedom
Residual deviance: 81.049  on 58  degrees of freedom
AIC: 85.049

Number of Fisher Scoring iterations: 4



Percentage decrease in deleted edits Test:

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
-0.2703  0.4309  0.9071  0.7058  1.0000  1.0000 

Percentage decrease in deleted edits Control:

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.0000  0.7770  0.9857  0.8320  1.0000  1.0000


  • Post Optimizely and Israeli data to data hub
  • email Aaron!
  • diff engine queries


Monday, March 5th

[edit]
  • Revisited Huggle analyses - began looking at strict edit counts after the posting
  • sent email to Aaron Swartz


Tuesday, March 6th

[edit]
  • Met with Erik M. to discuss pulling metrics from db42
  • Wrote scripts to generate and plot sample counts for Huggle tests


Wednesday, March 7th

[edit]
  • Retooled scripts to produce plots for measuring edits up to 30 days after
  • Meeting with Odiago and Zack
  • Gathered together all Huggle analysis into meta-page (see links section below)
  • executed time-bound query on alpha (still running)


Thursday, March 8th

[edit]
  • Communicate with Yusuke / Diederik regarding diff DB on alpha
  • Interpret Huggle results - post to meta
  • Optimizely Data - data hub


Friday, March 9th

[edit]


Monday, March 12th

[edit]
  • Look at additional experiments - what can be said about Twinkle, ImageTaggingBot, CorenSearchBot
  • PT Editor List
  • Better define metrics on Huggle Analysis - what do they mean?


Tuesday, March 13th

[edit]
  • Summarized and corrected some mistakes with Huggle Analysis [1] - this included a better definition of the metrics used to measure the performance of the new templates in the experiment
  • Tried to describe the experiments - I still need to understand how the experiments fundamentally differ


Wednesday, March 14th

[edit]
  • Analyses Twinkle / ImageTaggingBot / CorenSearchBot - working through Twinkle results
  • Meeting with Dario and Judd on micro on effects of post-transaction


Thursday, March 15th

[edit]
  • Continue Twinkle analysis
  • Stanford Brown bag + pre-meeting
  • Huggle results w/ Steven and Maryana - it turns out the Twinkle results may be tougher to suss out than first imagined. The problem lies in the for PROD templates we need to measure whether deleted revisions actually correspond to the article on which the PROD tag was places ... similarly, for AFD articles revisions in namespace 4 should be measured


Friday, March 16th

[edit]
  • Begin work on Shared IP
  • Gather metrics for all namespaces for Huggle experiments
  • Count Template instances for Steven / Maryana using postings script


Week of March 19th

[edit]


Monday, March 26th

[edit]
  • Determine which tested users are still editing - query running
  • Twinkle / Shared IP / ImageTaggingBot - work on one of these experiments
  • generated namespace 4 edit metrics for Twinkle tests
  • fix editor threshold notify - done
  • revision caching - scripts active cron on aluminium


Tuesday, March 27th

[edit]
  • Work on revision, bot, and active editor caching
  • Template Report review


Wednesday, March 28th

[edit]
  • Work on revision, bot, and active editor caching
  • Meet with Dario / Karyn / Steven / Maryana to begin the process of defining E3 projects
  • Meet with Dario / Karyn / Howie to discuss analytics specifications


Thursday, March 29th

[edit]
  • work on active editor caching
  • Data munging for SharedIP - analyzing Stu's tables
  • Analytics specing and use cases with Dario on E3 backlog projects
  • Review report, pick out some plots, add content


Thursday, March 29th

[edit]
  • Shared IP Analysis
  • Meet with Howie, Dario, Diederik, Karyn to discuss plans for a free form data store for the E3 team


Monday, April 2nd

[edit]
  • Largely focused on writing scripts for Shared IP testing - tried to create a table for first revision by user but this proved to be too resource intensive
  • Began implementing the "SlaveLoader" python class to handle data processing from enwiki slave tables


Tuesday, April 3rd

[edit]
  • Meeting with Steven and Maryana regarding Huggle Report
  • Sketched out data collection for Twinkle PROD testing
  • Finished off shared IP analysis:
  • compared archive group before bot was active and after .. it seemed that the non-archive group had more editing however the different time periods added a large amount of bias
  • Looked again at groups based on header templates. Mismatched sample sizes along with large differences in pre-bot edit activity among groups introduces a bias that would make it difficult to isolate the effect of archiving


Wednesday, April 4th

[edit]
  • PROD data collection implementation
  • WikiSym Submission
  • Input on report


Retrieved from "https://meta.wikimedia.org/w/index.php?title=User:Renklauf/Archive_2&oldid=3656570"