Research talk:Revision scoring as a service/Work log/2016-02-24

Wednesday, February 24, 2016

Arabic

>         revscoring train_test \
>                 revscoring.scorer_models.RF \
>                 editquality.feature_lists.arwiki.reverted \
>                 --version 0.1.0 \
>                 -p 'max_features="log2"' \
>                 -p 'criterion="entropy"' \
>                 -p 'min_samples_leaf=5' \
>                 -p 'n_estimators=640' \
>                 -s 'pr' -s 'roc' \
>                 -s 'recall_at_fpr(max_fpr=0.10)' \
>                 -s 'filter_rate_at_recall(min_recall=0.90)' \
>                 -s 'filter_rate_at_recall(min_recall=0.75)' \
>                 --balance-sample-weight \
>                 --center --scale \
>                 --label-type=bool > \
>         models/arwiki.reverted.rf.model
2016-02-24 19:00:50,796 INFO:revscoring.utilities.train_test -- Training model...
2016-02-24 19:01:07,650 INFO:revscoring.utilities.train_test -- Testing model...
ScikitLearnClassifier
 - type: RF
 - params: warm_start=false, class_weight=null, bootstrap=true, oob_score=false, scale=true, min_samples_split=2, max_leaf_nodes=null, balanced_sample_weight=true, n_estimators=640, center=true, min_samples_leaf=5, criterion="entropy", max_depth=null, min_weight_fraction_leaf=0.0, verbose=0, random_state=null, n_jobs=1, max_features="log2"
 - version: 0.1.0
 - trained: 2016-02-24T19:01:07.650387

         ~False    ~True
-----  --------  -------
False      3728      129
True         42       94

Accuracy: 0.9571750563486101

Filter rate @ 0.9 recall: threshold=0.214, filter_rate=0.902, recall=0.904
Recall @ 0.1 false-positive rate: threshold=0.957, recall=0.007, fpr=0.0
Filter rate @ 0.75 recall: threshold=0.434, filter_rate=0.94, recall=0.75
PR-AUC: 0.445
ROC-AUC: 0.95

Sincerly Amir (talk) 19:37, 24 February 2016 (UTC)