Research:Mobile editor engagement/Calls to action

Overview

Different calls to action are being tested to invite mobile users to register an account in order to perform some actions such as adding a page to a watchlist or uploading an image.

mobile_watchPageActionCta: invite mobile readers to register an account and save a page to the watchlist
mobile_editPageActionCta: invite mobile readers to register an account in order to edit a page

Research questions

How effective are mobile calls-to-action (CTA) at increasing the volume of new registrations?
What is the activation rate of these users as a function of the CTA they came from?

Data

We created a sample of users registered on a mobile device in a 2-week period in September 2013 (2013-09-16 00:00:00 - 2013-09-30 00:00:00).

Registered users by campaign
ID	Unique users
Mobile registrations
mobile_reg_no_cta_september_2013_16-30	5,500
mobile_reg_watchlist_cta_september_2013_16-30	2,521
mobile_reg_edit_cta_september_2013_16-30	4,803
Total users	12,824
Desktop registrations (baseline)
mobile_reg_desktop_september_2013_16-30	57,534

Note Due to a bug with campaign logging, we considered users as part of a CTA cohort whether they had a CTA campaign or the corresponding article_action parameter:

#Mobile reg watchlist CTA
SELECT DISTINCT(event_userId) FROM log.ServerSideAccountCreation_5487345 WHERE wiki='enwiki' AND event_displayMobile = 1 AND event_isSelfMade = 1 AND (event_campaign = 'mobile_watchPageActionCta' OR event_returnToQuery = 'article_action=watch') AND timestamp >=20130916000000 AND timestamp < 20130930000000;

#Mobile reg edit CTA
SELECT DISTINCT(event_userId) FROM log.ServerSideAccountCreation_5487345 WHERE wiki='enwiki' AND event_displayMobile = 1 AND event_isSelfMade = 1 AND (event_campaign = 'mobile_editPageActionCta' OR event_returnToQuery = 'article_action=edit') AND timestamp >=20130916000000 AND timestamp < 20130930000000;

#Mobile reg no CTA
SELECT DISTINCT(event_userId) FROM log.ServerSideAccountCreation_5487345 WHERE wiki='enwiki' AND event_displayMobile = 1 AND event_isSelfMade = 1 AND (event_campaign IS NULL OR event_campaign NOT IN ('mobile_editPageActionCta', 'mobile_watchPageActionCta')) AND (event_returnToQuery IS NULL OR event_returnToQuery NOT IN ('article_action=edit', 'article_action=watch')) AND timestamp >=20130916000000 AND timestamp < 20130930000000;

#Desktop registrations (baseline)
SELECT DISTINCT(event_userId) FROM log.ServerSideAccountCreation_5487345 WHERE wiki='enwiki' AND event_displayMobile = 0 AND event_isSelfMade = 1 AND timestamp >=20130916000000 AND timestamp < 20130930000000 GROUP BY 1;

Cohort analysis

1 edit in 24 hours

1+ ns0 edit 24h threshold
ID	Unique users	Editing users	%
Mobile registrations
mobile_reg_no_cta_september_2013_16-30	5,500	620	11.2%
mobile_reg_watchlist_cta_september_2013_16-30	2,521	51	2.0%
mobile_reg_edit_cta_september_2013_16-30	4,803	1,726	35.9%
Desktop registrations (baseline)
mobile_reg_desktop_september_2013_16-30	57,534	11,745	20.4%

1 edit in 7 days

1+ ns0 edit 7d threshold
ID	Unique users	Editing users	%
Mobile registrations
mobile_reg_no_cta_september_2013_16-30	5,500	651	11.7%
mobile_reg_watchlist_cta_september_2013_16-30	2,521	54	2.1%
mobile_reg_edit_cta_september_2013_16-30	4,803	1,751	36.4%
Desktop registrations (baseline)
mobile_reg_desktop_september_2013_16-30	57,534	12,355	21.5%

5 edits in 24 hours

5+ ns0 edit 24h threshold
ID	Unique users	Editing users	%
Mobile registrations
mobile_reg_no_cta_september_2013_16-30	5,500	56	1.0%
mobile_reg_watchlist_cta_september_2013_16-30	2,521	5	0.2%
mobile_reg_edit_cta_september_2013_16-30	4,803	124	2.6%
Desktop registrations (baseline)
mobile_reg_desktop_september_2013_16-30	57,534	1,533	2.7%

5 edits in 7 days

5+ ns0 edit 7d threshold
ID	Unique users	Editing users	%
Mobile registrations
mobile_reg_no_cta_september_2013_16-30	5,500	74	1.3%
mobile_reg_watchlist_cta_september_2013_16-30	2,521	5	0.2%
mobile_reg_edit_cta_september_2013_16-30	4,803	153	3.2%
Desktop registrations (baseline)
mobile_reg_desktop_september_2013_16-30	57,534	2,017	3.5%

Here is the above data in a spreadsheet.

Editor productivity

We compared the rate of productive editors (defined as those new editors with at least one non-reverted main namespace edit within the first week of contributions).

7-day productivity
ID	Unique users	Editing users	Productive users	%
Mobile registrations
mobile_reg_no_cta_september_2013_16-30	5,500	653	379	58.0%
mobile_reg_watchlist_cta_september_2013_16-30	2,521	54	27	50.0%
mobile_reg_edit_cta_september_2013_16-30	4,803	1752	879	50.2%
Desktop registrations (baseline)
mobile_reg_desktop_september_2013_16-30	57,534	12,393	8,744	70.5%