Research:Community portal redesign

Contact

Growth team

---

Research:Projects

This page documents a completed research project.

The purpose of this experiment was to test various redesigns to the Community portal on English Wikipedia, in order to identify the elements that will make it a more useful space for new and/or existing editors.

Background

Questions asked by new users at the Teahouse on English Wikipedia; the fifth most common question is where to find task suggestions or projects

The Community portal was created in 2004 on the English Wikipedia, as an editor-focused (as opposed to reader-focused) offshoot of the main page, and was originally a simple task list and a few links to fundamental policies and guidelines. Over time, as the number of editors grew and other pages like Village pumps and WikiProjects became more lively centers of community discussion, the Community portal became less important. However, the page is still linked from the sidebar and draws in around 10,000 pageviews a day.

A qualitative assessment of current Community portal viewers demonstrated that many of these pageviews probably come from readers and very new users who are looking for help or information from experienced editors, but most experienced editors no longer watch the page. As an additional datapoint, many of the new users who come to the Teahouse ask where to find task suggestions and/or general to-do lists and are guided to the Community portal's open task list. (See fig. right)

Unfortunately, the bot that updated the open tasks template with new tasks stopped running in February 2012.

These findings suggest that the Community portal might potentially fill a vital role for newer and less experienced users, but the current format is not serving this purpose effectively. Because the page is no longer used by most of the community of established editors, it may be an excellent space to iteratively test changes to design and layout and measure reader conversion and new editor engagement/retention.

Research questions

In this experiment, we are testing our hypotheses that:

1. It is mainly new and less experienced users of Wikipedia who are landing on the Community portal, and improving the information that is presented there will educate and empower these users to potentially become established Wikipedians.
2. Reducing and streamlining the information presented on the Community portal will make it a powerful resource for new and current Wikipedians alike.

Assumptions

1. The Community portal could potentially be a useful starting place/landing page for newer and less experienced users, but it is currently not set up with any target audience in mind.
2. The overwhelming number of links on the Community portal makes it unnavigable and unusable.
3. The redundant and outdated links on the Community portal confuse and disorient newer and less experienced users.
4. The lack of focus on any one audience (e.g., readers, new editors, established Wikipedians) makes the Community portal not useful for anyone.
5. The lack of focus on any one aspect of community engagement (e.g., suggestions for tasks to do, projects to join, help spaces to visit, etc.) makes the Community portal not useful for any one of these calls to action.
6. The design/layout of the Community portal does not effectively highlight the important information that is contained there and needs to be updated.

Methodology

• Make weekly changes to the design/layout of the Community portal page, bringing one main page element into focus (e.g., task recommendation, Wikiprojects, help pages, adopt-a-user program...)
• Measure clicks and impressions on the page and rate of successful conversions (e.g., tasks attempted/completed, WikiProjects joined, help pages used, new users adopted, etc.)
• Identify successful changes and discuss incorporating them into a stable new version of the page

Control data-gathering

The Community portal as of July 24, 2012, before testing

The opentask list as of July 24, 2012

• Measure current clickrate on page for one week to determine which elements are being used

Initial results

After three days of clicktracking, there were a total of 3212 links clicked (approx. 9% of the pageviews for that period, roughly 36,000). Below is some more fine-grained data:

Target

• 87% (2802) of the clicks came from unregistered users
• 13% (410) came from registered users
• As expected, there was no one overwhelming area where users clicked most. Among the more popular links were the Signpost (7%), the featured WikiProjects and Portals (each 6%), and, oddly, the Community portal itself (5%) – possibly because users were expecting something different on the page and clicked on the sidebar link again
• Just 3% of the clicks were to items in the Open tasks box, and only one of the articles clicked on was edited around the time of the experiment, meaning that none of these clicks were successful conversions

Referrer

• 37% (3425) of visitors who clicked on a link in this period were coming from the article namespace on Wikipedia
• 18% (1657) of visitors who clicked on a link were coming from the Main page
• 13% (1217) of visitors who clicked on a link were coming from a Special page (preferences, recent changes, watchlist, etc.)

One week results

After a week of clicktracking, there were a total of 9156 clicks on links in the Community portal, representing 13% of the approximately 70,000 pageviews for the period of July 27-August 2.

Users

• 86% (7847) of the clicks came from unregistered users
• 14% (1299) of the clicks came from registered users
• The majority of the registered editors (903, 70%) had made less than 1,000 edits to English Wikipedia

Links clicked

The majority of links clicked are located on the lefthand navigation toolbar. Some of the most popular included:

• The Community portal itself - 10% (888)
• Portals (e.g., Portal:Featured content) - 9% (818)
• The Main page - 9% (805)
• Special (e.g., Special:Random, Special:RecentChanges) - 9% (805)

Of the actual page content clicked, the most popular were:

• The Signpost content - 7% (635)
• Discussions (Requests for comment) - 6% (553)
• Articles in the open tasks area - 5% (418)
• WikiProjects - 4% (411)

Importantly, the general trend of clicks was highest at the top of the page and lowest at the bottom, and higher on the left than the righthand side, which is consistent with most eye-tracking studies of screen reading.

Conclusions

These quantitative findings support the qualitative feedback initially received: most visitors to the Community portal who click on any of its page elements are unregistered or relatively inexperienced editors. The haphazardness of elements clicked, and especially the very high number of clicks on the Community portal and other sidebar links, suggest that these users are not currently finding the page useful, and that there is much need for improved layout and design. The higher volume of clicks on links located in the top lefthand quadrant of the page suggests that more salient information should be placed there, so any topical redesign should be focused primarily in that area.

First iteration: tasking (8/6-813)

First iteration of opentask redesign, August 6, 2012

See also: Research:Community portal redesign/Opentask

Hypothesis: new users who come to this page might be looking for things to do/ways to help (per the Teahouse survey), but that information is not prominently displayed. This redesign will test the effect of placing a task list in the area where more visitors to the page are likely to see/interact with it.

Changes made:

• the header bars were removed/collapsed as much as possible
• the open tasks list was moved to the top lefthand corner of the page
• the task list was split into easier new-user-directed tasks (top) and more difficult experienced-user-directed tasks (bottom)

Initial results

The following was observed after a day of clicktracking:

• The overall number of clicks on the page increased (from an average of ~1308 a day during the control period to 1524 a day during the test period)
• The clickrate on open task articles went up from 3% of all clicks during the control period to 8% during the test period – a statistically significant increase
• However, the rate of task completion was also the same: no open task articles were edited during the experimental period
• Clicks on the open tasks list occurred mostly on articles in the first two task categories ("Add wikilinks" and "Copyedit", 18 and 20 of 123 open task clicks, respectively), followed in frequency by the two bottom task categories, "Requested articles" (12 of 123) and "Requested images" (10 of 123)

Second iteration: reordered task list (8/17-8/23)

To test for interest-dependence in tasking, the list of tasks was reordered to move high-click topics to the middle and low-click topics to the top - if clicks on tasks by topic remain the same, this will suggest that task list order does not play a significant confounding role in what kinds of tasks users are choosing to click on.

Results

Task	# of total clicks	# of unique clicks	# of returned clicks
Wikify	49	92%	8%
Copyedit	60	82%	18%
Remove original research	37	95%	2%
De-orphan	183	88%	12%
Verify	41	90%	10%
Merge	61	90%	10%
Split	91	93%	7%
Expand	166	90%	10%
Make neutral	36	94%	6%
Update	78	92%	8%
Discuss AfD	178	93%	7%
Clean up	55	87%	13%
Add image	7	100%	0%
Translate	104	95%	5%
Create	60	88%	12%
Total	1206

After one day of clicktracking, it appeared that people were still almost twice as likely to click on the two top-performing tasks, wikifying and copyediting, than on other tasks on the list, despite the fact that these two tasks no longer appeared in the eye-catching top area.

However, after 7 days, these results were reversed. Orphan and stub expansion, which were at the top of the new reordered list, became the most popular click area. The third most popular was AfD discussion, which was at the bottom:

Conclusion

It appears there is a weak relationship between task type and clicks, with people generally preferring wikifying and copyediting (two relatively simple and straight-forward tasks). However, there is a strong relationship between order of tasks and clicks—with items at the very top of the list receiving the most attention.

Third iteration: tasking (9/2-9/8)

Redesigned page and task list, September 2012

Collaboration-focused redesign, October 2012

Because clicks on articles went up significantly, but task completions did not, more design work on the task list seems promising.

Changes made:

• the task list was redesigned to place easier tasks in the high-priority page area (top left)
• more whitespace was added, tasks were placed on a grid, heavy formatting was removed
• prominent links to tutorial pages were added next to each task for receiving instruction on how to complete the tasks correctly

Results

Task	# of total clicks	# of unique clicks	# of returned clicks
Wikify	435	90%	10%
Copyedit	345	85%	15%
Remove original research	186	86%	14%
De-orphan	148	89%	11%
Verify	101	92%	8%
Merge	74	89%	11%
Split	80	90%	10%
Expand	94	93%	7%
Make neutral	101	90%	10%
Update	86	95%	5%
Discuss AfD	72	90%	10%
Clean up	128	96%	4%
Add image	0	0	0
Translate	92	92%	8%
Create	183	85%	15%
Total	2125

High-level data tables

Control period

August 7-13 (1 week)

• Pageviews: ~69,000
• Clicks (clicks on at least one outgoing link, did not close browser or click back/home button on browser): 10,522
• Clicks per pageview: 15%
• % of unique user clicks: 69%

Breakdown by user type and return/unique:

User type	# total clicks	% per pageviews	% per total clicks	# unique visitor clicks	% unique per user type	# returned visitor clicks	% returned per user type	% returned per pageviews
Anonymous	9038	13%	86%	6478	72%	4044	45%	6%
Registered 0 edits	355	1%	3%	265	74%	90	25%	0.1%
Registered 1-10 edits	236	0.3%	2%	126	53%	110	47%	0.2%
Registered > 10 edits	889	1%	8%	446	50%	443	50%	0.6%

Referrer page	# of referrals	% per total clicks	# unique referrals	% per total clicks
Articles	3537	34%	2827	27%
Main Page	2037	19%	1451	14%
Special:Search	970	9%	625	6%

Test 1 (reorganizing page, task list at top)

August 17-23 (1 week)

• Pageviews: ~56,000
• Clicks (clicks on at least one outgoing link, did not close browser or click back/home button on browser): 13,072
• Clicks per pageview: 23%
• % of unique user clicks: 71%

Breakdown by user type and return/unique:

User type	# total clicks	% per pageviews	% per total clicks	# unique visitor clicks	% unique per user type	# returned visitor clicks	% returned per user type	% returned per pageviews
Anonymous	11515	21%	88%	8343	72%	3172	28%	6%
Registered 0 edits	447	8%	3%	293	65%	154	34%	0.3%
Registered 1-10 edits	245	0.4%	2%	129	53%	116	47%	0.2%
Registered > 10 edits	892	2%	7%	491	55%	401	45%	0.7%

Referrer page	# of referrals	% per total clicks	# unique referrals	% per total clicks
Articles	3844	29%	2980	23%
Main Page	3666	28%	2781	21%
Special:Search	1363	10 %	965	7%

Test 2 (redesigned task list)

September 3-8 (1 week)

• Pageviews: ~52,500
• Clicks (clicks on at least one outgoing link, did not close browser or click back/home button on browser): 13,105
• Clicks per pageview: 25%
• % of unique user clicks: 68%

Breakdown by user type and return/unique:

User type	# total clicks	% per pageviews	% per total clicks	# unique visitor clicks	% unique per user type	# returned visitor clicks	% returned per user type	% returned per pageviews
Anonymous	11753	22%	90%	8105	69%	3648	31%	7%
Registered 0 edits	401	1%	3%	300	75%	101	25%	0.6%
Registered 1-10 edits	203	0.4%	2%	116	57%	87	43%	0.2%
Registered > 10 edits	797	2%	6%	406	51%	391	49%	1%

Referrer page	# of referrals	% per total clicks	# unique referrals	% per total clicks
Articles	4069	31%	3232	25%
Main Page	2105	16%	1518	12%
Special:Search	2526	19%	1732	13%

Test 3 (redesigned task list)

September 9-15 (1 week)

• Pageviews: ~55,000
• Clicks (clicks on at least one outgoing link, did not close browser or click back/home button on browser): 11981
• Clicks per pageview: 22%
• % of unique user clicks: 68%

Breakdown by user type and return/unique:

Possible next steps

Task suggestions/recommendations

to make the open task list more prominent and useful

Potential changes:

1. Move "Help out" section above the fold and move "Open tasks" to the left (higher visibility and importance) Done
2. Streamline and simplify the task recommendation list to make it more user-friendly (fewer categories and tasks, more white space between links) Done
3. Test random articles (current) against targeted articles (low quality/high importance articles, similar to the "high priority" algorithm used by Suggestbot) In progress…

Featured WikiProjects/collaborations

to surface current collaborations of note

Potential changes:

1. Remove Signpost and CENT
2. Replace the Community bulletin board with a reduced number of featured projects/collaborations, presented in a more newbie-friendly way
3. Add images
4. Reload with a different project/projects with each pageload

Help/mentoring

to surface useful help and mentoring spaces in an non-overwhelming way

Potential changes:

1. Remove all sidebars, navbars, boxes, etc., except for "Guidelines and resources"
2. Reduce number of links to help spaces to just three or four most essential/active ones (e.g., Teahouse, Help desk, Reference desk)
3. Feature three or four experienced users seeking adoptees