Learning and Evaluation/Archive/Boiler room/Portal plan

Ask questions and discuss program evaluation

The project planning document

This page is for planning the design of the Evaluation Portal that will be maintained by the Program Evaluation & Design (Programs) and the Learning & Evaluation (Grantmaking) teams in the Wikimedia Foundation. The evaluation portal will contain resources that help Wikimedia movement entities (Chapters, user groups, and individuals) pursue mission-aligned activities successfully and measure the impact of those activities effectively.

We're using this page to specify the information architecture and functional requirements for the project space itself.

Nomenclature

(TL;DR most of this is basic, but including lots of terms for thoroughness)

Landing page. The page people land on! The top-level in the navigation hierarchy; links to the pages in the next level. Generally hosts very little longform content of its own.
Content page. One of the reader-facing pages of the Evaluation Portal (e.g. "Printing press", Library/Learning_patterns", "Parlor/Introductions").
Learning resource. A page that presents information related to evaluation in a consistent format, such as a learning module or design pattern.
- Learning module. A step-by-step, multi-page instruction resource. (example)
- Learning pattern. A single-page design pattern that offers an actionable solution to a well-defined problem. (example)
- Case study. A longer report or post-hoc analysis of an event, project or program. (example)
Call to action. A widget or message (such as a button or "see more…" link) that encourages someone to take an action, such as navigating to another page to view something, or leaving a comment.

Basics

This portal will be used by a diverse set of participants with a diverse set of needs. The content the portal provides reflects that diversity. Unlike other interactive project spaces like the Teahouse and IdeaLab, which are focused on particular tasks, we are inviting people to participate in and through the portal in quite a few different ways: people can ask and answer questions on the Questions page, create a profile on the People page, make an announcement in the Press room, create a Learning Pattern, join social media or IRC groups, write a blog post, and of course... post on the talk page.

But most people will just want to read the information resources we've created. So making it easy for people to find the right resources is important. We can use several different approaches to make stuff easier to find:

good global navigation menus.
no more than three levels of hierarchy before you get to a resource page.
standardizing the format of resource pages as much as possible. A standard format makes the documents more readable, by both people and bots. It also gives the impression that this is a curated collection of resources, not just a jumble.
surfacing relevant content through galleries, feeds and guides.
keeping non-resource pages as short as possible, so that people don't have to scroll a lot and read through lots of text for important links.
encouraging people to ask questions.

Sitemap

The pages in the evaluation portal. Does not include pages for learning resources (such as individual case studies, learning patterns or learning modules), which may exist outside the Programs namespace entirely.

Grants:Evaluation: the portal landing page. (full description)
- Grants talk:Evaluation: the primary portal talk page. Other talk pages redirect here. (full description)
/Boiler room: the back-end project workspace, for coordination and maintenance activities and information related to the Evaluation portal itself.
- Grants talk:.../Boiler room: the primary back-end talk page for project coordination.
- /Boiler room/Portal plan: the planning document for the Evaluation portal.
/About: describes the purpose of the portal and the teams that run it. (full description)
- /About/FAQ: a list of Frequently Asked Questions about the portal, and evaluation in general. (full description)
/News: the Printing Press, a place for news and outreach. (full description)
/Parlor: the hub for social interaction on the portal. (full description)
- /Parlor/Questions: a Q&A forum where anyone can ask and answer questions related to evaluation. (full description)
- /Parlor/Introductions: a page where anyone can introduce themselves, or affiliate themselves with the project. (full description)
/Tools: the Machine Shop, a hub for learning resources related to WMF-supported evaluation tools. (full description)
- /Shop/Wikimetrics: information on Wikimetrics, and a guide of related learning resources. (full description)
- /Shop/Qualtrics: information on Qualtrics, and a guide of related learning resources. (full description)
- /Shop/Community visualization: information on the community visualization tool, and a guide of related learning resources. (full description)
/Library: the hub for finding learning resources, and browseable guides of all types of learning resources. (full description)
- /Library/Case studies: information on case studies, and a guide of existing case studies.
- /Library/Learning modules: information on learning modules, and a guide of existing modules. (full description)
Grants:Learning patterns: information on learning patterns, a guide of existing patterns, and a call to action to add a pattern to the library. (full description)

Primary pages

Detailed descriptions of the main content pages of the Evaluation portal, with a brief rationale for each page.

Programs:Evaluation portal

The landing page for the whole site. The place we expect most people to visit first.

Content

primary navigation, with major links (maybe: About - Library - Machine shop - Parlor - People - Press)
brief summary of the goals and scope of the Evaluation portal (a few sentences at most).
One (or two?) galleries that feature dynamic profiles of learning resources and people. Similar to the galleries on the IdeaLab.
Activity feed that shows a variety of recent events (resources recently added, people who joined, etc.)
Calls to action, in the form of mid-length (2 sentences and a picture?) static profiles that advertise and link to the first-level subpages of the portal: Parlor, Library, Press, About, and Machine Shop.

Considerations

We don't expect people to linger, so we don't need a lot of extraneous text beyond what's in the profiles.
the 'about' Call to Action could be visually separated from the other four, since that page presents a different kind of information. Maybe a short sentence is all that's needed "Find out about Program Evaluation & Design".
the gallery and activity feed should be near the top, above the fold.

Programs_talk:Evaluation portal

This is the primary talk page for the whole portal, and is available behind the 'Discussion' tab on the landing page.

Content

General discussion forum, so minimal formatting. Maybe a banner at the top advertising the Q&A forum to people who want to ask questions?

Considerations

We probably don't have to advertise this page; Wikipedians know to look for it. All other talk pages will be redirected to this one, to keep discussions centralized.

/About

This page offers a simple summary of the Portal and its purpose.

Content

Paragraph explaining how the Portal came to be, what it's for, and the teams that run it.
Links to the Program Evaluation and Learning & Evaluation spaces on Meta, maybe to the Grantmaking & Programs section on Foundation wiki.
Prominent Call to Action to browse the Frequently Asked Questions

Considerations

I prefer to make the FAQ a separate subpage (see below) and then transclude it onto this page. I prefer to keep different kinds of content on separate pages. But we could also just make the FAQ part of the /About page itself.
Although this page will talk about the two teams that are running the portal, I don't think it makes sense to advertise individual team members or put contact information here. We should direct people to ask questions in public (the Q&A board, the talk page). The team members will have profiles on Parlor/People, just like everyone else. Contact info, if needed, could be put in the /Press page.

/About/FAQ

Frequently asked questions about evaluation, the role of PED or L&E, etc.

Content

To start with, will contain whatever we anticipate people will ask.
Eventually, we can add in questions and answers directly from the Q&A board, if people seem to be asking the same question over and over in that forum.

Considerations

We expect this list to grow, which is one of the reasons it might be better to have it on its own dedicated page to begin with.
Should this be linked to directly from the landing page? Or in the global navigation?

/Press

The printing press, a news and communication hub.

Content

News bites. A manually curated list of recent/upcoming events. Unlike the activity feed, these items can have longer summaries.
Perhaps: a simple call to action encouraging other stakeholders to add their own news bites (for instance, a chapter could advertise an upcoming workshop here).
Links to communication channels. Examples: WMF blog, Facebook group, IRC channel, external event calendar (an open google calendar?).
Call to action to write a blog post about a relevant activity or experience.
Contact info for media-related requests, like you would find on a "Press" or "Media" page of a normal website. Email or User talkpage for contact person.

Considerations

If we have a call to action encouraging people to write a blog post, we should link to instructions on how to write a post for the WMF blog, and who to contact.
If we link to the blog itself, would be useful to link directly to a page that displays a list of all previous posts that deal with PE&D (improves findability).

/Parlor

A kind of secondary landing page, which directs people to our two primary community pages:/Parlor/People and /Parlor/Questions.

Content

Short summary (1-2 sentences) about what the parlor is, and what you will find there.
Two galleries: one for recent questions, one for profiles of recently-created (or recently active) people. Similar to the Teahouse landing page.
Calls to action for each gallery: "create a profile" and "ask a question".

Considerations

none, yet

/Parlor/Introductions

The profile page, where people can introduce themselves and talk about their role or interest in PE&D.

Content

A guide containing user profiles, sorted for recent activity by GrantsBot.
A prominent call to action to create a profile, which triggers a simple profile-creation workflow similar to the one on the Teahouse and IdeaLab.

Considerations

We need to decide what kind of information we want to include in these profiles. Rather than asking people for their general interests or biographical information, could be useful to ask them to list skills, affiliations, etc. Could even encourage people to advertise events they have led, projects they have participated, or resources they have contributed to (like patterns, or project reports) in their profile.

/Parlor/Questions

A Q&A board for all things Evaluationary!

Content

Questions. And (hopefully) answers.
Call to action: "Ask a question."
Short encouraging statement letting people know they can ask any sort of question here, and anyone can answer it.
Prominent link to FAQ, in addition to the one that may be present in the global nav.

Considerations

The Teahouse Gadget would be fine here. It surfaces recent questions, and reduces barriers to participation for more peripheral movement partners--which is important since this is meant to be an inclusive space, after all. People who don't want to use it can edit directly, or post to the main portal talk page instead.
We'll eventually want an archiving bot. Not sure if there's one active on Meta already?
We'll want to encourage people who respond to link to the questioner's talk page in their response, which triggers a Notification for the user, letting them know someone has responded.

/Library

The main hub for our various learning resources related to evaluation.

Content

Short explanation of the types of resources we offer.
Guides for each of our various types of learning resources that we can make profiles for. Currently I believe we can do this for Learning Patterns and Learning Modules. In the future, we could do it for Case Studies too. Each guide is a list of profiles generated from the individual resource pages. Similar to the way the IdeaLab Ideas page contains several different guides for different classes of idea (New, IEG Draft, Participants wanted).
Prominent link to the Machine shop, which will also display information about learning resources that are related to our tools.
Call to action to create a new learning resource (Pattern?)
List of links to other learning resources, like presentations, glossary, externally-created resources or to a /Resources subpage that hosts such a list of links.

Considerations

We want to keep this page from becoming a forest of links!
There will likely need to be subpages of this page that describe each type of learning resource in detail. We probably shouldn't try to host that content directly on this page, because that would likely make the page much longer than it should be.
There won't be a large number of Learning Patterns or Learning Modules available at the beginning, but that's okay. The trick is designing the page so that it is useful with a little content, but scales well to host a lot more content in the future.

/Library/Learning_modules

Contains the guide of all existing learning modules (which is transcluded onto /Library) as well as basic information about learning modules.

/Library/Learning_patterns

Contains the guide of all existing learning patterns (which is transcluded onto /Library) as well as basic information about learning patterns.

/Shop

The Machine shop is kind of a specialized Library page, focused on the evaluation tools we offer and how to use them.

Content

Brief summary of what these tools are for, letting people know that they are available for any stakeholders to use, not just Foundation staff.
Contact information for a 'technical lead' who should be contacted if someone has trouble setting up one of the tools.
Call to action to ask questions on the /Parlor/Questions page.
A brief description of each of the tools we have available.
Calls to action to browse the specific pages devoted to the description of each of the tools we currently offer: WikiMetrics, Community Visualization(?), and Qualtrics surveys.

Considerations

none, yet.

/Shop/Wikimetrics

Contains a basic description of Wikimetrics, what it's used for, and links to the Wikimetrics learning module and other related resources.

/Shop/Qualtrics

Contains a basic description of Qualtrics, what it's used for, and links to survey-related learning patterns and other related resources.

/Shop/Community_visualization

Contains a basic description of the Community Visualization tool, what it's used for, and links to any modules, patterns or other related resources.

/Boiler room

The boiler room is the equivalent of the Teahouse Hosts Lounge. It's a space for project coordination, and will have it's own dedicated talk page. It should be linked to from some other pages on the Portal for transparency, but it may be desirable to leave it out of the primary navigation.

/Boiler room/Portal plan

This page! Project documentation.

/Boiler room/Forge

Coordinating space for evaluation teams to develop resources, reports, etc. Mainly used by staff.

Maintenance pages

These pages are relevant to the design and functionality of the Evaluation portal, but are not intended to be viewed or edited directly by end users.

Templates

Template:Evaluation/Pattern/Infobox: infobox for all learning patterns.
Template:Evaluation/Pattern/Infobox/Links: header links for Template:Evaluation/Pattern/Infobox.
Template:Evaluation/Pattern/Endorse: template button for endorsing a pattern. Used in Template:Evaluation/Pattern/Infobox.
Template:Evaluation/Pattern/Button: button that triggers the creation of a new pattern. Accessed through Grants:Learning patterns and possibly elsewhere in the portal TBD.
Template:Evaluation/People/Button: button that triggers the creation of a new participant profile. Accessed through Grants:Evaluation/Parlor/Introductions and possibly elsewhere in the portal TBD.
Template:Evaluation/Questions/Header: the page header for Grants:Evaluation/Parlor/Questions
Template:Evaluation/Activity/Feed: template for the activity feed. currently transcludes Grants:Evaluation/Activity.
Template:Evaluation/Activity/Entry: template for activity feed profiles.
Template:Evaluation/Feature: template for featured content.
Template:Evaluation/Questions/Button: template for the "Ask a question" button.
Template:Evaluation/Body: the body of each primary content page. Contents go in the "content=" parameter.
Template:Evaluation/Header: the navigation and header for each portal content page.
Template:Evaluation/Introductions/Profile: the participant profile template. the profile template for introductions in /Parlor/Introductions. Used in Grants:Evaluation/Parlor/Introductions/Preload.
Template:Evaluation/Pattern/Type: uses switch to categorize the pattern by type based on the 'type=' infobox parameter in Template:Evaluation/Pattern/Infobox. Category selected must exist. All available categories should be listed in Grants:Learning patterns/Categories#Category keywords.
Template:Evaluation/Resource/Summary: the profile template for lists (guides) of evaluation resources, like the one on Grants:Learning patterns.
Template:Evaluation/Box: callout box for drawing attention to short messages. Created because tables don't work inside Template:Evaluation/Body. Currently used in Grants:Learning patterns.
Template:Evaluation/List frame. Another table for scrolling lists. Created for similar reasons as Template:Evaluation/Box. Currently used in Grants:Learning patterns/Introduction/Preload.
Template:TH guillemet: because a life without guillemets is no life at all.
Template:Evaluation/Information:allows for a small "i" icon that can be loaded with both a link and a tooltip that displays when you hover.
Template:Evaluation/Introductions/Profile: template for user-created profiles.

NEEDED TEMPLATES

Template:Evaluation/Resource/Infobox: a generic infobox for all learning resources that we want to feature, other than patterns (which have a special infobox 'cause they are more dynamic). Put in a couple parameters like "summary" and "image" so that at some point in the future GrantsBot can automatically make profiles for each new resource that is created.

MYSTERY TEMPLATES

Template:Evaluation/Resource: what does this do?
Template:Evaluation/Categories/Template: what is this for?

Process pages

Grants:Learning patterns: top level page under which all patterns created through Template:Evaluation/Pattern/Button workflow are organized.
Grants:Evaluation/Parlor/Questions/Question-form: the formatted question button.
Grants:Evaluation/Survey landing: landing page where people who take Grantmaking/Programs surveys on Qualtrics are left at.
Grants:Evaluation/Translate: information on how/when to translate a pattern (not currently in use)

Content creation workflows

Grants:Learning patterns/Introduction/Preload: preloaded content when Template:Evaluation/Pattern/Button is triggered.
Grants:Learning patterns/Introduction/Editintro: preloaded instructions when Template:Evaluation/Pattern/Button is triggered.
Grants:Evaluation/Library/Learning patterns/Categories: list of pattern categories transcluded into Grants:Learning patterns/Introduction/Preload via Template:Evaluation/List frame.
Grants:Evaluation/Parlor/Introductions/Preload: preloaded content when Template:Evaluation/People/Button is triggered.
Grants:Evaluation/Parlor/Introductions/Editintro: preloaded instructions when Template:Evaluation/People/Button is triggered.
Grants:Evaluation/Endorse/Editintro: preloaded instructions when Template:Evaluation/Pattern/Endorse is triggered.
Grants:Evaluation/Translate/Editintro: preloaded instructions when someone signals that a pattern is ready to be translated.

Activity surfacing

Grants:Evaluation/Activity: This will be the activity feed for the Evaluation Portal. it will be transcluded onto Grants:Evaluation, and possibly onto the Grants:Evaluation/Library and Grants:Evaluation/Parlor pages.
Grants:Evaluation/Library/Featured: Index page for featured learning resources. Transclude a random subpage (1-6) of this page to power a rotating content gallery.
Grants:Evaluation/Parlor/Featured: Index page for featured participant intros & recent questions. Transclude a random subpage (1-6) of this page to power a rotating content gallery.

To do

Done update all existing patterns with correct template parameters.
assure activity feed entries have correct parameters (Template:Evaluation/Activity/Entry]])
Done add header & remove "draft" template to/from all main pages
summary content for all pages
Done update the text in the question box so it doesn't say wikipedia
Donecreate real editintro for Programs:Evaluation portal/Endorse/Editintro
Done create real editintro for Programs:Evaluation_portal/Library/Learning_patterns/Editintro
redirect all the dang talk pages, at least the ones in the Program:namespace.
feature some resources other than patterns
Done update pattern-creation Editintro
Done add in type= param to learning pattern profiles
Done hide community visualization stuff
make profiles for the other learning modules: Meta:Training
Done Add MiszaBot to Q&A forum.
Add Archives box to Q&A forum. See Template:Archive_box_auto (@ User:Jmorgan (WMF) can you give this a go? I fail at it every time I do it.
Done Remove test Q&A, populate with Q&A from old forum.
Done Create Forge (Sarah)
Done Create other resources section (Sarah)
Redirect shortcuts from old PE&D portal
FIgure out what the hell to do with the old share space content.

Heather is gonna work on this

Done update Template:Evaluation/Pattern/Infobox and Template:Evaluation/Pattern/Infobox/Links to match portal theme (colors, fonts & iconography).
set up /Library with transcluded lists of resources

Heather: I didn't see that you'd taken this on, and I started to do this. Feel free to change everything I did. I'll keep my hands off for now :) Jmorgan (WMF) (talk) 01:02, 1 October 2013 (UTC)[reply]

Post launch

Check out questions and ideas in the boiler room talk page
Have both teams add to the FAQ
Make it possible for people to suggest categories?
badges/stickers to signify endorsements, translations, quality (as per AFT)
sublists of patterns by category (either in category trees, or transcluded subpages)
Make it more obvious (in the edit window perhaps) that the talk page in the portal space is for the portal itself not to talk about evaluation. Take that to the Parlor.
Design an invitation for talk pages to invite portal visitors to create profiles. SarahStierch (talk) 19:52, 16 December 2013 (UTC)[reply]