Jump to content

Learning and Evaluation/Archive/Boiler room/Portal Redesign Plan

Add links
From Meta, a Wikimedia project coordination wiki

This page is for planning the redesign of the Evaluation Portal that will be maintained by the Learning and Evaluation team (Grantmaking) of the Wikimedia Foundation. The evaluation portal will contain resources that help Wikimedia movement entities (Chapters, user groups, and individuals) pursue mission-aligned activities successfully and measure the impact of those activities effectively.

We're using this page to specify the information architecture and functional requirements for the project space itself.

Nomenclature

[edit]
  • Landing page. The page people land on! The top-level in the navigation hierarchy; links to the pages in the next level. Generally hosts very little long-form content of its own.
  • Content page. One of the reader-facing pages of the Evaluation Portal (e.g. "News", "Library", "Parlor," "Tools").
  • Learning resource. A page that presents information related to evaluation in a consistent format, such as a learning module or design pattern.
    • Learning module. A step-by-step, multi-page instruction resource. (example)
    • Learning pattern. A single-page design pattern that offers an actionable solution to a well-defined problem. (example)
    • Case study. A longer report or post-hoc analysis of an event, project or program. (example)
    • Survey Objective. A Survey Question Bank page of example survey items by question objective. New needs design of templates
    • Program Toolkit. A toolkit with program description, resource strategies, best practices for design and evaluation including samples and templates and program support information. New needs design of templates
  • Call to action. A widget or message (such as a button or "see more…" link) that encourages someone to take an action, such as navigating to another page to view something, or leaving a comment.

Basics

[edit]

This portal will be used by a diverse set of participants with a diverse set of needs. The content the portal provides reflects that diversity. Unlike other interactive project spaces like the Teahouse and IdeaLab, which are focused on particular tasks, we are inviting people to participate in and through the portal in quite a few different ways: people can ask and answer questions on the Questions page, create a profile on the People page, make an announcement in the Press room, create a Learning Pattern, join social media or IRC groups, write a blog post, and of course... post on the talk page.

But most people will just want to read the information resources we've created. So making it easy for people to find the right resources is important. We can use several different approaches to make stuff easier to find:

  • good global navigation menus.
  • no more than three levels of hierarchy before you get to a resource page.
  • standardizing the format of resource pages as much as possible. A standard format makes the documents more readable, by both people and bots. It also gives the impression that this is a curated collection of resources, not just a jumble.
  • surfacing relevant content through galleries, feeds and guides.
  • keeping non-resource pages as short as possible, so that people don't have to scroll a lot and read through lots of text for important links.
  • encouraging people to ask questions.


Sitemap

[edit]

The pages in the evaluation portal. Do not include pages for learning resources (such as individual case studies, learning patterns or learning modules), which may exist outside the ProgramsGrants namespace entirely.

Formerly: Programs:Evaluation portal| see (full description)
Formerly: Programs talk:Evaluation portal see (full description)

All the following under Grants:Evaluation:

"ABOUT"

  • /About: describes the purpose of the portal and the teams that run it.
Formerly: Programs:Evaluation portal/About see (full description)
  • /About/FAQ: a list of Frequently Asked Questions about the portal, and evaluation in general.
Formerly:Programs:Evaluation portal/About/FAQ: a list of Frequently Asked Questions about the portal, and evaluation in general. (full description)

"NEWS AND EVENTS"

Formerly: Programs:Evaluation portal/Press: the Printing Press, a place for news and outreach. see (full description)

"COMMUNICATE"

  • /"Communicate": the hub for social interaction on the portal. Needs a name
Formerly: Programs:Evaluation portal/Parlor: see (full description)
  • /Parlor/Questions: a Q&A forum subpage to main communicate page where anyone can ask and answer questions related to evaluation.
Formerly: Programs:Evaluation portal/Parlor/Questions see (full description)
  • /Parlor/Introductions: a subpage to "Communicate" page where anyone can introduce themselves, or affiliate themselves with the project.
Formerly:Programs:Evaluation portal/Parlor/Introductions: see (full description)

"LEARN"

  • /"Learn": the hub for finding learning resources, and browsable pages of all types of learning resources (i.e. Patterns, Modules, Glossary - w/ less prominent linkage to News & Events and Case Studies ad Reports pages). Possible name with other verb for "learning"?
Formerly: Programs:Evaluation portal/Library: the hub for finding learning resources, and browseable guides of all types of learning resources. (full description)
Formerly: Programs:Evaluation portal/Library/Learning modules: information on learning modules, and a guide of existing modules. (full description) Need a new name?
  • Learning patterns: information on learning patterns, a guide of existing patterns, and a call to action to add a pattern to the library.
(see full description)
  • /"Learn"/Glossary: Glossary of evaluation terms for WIkimedia program evaluation. (Needs elevation more than among "other resources")
Formerly: Programs:Evaluation portal/Library/Glossary: Glossary of evaluation terms for WIkimedia program evaluation.

"SHARE"

  • /"Share"Learning patterns, case studies, and reports and a searchable guide of each of those subpages. This takes case studies page from part of library to inclusion in a higher prominence category to let people know to contribute or "Share" Possible other verb for "sharing"?
NEW ORGANIZATION CATEGORY/LEVEL Name with another verb for "planning"?
Formerly: Programs:Evaluation portal/Library/Case studies: information on case studies, and a guide of existing case studies.
Formerly: Programs:Evaluation portal/Library/Overview with subpages /Edit-a-thons, /Editing workshops, /GLAM content donations, /WLM, /On-wiki writing contests, /Other photo initiatives, /WEP. Each of the subpages will need to have a template connection at the bottom of each program report to program toolkits.
NEW PAGE IN PLANNING
  • /"Plan": Templates for planning a program, Program Directory, Program Guides, Process tools (i.e., logic model guide, tracking and reporting toolkit, goals and objectives)
NEW ORGANIZATION CATEGORY/LEVEL Name with another verb for "planning"?
  • /"Plan"/Program Toolkits: Program tool kits and guides for planning, implementation, and evaluation of select Wikimedia programs.
This page needs design and template creation

"MEASURE"

  • /"Measure": the tool shed, a hub for learning resources related to WMF-supported evaluation tools.
Formerly: Programs:Evaluation portal/Shop: the Machine Shop, a hub for learning resources related to WMF-supported evaluation tools. see (full description) Possible name with other verb for "measuring"?
  • /Tools/Wikimetrics: information on Wikimetrics, and a guide of related learning resources.
Formerly: Programs:Evaluation portal/Shop/Wikimetrics: see (full description)
  • /Tools/Qualtrics: information on Qualtrics, and a guide of related learning resources.
Formerly: Programs:Evaluation portal/Shop/Qualtrics: see (full description)
Formerly: Programs:Evaluation portal/Shop/Community visualization: see (full description)

BACK-END

  • Boiler room: the back-end project workspace, for coordination and maintenance activities and information related to the Evaluation portal itself.
Formerly: Programs:Evaluation portal/Boiler room: Need to rename - portal workspace
  • [[Grants talk:Evaluation/ Need to rename|Grants talk:.../Boiler room]]: the primary back-end talk page for project coordination.
Formerly: Programs talk:Evaluation portal/Boiler room|Programs talk:.../Boiler room]]:
Formerly: /Boiler room/Forge
Formerly: /Boiler room/Portal plan:

Primary pages

[edit]

Detailed descriptions of the main content pages of the Evaluation portal, with a brief rationale for each page.

Grants:Evaluation

[edit]

The landing page for the whole site. The place we expect most people to visit first.

Content
  • Primary navigation, with major links (maybe: About - Library - Tools - Parlor - News)
  • Brief summary of the goals and scope of the Evaluation portal (a few sentences at most).
  • Who we are (L&E) with pics and our titles so they know who is PE and who is learning strategy?
  • Definition of program leaders (organizers, funders, implementers, evaluators) as an evaluation community with surfacing of other profiles from parlor.
  • Calls to action and links to the first-level subpages of the portal: Parlor, Library, News, About, and Tools.
  • Links to other social media outlets: IRC, Facebook, Google+, Twitter and mailing list sign up
  • Two galleries that feature dynamic profiles of learning resources and people. (Could we integrate to include the latest news item?)
  • Activity feed that shows a variety of recent events (resources recently added, people who joined, etc.)
Considerations
  • Need easy-to-read font and size
  • Social networking links not logos, just links - while this redirects traffic, it helps bring people back to the portal when they sign up for these.
  • A simple landing page, not noisy - aiming to not be overwhelming and easy guidance to the appropriate pages. (I really like the top of Grants:Start)
  • We don't expect people to linger, so we don't need a lot of extraneous text beyond what's in the templated profiles.
  • the 'about' Call to Action could be visually separated from the other four, since that page presents a different kind of information. Maybe a short sentence is all that's needed "Find out about Program Evaluation & Design".
  • The main portion of the landing page should be static. With the galleries and feeds to follow so if any breaks,the main functionality of the page will not be interrupted by any distractions.

Grants talk:Evaluation

[edit]

This is the primary talk page for the whole portal, and is available behind the 'Discussion' tab on the landing page and all main (Parlor, News, Library, About, Tools) pages.

Content
  • General discussion forum, so minimal formatting for now - would like flow if it ever becomes functional
  • A banner at the top advertising the Q&A forum to people who want to ask questions
Considerations
  • We don't have to advertise this page; Wikipedians know to look for it. All other talk pages will be redirected to this one, to keep discussions centralized.

/About

[edit]

This page offers a simple summary of the Portal and its purpose.

Content
  • Paragraph explaining how the Portal came to be, what it's for, and the teams that run it.
  • Links to the Program Evaluation and Learning & Evaluation spaces on Meta, maybe to the Grantmaking & Programs section on Foundation wiki.
  • Prominent Call to Action to browse the Frequently Asked Questions
Considerations
  • FAQ is a separate subpage (see below) and transcluded onto this page. But we could also just make the FAQ part of the /About page itself.
  • Individual team members or contact information here? We should direct people to ask questions in public (the Q&A board, the talk page). The team members will have profiles on Parlor/People, just like everyone else. Contact info, if needed, could be put in the /Press page.

/About/FAQ

[edit]

Frequently asked questions about evaluation, the role of PED and L&E more generally

Content
  • Anticipated or asked questions and answers including from the Q&A board, if people seem to be asking the same question over and over in that forum.
Considerations
  • We expect this list to grow, which is one of the reasons it might be better to have it on its own dedicated page to begin with.
  • Should this be linked to directly from the landing page? Or in the global navigation?

/News_and_Events

[edit]

The press and announcement page, a news and event communications hub.

Content
  • Link to other social media - facebook, #wikimedia-PED on IRC, @WikiEval on twitter, and google plus
  • News bites. A manually curated list of recent/upcoming events. Unlike the activity feed, these items can have longer summaries.
  • Perhaps: a simple call to action encouraging other stakeholders to add their own news bites (for instance, a chapter could advertise an upcoming workshop here).
  • Links to communication channels. Examples: WMF blog, Facebook group, IRC channel, external event calendar (an open google calendar?).
  • Call to action to write a blog post about a relevant activity or experience.
  • Contact info for media-related requests, like you would find on a "Press" or "Media" page of a normal website. Email eval@wikimedia.org.
  • Link to archived news page along with an archived news page
Considerations
  • NEED TO DISCOURAGE TRANSLATION OF ANYTHING ON THIS PAGE THAT IS NOT A PAST EVENT AND?OR STRATEGY TO MIGRATE TRANSLATION MARK-UPS TO ARCHIVED NEW PAGE

These are things that were factored into the current templates:

  • If we have a call to action encouraging people to write a blog post, we should link to instructions on how to write a post for the WMF blog, and who to contact.
  • If we link to the blog itself, would be useful to link directly to a page that displays a list of all previous posts that deal with PE&D (improves findability).
Template for /News_and_Events
  • Can we delete the menu on the top? The idea is to have all the sections as modules, collapsed, that can be expanded by the user. I would do the same for Library and Tools. (María posted this.)
Template for /News#hot (same as /News/Archive)
  • Can we adjust the current template? Suggestions:
  • Date on top, with small fonts.
  • Title with link.
  • Brief description (as it is now).
  • Photo aligned with the title (not higher), and with no border.

(María posted this.)

/"Communicate"

[edit]

A kind of secondary landing page, which directs people to our two primary community pages:/Parlor/People and /Parlor/Questions.

Edward's thoughts:

  • There are over 100 people in the facebook group and having a paragraph about each person is probably too much since we expect it to grow. The social page should be more of a list or gallery of faces that includes names, country, affiliations, that can be sorted by program/org affiliation (GLAM, WEP, Wiki Loves Monuments, Chapters, etc.). This might help people easier identify who they can/should be connecting with on/off wiki.
Content
  • Short summary (1-2 sentences) about what the parlor is, and what you will find there.
  • Two galleries: one for recent questions, one for profiles of recently-created (or recently active) people. Similar to the Teahouse landing page.
  • Calls to action for each gallery: "create a profile" and "ask a question".
Considerations
  • none, yet

/"Communicate"/Introductions

[edit]

The profile page, where people can introduce themselves and talk about their role or interest in PE&D.

Content
  • A guide containing user profiles, sorted for recent activity by GrantsBot.
  • A prominent call to action to create a profile, which triggers a simple profile-creation workflow similar to the one on the Teahouse and IdeaLab.
Considerations
  • We need to decide what kind of information we want to include in these profiles. Rather than asking people for their general interests or biographical information, could be useful to ask them to list skills, affiliations, etc. Could even encourage people to advertise events they have led, projects they have participated, or resources they have contributed to (like patterns, or project reports) in their profile.

/"Communicate"/Questions

[edit]

A Q&A board for all things Evaluation!

Content
  • Questions and answers.
  • Call to action: "Ask a question."
  • Short encouraging statement letting people know they can ask any sort of question here, and anyone can answer it.
  • Prominent link to FAQ, in addition to the one that may be present in the global nav.
  • Q&A curation?
Considerations

These are things that were factored into the current templates:

  • The Teahouse Gadget would be fine here. It surfaces recent questions, and reduces barriers to participation for more peripheral movement partners--which is important since this is meant to be an inclusive space, after all. People who don't want to use it can edit directly, or post to the main portal talk page instead.
  • We'll eventually want an archiving bot. Not sure if there's one active on Meta already?
  • We'll want to encourage people who respond to link to the questioner's talk page in their response, which triggers a Notification for the user, letting them know someone has responded.

/"Learn"

[edit]

The main hub for our various learning resources related to evaluation.

Content
  • Short explanation of the types of resources we offer.
  • Guides for each of our various types of learning resources that we can make profiles for. Currently I believe we can do this for Learning Patterns and Learning Modules. Each guide is a list of profiles generated from the individual resource pages. Similar to the way the IdeaLab Ideas page contains several different guides for different classes of idea (New, IEG Draft, Participants wanted).
  • Prominent link to learning modules
  • Call to action to create a new learning pattern
  • List of links to other learning resources, like presentations, glossary, externally-created resources or to a /Resources subpage that hosts such a list of links.
Considerations
  • We need curation and additional layering, bots?, to clean up the "forest" here - we need to still do the second bullet point below as well - we are not scaling well at all currently
/"Learn" template

Can we change the current template? See Template comment on News section. --MCruz (WMF) (talk)

These are things that were factored into the current templates:

  • We want to keep this page from becoming a forest of links!
  • There will likely need to be subpages of this page that describe each type of learning resource in detail. We probably shouldn't try to host that content directly on this page, because that would likely make the page much longer than it should be.
  • There won't be a large number of Learning Patterns or Learning Modules available at the beginning, but that's okay. The trick is designing the page so that it is useful with a little content, but scales well to host a lot more content in the future.

/"Learn"/Learning modules

[edit]
Formerly Programs:Evaluation portal/Learning modules

Former plan: Contains the guide of all existing learning modules (which is transcluded onto /Library) as well as basic information about learning modules.

"Learn"& "Share"/Learning patterns

[edit]

Contains the guide of all existing learning patterns (which is transcluded onto /Library) as well as basic information about learning patterns.

  • This is too long a listing in the library, could we layer deeper and use a bot to surface popular and/or new learning patterns on the library page somewhat like we do on the main landing page?


/"Plan"/Program Toolkits

[edit]
  • Program toolkits with guides for planning, implementation, and evaluation of select Wikimedia programs.

This page will also need design and template creation.


/"Share"/Case studies and Reports

[edit]
  • Wiki-based evaluation reports and case studies of programs.


/"Share"/Case Studies and Reports/Case Studies

[edit]
  • Searchable list of Case Studies and call to action to share a case study of a program.

/"Share"/Case Studies and Reports/Program Evaluation Reports

[edit]
  • High-level reports on programs and a searchable guide of existing program reports and overview of reporting.
Formerly posted as case studies and housed at:

Programs:Evaluation portal/Library/Overview
Programs:Evaluation portal/Library/Edit-a-thons
Programs:Evaluation portal/Library/Editing workshops
Programs:Evaluation portal/Library/GLAM content donations
Programs:Evaluation portal/Library/WLM
Programs:Evaluation portal/Library/On-wiki writing contests
Programs:Evaluation portal/Library/Other photo initiatives
Programs:Evaluation portal/Library/WEP

Each of the subpages will need to have a template connection at the bottom of each program report to program toolkits.


/"Measure"

[edit]

The tools page is kind of a specialized Library page, focused on the evaluation tools we offer and how to use them.

Content
  • Brief summary of what these tools are for, letting people know that they are available for any stakeholders to use, not just Foundation staff.
  • Contact information for a 'technical lead' who should be contacted if someone has trouble setting up one of the tools.
  • Call to action to ask questions on the /Parlor/Questions page.
  • A brief description of each of the tools we have available.
  • Calls to action to browse the specific pages devoted to the description of each of the tools we currently offer: WikiMetrics, Community Visualization(?), and Qualtrics surveys.
Considerations
  • none, yet.
/Tools template

Can we change the current template? See Template comment on News section. --MCruz (WMF) (talk)

"Measure"/Survey Question Bank

[edit]

Content:

  • Curated list of survey items by question survey objective .
  • See api design plan google doc. The Survey Question Bank will be a curated list of survey items by question survey objective. The main page will be a curated list of links to subpages of items lists by survey and question objective, with the ability to browse questions by program type, goal and theme, using hidden categories. Since question objectives and questions may be duplicated across survey objective pages we likely will want to create a template and index page of all questions from which questions and responses may be transcluded on each survey objective subpage.
  • This page will need design and template creation.

"Measure"/Process and Tracking Tools

[edit]

Contains a basic description of process and tracking tools, links to the tracking and reporting tools and guides.

/"Measure"/Wikimetrics

[edit]

Contains a basic description of Wikimetrics, what it's used for, and links to the Wikimetrics learning module and other related resources.

/"Measure"/Qualtrics

[edit]

Contains a basic description of Qualtrics, what it's used for, and links to survey-related learning patterns and other related resources.

STILL EDITING FROM PREVIOUS PLAN: (Not presently there)

/"Measure"/Community visualization

Contains a basic description of the Community Visualization tool, what it's used for, and links to any modules, patterns or other related resources.

/"Work Space"

[edit]

(Should be renamed to something more clear and the "Leave feedback" lead in to this page may need to be more clearly distinguished from "Discussion"

The boiler room is the equivalent of the Teahouse Hosts Lounge. It's a space for project coordination, and will have it's own dedicated talk page. It should be linked to from some other pages on the Portal for transparency, but it may be desirable to leave it out of the primary navigation.

/"Workspace"/Portal Redesign Plan

[edit]
Formerly at Old documentation

===/"Workspace"/"Drafts" === (Also needs rename - is sandbox used universally? - deletion?

Coordinating space for evaluation teams to develop resources, reports, etc. Mainly used by staff.

Maintenance pages

[edit]

These pages are relevant to the design and functionality of the Evaluation portal, but are not intended to be viewed or edited directly by end users.


Templates

[edit]

IMPLEMENTED FOR PREVIOUS PLAN:


NEEDED TEMPLATES

[edit]
  • Template:Evaluation/Resource/Infobox: a generic infobox for all learning resources that we want to feature, other than patterns (which have a special infobox 'cause they are more dynamic). Put in a couple parameters like "summary" and "image" so that at some point in the future GrantsBot can automatically make profiles for each new resource that is created.

MYSTERY TEMPLATES

[edit]
I think it was used in the old /News/Archive page. I changed that recently, and used the /News template. See the section Grants:Evaluation/News/Archive --MCruz (WMF) (talk)

Categories

[edit]
Question Bank, Reports, and Toolkit page categories

need for all programs and program goals

IMPLEMENTED FOR PREVIOUS PLAN:

Primary page categories
Learning pattern categories
Template categories

NEEDED CATEGORIES

[edit]

Process pages

[edit]

IMPLEMENTED FOR PREVIOUS PLAN:

Content creation workflows

[edit]

IMPLEMENTED FOR PREVIOUS PLAN:

Activity surfacing

[edit]

IMPLEMENTED FOR PREVIOUS PLAN:

To do (Remaining to do's from previous plan)

[edit]
  • feature some resources other than patterns
  • Add Archives box to Q&A forum. See Template:Archive box auto (@ User:Jmorgan (WMF) can you give this a go? I fail at it every time I do it.
  • FIgure out what the hell to do with the old share space content.
  • set up /Library with transcluded lists of resources
  • Check out questions and ideas in the boiler room talk page
  • Have both teams add to the FAQ
  • Make it possible for people to suggest categories?
  • badges/stickers to signify endorsements, translations, quality (as per AFT)
  • sublists of patterns by category (either in category trees, or transcluded subpages)
  • Make it more obvious (in the edit window perhaps) that the talk page in the portal space is for the portal itself not to talk about evaluation. Take that to the Parlor.
  • Design an invitation for talk pages to invite portal visitors to create profiles. SarahStierch (talk) 19:52, 16 December 2013 (UTC)[reply]
Retrieved from "https://meta.wikimedia.org/w/index.php?title=Learning_and_Evaluation/Archive/Boiler_room/Portal_Redesign_Plan&oldid=25035771"