Grants:APG/Proposals/2017-2018 round 2/Wikimedia Norge/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$234,950	$250,190	6.49%
Budget	$256,591	$286,814	11.78%
Staff (FTE)	2.3	2.55	0.43%

Overview

This section summarizes the themes that emerged from the proposal, which are explained in the detailed narrative section of this assessment.

Themes

• The proposed plan is much more coherent than in past years, with a strong emphasis on gender gap work.
• WMNO is requesting an increase that does not seem to be linked directly to improved results.
• Results are improving, but are low overall when considered alongside the requested funding amount.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

WMNO is taking advantage of momentum in their society around issues of gender equality, and has been able to design their programs and activities in a way that effectively harnesses this momentum. They have also been able to tap into regional interest in working with the indigenous Sami communities in Norway, and are working to leverage this work to secure external funding.

Nevertheless, WMNO still struggles to secure enough external funding to support the organization outside of FDC funding. While they are taking some steps to address this, for example by attempting to bring more visibility to the organization and its work in Norwegian society, it has taken a long time for WMNO to gain traction in this area. For example, WMNO has secured a partnership opportunity for a GLAM cooperation but it is necessary for them to contribute matching funds in order to make this work.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

WMNO’s targets have been low year after year, and so have their achievements. While this may be aligned with what they can realistically achieve given the size of their communities, results have not been commensurate with the amounts of funding received, which have included an increase in the current year and could include a proposed increase in the upcoming year.

WMNO is increasing their targets in several areas on the basis of improved performance during the current year. For example, they achieved 843 participants at the midpoint of the current year, which is an improvement from 520 participants in the entire previous year. Accordingly, they have set a more ambitious target of 1,350 participants in the upcoming year, which they may be able to achieve given these improvements. Although they are on track to meet their target for new editors in the current year, the overall number of new editors involved is still very low, with 94 new editors involved at the midpoint of the current year. The midpoint result of 3,218 articles in the current year is just a fraction of their target of 47,000, although WMNO is expecting some mass uploads in the second half of the current year. This indicates that a large part of that target might be dependent on a single partnership. WMNO could be on target to achieve 1,100 diversity articles in the current year, with an achievement of 400 at the midpoint of the current year.

Despite lack of performance in quantitative metrics, we would like to highlight important qualitative achievements in the area of gender gap work. We believe that WMNO’s ability to generate large amounts of public interest in this topic is something that can potentially lead to long term benefits for the Wikimedia projects and the communities in Norway.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

It is clear from improvements to this year’s proposal that WMNO is growing into a more effective organization. Despite the lack of tangible results, it is likely that increasing funding for community staff is helping them to improve this focus. We appreciate that WMNO has revamped their financial structures and processes in order to function more effectively.

We note that WMNO is trying to grow more effective in the area of external partnerships and funding, as evidence by their proposed partnership including matching funding.

As an aspect of effectiveness, we note here that WMNO is now measuring volunteer hours. We think this shows a commitment to increasing volunteer engagement and involvement, to increase this organization’s capacity and sustainability.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

WMNO is attempting to better align their strategy with the movement strategic direction by focusing on diversity, with their work on gender gap and with Northern Sami communities.

We recognize an improved approach to strategy implemented in their annual planning. The proposal presented here is far more coherent, and provides good explanations of the strategy behind each program area. The plan is still greatly focused on activities, however, and it is sometimes difficult to draw links between these activities in order to understand the broader logic behind each program.

We appreciate the thoughtful risk assessments included here, which were useful in understanding WMNO’s deeper thinking about each program area.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

WMNO has presented a more coherent program plan than in previous years, and we appreciate that. At the same time, the plan still seems focused on a large number of activities, which may be a challenge for implementing programs and evaluation for an organization of WMNO’s size. We greatly appreciate that WMNO is committed enough to making gender gap work central to its mission that they are featuring a content diversity metric as one of their grantee-defined metrics.

WMNO’s approach to diversity is intentional and central to their plan. We hope this will also be an opportunity for WMNO to gain more partners and more funding, and we see that they are leveraging this through the Wikicamp application at NIKK (Nordic Information on Gender). We see that WMNO intends to build upon the public's goodwill around gender gap work. Of particular interest is the activity Science Days by the research council, which combines gender gap initiatives with tackling content gaps.

WMNO's community support program includes small innovative projects like the drone project as well as long term initiatives such as ongoing support to editors and readers and recurring online editing contests. At midpoint, WMNO has achieved more than half of their target (472 for 700 hours) in volunteer hours, which could be due a positive response from the community to these programs and a certain interest in WMNO's activities, also shown by the response to their survey in 2016 (121 responses for a total of about 100 regularly active editors in the Norwegian Wikipedia).

The survey on Northern Sami surfaces interesting challenges inherent to work done in smaller languages, such as questions around the modernity of the language base, or the difficulty of ensuring quality for a language with few speakers). The Northern Sami project seems unlikely to lead to significant measurable results, but it may generate interest from institutional partners and help WMNO grow their partnership base.

WMNO collaborates with other organizations in the movement. For example, their ongoing relationship with Wikimedia Armenia has led to important work in bridging content gaps about the Armenian genocide. They also contribute to other international initiatives, such as the Art+Feminism campaign, and are looking at developing a common wikicamp focused on diversity, for which they have applied for outside funding, with Wikimedia Sverige and Wikimedia Suomi.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

WMNO has been operating with a sizeable budget compared to their results and outcomes. While we believe that WMNO is directing resources effectively within this budget, it is still large with respect to what they are planning to achieve, even with this improved plan.

WMNO is requesting an increase in the current year, so that they can include a position for which they are also receiving matching funds. While we recognize the importance of securing significant support from an external partner, it is not clear that further investment will lead to commensurate impact.

As in previous years, we note that WMNO will need to secure external funding in order to maintain their organization in the long term and we would like to see more progress in this area.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Neither	While WMNO lacks a comprehensive strategic plan, we see significant improvements in the implementation of the strategy, including the integration of concepts from the movement strategy discussions.
P2. Potential for impact at scale	Concern	WMNO's programs, while interesting, do not seem to have great potential to achieve impact beyond Norway, given the level of funding they are requesting.
P3. Objectives and evaluation methods	Neither	WMNO has improved in this area since last year. We appreciate their selection of relevant grantee-defined metrics that will measure how the organization is doing in terms of addressing some of their key challenges. We look forward to ongoing improvements in metrics that can help WMNO better showcase their achievements.
P4. Diversity	Strength	We recognize that WMNO has gone a step further in this plan; beyond integrating the idea of diversity (especially gender diversity) in their program plan, they are also taking concrete steps to measure their progress in this area.
Organizational effectiveness
O1. Past results	Neither	We recognize that the changes WMNO has made to their plan and approach are likely to lead to improved results, but we have not yet seen evidence of these results despite an increase in funding given last round.
O2. Learning	Strength	We continue to appreciate WMNO's work in this area, which is demonstrated by the improvements they have made to their annual planning approach.
O3. Improving movement practices	Neither	We see that WMNO is continuing their collaborations with other affiliates in their program work. Standards are high at this level of funding.
O4. Community engagement	Neither	We appreciate that WMNO continues to focus on this area, which has been a challenge for them. We like that WMNO has highlighted the grantee-defined metric of volunteer hours.
O5. Capacity	Neither	In the current year, WMNO brought on increased capacity in the area of community management, and we hope this results in more impact in the future. WMNO is putting in a good effort to involve and motivate their volunteer community, and to measure engagement. These things may result in an increased capacity in the long term.
Budget
B1. Past budgeting and spending	Neither	WMNO is managing their budget effectively and doing their best to direct their resources toward impact.
B2. Budget is focused on impact	Concern	We appreciate this improved plan; however, WMNO must secure more external funding to support its maintenance and growth in the long term.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 07:11, 15 May 2018 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.