Grants:APG/Proposals/2014-2015 round2/Wikimedia Italia/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal. Members of the FDC review each proposal in its entirety, including relevant background documents, inputs from WMF grants officers, the learning and evaluation team, and finance staff. They also analyze the detailed discussions surrounding each proposal that include questions and comments from community members. FDC staff do not participate in the actual decision-making process of the FDC; their primary role is to identify key strengths and concerns for the FDC to consider while making funding recommendations.

This staff proposal assessment, which was refined after the first two years of the FDC process, consists of three parts: (1) Overview, including a financial summary and a summary of strengths and concerns; (2) Staff proposal assessment narrative, including a nuanced analysis of the programs, context, and the feasibility of each plan; (3) Staff proposal assessment rubric, assessing each proposal across three dimensions by identifying areas of strength and concern. Read more about the methodology behind the Staff proposal assessment rubric on this page.

Staff proposal assessments do not represent the views on any one staff person, since all FDC staff (unless one of them recuses for some reason) work together to complete each staff proposal assessment, and may consult with other key WMF staff and other experts as needed. While assessments will not be changed after they are published except to correct significant factual errors, organizations are welcome to share responses on the discussion page of each staff proposal assessment if they would like to provide the FDC with additional information or perspectives on the assessments.

Overview for all assessments

As in Round 1 of 2014-2015, proposals in 2014-2015 Round 2 are being assessed in the context of some preliminary impact data for grants and programs over the last two years, and our current and future approach is being guided by this analysis. In particular, we have been rigorous with our assessments of the Wikimedia organizations with the largest budgets and requests, as we expect them to be delivering outcomes to scale.

For example, we were able to access data from the first round of impact reports for grantees entering the APG process in 2012-2013 Round 2 (i.e. Wikimedia Norge and Wikimédia France), as well as progress reports received from the current grants (i.e. Q1 and Q2 reports) and project grant reports for all organizations but Wikimedia Italia, which is new to the APG program and did not receive significant project funding in the past. This allowed for a more detailed assessment of proposed goals against progress in the current year, as well as the current year’s progress against the current year’s goals. In some cases, we were also able to compare the current year’s goals and performance with past achievements. A move toward the inclusion of more easily comparable metrics in reports also assisted with this analysis.

There are a few overarching themes we noticed in this year’s proposals that we want to highlight. Please note that any examples used are meant to be illustrative, not exhaustive.

Train the Trainer approaches

In this round, we noticed a focus on Train the Trainer (TTT) approaches, which builds on the interest from Round 1. With a TTT approach, organizations target and train a group of participants on specific topic areas so they may in turn train others. If designed and implemented well, this program approach may have potential to expand reach in the movement and improve relevant skills beyond an organization’s direct sphere of influence.

This approach, which takes significant resources and organizational focus, requires more clearly articulated strategies, documented/centralized resources, and measurement and evaluation of the results. Most organizations are not yet evaluating skills developed, results of trainings after the original trainings, or attitudinal shifts, and so it is difficult to understand the impact of these programs. Furthermore, we are concerned not to see plans for sustaining engagement with these trainees after trainings.

We encourage organizations to document results and resources invested in these programs. We also encourage organizations to develop engagement plans before new training programs are implemented. Finally, we also encourage organizations start small before making the decision to invest in programs more significantly.

Metrics and evaluation

As in Round 1, we realize that all Wikimedia organizations in this round are improving their measures of success and attempting to demonstrate impact. This work is not easy; we recognize the effort. For example, we are pleased to see some (including Wikimedia Armenia, Wikimedia Norge, and CIS) offer baselines and projected outputs and outcomes. This is a significant improvement from 2013-2014 Round 2 proposals.

However, not all programs include metrics, particularly around outcomes. We are very concerned, particularly in the case of the larger organizations, about the lack of a baseline and project outcomes beyond outputs. Quantitative and qualitative evaluation methods are both important. The Wikimedia Foundation will provide continued support to organizations who request it to develop capacity in evaluation. On the quantitative side, we encourage organizations to establish systems to track the global metrics, as they will need to report on in order to understand their work in a broader context, and also to continue to carefully consider what metrics will be most relevant in their contexts.

Diversification of funds and resources

Especially for organizations with larger budgets, we are monitoring reliance on APG funding. Organizational reliance on APG funds (expressed as a total percentage of revenue) may be increasing for some, and in other cases may not be decreasing at a sufficient pace. In some cases, large organizations are not meeting fundraising targets or cultivating sustainable longer term institutional partnerships. We are particularly impressed with organizations in the global south who have been able to cultivate donors.

All organizations in this round are proposing significant total budget growth. Each organization’s growth must be examined in context, but in some cases this rapid growth does raise capacity concerns.

As we noted in Round 1, the diversification of resources is not only needed in order to maintain the sustainability of each individual organization and the global movement, it is also a way to build awareness of the Wikimedia mission.

Lobbying and political advocacy

In Round 1, we saw organizations express significant interest in policy, advocacy, and awareness.

In Round 2, Wikimedia Italia has proposed to engage a lobbyist to address legal threats to its program. Any lobbying work requires some oversight from the Wikimedia Foundation. We will monitor this work over time.

Increase in program activities and organizational capacity

In many cases, organizations are planning a large number of programs, with a many activities. This raises concerns around organizational capacity. Beyond the activity and staff expenses for each program, programs represent a significant investment of organizational focus. We encourage organizations to group programs or prioritize them appropriately in order to present a clear picture of what they are trying to achieve and why, and in order to reduce strain on volunteer boards, staff, online contributors, and offline volunteers active in program areas that can result from too many dispersed activities. We encourage organizations to focus on their strengths rather than to experiment in every area. In this way, organizations can experiment with thoughtful adaptations of existing approaches based on local context and the organization’s expertise, and with small scale pilots with potential to scale up.

As we noted in Round 1 around proactive and responsive community engagement strategies, we realize it is not easy to achieve a good balance of responding to community needs and leading strategic work for the movement. Refining focus will be an ongoing process.

Diversity and work on projects other than Wikipedia and Commons

As in Round 1, an interesting development is the increasing focus on Wikimedia projects other than Wikipedia. For example, we have seen CIS and Wikimedia Armenia integrate work on Wikisource and Wiktionary into key programs.

As in Round 1, we are also assessing proposals across the dimension of diversity, and we have seen Round 2 organizations focusing on this in their contexts through both a regional and a gender-focused lens. Some organizations have diversity at the core of their missions (e.g. CIS, Wikimedia Armenia), some are approaching this work through a cross-cutting approach (e.g. Wikimedia Italia) and others have designed specific programs to address diversity (e.g. Wikimedia Norge).

Overview

Overall assessment and eligibility

Organization	Wikimedia Italia
Eligibility	Confirmed

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or PEG funding	0	$166,680	NA
Budget	$292,349	$451,209	54%
Staff	2.5	3.5	40%

Overview of strengths and concerns

This section summarizes the key strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths

Stable and effective leadership
Growing an evaluation framework, in line with a detailed strategic plan
Many thoughtful adaptations of programs based on context (e.g. work with libraries, affiliation with the OpenStreetMap Foundation)

Concerns

Significant growth in budget and staff will be a challenge to manage, and may not correspond to impact
No baselines are yet available to assess past performance with respect to what is proposed for the coming year (a particular challenge given the size of the grant request)
Some programs do not seem strongly linked to online impact or well-adapted to context (e.g. education, museums)

Staff proposal assessment narrative

This section takes an in-depth look at the strategy behind this proposal, this organization's broader context, and the feasibility and risks of this plan.

Context and potential for impact

This section takes a close look at this organization's context, since there will be unique factors specific to this organization or to its environment that enable this plan to have impact or that make this plan less likely to have impact. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact? Are there risks and opportunities presented by this organization's environment? Are there extraordinary events or other factors that need to be considered as part of this organization's context?

Wikimedia Italia has a strong core of active volunteers who drive its program activities and have actively engaged with the chapter through meetings and surveys. We would like to see Wikimedia Italia deepen its engagement with online communities.
Wikimedia Italia has the opportunity to participate in the “cinque per mille” program, accessing an Italian government funding stream that Wikimedia would not otherwise have access to. At the same time, this funding stream has not proven to be consistent or reliable. Therefore, rapid fluctuations could also present a threat to Wikimedia Italia, making it more reliant on APG funding or even preventing it from executing planned programs, if expected revenue is not raised.
Wikimedia Italia is seizing opportunities in its environment by working strategically with partnerships to leverage networks within Italy. The organization is also working with an open knowledge ally in Italy with significant networks (OpenStreetMap Foundation).
Wikimedia Italia has also assessed legal risks in its context, as well as opportunities to address those risks through lobbying work. Legal risks may threaten the success of content drives such as Wiki Loves Monuments, but Wikimedia Italia’s planned lobbying work could have longer term impact by changing the legal environment in Italy.
Wikimedia Italia sees significant opportunities in its context for liberating content, especially through partnership work with libraries and museums.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed? Does it raise any concerns?

We are not able to evaluate Wikimedia Italia’s past performance because we do not have data about the impact of past programs.
Wikimedia Italia has experience in many of the program areas it plans to grow, so we do not have concerns around its ability to do this plan. However, we will need data about Wikimedia Italia’s current year (which is included in the proposal only as “projected”) and proposed funding period as a baseline for future grants.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed? Does it raise any concerns?

Wikimedia Italia has stable and effective leadership, including a new executive director and a dedicated volunteer board.
Wikimedia Italia is developing a more structured approach to volunteer roles in its programs, but expecting core volunteers to increase their commitments may not be realistic.
Wikimedia Italia has produced detailed activity reports twice each month for many years. These activity reports do not report about the impact of Wikimedia Italia’s activities using metrics, so we are unable to establish baselines or understand Wikimedia Italia’s past performance at a high level, nor are we able to compare it to the work of organizations.
Wikimedia Italia has a strong track record around regular and transparent financial reporting and budgeting, and has shown strong capacity in this area.

Strategy, program design, and program implementation

This section takes a close look at this organization's plans for its programs, including its past performance with program implementation. In this section, we may also identify specific programs that seem to have particularly high or low potential for impact. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan? Is this organization proposing significant changes to its strategy, or are the strategic plan and annual plan consistent with past strategies? Is this organization's strategy and are this organization's programs aligned with the Strategic Priorities of Increasing participation, Improving quality, or Increasing reach?

Wikimedia Italia produced its first strategic plan for 2015-16. We are impressed by the detail and quality in this plan, and found the background that explains the continuity behind its programs extremely helpful, especially in the absence of past grant reports.
We are impressed that Wikimedia Italia included detailed SWOT (Strengths / Weaknesses / Opportunities / Threats) analysis, Gantt charts, and analysis of program objectives at multiple levels for each of its programs, thus incorporating aspects of its strategy into its APG proposal.
While we are impressed by this first effort, we find that strategic alignment was better articulated for some programs than for others. For example, we have difficulty understanding the strategy behind Wikimedia Italia’s Education program and GLAM - Museums work. We find the OpenStreetMap work very interesting, but we also struggle to make clear links to the movement strategic priorities for this program (based on the objectives provided).
The emphasis of this proposal is on quality content, as the most clearly articulated programs are focused on content and fewer targets are set around newly registered users, active editors, or retained editors. Wikimedia Italia is also doing work to support online contributors through the Italian Wikipedia program.

Program design

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked?

Wikimedia Italia provides an evaluation framework for each of its programs, which will enable Wikimedia Italia to understand its impact and learn from results.
On the other hand, the quality of the objectives for each program is not consistent, and many programs do not include relevant SMART objectives with targets that draw a clear link to online impact. For example, targets around the number of pages or articles created and improved are not included for most programs other than the GLAM - Libraries program. Newly registered users seem only to be tracked for the OpenStreetMap work. Across programs, Wikimedia Italia is tracking numbers of partners and participants, but this information does not clearly show the impact of its work.

Specific programs

Based on proposed plans and past work, which programs may have the highest potential for impact corresponding to the amount of funding requested, and which programs may not have impact corresponding to the funds requested?

Wikimedia Italia’s GLAM work with libraries, work with OpenStreetMap, and Italian Wikipedia work are high potential areas with thoughtful plans.
We like the focus on online communities through the Italian Wikipedia program, and particularly the support offered for OTRS volunteers. We look forward to seeing online engagement and volunteer engagement deepen.
Along these lines, Wikimedia Italia’s strategies for volunteer engagement have the potential for impact. We look forward to seeing more clear links established with Wikimedia Italia’s other program work as this approach gains traction.
Wikimedia Italia’s lobbying work is an interesting experiment. While this work is still experimental for Wikimedia Italia, it may develop into a high potential area. It will need to be monitored closely.
Wikimedia Italia’s GLAM work with museums and its education program may be the programs with the least potential for impact as they are currently articulated. We see few relevant targets presented for these programs and we are not clear how they benefit the Wikimedia movement or online communities overall (e.g. we are not clear how education program targets participation or content).

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

Wikimedia Italia is requesting a very significant increase in overall budget and in number of staff. We will need to closely monitor the growth of Wikimedia Italia’s volunteer base as staff, budget, and programs expand, to ensure that volunteer engagement is growing in proportion to this expansion.
While Wikimedia Italia has proposed many thoughtful programs, we have concerns about the budget focus of this proposal. Particularly, communications costs (including 98,048 EUR for program-related communications expenses and 33,211 EUR for operational communications expenses, for a sum of 131,259 EUR), are an unusually high percentage of the budget and correspond to a very significant dollar amount. This may also have to do with the way Wikimedia Italia classifies communications expenses related to programs as part of overall communications expenses according to local regulations.
Overall, Wikimedia Italia is requesting a significant budget increase as it enters the APG process, which in turn raises the expectations for how much impact it will need to demonstrate through its program activities. Without clear baselines, future impact is difficult to predict, but the amount of funding seems high with respect to the targets set for the proposed funding period. In most cases, we believe this is because relevant targets are missing for many programs (e.g. 38 newly registered users are targeted across all programs because this metric is only used for one program while it is presumably relevant for many other programs). On the other hand, without clear targets set that are logically linked to online impact, the large amount of funding does not seem to correspond to the potential for impact.

Feasibility and risks

How likely is it that this organization can implement this plan successfully if funded? Is this plan likely to achieve impact corresponding to the funds requested?

While Wikimedia Italia has significant capacity in some important areas, we still have concerns about its ability to implement a plan of this scope.
Wikimedia Italia has presented a quality strategy and a clear plan for evaluating the success of its work. It has committed and stable leadership in place, including a dedicated volunteer board and an effective, though relatively new, executive director.
Wikimedia Italia is considering the state and future of its volunteer community as integral to its plan, and is pursuing several programs and approaches that have strong continuity with its past work. This may enable the long term success of its programs.
Participation in the “cinque per mille” revenue program, enables Wikimedia Italia to have a significant budget to support its work. Wikimedia Italia has a relatively diverse funding stream and APG is not its primary revenue source, although it still lacks significant support from institutional partners. Increasing staffing may make Wikimedia Italia more reliant on movement funding in the long term, especially if it cannot find other significant sources of revenue through institutional partners.
Wikimedia Italia has conducted a detailed assessment of strengths, weaknesses, opportunities and threats and have identified some important risks and opportunities in its own context. For example, Wikimedia Italia’s growing involvement with lobbying presents both an opportunity for developing new expertise and a threat, since resources in the movement for lobbying are limited. Wikimedia Italia is engaging in this work as a result of its assessment of legal threats in its context.
During the course of this plan, Wikimedia Italia and many of its volunteers will be engaged with the supporting of Wikimania 2016 in Esino Lario. While this is addressed briefly in the plan, we believe that Wikimedia Italia may be underestimating the costs (both financial and otherwise) of this activity. It seems unlikely that Wikimedia Italia will be able to execute such an ambitious plan while also supporting Wikimania.
In this proposal, Wikimedia Italia includes interns, contractors, and temporary staff in its FTE counts; although these numbers may not give an accurate picture of overall staff growth (especially because some interns and contractors are temporary), Wikimedia Italia is managing a significant number of staff. In terms of permanent FTEs, Wikimedia Italia has grown significantly from 1.5 FTE in January 2015 to 2.5 FTE at the time of this proposal, to 3.5 FTEs by the end of the upcoming funding period. At the same time, Wikimedia Italia is engaging numerous interns (paid and unpaid) and contractors working on specific projects. While this approach to hiring contractors for specific projects rather than bringing on full time staff appears to work well in Wikimedia Italia’s context, we do note that hiring and managing interns and temporary staff is also a significant burden for the organization as they are growing. We encourage Wikimedia Italia to consider a measured approach to staff growth as it moves forward.
This plan includes significant growth in programs as well as staff, rather than a focus on gradual staff growth or a focus on developing a few key programs. Due to the scope of this plan, it may be difficult to execute. Wikimedia Italia is growing its capacity to measure its work through a detailed evaluation framework, and that is likely to consume the energy of its staff and volunteers, making it less likely that Wikimedia Italia will be able to execute on a plan at this scale. Finally, Wikimedia Italia’s ability to measure the impact of its work is not yet proven and clear relevant targets have not been set across all programs. Therefore, it is possible that after a significant investment we will still not have a clear picture of impact.

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design and strategic alignment, (2) Organizational capacity and effectiveness, and (3) Budgeting by identifying strengths and concerns in each of these areas.

In each assessment, an organization’s context is taken into account and so criteria may be applied differently in different contexts. For example, a smaller organization with less experience will not be expected to have a formal learning and evaluation program that a larger more experienced organization might have, and so what may constitute a major strength for a smaller organization in the same category may not constitute a major strength for a larger organization. With this approach, the criteria listed here are not weighted equally, and so an “overall assessment” cannot be determined simply by adding up the numbers of strengths and concerns identified for each dimension. Also note that the scale here is not linear and so we do not assign any numerical values to the assessments given for each criterion. More information about each criterion is shared in the assessment. We believe a framework like this is helpful, as it ensures that each proposal is assessed consistently, even as they are applied with contextual nuance.

The dimension “Program design and strategic alignment” addresses many high-level elements, focusing on how likely programs are to achieve progress toward some of the Wikimedia movement's strategic priorities. Programs need to show clear links between strategic priorities and program objectives, program objectives and targets, and targets and program activities. In addition, programs should focus on areas that will impact participation, reach, and quality on the Wikimedia projects, with an emphasis on approaches and programs that increase diversity.

Meanwhile, the dimension “Organizational capacity and effectiveness” looks at the ability of an organization to achieve its plan based on both past experience (as an indicator of future success) and current practice. An effective organization both learns from past experience and contributes to movement learning in the process. This dimension also includes effective and stable leadership that models good practices, and a rational approach to growth that is supported by the organization’s strategy and context.

Finally, we analyze the organization’s past experience with and current plan for budgeting. Here, we look at how the organization has performed against its plans and budgets in the past.

To complete the assessment, we assess each of the criteria in each category according to the following scale, which is not necessarily linear:

Major strength
Strength
Neither a strength nor a concern
Concern
Major concern

Criterion	Assessment	Description
Program design and strategic alignment
P1. Strategy and alignment	Strength	Most program objectives are strongly aligned with the Wikimedia strategic priorities of improving quality or increasing participation, but strategic alignment is not clearly articulated for some programs (e.g. education). Wikimedia Italia has developed a strong and comprehensive two-year strategic plan for 2015-16.
P2. Targets and logic models	Neither	Some programs have specific, measurable, time-bound objectives with clear targets, and some programs present clear links between planned activities and desired results. However, some programs are lacking meaningful targets, especially regarding outcomes. Instead, some targets for large programs focus on outputs.
P3. Needs assessments	Strength	Program objectives are based on needs assessed in consultation with stakeholders, including the communities and volunteers that are working together with Wikimedia Italia to achieve its program objectives. Wikimedia Italia has done a strong assessment of strengths, weaknesses, opportunities, and threats for each program in addition to detailed surveys conducted about its programs.
P4. Potential for impact at scale	Neither	While some programs could lead to significant impact (e.g. improving quality), potential for global impact may not correspond to the amount of funding requested.
P5. Evaluation methods	Strength	Wikimedia Italia has presented detailed plans for measuring results and ensuring learning, and is likely to establish baselines for understanding the impact of its work in the coming year.
P6. New ideas and approaches, or thoughtful adaptations	Neither	Programs will test new ideas and approaches or will thoughtfully adapt ideas and approaches that may work in Wikimedia Italia’s context (e.g. OSM, regional networks for partnerships, libraries, lobbying). Other programs do not appear to be thoughtfully adapted in context (e.g. work with museums and education).
P7. Diversity	Concern	Programs may not expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served. While some work addressing the gender gap is included, it is not a significant focus of the proposal, which is a concern for a movement organization with this budget size.
Organizational capacity and effectiveness
O1. Past results	Neither	We do not have baselines available to assess the past results of Wikimedia Italia's work.
O2. Stability and leadership	Major strength	Wikimedia Italia has a long history of stable, effective volunteer leadership and good governance that will enable this plan to succeed. Transition to an organization with more staff appears to be going smoothly.
O3. Learning	Neither	Wikimedia Italia is addressing risks and challenges effectively, is learning from and documenting past experiences, and is applying learning to improve its current and planned programs. Wikimedia Italia has done a SWOT analysis for each program to assess risks and challenges, and has reflected on its programs in the context of its strategic annual plan. On the other hand, we have not yet seen strong evidence of Wikimedia Italia's abiity to apply past learning to program design.
O4. Improving movement practices	Strength	Wikimedia Italia effectively shares learning about its work with the broader movement and beyond. Wikimedia Italia has consistently reported about its activities on twice-a-month basis and has also hosted movement-wide events such as the Wikimedia Conference and Wikimania (planned for 2016).
O5. Community engagement	Neither	Wikimedia Italia is developing its strategy for volunteer engagement, but can deepen its online engagement.
O6. Capacity	Strength	While Wikimedia Italia has expertise and strong systems in place to do the proposed work, the scope of this plan and the scope of the proposed overall budget growth may not be realistic with the resources requested.
Budget
1. Past budgeting and spending	Strength	Wikimedia Italia has a history of budgeting realistically and managing funds effectively in the past.
B2. Proposed budget is realistic	Strength	Wikimedia Italia's proposal includes a budget that is comprehensive and clear, and that corresponds to the activities proposed. To better understand Wikimedia Italia's plan, allocations of staff time would be useful.
B3. Budget is focused on programmatic impact	Neither	Wikimedia Italia is significantly increasing its overall expenses, and expectations for programmatic impact are also significantly increasing. Wikimedia Italia has significant amounts dedicated to communications expenses, which are both operational and programmatic.

This staff proposal assessment is the work of FDC staff and is submitted by: Winifred Olliff (WMF Program Officer) talk 22:04, 8 May 2015 (UTC)

Staff proposal assessment rubric description

This rubric assesses annual plan grant proposals across the three dimensions of (1) Program design and strategic alignment, (2) Organizational capacity and effectiveness, and (3) Budgeting by identifying strengths and concerns in each of these areas.

In each assessment, an organization’s context is taken into account and so criteria may be applied differently in different contexts. For example, a smaller organization with less experience will not be expected to have a formal learning and evaluation program that a larger more experienced organization might have, and so what may constitute a major strength for a smaller organization in the same category may not constitute a major strength for a larger organization. With this approach, the criteria listed here are not weighted equally, and so an “overall assessment” cannot be determined simply by adding up the numbers of strengths and concerns identified for each dimension. Also note that the scale here is not linear and so we will not assign any numerical values to the assessments given for each criterion. More information about each criterion is shared in the assessment. We still believe a rubric like this is helpful, as it ensures that staff analysis of each proposal is based on the same criteria, even as they are applied in a nuanced way.

The dimension “Program design and strategic alignment” addresses many high-level elements, focusing on how likely programs are to achieve progress toward some of the Wikimedia movement's strategic priorities. Programs need to show clear links between strategic priorities and program objectives, program objectives and targets, and targets and program activities. In addition, programs should focus on areas that will impact participation, reach, and quality on the Wikimedia projects, with an emphasis on approaches and programs that increase diversity.

Meanwhile, the dimension “Organizational capacity and effectiveness” looks at the ability of an organization to achieve its plan based on both past experience (as an indicator of future success) and current practice. An effective organization both learns from past experience and contributes to movement learning in the process. This dimension also includes effective and stable leadership that models good practices, and a rational approach to growth that is supported by the organization’s strategy and context.

Finally, we analyze the organization’s past experience with and current plan for budgeting. Here, we look at how the organization has performed against its plans and budgets in the past.

To complete the assessment, we assess each of the criterion in each category according to the following scale, which is not necessarily linear:

Major strength
Strength
Neither a strength nor a concern
Concern
Major concern

Criterion	Description
Program design and strategic alignment
P1. Strategy and alignment	Programs objectives are strongly aligned with the Wikimedia strategic priorities of increasing reach, improving quality, or increasing participation. The organization has a strategic plan in place.
P2. Targets and logic models	Programs have specific, measurable, time-bound objectives with clear targets, and programs present clear links between planned activities and desired results.
P3. Needs assessments	Program objectives are based on needs assessed in consultation with stakeholders, including the communities and volunteers that are working together with this organization to achieve its program objectives.
P4. Potential for impact at scale	Programs could lead to significant impact on the Wikimedia strategic priorities of increasing reach, improving quality, or increasing participation, at scale and corresponding to the amount of funds requested.
P5. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems.
P6. New ideas and approaches, or thoughtful adaptations	Programs will test new ideas and approaches or will thoughtfully adapt ideas and approaches that may work in this organization’s context.
P7. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational capacity and effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Stability and leadership	Stable, effective leadership and good governance will enable this plan to succeed. Any leadership transitions are managed effectively.
O3. Learning from the past	This organization is addressing risks and challenges effectively, is learning from and documenting past experiences, and is applying learning to improve its current and planned programs.
O4. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond.
O5. Community engagement	This organization effectively engages key communities and volunteers in the planning and implementation of its work.
O6. Capacity	This organization has the resources and ability (for example, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Proposed budget is realistic	This proposal includes a budget that is comprehensive and clear, and that corresponds to the activities proposed.
B3. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.