Grants:APG/Proposals/2014-2015 round1/Wikimedia UK/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal. Members of the FDC review each proposal in its entirety, including relevant background documents, inputs from WMF grants officers, the learning and evaluation team, and finance staff. They also analyze the detailed discussions surrounding each proposal that include questions and comments from community members. FDC staff do not participate in the actual decision-making process of the FDC; their primary role is to identify key strengths and concerns for the FDC to consider while making funding recommendations. This staff proposal assessment, which was refined after the first two years of the FDC process, consists of three parts: (1) Overview, including a financial summary and a summary of strengths and concerns; (2) Staff proposal assessment narrative, including a nuanced analysis of the programs, context, and the feasibility of each plan; (3) Staff proposal assessment rubric, assessing each proposal across three dimensions by identifying areas of strength and concern. Read more about the methodology behind the Staff proposal assessment rubric on this page.

Staff proposal assessments do not represent the views on any one staff person, since all FDC staff (unless one of them recuses for some reason) work together to complete each staff proposal assessment, and may consult with other key WMF staff and other experts as needed. While assessments will not be changed after they are published except to correct significant factual errors, organizations are welcome to share responses on the discussion page of each staff proposal assessment if they would like to provide the FDC with additional information or perspectives on the assessments.

Overview for all assessments

This year’s proposals are being assessed in the context of some preliminary impact data for grants and programs over the last two years, and our current and future approach is being guided by this analysis. In particular, we have been rigorous with our assessments of the Wikimedia organizations with the largest budgets and requests in this round, as we expect them to be delivering outcomes to scale. In a context where six of the largest (in budget and staff) Wikimedia organizations get nearly 60% of all funds (approximately about $8M) allocated by our community-led review committees (annual plan grants, project and event grants and individual engagement grants together), we believe it is our shared responsibility to be accountable for the impact of these funds. Five of these organizations are in the current round of review by the FDC: WMDE, WMUK, WMCH, WMNL and WMSE; WMFR will be in the April 2015 round of FDC review.

There are a few overarching themes we noticed in this year’s proposals that we wanted to call out. Please note that any examples used are meant to be illustrative, not exhaustive!

Proactive versus responsive strategies and community engagement

Strategy is about trade-offs and priorities: what do you choose to do with a certain set of resources, whether people, money or ideas? Wikimedia organizations, primarily built of, by, and for volunteers, often make decisions that respond immediately and always to all volunteer needs and interests. Such an approach can help volunteers feel appreciated and recognized, and give volunteers the resources to enjoy their own specific interests. At the same time, this approach can feel ad hoc for the organization overall, and may leave the organization without clear aggregate outcomes it can point towards that justify its work. For the significant level of resources (grant funding, donations, volunteer time, in-kind donations) being utilized by each organization, therefore, it is important to have clarity on the balance between such a responsive strategy and a proactive one, in which an organization charts a specific course of action focused on some critical outcomes that have value (perhaps for both locally and globally), and engages volunteers to support those activities in a way that increases their enjoyment and motivation. These are not easy choices, and there are no simple answers to this challenge.

However, it seems critical to recognize that our incredibly passionate communities of volunteers appear less motivated by money (or its lack) than by peer appreciation, recognition and by having the tools needed to contribute more effectively and enjoyably. How do we design our ‘proactive’ strategies to include elements of enjoyment and interest for volunteers, while also driving specific and substantial outcomes around content or contributor growth? Organizations like WMEE and WMAT have organized well-designed and integrated offline and online initiatives specifically geared towards challenging identified content gaps, and have succeeded in achieving online outcomes toward a key movement goal, and at the same time increasing the joy and sense of achievement of volunteers.

Finally, we believe that it is important to design programmatic approaches based on opportunities for scale and opportunities to address key content or contributor gaps, as well as volunteer-led interests. We appreciate that some organizations are working to deeply understand their communities to support them appropriately (like WMNL with its needs assessments, WMSE in its advocacy work and WMIL in its work to map its volunteer community). We hope this will lead to a much more directed and substantial set of outcomes.

Policy awareness, advocacy and collaboration

A separate but related theme we see evolving across the movement is work on policy change at the local or national level (for instance, with changing copyright laws or influencing education policies to include Wikipedia as a pedagogical tool) or regional and global levels (for instance, increasing awareness of free and open content and protocols to enable it). This is an important way in which Wikimedia organizations can support an enabling environment for free knowledge, and this work demands greater and more effective alliances with external partners (such as sub-national, national or regional organizations interested in free knowledge) and across the movement (such as a group of European Wikimedia chapters working in Brussels).

Not only at the level of policy, but at the level of specific programmatic activity, we are encouraged to see far greater cooperation and collaboration planned, and we believe this is one more way in which we can sustainably scale our efforts and our outcomes.

Measures and metrics

All Wikimedia organizations in this round are working to improve their measures of success and to find ways to offer reliable metrics that demonstrate impact. We know this is not easy; we recognize the effort it takes, and appreciate those organizations that have made particular effort to advance their own understanding and articulation (including WMDE, WMUK, WMSE, WMEE). At the same time, we want to point out that it has been difficult for us while reading these proposals to be able to point to clear evidence of achievement in many cases: we all (including WMF) have a lot more to do in order to claim with some confidence, the evidence of significant positive outcomes for the levels of resources being used.

This challenge is particularly understandable when some of the most critical work done by Wikimedia organizations is on partnerships for content or policy change; the progress and outcomes of relationship building are not easily ‘measured’. However, we do need to set some targets, and, at the very least, establish specific milestones to monitor progress, in order to be able to assess and demonstrate our collective efforts and justify the significant resources spent on this work. We know organizations like WMDE, WMSE and others are seriously working on this, as are we at WMF. Global metrics are a start, but we need to work together to look at more specific ways to tell the layered story of partnerships and their outcomes for the movement. Importantly, for successful implementation, we need to be focused about which kind of partnerships and which partners, have the most potential to lead to impact at scale; in only a few cases did we find clear rationale for why certain partnerships were being pursued and what content gaps were being specifically addressed through partnerships.

Diversification of funds and resources

We would like to appreciate that organizations are working seriously towards the diversification of their funds and other resources. A number of organizations have been much more successful in the past year at raising funds from outside the movement (including WMEE, WMSE, WMCH, and WMNL) and many have set ambitious targets for 2015 (including WMUK, WMSE, WMDE, and WMEE). Many organizations are moving towards being less dependent on APG funds for their work. As we have said before, it is important that as a movement, we also acknowledge and assess the range of non-monetary or in-kind contributions and resources we mobilize (for example, Amical works rent-free in a shared office, and WMCH is currently working on pro bono communications and marketing support). Overall, the diversification of resources is not only needed in order to maintain the sustainability of each individual organization and the global movement, it is also a significant way in which individuals and organizations in each local or regional context can express greater awareness of and affinity with the Wikimedia mission. In this way, Wikimedia organizations can build leverage in their environments over time.

Software development and ‘technology pools’

Not all Wikimedia organizations need to be doing the same set of activities or have the same set of approaches: the strength of a plural, diverse, global movement is that individuals and groups can focus on their core competencies and interests, while these complement those of other individuals or groups.

We greatly appreciate the critical success of Wikidata, WMDE’s award-winning software project. Apart from WMDE, we found that a few organizations in this round included elements of software development in their proposals (including WMCH and WMUK). While we do believe that software development for our movement should be driven by multiple actors, not only by WMF, we also ask organizations to ensure that they are prepared for the significant leadership and management skills and capacities it takes to focus on building large-scale technological tools for the movement. This is a competency that can be specifically planned and mentored across a few chapters in service to the rest of the movement. As Erik Möeller, WMF VP of Product & Strategy, put in his expert commentary: “Wikimedia Deutschland has developed a mature engineering department, which puts it in an excellent position to mentor other chapters which are looking to take on software development work”.

Specifically, we find that many groups have successfully used ‘technology pools’ (loaning equipment to volunteers - for example, photo equipment as well as accompanying press accreditation) to support the creation and contribution of high-quality content, often about significant local or regional events or themes. This appears to be a relatively easy, low-cost approach to supporting volunteers that achieves good outcomes. It is, of course, contingent on factors like geographic proximity to the organization, but there may be some innovative approaches - including in-kind support from other groups - that can be adapted as Wikimedia organizations grow their outreach efforts.

Moving beyond Wikipedia to the sister projects

An interesting development we have been observing over the past few years is exemplified by some of the proposals in this round: the increasing focus of Wikimedia organizations on Wikimedia projects other than Wikipedia. WMNL for instance, has a focus on Wikimedia projects “other than Wikipedia and Commons”, while WMIL and WMRS are specifically interested in supporting more work on Wiktionary.

While this may lead to some useful outcomes, we ask organizations to offer more detailed rationale for these choices, particularly about how well these goals will scale for projects that currently have relatively small communities. We also encourage them to document the progress of their work on these projects. For example, do these projects help us motivate and focus newbie editors more easily because of the type of content, the size of community, and the possible focus on language preservation? Is this a useful strategic choice both locally and globally, and if yes, why? Do contributors see this as an entry-point to Wikipedia or as their home community where they spend the majority of their time? In particular, as we turn our attention to key content and contributor gaps more globally, will this approach support us effectively?

Diversity

Finally, for the first time in the FDC process, we specifically analyzed annual plan grant proposals on the dimension of ‘diversity’. This may be interpreted differently in each context: for instance, while this is a cross-cutting issue for many communities, we at WMF currently prioritize Global South and gender diversity in our movement. Our specific emphasis on this dimension, which many of our partner organizations also care about, is based on the perspective that this is a critical strategic responsibility for us to share in order to achieve our vision of freely shared knowledge for every single human being. We encourage Wikimedia organizations to think about diversity in multiple ways, both within their own organizational structures and processes (for instance, the gender diversity in their boards and staff) and within their programs (for instance, supporting the acquisition and improvement of geographically diverse content).

We are looking forward to seeing the results from the cross-cutting programmatic focuses some applicants have described in this round on gender and diversity (including WMAR and WMAT) and note that other organizations have put an internal emphasis on diversity in their volunteers and Board (including WMUK). Others have created programs specifically designed to target diverse participation or content gaps related to gender in their work or are working to identify those gaps (including WMNL, WMDE, WMRS, WMSE). While we did not explicitly seek goals and targets against this, it would be worthwhile for those organizations for whom this is a focus, to offer specific measures in the next round and to consider targets around online outcomes as well as offline outcomes.

We look forward to our continued work together, supporting you, and learning with you all.

Anasuya, with Katy and Winifred

Overview

Overall assessment and eligibility

Organization	Wikimedia UK
Eligibility	YES

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or PEG funding	$586,051	$672,381	14.73%
Budget	$1,212,562	$1,347,860	11.16%
Staff	9.4	12	31.91%

Editors	Country	Wikipedia	October 1, 2012	October 1, 2013	October 1, 2014
All editors	United Kingdom	English	14,921	13,929	14,280
Active (5+/mo)			4,720	4,128	4,365

Overview of strengths and concerns

This section summarizes the key strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths

• Shift toward targets and better systems for measurement and evaluation
• Opportunities for significant global impact through GLAM partnerships
• Good results through focused and integrated programs like its work on Welsh Wikipedia

Concerns

• Significant growth in budget, staff, and FDC funding is not sustainable
• The high potential for GLAM has been leveraged adequately
• Some programs are not well-articulated (like policy work) or do not match the organization’s current capacities (like technology work)

Staff proposal assessment narrative

This section takes an in-depth look at the strategy behind this proposal, this organization's broader context, and the feasibility and risks of this plan.

Context and potential for impact

This section takes a close look at this organization's context, since there will be unique factors specific to this organization or to its environment that enable this plan to have impact or that make this plan less likely to have impact. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact? Are there risks and opportunities presented by this organization's environment? Are there extraordinary events or other factors that need to be considered as part of this organization's context?

• WMUK’s environment makes it well-positioned for global impact in some ways. WMUK has the opportunity to have impact on English Wikipedia, the Wikimedia project with the largest reach of any project.
• WMUK has great potential to work with GLAM institutions in the UK that could result in significant content donations, with potential usage across Wikimedia communities beyond English Wikipedia.
• Overall editors and active editors on English Wikipedia in the UK are increasing from 1 October 2013 to 1 October 2014.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed? Does it raise any concerns?

• Significant results have been achieved in terms of content added to the projects, particularly in the Wales program, and WMUK is now better placed to articulate them as a result of its strategic shift.
• We are pleased to see the number of images uploaded increase by 40% in the first two quarters of the year over the last year in total. At the end of Q2, the number of articles was 20% above last year’s number in total. In addition, WMUK has more than doubled the number of GLAM partners and has a growing number of active volunteers -- an increasing percentage of whom are women (at Q2 the percentage was 41%).
• WMUK supported an impressive Wikimania with extensive participation by mission-aligned individuals and organizations, as well as a large number of Wikimedians from across the world. We look forward to seeing the outcomes of this conference in the long term in the UK.
• However, we also have not yet seen impact information or even number of classes/professors supported through the education program. Nor have we been able to see impact evidence of the EduWiki conference, for instance, on the number of teachers using Wikipedian in the classroom or the amount of content produced. Therefore, it is difficult for us to understand what and how WMUK plans to grow in the education program.
• In addition, WMUK has not demonstrated success in areas like software development and an emphasis on QRpedia may not have high potential for impact.
• Furthermore, WMUK is proposing some work that seems to be outside its scope of expertise (based on previous work), such as removing barriers to technology access in order to get more people to participate in Wikimedia projects.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed? Does it raise any concerns?

• WMUK’s board is one of the most diverse in the Wikimedia movement, with a number of external experts joining long term Wikimedians. It is enacting positive shifts in the organization, including increased work on strategy and evaluation.
• WMUK has also improved, over the past several years, its capacity to document its learning for internal use and to share learning with others to improve Wikimedia movement practices.
• WMUK is sharing movement practices about programs like Wikipedians in Residence, and is strong in documenting its work to apply learning to its program design.
• WMUK has made significant improvements in its ability to measure and adapt programs (e.g. narrowing focus in education to focus on impact at university level, adapting Scotland program to build on learning from Wales program).
• WMUK is raising its capacity to secure funding from other sources to support its work, and has set an ambitious fundraising target for 2015.
• However, the shifts in organizational strategy and learning are not yet demonstrating the outcomes at scale that one would expect from an organization of WMUK’s size and scope.

Strategy, program design, and program implementation

This section takes a close look at this organization's plans for its programs, including its past performance with program implementation. In this section, we may also identify specific programs that seem to have particularly high or low potential for impact. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan? Is this organization proposing significant changes to its strategy, or are the strategic plan and annual plan consistent with past strategies? Is this organization's strategy and are this organization's programs aligned with the Strategic Priorities of Increasing participation, Improving quality, or Increasing reach?

• WMUK’s leadership is shifting its priorities to emphasize strategy and evaluation. In this vein, WMUK has created a new strategic plan that is detailed. It has worked to form its proposal and annual budget according to this plan.
• Beyond this, some of WMUK’s work is well-aligned with the strategic priorities of Improving quality (GLAM, WiRs, work in Wales). Despite this, not all work proposed has equally high potential to be effective and WMUK should consider a more focused approach to its annual planning.

Program design

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked?

• We appreciate WMUK’s significant work to improve its goal-setting and to include SMART objectives with targets in its plan. We realize that WMUK has made great progress in this area, and at this funding level, we have high expectations.
• We appreciate the improvements in WMUK’s reporting to the FDC. In the last few quarterly progress reports, WMUK has been measuring outputs and outcomes against baselines and targets, which is a big step in the right direction. We also appreciate the color coding they have provided on progress against their objectives. WMUK can still work to ensure that the targets selected are the most relevant ones for measuring results. For example, we note that WMUK would like to do more work around diversity but that diversity-related targets in the proposal are focused solely on its volunteer base and not on its base of online contributors or on relevant content gaps. Therefore, these targets won’t give us a sense of the impact of this work online.
• We understand that WMUK draws a link between some offline outcomes and online outcomes, and including more online outcomes in some areas would help us understand the direct impact of its work. In a few cases, targets around content (such as # of files uploaded and used, # of articles created or improved) are not present when content seems to be a key focus of the program.
• Some of WMUK’s targets are lower than last year, and it’s unclear why the target has decreased. For instance, WMUK has a goal of just 200 articles in English when it supported 400+ articles in the first half of this year.
• Other useful metrics to consider as targets, if they make sense in WMUK’s context, would be bytes added and # of active editors involved in its work. We realize that it will take time for organizations to implement suggestions around the use of global metrics, and we appreciate WMUK’s improvements and continued efforts in this area.
• We are pleased to see the GLAM/Wikipedians in Residence generating content with very high rate of files in use (48% and 38% at Q2).

Specific programs

Based on proposed plans and past work, which programs may have the highest potential for impact corresponding to the amount of funding requested, and which programs may not have impact corresponding to the funds requested?

• We see WMUK’s strongest program potential is in the area of GLAM partnerships.
• We see WMUK’s program with the least potential - without a clear strategy and capacity for management - is its proposed technology work.
• WMUK has proposed some policy advocacy work, and has engaged in this work, but the work described is not specific enough for this to be considered a high potential area yet.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

• WMUK’s proposed budget of $1,347,860 and proposed staff at 12 FTE is very large with respect to the outcomes it is achieving. This budget will rely on a 14.73% increase in FDC funding from $586,051 to $672,381.
• WMUK is planning to hire additional staff in fundraising. While raising other funding to support WMUK’s rapid growth in budget, staff, and programs would be useful to sustain future growth, we hope WMUK is considering its need to grow carefully. For example, focusing on fewer programs in areas where WMUK has strong capacity may allow WMUK to focus on achieving more through those programs without rapid growth. We appreciate that WMUK has already made some progress here by refocusing the efforts of its education program staff.
• WMUK also plans to hire a Chief Technology Officer, a costly staff position devoted to a program that does WMUK may not be well placed to deliver on.
• Although WMUK does plan to raise significant other funding in the coming year, it may not be prudent to hire more staff until significant other funding is secured. We laud Wikimedia UK’s efforts to reduce its reliance on FDC funding, and note that they have set an ambitious target of raising half of their revenue in 2015 from sources other than the FDC.

Feasibility and risks

How likely is it that this organization can implement this plan successfully if funded? Is this plan likely to achieve impact corresponding to the funds requested?

Feasibility

What enables this plan?

• WMUK has a strong and active volunteer community, and the organization has competency in recruiting and managing volunteers.
• WMUK has a strong and diverse board to drive its success.
• WMUK is building on past successes (with programs like Wikimedians in Residence) and applying its learning to adapt new projects (like its Scotland work).
• WMUK is building its capacity to raise funding from other sources.

Risks

What are the risks?

• The size of the grant requested is very large without commensurate impact.
• WMUK aims to increase its staff by 3 FTEs without enough external funding secured to support this staff growth, which could result in an ongoing reliance on APG funding.
• Since WMUK was unable to hire a developer in the previous year, we have concerns around its plan to hire a Chief Technology Officer, as well as the organization’s interest in shifting its focus to technology without proven ability to manage this.
• WMUK’s role in policy advocacy work is unclear.

Summary of expert opinions and community commentary

This section will summarize any expert opinions or community commentary offered through the APG proposal process, or through additional interviews and research. Wikimedia CH’s proposal was discussed by the community, FDC, and FDC staff during an open comment period between 1 October and 31 October. To view comments, questions, and responses, please visit the proposal form discussion page.

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design and strategic alignment, (2) Organizational capacity and effectiveness, and (3) Budgeting by identifying strengths and concerns in each of these areas.

In each assessment, an organization’s context is taken into account and so criteria may be applied differently in different contexts. For example, a smaller organization with less experience will not be expected to have a formal learning and evaluation program that a larger more experienced organization might have, and so what may constitute a major strength for a smaller organization in the same category may not constitute a major strength for a larger organization. With this approach, the criteria listed here are not weighted equally, and so an “overall assessment” cannot be determined simply by adding up the numbers of strengths and concerns identified for each dimension. Also note that the scale here is not linear and so we do not assign any numerical values to the assessments given for each criterion. More information about each criterion is shared in the assessment. We believe a framework like this is helpful, as it ensures that each proposal is assessed consistently, even as they are applied with contextual nuance.

The dimension “Program design and strategic alignment” addresses many high-level elements, focusing on how likely programs are to achieve progress toward some of the Wikimedia movement's strategic priorities. Programs need to show clear links between strategic priorities and program objectives, program objectives and targets, and targets and program activities. In addition, programs should focus on areas that will impact participation, reach, and quality on the Wikimedia projects, with an emphasis on approaches and programs that increase diversity.

Meanwhile, the dimension “Organizational capacity and effectiveness” looks at the ability of an organization to achieve its plan based on both past experience (as an indicator of future success) and current practice. An effective organization both learns from past experience and contributes to movement learning in the process. This dimension also includes effective and stable leadership that models good practices, and a rational approach to growth that is supported by the organization’s strategy and context.

Finally, we analyze the organization’s past experience with and current plan for budgeting. Here, we look at how the organization has performed against its plans and budgets in the past.

To complete the assessment, we assess each of the criteria in each category according to the following scale, which is not necessarily linear:

• ' Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design and strategic alignment
P1. Strategy and alignment	Neither	WMUK has created a detailed strategic plan and a proposal that is in line with its strategy. Some program objectives are strongly aligned with Wikimedia strategic priorities (ex: significant potential for GLAM partnerships), while others are not strongly aligned (ex: focus on QRpedia, work on removing barriers to access, advocacy work that is not specific). An overall lack of focus detracts from WMUK's ability to realize its strategic potential.
P2. Targets and logic models	Concern	Some programs have SMART objectives with clear targets, but not all targets are relevant. There are gaps in WMUK's targets in key areas of organizational focus such as article content on English Wikipedia and use of files uploaded through its work. While we appreciate the improvements WMUK has made in terms of tracking and measuring its work, more is needed at this level of funding.
P3. Needs assessments	Neither	WMUK conducts needs assessments to determine content gaps and in some cases leads programs that are based on clear statements of need. In other cases,WMUK implements programs that do not seem to be based on clear strategic needs. At this level of funding, WMUK should have a more sophisticated approach to determining whether, where and why its programs are needed.
P4. Potential for impact at scale	Concern	The funds requested are not comensurate to potential impact. We do see great potential in WMUK's GLAM partnerships, but this could be a stronger focus in its proposal.
P5. Evaluation methods	Strength	We have been pleased to see significant improvement in WMUK's evaluation capacities over the past year and note that WMUK is now tracking and reporting on many outcomes.
P6. New ideas and approaches, or thoughtful adaptations	Neither	WMUK is not always thoughtful about adapting new approaches that best match its strengths and capacities (ex: technology work); in other areas, WMUK has thoughtfully refined its approaches (ex: Wikipedians in Residence).
P7. Diversity	Strength	WMUK has done some encouraging work in the gender gap area, such as the 'Women in Science' editathons and the Ada Lovelace days, that have focused on growing the proportion of women participating in its events. However, we encourage them to have a focus and targets around gender diversity. In addition, we appreciate the diversity on its Board.
Organizational capacity and effectiveness
O1. Past results	Strength	WMUK has demonstrated significant success through its GLAM and Wales work in particular in 2014, and considerable improvement in measuring and documenting its success. On the other hand, WMUK is pursuing some initiatives (such as software development) where it has not had past success, and is not leveraging its current strengths adequately. WMUK's results should be considered in context of the significant funding requested.
O2. Stability and leadership	Neither	This may be considered a strength and a concern.WMUK's board has undergone changes that include a shift toward a focus on strategy and evaluation, which we appreciate. We are awaiting the impact of this transition over time.
O3. Learning	Strength	WMUK has developed good systems for documenting learning and often applies learning to its programmatic approaches. WMUK has shown a willingness to change as a result of learning.
O4. Improving movement practices	Strength	WMUK has shown an eagerness to share information about its work with others (for example, the report produced on Wikipedians in Residence), and have also been increasing collaboration with other chapters.
5. Community engagement	Neither	While WMUK has a responsive strategy to address the interests of community members, WMUK needs to be more proactive in engaging its vast and active community for strategically focused programs.
O6. Capacity	Concern	WMUK has proposed several programs without having proven capacity to do them (software development, work on barriers to access) in addition to programs in which it has strong experience and high potential (GLAM partnerships). WMUK's plan is not focused in areas where it is likely to have the most impact.
Budget
1. Past budgeting and spending	Concern	WMUK has spent inconsistently and has used money from its reserves to maintain budget growth and staff growth, even in years when the FDC did not grant funding to support its growth.
B2. Proposed budget is realistic	Concern	While the budget is comprehensive and clear, the resources needed to implement WMUK's programs (particularly staff resources) do not correspond to potential impact based on the current proposal targets.
B3. Budget is focused on programmatic impact	Major concern	The grant request is not likely to yield commensurate impact, as the program targets do not merit such a large increase. The large staffing increase to 12.4 FTEs (from current 9.4) proposed is not likely to lead to significantly increased impact.

This staff proposal assessment is the work of FDC staff and is submitted by: Winifred Olliff (FDC Support Team) talk 20:47, 8 November 2014 (UTC) WMUK

Staff proposal assessment rubric description

This rubric assesses annual plan grant proposals across the three dimensions of (1) Program design and strategic alignment, (2) Organizational capacity and effectiveness, and (3) Budgeting by identifying strengths and concerns in each of these areas.

In each assessment, an organization’s context is taken into account and so criteria may be applied differently in different contexts. For example, a smaller organization with less experience will not be expected to have a formal learning and evaluation program that a larger more experienced organization might have, and so what may constitute a major strength for a smaller organization in the same category may not constitute a major strength for a larger organization. With this approach, the criteria listed here are not weighted equally, and so an “overall assessment” cannot be determined simply by adding up the numbers of strengths and concerns identified for each dimension. Also note that the scale here is not linear and so we will not assign any numerical values to the assessments given for each criterion. More information about each criterion is shared in the assessment. We still believe a rubric like this is helpful, as it ensures that staff analysis of each proposal is based on the same criteria, even as they are applied in a nuanced way.

The dimension “Program design and strategic alignment” addresses many high-level elements, focusing on how likely programs are to achieve progress toward some of the Wikimedia movement's strategic priorities. Programs need to show clear links between strategic priorities and program objectives, program objectives and targets, and targets and program activities. In addition, programs should focus on areas that will impact participation, reach, and quality on the Wikimedia projects, with an emphasis on approaches and programs that increase diversity.

Meanwhile, the dimension “Organizational capacity and effectiveness” looks at the ability of an organization to achieve its plan based on both past experience (as an indicator of future success) and current practice. An effective organization both learns from past experience and contributes to movement learning in the process. This dimension also includes effective and stable leadership that models good practices, and a rational approach to growth that is supported by the organization’s strategy and context.

Finally, we analyze the organization’s past experience with and current plan for budgeting. Here, we look at how the organization has performed against its plans and budgets in the past.

To complete the assessment, we assess each of the criterion in each category according to the following scale, which is not necessarily linear:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Description
Program design and strategic alignment
P1. Strategy and alignment	Programs objectives are strongly aligned with the Wikimedia strategic priorities of increasing reach, improving quality, or increasing participation. The organization has a strategic plan in place.
P2. Targets and logic models	Programs have specific, measurable, time-bound objectives with clear targets, and programs present clear links between planned activities and desired results.
P3. Needs assessments	Program objectives are based on needs assessed in consultation with stakeholders, including the communities and volunteers that are working together with this organization to achieve its program objectives.
P4. Potential for impact at scale	Programs could lead to significant impact on the Wikimedia strategic priorities of increasing reach, improving quality, or increasing participation, at scale and corresponding to the amount of funds requested.
P5. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems.
P6. New ideas and approaches, or thoughtful adaptations	Programs will test new ideas and approaches or will thoughtfully adapt ideas and approaches that may work in this organization’s context.
P7. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational capacity and effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Stability and leadership	Stable, effective leadership and good governance will enable this plan to succeed. Any leadership transitions are managed effectively.
O3. Learning from the past	This organization is addressing risks and challenges effectively, is learning from and documenting past experiences, and is applying learning to improve its current and planned programs.
O4. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond.
O5. Community engagement	This organization effectively engages key communities and volunteers in the planning and implementation of its work.
O6. Capacity	This organization has the resources and ability (for example, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Proposed budget is realistic	This proposal includes a budget that is comprehensive and clear, and that corresponds to the activities proposed.
B3. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.