Jump to content

Grants:APG/Proposals/2013-2014 round1/Wikimedia Deutschland e.V./Staff proposal assessment

Add links
From Meta, a Wikimedia project coordination wiki


Wikimedia Deutschland e.V. received a funds allocation in the amount of €1,296,000 (equal to about US$1,750,000) in 2013-2014 Round 1. This funds allocation was recommended by the Funds Dissemination Committee on 15 November 2013 and approved by the WMF Board of Trustees on 31 December 2013.

Applicants should read this document to familiarise themselves with the assessment criteria; however, it should be edited only by FDC-related staff. The staff will use the document to set out their objective assessment of each funding request from eligible entities – specifically, to determine the extent to which plans align with the mission goals of the Wikimedia movement, and the extent to which the entity is well-placed to execute those plans effectively. The staff will base this report on proposals from entities and input from the community and other stakeholders (including, where required, deeper due-diligence on the entity's previous track record within the Wikimedia movement). The staff will provide completed reports to the FDC, which will then make a recommendation to the Board of Trustees on funding allocations. See a note from FDC staff explaining the process is more detail here: FDC portal/Note on staff proposal assessments 2013-14 Round 1.


Methodology behind the 2013-2014 Round 1 FDC staff assessments

The FDC staff assessments provide a snapshot of the staff review of Annual Plan Grant (FDC) proposals, and identify both strengths and concerns with the work proposed.

  • We did this through both portfolio analysis - looking across proposals and calibrating across different organizations - as well as assessing the context of each individual organization and its proposal. We did our best to be as nuanced as possible. For example, we understand that an organization that is beginning to formalize structures and bring on staff may have far fewer resources for learning and evaluation processes than an organization with a longer history and full-time staff. At the same time, a larger organization is also likely to have complex program structures and processes, that reflect in its use of metrics and measures.
  • We significantly increased the inputs to the staff assessment process from last year. They include:
    • The proposals themselves, with the attached documents (strategic plans, annual plans, detailed budgets etc)
    • Past and current reports (FDC quarterly progress reports, grant reports etc)
    • A portfolio view of financials across proposals: FDC_portal/Proposals/2013-2014_round1/Financial_overview.
    • Internal financial, evaluation and compliance inputs from the WMF Finance, Programs, Legal and Grantmaking teams.
  • We used a few key principles for the assessment, across all proposals. These are not new, and are detailed in the narrative and scoring sections of the assessment. They include:
    • Impact on Wikimedia projects and the global movement. We looked for the rationale behind the work organizations do, aligning them to the Wikimedia strategic priorities and goals. We looked particularly for direct and indirect impact of this work on Wikimedia projects; for example, growth in contributors and content donation. Most importantly, we looked at whether the funds requested were justified in the context of current and potential impact of the proposed work.
    • Organizational strategy, leadership and governance. We looked for evidence that an organization had a good understanding of its own context, and the needs, opportunities and challenges of the communities it seeks to serve or partner with. We looked for board, staff and volunteer leadership that is committed, effective and engaged with these communities and practising movement values of openness and transparency.
    • Rates of growth and financial management. In this round, only two returning applicants asked for funds within the recommended maximum growth rate in movement resources (annual plan grants/FDC allocations) of 20%, which is itself a significant rate in many contexts. In addition, many organizations are underspending on previous grant allocations and/or have extensive reserves, which makes overall growth rates even higher. We appreciated organizations that were not spending for spending’s sake, and being careful and prudent about resources; we hope this continues. However, significant underspends (particularly on programs) across a number of organizations flag the challenges of budgeting, planning and program design. It may be that the most effective and impactful programs do not always need significant resources. We specifically looked at the nature of long-time underspends in the context of organizations with a history of overbudgetting and underspending.
    • Program design, learning and evaluation. We recognize that the Wikimedia movement at large is still developing a more nuanced understanding of how we measure our impact, as volunteers and as organizations. We looked at each organization’s self-defined metrics and measures for success, both qualitative and quantitative. We searched for indicators that went beyond process (or outputs) and looked at intended outcomes and specific goals or targets. We recognized, however, that many organizations are in the process of establishing baselines for their work, and incorporating learning into program design and re-design. We appreciated organizations that were open about these challenges and were honest and reflective about their own capacities and systems.

Note: We shared our nearly final draft assessments with all FDC applicants a day ahead of publishing them, so that organizations had some lead time for sharing these with their Board and staff. We then took all responses into consideration where possible, making factual changes and clarifying our statements where necessary. We encouraged organizations to note all substantive comments and concerns on the Discussion pages of these assessments.

Proposal overview

Name of eligible entity Wikimedia Deustchland
Summary staff assessment [substantial concerns: none/some/many] Many concerns
Total amount proposed to the FDC for the upcoming year annual plan in currency requested 1,800,000 EUR
Estimate of total amount proposed to the FDC for the upcoming year annual plan in US dollars 2,431,458 USD
Exchange rate used to calculate estimate in US dollars and date accessed (mm/dd/yy) 1.35081 US dollars to 1.00000 Euro (calculated 1 October 2013)
Date of proposal submission 1 October 2013
Adherence to proposal process [yes/no] yes
Adherence to legal, regulatory, and policy regulations [yes/no] yes
Adherence to US anti-terrorism laws [yes/no] yes


Financial summary

A detailed summary of the financial information from all proposals is here: financial overview document. Movement resources are defined as what the entity received through FDC allocations or grants or what the entity retained via payment processing in the prior funding cycle.

Projected 2013 Proposed 2014 Proposed change (as a +/− %)
Movement resources* 1,376,923 EUR 1,800,000 EUR 31%
Overall budget 4,218,359 EUR 5,610,000 EUR 33%
Number of staff 42.95 51.45 20%

Annual plan

Overview of staff assessment

Strengths

What are the top strengths of the plan?

Concerns

What are the top concerns with the plan?

  • Impact of proposed programs not commensurate with proposed request of US$ 2.4 million
  • Proposed growth in movement resources is too rapid in the context of underspending
  • Proposed growth of both budget and staff is not sustainable if funding options become limited

Strategic alignment

To what extent does the plan address the strategic goals and priorities of the movement?

  • Clear alignment with the strategic priorities across most programs along with clear rationale for this alignment is expected of an entity of WMDE’s scope and budget. WMDE’s Volunteer Support program is strongly aligned with Participation and its Institutional Partnerships program is closely aligned with Quality, and WMDE presents reasonably clear rationale for this alignment. The rationale for strategic alignment for Legal and Social framework is not clear in this proposal, however.
  • WMDE has developed detailed strategy documents to support its work, and has been thinking strategically about how to best focus its work.

Context of the entity and community

To what extent does the context of the entity (e.g., community participation, fundraising context, sociopolitical environment) enable or inhibit the entity's potential for impact? To what extent does the entity's experience enable or inhibit the entity's potential for impact?

  • WMDE is an experienced organization that has been entrusted in the past with a significant proportion of movement resources. This year’s proposed request of US$ 2.4M is 41% of the overall amount requested this round by 11 organizations.
  • WMDE plays a leadership role in the movement, and often provides guidance and mentorship to other chapters. WMDE is also starting to work on issues of diversity, particularly gender. This work may also be useful to the wider movement.
  • WMDE seeks to serve a large and very active community of contributors to German Wikimedia projects, although it has acknowledged challenges around its relationship with the community. However, the overall number of Wikipedia editors continues to decline.
  • The cooperation and complementarity between WMDE and neighbouring Wikimedia organizations working with the same language group (like WMCH and WMAT) is not clear in the proposal.
  • WMDE is positioned for global impact on the Wikimedia projects through programs like software development.

Feasibility and risks for the annual plan

How feasible is the annual plan? Specifically:

  • To what extent are the budget, staffing, availability of volunteers, and other resources realistic for achieving the anticipated outcomes?
  • Are the timelines realistic for the proposed initiatives and associated activities?
  • Are there acknowledged and/or unacknowledged risks in the proposal?

WMDE is proposing a 31% increase in movement resources in the context of a planned underspend in its current budget of roughly US$675,000. This growth in movement resources is beyond the maximum 20% increase recommended in the guardrails.

Feasibility

  • WMDE is strategic in consolidating and prioritising its activities into four focus areas, including software development, and is discontinuing some of its other programs in order to focus on these areas.
  • WMDE has a relatively stable and well-resourced staff and board, and WMDE is building capacity to measure its work.
  • WMDE has had successes with some past programs, including its book grants program, partnerships, and E.U.-level policy work. The software development program is enabled in particular by WMDE’s past year’s success with Wikidata.
  • WMDE is building its evaluation framework and has developed ‘change models’ that seek to explain the rationale behind each program. This may help with better measurement of impact.

Risks

  • WMDE’s proposed 2014 budget would reach over US$ 7.5 million, which may be unsustainable. A proposal for a request of US$ 2.4 million should have stronger direct impact on Quality (for example, enabling more and better content on the projects) and Participation (for example, recruiting and retaining new editors).
  • WMDE is planning to hire nine new positions in addition to increasing two positions to full-time status, bringing total staff to 51.45 part- and full-time staff. Adding US$ 742,500 in staffing cost seems neither prudent nor sustainable, given its current revenue plan. Only one of nine new positions is for its new focus area of software development, and five are devoted to evaluation, communications, and administration. This raises questions about how this staff growth corresponds to WMDE’s new focus areas.
  • Nine employees and more than US$ 1,000,000 are to be allocated to the Volunteer Support program, but this significant investment may not have commensurate impact on the Wikimedia projects, especially considering WMDE’s mixed record in past community support work (for example, Fact Check and the community budget have had uneven results and response).
  • Relationships among board, staff, and community have been challenging.
  • WMDE is not always forthcoming with sharing its challenges and lessons learned in its reports and is still working to find an appropriate level of detail to include in its reports. We recognize that this is a challenge for an organization that is reporting on so many activities. See, progress report for Q1 and progress report for Q2.
  • WMDE has provided change models for its programs; on the other hand, the metrics presented with these change models do not include clear targets or a strong focus on outcomes, which is expected for an organization of its scope and budget.

Summary of expert opinions and/or community commentary

The WMDE proposal received a fair number of comments from the community, as well as one expert subject matter review.

Summary of the opinions of subject-matter experts on the plan

  • Erik Moeller, Vice President of Product/Development at the Wikimedia Foundation, provided his review of Wikidata and on the WMDE plan more generally. He endorsed WMDE’s decision to resource software engineering noting that this work can have a large impact on the movement as a whole. He noted the budget presented for Wikidata is feasible and informed by past spending. He strongly supported continued investment in Wikidata, noting that WMF would not have the ability to to take on the project if WMDE did not invest in it.
  • He wrote: “Wikimedia Germany has demonstrated its ability to not only build a high-performing team working on the Wikidata project, but to do so in close partnership with the Wikimedia Foundation to ensure that we coordinate on architectural issues, deployment, and operations support.” He noted he’s a bit more skeptical about other software engineering work included in the plan by WMDE, but trusts the chapter to prioritize.

Summary of community commentary on the plan

  • For detailed comments from the community, the FDC and FDC staff, as well as responses from the organizations themselves, please visit the proposal form discussion page. The summary offers a brief overview of the questions or comments from community members, and is not an endorsement of the views expressed on the proposal form discussion page.
  • One member of the community asked questions about World Book and Copyright Day as well as about the fundraising model of WMDE.
  • Another member of the community endorsed the proposal, particularly WMDE’s work on Wikidata and Commons.
  • One member expressed concern about the size of WMDE’s grant request, citing that it could be unfair to other chapters if WMDE receives half of the funds available.
  • Another community member noted some degree of concern around the high office and staff expenses of WMDE but highlights its support to German Wikimedians as well as to the movement world-wide.
  • Several expressed concern at the high level of funds requested by WMDE. One of them suggests comparing the context of the UK to Germany and looking at the number of dollars spent per citizen.
  • One user inquired about how WMDE would reposition itself if it did not receive the full requested amount.
  • Others discussed the Fact Check Program.

Scores, comments and feedback

Scores provide only a preliminary assessment for consideration by the FDC. The rubric for scoring is below.

Dimensions Criteria Score (1–5) Comments and feedback
Potential for impact against strategic priorities (A) The plan addresses the Wikimedia movement strategic priorities of the Wikimedia movement (infrastructure, participation, quality, reach, and innovation) 3.50 Programs are focused and most programs are strongly aligned with the strategic priorities, particularly Quality and Participation, but the rationale presented is not always clear or commensurate with the size of this request.
(B) TThe initiatives in the plan have the potential to lead to movement strategic priorities, if executed well 4.00 Significant potential impact on German language community and on the global movement through projects like Wikidata.
(C) The initiatives in the plan have the potential to lead to significant impact, given context and amount of funds proposed 2.50 Initiatives are likely to have significant impact in the broader movement context, but some initiatives may not have impact commensurate with the size of this request (US$2.4 million).
Ability to execute (D) The entity has the resources, skills, and capacity (staff, volunteer, non-staff) for the plan to succeed 3.00 Committed and well-resourced staff, however proposed staff growth does not reflect the four strategic focus areas. The entity is well-resourced, with strong commitment from board and staff.
(E) The entity has a record of success in similar initiatives 3.50 With projects like Wikidata, partnerships, and policy work, the entity is building on past successes; in some cases, they are continuing programs that may not be successful (community budget, Fact Check)
(F) The entity's leadership is effective, committed, and relatively stable 3.00 Relations between board, staff, and community can be challenging.
Efficient use of funds (G) The funding requested is reasonable given the proposed initiatives 2.50 Amount requested for this proposal overall is too large with respect to impact; request for Wikidata is reasonable but prioritizing staff growth is not prudent at a time when WMDE is redefining its strategic focus.
(H) The entity has a record of using funds efficiently and staying on budget 2.50 Projecting a significant underspend on its current FDC funds allocation.
Quality of proposed measures of success (I) Clear indicators of success are outlined in the plan 3.50 Clear indicators are focused on outcomes and explained with change models, but do not include clear targets to measure progress against goals.
(J) A plan is in place to track the proposed metrics 4.00 Provides change models to explain plans.
(K) The entity has a feasible plan in place to track the proposed metrics 4.00 Staff resources are in place to monitor and track the plan.
Potential to add knowledge and/or other benefits to the movement (L) The initiatives in the plan, if successful, could be helpful and/or productively adapted elsewhere in the movement 4.00 WMDE's focused approach can be adapted by others in the movement.
(M) The entity is ready to share with the movement the lessons learned from the initiatives 4.00 Active within the movement and reports regularly in German and English and learns and shares with other movement groups; is a leader in the movement. However, WMDE can be more open about sharing challenges and is working to balance the level of detail included in its reports.

Scoring rubric

Dimensions Evaluation criteria 1 = Weak or no alignment with criterion 3 = Moderate alignment with criterion 5 = Strong alignment with criterion
Potential for impact against strategic priorities (A) The plan addresses the Wikimedia movement strategic priorities The plan will not directly address any strategic priorities The plan will partially address strategic priorities or other goals that are closely related to the strategic priorities The plan will directly address several strategic priorities
(B) The initiatives have the potential to lead to the strategic priorities, if executed well The initiatives have no clear connection to strategic priorities Some or all of the initiatives have an unclear relationship to strategic priorities The initiatives have been shown (by the entity or the movement) to lead to strategic priorities
(C) The initiatives have the potential to lead to significant impact, given context and amount of funds proposed The initiatives will have marginal impact in the local community and the broader Wikimedia community, given context and amount of funds proposed The initiatives proposed may have a significant impact in the local community, given context and amount of funds proposed The initiatives could have significant impact at scale, both in the entity's local community and in the broader movement, given context and amount of funds proposed
Ability to execute (D) The entity has the resources, skills, and capacity (staff, volunteer, non-staff) for the plan to succeed The entity has neither the number nor the amount of resources, skills, and capacity needed for success. The entity has some relevant staff and volunteer capacity, but may be under-resourced for some initiatives The entity is well-resourced with the relevant skills and capacity for success
(E) The entity has a record of success in similar initiatives The entity has had trouble succeeding in similar initiatives The entity has not engaged in similar initiatives in the past The entity has conducted similar initiatives in the past and has been successful in achieving desired results
(F) The entity's leadership is effective, committed, and relatively stable Leadership has been unstable (e.g., changing on an annual basis) Current leadership is committed to the entity; some concerns exist around stability or effectiveness of leadership Current and past leaders have demonstrated the ability to develop and execute on strategic plans; leadership team has been stable and transitions have been smooth
Efficient use of funds (G) The funding requested is reasonable given the proposed initiatives Based on FDC experience and relative to other proposals, the plan contains significant over- or under-budgeting, and/or has not thoroughly accounted for costs Based on FDC experience and relative to other proposals, the budget is generally consistent with the initiatives; some initiatives are likely to be over- or under-budgeted Based on FDC experience and relative to other proposals, the budget is reasonable, and rigorous cost projections have been presented
(H) The entity has a record of using funds efficiently and staying on budget The entity has a poor record of budgetary management: e.g., consistently and significantly below budget (such as 50% underspending), and drawing on organizational reserves due to overspending The entity has a middling record of budgetary management; e.g., generally on-budget for its biggest initiatives and cost items, properly recording any over- and under-spending, and with plans in place for improvements The entity has consistently strong record of financial management; e.g., consistently staying on or below budget (within ~5% of budget), and putting surplus funds towards the entity's and/or movement's goals
Quality of proposed measures of success (I) Clear indicators of success are outlined in the plan Initiatives have no associated indicators of success, or the proposed indicators are unrealistic and/or unmeasurable Each initiative has indicators of success, but at least some indicators are unrealistic or are not clearly connected to the initiative Each initiative has a realistic and carefully framed set of indicators of success associated with them, which are rigorous and time-bound
(J) A system is in place to track the proposed metrics The plan does not include methodology or timing for tracking and collecting metrics to measure progress against proposed indicators of success A proposal to track progress against indicators exists, but has flawed methodology or unrealistic timelines and initiatives The plan includes timing and tracking of proposed metrics, and additional initiatives (e.g., surveys, research) to track indicators
(K) The entity has a feasible system to track the proposed metrics No systems are in place to track metrics, or staff or volunteer capacity to manage metrics tracking There is some overall capacity (e.g., staff, volunteers) to track and monitor metrics, but the entity does not clearly have the relevant skills to do this High-quality systems, and the necessary skills and capacity to manage these systems, are in place
Potential to add knowledge and/or other benefits to the movement (L) The initiatives, if successful, could be helpful and/or productively adaptedelsewhere in the movement The initiatives would not be relevant to other movement groups Some initiatives may be relevant to other movement groups Many initiatives proposed are new or have seldom been tried elsewhere in the movement; if successful, other movement entities would benefit from doing similar initiatives
(M) The entity is ready to share with the movement the lessons learned from the initiatives The entity is isolated from other movement groups and rarely shares information; the entity's leaders and members seldom participate in movement-wide functions The entity is somewhat active within the movement, and is developing more relationships with other movement entities with which it could share lessons learned The entity has regular communication with other movement entities, is a regular participant in Wikimedia events, and has a track record of sharing with others

This form was created by: KLove (WMF) (talk)


Retrieved from "https://meta.wikimedia.org/w/index.php?title=Grants:APG/Proposals/2013-2014_round1/Wikimedia_Deutschland_e.V./Staff_proposal_assessment&oldid=15333575"