Grants talk:Project/Fieldback Catalytics

Scope

This proposal is extremely confusing. 80 k$ to install some analytics software somewhere, ok. What kind of software does it need to be, why is it so complicated to install? Have you assessed existing analytics software and why is not fit for purpose? Are you going to develop an existing one and if yes which and how?

What does "commit to Github" mean? Github is not synonymous with git; do you say Github because there's already some existing project you have in mind? What will the license be?

The description leaves this entirely to the imagination, but it seems you're saying you'd like to install something like Matomo on the various machines. If each community is going to have its own analytics server fed and consulted over an existing local network, I fail to see how this is complicated. If the problem is aggregation of such statistics from places which have poor or no connectivity to each other, that's interesting but I find it very hard to understand what you're going to do about it or how you concluded that it's going to need that amount of work. I suppose there are already feature requests or previous attempts for some proxy system to aggregate statistics over the network or other methods. Nemo 12:19, 23 February 2020 (UTC)

@Nemo bis: The organizers can speak for themselves but I will share my perspective as an advisor to this project.

An outcome of this project is an educator's guide for best social, ethical, and cultural practice in applying user analytics in offline Wikipedia readers. As you say, the challenge here is not technical or in the software. Instead, the challenge is in presenting ethical recommendations for how to report these analytics. On the ground, organizations which use offline Wikipedia devices raise many challenging issues which are hard to discuss.

Various groups of Wikipedians with off-wiki partner organizations have been deploying offline copies of Wikipedia for several years. Among the most common feedback from community partners is that they want traffic reports comparable to the online Wikipedia's Pageviews Analysis, and right now offline deployment teams do not provide this information because it seems harmful to users. The problem is that tracking software is too easy to use, and by default tracks everything that everyone does including reporting the reading history of each user. In online Wikipedia community culture, we value the privacy of readers and do not track individual users' reading activity. It will not be possible to track offline devices exactly as we do tracking in online Wikipedia, so someone needs to make an offline tracking plan then communicate to the Wikipedia community how this differs from online tracking. Addressing the ethical challenge of producing fair metrics reporting for offline readers would lower the barrier for local partners like schools and clinics to get the reports they need to justify their own engagement and investment in Wikipedia.

The sooner someone does this kind of research and publication, the sooner that the Wikipedia community can consider the issues and discuss best practices for offline deployment, and the sooner that we can collect data about the impact of offline programs. Blue Rasberry (talk) 14:31, 27 February 2020 (UTC)

What you describe are very nice goals, but I see no connection between the current proposal and the goals described above, or at least no reasonable path which would bring the proposed activities to the outcomes you hope. Writing a report is only one of a dozen subpoints and Kiwix is barely mentioned.

If this is primarily a research project, I would expect more background information on what literature the authors have studied, how they plan to build on it, what their relevant experience is. As it happens, there's plenty of research in this space: I can recommend our friend's A reading list on differential privacy and Desfontaines, Mohammadi, Krahmer and Basin (2019) for a formal version.

If the focus is to measure the usage of Kiwix, I would suggest to start smaller by mentoring a mw:GSoC project to add some such features to Kiwix itself. It may not be particularly hard to add a couple simple counters such as number of pages shown, searches performed etc., and then send such data to appropriate servers, similar to the statistics collection in several GNU/Linux distributions. Nemo 06:37, 28 February 2020 (UTC)

@Nemo bis: I will continue to answer. Can you say under what circumstances you could support this proposal?

As you say, this proposal barely mentions Kiwix. The explanation is that offline Wikipedia has several parts, Kiwix is one part, and this project has another focus.

You asked for the researcher's relevant experience and backgrounds.

Eric has been managing non-Wikipedia offline education projects in Africa for some years through wakoma.co/ and other projects. Adam has been representing en:One Laptop per Child for 15 years and is now managing unleashkids.org/ and bringing in kidsoncomputers.org for continued Wikipedia engagement in this project. This proposal is more of a development project than a research project. If you have other questions about the organizer's background, perspective, and experience, then I can get that information. I expect that you can appreciate that they have many years and an established social network in this space. I appreciate that these various projects have been turning from various past models of management to the Wikipedia model of community engagement for their futures.

Those papers you shared are rather academic. This project has a more practical focus of reporting how things are now, what software is available, and what is the cheapest, easiest, and best way to model a default practice recommendation. This project is for the benefit of on-the-ground users and less about the academic ideal practice of what we should desire if there were long term development funds.

And there are long term development funds as the Wikimedia Foundation seems to have spent a few million dollars on this already. If we had a small community project like this to complement that WMF investment, then the precedent of this project would also assist in opening up the WMF's programs to community participation. I would like to get more resources to the community in this space so that community, WMF, and external partners can have better communication with each other about goals and standards.

Last I will list some various aspects of offline engagement to clarify that this project is and is not.

1. content editing - typical Wikipedia.org experience
2. content curation - Wikipedia 1.0, which is selecting certain content to include. Kiwix does this in a more automated way, but 1.0 is the human-centered project for this
3. platform - Kiwix
4. hardware - Internet in a Box or the Wiki Project Med/App
5. analytics - reports of user activity
6. deployment - on the ground distribution, as with en:Wikipedia:Offline Internet Resources for Latin America
7. documentation / communication
8. ethics - develop the discourse of thought specific to Wikipedia's practice

This project will mostly examine hardware, analytics, deployment, documentation, and ethics. It will report what various players already do, document what is normal for now, and report what seems to align with wiki ethics and what is a challenge. This will happen in the context of live field deployment. Kiwix participates in Google Summer of Code, which is great, but projects in Google's program are more technical than about field deployment or establishing ethical practice so not the right fit for this project's focus.

i know this is a lot. Please be direct in asking the hardest questions you have. Thanks for the feedback till now. Blue Rasberry (talk) 14:55, 9 March 2020 (UTC)

Eligibility confirmed, Round 1 2020

This Project Grants proposal is under review!

We've confirmed your proposal is eligible for Round 1 2020 review. Please feel free to ask questions and make changes to this proposal as discussions continue during the community comments period, through March 16, 2020.

The Project Grant committee's formal review for Round 1 2020 will occur March 17 - April 8, 2020. We ask that you refrain from making changes to your proposal during the committee review period, so we can be sure that all committee members are seeing the same version of the proposal.

Grantees will be announced Friday, May 15, 2020.

Any changes to the review calendar will be posted on the Round 1 2020 schedule.

Questions? Contact us at projectgrants wikimedia  · org.

I JethroBT (WMF) (talk) 20:15, 2 March 2020 (UTC)

Notifications

It is challenging to find communities which understand the idea of Offline Wikipedia. It is more challenging to find people who understand Offline Wikipedia, and the demand for analytics monitoring, and the ethical challenges for doing this. I did notifications in these places:

• English Wikipedia WikiProject Medicine
• English Wikipedia non-discrimination policy

If anyone has ideas for whom I should contact then let me know. This one is difficult.

Thanks. Blue Rasberry (talk) 15:03, 9 March 2020 (UTC)

Please complete survey for your Project Grants proposal

Dear Eric Nitschke, Adam Holt, Bluerasberry, and Dr. David Johnson,

We have sent you a survey link to the email address you provided for this Project Grants proposal. We need you to open the email and fill out the survey as soon as possible. We have emailed you twice without response (on March 20 and March 23), and we are not sure if you still wish for your proposal to be reviewed. If we do not receive your survey response by March 31, 2020, we will mark your proposal withdrawn.

We hope to hear from you!

Warm regards,

--Marti (WMF) (talk) 02:47, 28 March 2020 (UTC)

Aggregated feedback from the committee for Fieldback Catalytics

Scoring rubric	Score
(A) Impact potential Does it have the potential to increase gender diversity in Wikimedia projects, either in terms of content, contributors, or both? Does it have the potential for online impact? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	3.6
(B) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support?	3.4
(C) Ability to execute Can the scope be accomplished in the proposed timeframe? Is the budget realistic/efficient ? Do the participants have the necessary skills/experience?	6.0
(D) Measures of success Are there both quantitative and qualitative measures of success? Are they realistic? Can they be measured?	2.4
Additional comments from the Committee: I don't see a huge impact. I agree with the concerns in the discussion page and I am quite confused by this statement in the discussion page: "an educator's guide for best social, ethical, and cultural practice in applying user analytics in offline Wikipedia readers" which is quite fascinating to read but it is unclear. Why does this guide cost so much? Why must it involve society, ethics and culture if it's a guide to collect data? Wishing for a more precision formulation of the problem(s) to be solved. Not sure how possible outcomes of this project fit with Wikimedia's strategic priorities. Not sure it is sustainable: not much real wiki community involved. We see here a third-party project who tries to align its activity with Wikipedia by stating some words from wikimedia strategy. But I see no value for Wikimedia and its community, so, I don’t believe in its sustainability as the community isn’t involved I agree knowing this information would be interesting, but I do not know how the information will further development or impact current efforts. This proposal is unnecessarily confusing. So, it's difficult to evaluate its impact. Low innovation. I don't see a big idea behind the proposal. The potential impact is not clear due to lack of detail and the measures of success are vague. Here we have a creation of some research framework as a goal by creating a software platform. But what hypothesis do we want to test? Why? How will it help the movement? Not clear The outcomes and measures are rather vague and not as specific as I would like to see for a proposal of this expense. I think the project can be accomplished in 12 months or less but the budget seems excessive. The skills and experiences are present but it is not clear how they will help to reach unclear goals? It looks like they have some experience but not from the technical side I am not clear on the goals and scope of the proposal. This all seems to be rather vague and preliminary. Perhaps the proposers need to develop a more concrete methodology. I think the team have the relevant experience working in this area. Confused about the statement in the endorsement. The community engagement is insufficient. The grantees should consult with other teams using internet-in-a-box approaches to deliver Wikimedia content. I see no real community involvement here I would expect to see larger community engagement, especially in communities which benefit from offline resources. I don't see a significant community engagement. I note that one of our long time volunteers who consider herself a mother of offline-wiki raised a concern about this insufficient or no community engagement. At the moment I don't see an impact. The project asks for 80'000$ while the proposal is lesser than a draft. I would expect more detail to know the impact that the project will produce and the specific problem they would solve with this project. It seems to me only a complication of a complicated situation but not a solution to a problem. The project does not seem to be well thought through. The problem(s) that it aims at solving are not clear. If it is the lack of information about how off-line users utilize the internet-in-a-box then why such information can be obtained by a survey of those users? Too expensive, not much value for money and I am not sure that we are working on wikimedia goals here, not goals of IIAB I like the idea of this proposal, but before I feel comfortable funding it, many areas of the proposal need to be further detailed and developed. The budget is very confusing. 1. It's not clear from the proposals how many online workshops (with grantee) they would be organizing and what the USD 4000 budgeted for that would cover. 2. It is not clear from the proposals how many hours the hardware/software developer would commit to the work so that we can effectively evaluate the remuneration. Or is the 24,000 USD budgeted for the purchase of some hardware or software ? This needs to be clarified. 3. How many hours would the analytic advisor would be committing to the project and what is the cost per hour? 12,000USD for analytic advisor is much. 4. 12,000 USD was budgeted for ongoing troubleshooting (online work with local champions to customize content and troubleshoot installs). It isn't clear from the budget what this money is for. Is it for paying developers as well or something like that? 5. 4000USD was budgeted for Wikimanian online workshop. It isn't clear from the proposal if they meant Wikimanian 2021 as Wikimanian 2020 has been cancelled. In addition, it isn't clear what this 4000USD would be used for. 6. 13,200USD was budgeted for project management and unexpected cost. The grantee needs to separate this line of budget. We do need to know the exactly amount for project management and the number of hours to be committed by the project manager must be stated. I am unsure if I could recommend funding for this project. So, I'd stay neutral for now.

Opportunity to respond to committee comments in the next week

The Project Grants Committee has conducted a preliminary assessment of your proposal. Based on their initial review, a majority of committee reviewers have not recommended your proposal for funding. You can read more about their reasons for this decision in their comments above. Before the committee finalizes this decision, they would like to provide you with an opportunity to respond to their comments.

Next steps:

1. Aggregated committee comments from the committee are posted above. Note that these comments may vary, or even contradict each other, since they reflect the conclusions of multiple individual committee members who independently reviewed this proposal. We recommend that you review all the feedback carefully and post any responses, clarifications or questions on this talk page by 5pm UTC on Tuesday, May 11, 2021. If you make any revisions to your proposal based on committee feedback, we recommend that you also summarize the changes on your talkpage.
2. The committee will review any additional feedback you post on your talkpage before making a final funding decision. A decision will be announced Thursday, May 27, 2021.

Questions? Contact us at projectgrants wikimedia  · org.

--Marti (WMF) (talk) 03:16, 10 May 2020 (UTC)

Round 1 2020 decision

This project has not been selected for a Project Grant at this time.

We love that you took the chance to creatively improve the Wikimedia movement. The committee has reviewed this proposal and not recommended it for funding. This was a very competitive round with many good ideas, not all of which could be funded in spite of many merits. We appreciate your participation, and we hope you'll continue to stay engaged in the Wikimedia context.

Comments regarding this decision:
We will not be funding your project this round. The aggregated comments from the Project Grant Committee, posted above, provide insight into their thinking about your project. Especially, many committee members felt the proposal was not sufficiently developed and that its potential for impact was not clear.

Next steps: Applicants whose proposals are declined are welcome to consider resubmitting your application again in the future. You are welcome to request a consultation with staff to review any concerns with your proposal that contributed to a decline decision, and help you determine whether resubmission makes sense for your proposal.

Over the last year, the Wikimedia Foundation has been undergoing a community consultation process to launch a new grants strategy. Our proposed programs are posted on Meta here: Grants Strategy Relaunch 2020-2021. If you have suggestions about how we can improve our programs in the future, you can find information about how to give feedback here: Get involved. We are also currently seeking candidates to serve on regional grants committees and we'd appreciate it if you could help us spread the word to strong candidates--you can find out more here. We will launch our new programs in July 2021. If you are interested in submitting future proposals for funding, stay tuned to learn more about our future programs.